AI features in Azure databases

A support engineer hits a fault. Somebody solved the same fault two years ago, wrote down what fixed it, and closed the ticket. The engineer searches, finds nothing, and works it out from scratch. The answer was in the database the whole time. The search could not reach it.

The cost of that is not abstract. It is the same diagnosis paid for twice, a vessel or a production line down for longer than it needed to be, and an organisation that cannot tell whether a fault is new or its fifth recurrence. The knowledge was captured. The organisation still could not use it.

That is a retrieval problem, not a model problem, and it is where this experiment starts. It also explains why a model alone does not fix it: the limiting factor is what the system can find, not how well it writes.

Azure Database for PostgreSQL and Azure Cosmos DB have both added vector search, full text search, hybrid ranking, and the ability to call a model directly. These arrived as feature announcements. What they do under a real question, what they cost, and where each one stops being the right tool are not in the announcements.

So the same 1000 records were loaded into both engines and into a graph, and every query was run live against Azure. Nothing here is a screenshot or a recorded result. Each claim is a query that executed, with its wall clock time and its request unit cost recorded.

Three questions were being answered:

• Where does keyword search beat vector search, and where does it collapse?


• What changes when the database calls the model itself, rather than an application doing it?


• Does a graph add anything that SQL over the same table cannot already do?

The third one produced the most uncomfortable answer. 

Both engines now ship the same broad capability set, at different stages of maturity.

The asymmetry is the interesting part. PostgreSQL gives the database its own identity and lets it call a model, which moves work out of the application entirely. Cosmos DB gives hybrid ranking as one function call and keeps latency low. Neither is a superset of the other.

Dataset. 1000 synthetic support tickets across maritime, energy, payments and retail, generated from a fixed seed so the results reproduce exactly. Each ticket has a title, a free text description written the way a person would write it, a resolution, a fault code, an asset, a team and a severity.

Two design choices decide whether the experiment says anything:

• The same fault is described as shuddering, juddering, trembling and knocking by different people. Without that variation there is nothing for vector search to beat keyword search at.


• The fault code is excluded from the embedded text. Include it and vector search half answers code lookups, which destroys the contrast being measured.

Stores.

Models. One Microsoft Foundry account. text-embedding-3-large for both stores, gpt-5.5 for the application, gpt-4.1-mini for generation inside PostgreSQL.

Authentication. Entra ID everywhere. Password authentication disabled on PostgreSQL, local authentication disabled on Cosmos DB and Foundry. No key or connection string exists anywhere in the build.

Measurement. A four page web application executes each query and reports wall clock time. A script runs every endpoint against every input and then clicks every control in a real browser. Numbers below come from that run.

The setup. Note the difference in the two lines on the right: for PostgreSQL the database calls the model itself, for Cosmos DB the application does it. 

The query contains no vector. azure_openai.create_embeddings() runs inside the SQL, authenticated by the server’s own managed identity.

Because a similarity search returns an ordinary relation, ordinary SQL aggregates over it: which teams, how many tickets, average hours to repair. This is the operation a separate vector store cannot perform, because the join does not exist across two systems.

With azure_ai.generate(), one statement embeds the question, searches, builds the prompt and calls the model. There is no application tier in the loop, so anything that can run SQL can do it: a scheduled job, a report, a trigger.

The cost of that convenience is latency. Every one of these timings includes the database going out to Foundry and back on the caller’s behalf, which is why the plain English search takes over a second. The next section shows what the same questions cost when the application does that work instead. 

Apache AGE runs inside the same PostgreSQL server used above. No second database and no sync job. Three of its four edge types restate columns that already exist: which asset a ticket is on, which team fixed it, which code it carried.

The fourth does not. SIMILAR_TO links each ticket to its two nearest neighbours by embedding, restricted to neighbours with a different fault code and a cosine distance under 0.40. That produced 209 edges over 1000 tickets. Same code neighbours are skipped because the existing CODED edge already connects those.

Measured against a retail point of sale fault:

The first walk is reproducible in SQL. SELECT DISTINCT asset FROM tickets WHERE error_code = … returns the identical rows, verified one by one. The second is not, because no column connects a checkout terminal to a ship’s bridge display.

Expanding from a retail point of sale fault. Grey links are the fault code, which SQL could also follow. Magenta links are the embedding-derived edge, and they reach four vessels and two substations that carry no ERR-6205 ticket at all.

Note also that the graph stores no text and no vectors, so it cannot be searched by meaning, only traversed. Vector search selects the entry node and the graph decides how far to walk from there.

The same 1000 tickets, in a document database rather than a relational one, with the application rather than the database calling the model.

Two results stand out.

The shaking query returns zero. The tickets describe that exact fault, using other words. Vector search then returns those tickets from a sentence that shares no vocabulary with them. Handed a fault code instead, vector search fails, because a code carries no meaning to embed. The two methods fail at opposite things.

Hybrid search costs roughly ten times a plain vector query, 70.5 RU against 7.1. It is one clause and it works well, but it is not free and should not be the default on every request.

The fusion itself is reciprocal rank fusion, which combines the two result lists by position rather than by score:

score = sum over lists of 1 / (k + rank)

This matters in practice, because it means the BM25 relevance score and the cosine distance never have to be made comparable. Only the ordering is used. It is also why the demo shows rank rather than the fused score: the score is an artefact of the constant k.

With both sets of numbers on the table, the latency gap is the clearest difference between the two engines. Cosmos DB answers a vector query in 190 ms where PostgreSQL takes 1.3 s, roughly seven times faster, because the application has already produced the vector before the query is sent. That is the trade: PostgreSQL removes the application and pays for it in latency.

The final test holds the model constant and varies only the retrieval. Same deployment, same prompt, same question. Each side receives whatever its search returned.

 

Same deployment, same prompt, same 1000 tickets. Only the retrieval differed.

The first answer is not a hallucination and not a weak model. It is accurate about what it was given, and wrong about reality, because the retrieval beneath it failed. A system that returns no results produces a confident denial rather than an error, which is why this failure mode reaches production.

It is also the expensive one. A wrong answer gets challenged. “We have no record of this” gets believed, and the engineer starts again from nothing. No alert fires, no error appears in a log, and the only visible symptom is work being repeated somewhere else in the organisation months later.

Findings that were not documented in anything read beforehand.

Cosmos DB autoscale has an irreversible floor. The maximum cannot later be set below one tenth of the highest value ever provisioned on that container. Raising throughput to 20000 briefly during a load permanently floored the container at 2000 RU/s.

Data plane and control plane RBAC are independent. A subscription Owner still receives 403 when reading documents without a Cosmos data plane role.

Throughput changes are control plane only. With local authentication disabled the data plane SDK cannot alter RU/s at all, regardless of identity.

azure_ai.generate() sends temperature = 0.2 with no override. Newer reasoning models accept only the default and reject the request, so in database generation required a separate gpt-4.1-mini deployment.

An embedding call inside ORDER BY is evaluated once per row. Over a cross join that is 1000 model calls per query. Moving it into its own CTE fixes it. The symptom presents as an unreliable network.

pg_diskann is limited to 2000 dimensions. text-embedding-3-large produces 3072, so PostgreSQL indexes a 1536 dimension prefix of the same model.

Apache AGE is preloaded on Azure. LOAD ‘age’, which most tutorials instruct, fails with a privilege error. shared_preload_libraries replaces rather than appends and changing it restarts the server.

The Gremlin API does not support Entra ID on the data plane. It requires an account key and a separate account, which is incompatible with a keyless deployment. AGE inside PostgreSQL provided a graph without a second database.

Role assignments take two to four minutes to propagate. During that window the error reads “Principal does not have access to API/Operation”, which is indistinguishable from a genuine misconfiguration.

The /models inference endpoint requires Azure AI Developer. Cognitive Services OpenAI User grants OpenAI/* but not MaaS/*.

A pooled connection survives a network change as a black hole. After the client machine changed address, the shared PostgreSQL connection remained open but unresponsive, and queries blocked indefinitely. connect_timeout, TCP keepalives and statement_timeout convert this into a recoverable error; measured recovery is 1.3 seconds.

Per day, in NOK, for the deployment described above:

The database dominates. Stopping the PostgreSQL server between sessions is the entire cost strategy. Model usage is consumption priced and negligible at this volume.

Two things follow from that. The search and retrieval capability itself is nearly free, because it is a feature of a database that is already being paid for rather than a separate product with its own licence and its own operations budget. And the meter that matters is compute hours on a server that can be stopped, not tokens, which is a more predictable line item than most AI spending.

Money spent on a better model does not buy a better answer if the retrieval is weak. Both sides of the final test held the same 1000 tickets. Only the search differed, and that alone separated a correct answer from a confident denial. The search layer is where the return on an AI investment is decided, and it is usually the cheapest part of the stack to improve.

The two search methods are complements, not competitors. Keyword search is exact and literal, and it is what a serial number, a part code or an invoice number needs. Vector search is approximate and semantic, and it is what a symptom described in someone’s own words needs. Each fails at what the other does well. Real questions contain both, which is the argument for hybrid rather than for choosing.

Hybrid search is not free. Ten times the request units of a vector query on this dataset. That is a per-query cost multiplier at production volume, so it belongs on the queries that mix a code with a description, not on every request by default.

PostgreSQL and Cosmos DB answer different questions, and the choice has consequences beyond performance. PostgreSQL keeps the semantic result inside SQL, where it can be joined, aggregated, and handed to a model without an application in between, which means a scheduled job or a report can use AI without anyone building a service first. Cosmos DB is the low latency path and gives hybrid fusion as a single clause. The first changes who in an organisation is able to deliver something. The second changes what the user waits for.

Keeping operational data and vectors in one engine removes a moving part. There is no second store to provision, secure, synchronise or reconcile, and no window in which the two disagree. The experiment never had to answer the question “which copy is right”, because there was only one.

A graph earns its place only through edges that are not columns. Traversing relationships that already exist as foreign keys is a GROUP BY in different syntax, and calling that a graph capability will not survive a competent question. An edge derived from embeddings is different: it reaches records that no WHERE clause can, and in this dataset it connected failures across four separate parts of the business that shared no code, no component and no domain.

Keyless is achievable end to end. Entra ID across both databases and the model account, with the database calling the model as its own managed identity, removed every key, password and connection string from the system. Nothing can leak from a repository or a configuration file, because nothing is there. The cost is understanding that data plane roles are assigned separately from control plane roles.

Silent failures dominate, and they are the ones that reach production. Every significant defect in this build produced no error: counts inflated by a cartesian product, unresolved tickets averaged as zero hour repairs, near duplicate generated text, a stale process serving old code, and a connection that had stopped responding. Nothing alerted. Assertions over the data and a scripted pass over every endpoint caught all of them, and both took under a day to write. On a system that answers questions rather than returning rows, that kind of check is not optional, because a wrong answer looks exactly like a right one.

Nothing in this experiment used Azure HorizonDB, but it changes the shape of the argument, so it belongs here.

The pattern behind the scaling question is familiar: teams want to stay on PostgreSQL and eventually hit a ceiling. HorizonDB is Microsoft’s own PostgreSQL, built for workloads past that point, and it is in public preview. It carries the same AI surface used above, pgvector, DiskANN indexing, hybrid search with reciprocal rank fusion, and the azure_ai extension, so the code in this experiment transfers rather than being rewritten.

What it adds is AI pipelines. Chunking, embedding, extraction, generation and ranking are declared in SQL as a pipeline that runs inside the database, on top of durable execution. The definition is a row in a system catalog. Execution survives a crash, retries failed steps, checkpoints partial work, and re-embeds only rows that are new or changed.

That closes the remaining gap. This experiment showed a database answering a question. A pipeline is the same idea applied to keeping the data ready to be asked: no orchestrator, no separate worker, no job that quietly stops running.

Both are preview features and should be read as direction rather than as something to put in front of customers this quarter.

Scope was limited to what the two databases do on their own. Three things were left out deliberately and would change the conclusions if included.

Agentic retrieval. Query planning, sub queries, iterative retrieval and reflection over multiple knowledge sources, as offered by Azure AI Search and Foundry IQ. That is a retrieval strategy built above the database, and it would have made the single query comparison meaningless.

Reranking. A semantic reranking model over the candidate set. PostgreSQL exposes azure_ai.rank() for this and it was verified as working, but nothing in the experiment used it.

Scale. 1000 rows is enough to show behaviour and not enough to show performance. The query planner in particular behaves differently: at this size it correctly ignores the vector index, because a sequential scan over a thousand rows is cheaper than the index startup cost.

Everything described here is published, including the Terraform for the Azure resources and the generator for the dataset.

https://github.com/Hanifff/ai-features-in-azure-databases

git clone https://github.com/Hanifff/ai-features-in-azure-databases.git cd ai-features-in-azure-databases az login ./scripts/setup.sh

That reads your current Azure context, provisions the resources, loads all three stores, and starts the application. The dataset is generated from a fixed seed, so the counts quoted in this post reproduce exactly.

The PostgreSQL server is the only meaningful cost. Stop it when you are not using it, and run terraform -chdir=infra/terraform destroy to remove everything.