What is Mosaic AI Vector Search?

Mosaic AI Vector Search is Databricks' native vector database. It indexes data directly from Delta tables, auto-syncs when source data changes, inherits Unity Catalog governance, and integrates with Databricks Foundation Model APIs for embeddings and generation.

Does it support hybrid search?

Yes. Hybrid search combines dense vector retrieval with keyword (BM25) retrieval, then re-ranks. For domains with technical terminology, identifiers, or exact-match needs, hybrid usually beats pure vector search. The 2026 release makes hybrid the default for most index types.

How do I choose an embedding model?

Three factors: dimensionality (smaller = cheaper at scale), recall on your domain (test it; don't trust benchmarks), and latency tolerance. For general text in 2026, BGE M3 and Mosaic's MXBai-large are strong defaults. For code, specialized models still win. The book has benchmark results on three domains.

What is the cost model?

Two main components: (1) embedding generation at ingestion time, billed per token; (2) index serving DBUs, billed by index size and traffic. Storage is incidental. A 10M-document index serving moderate traffic typically costs hundreds, not thousands, of dollars per month, but it scales non-linearly above that.

Can I update or delete individual documents?

Yes. Delta sync mode keeps the index in lockstep with the source Delta table, inserts, updates, and deletes propagate automatically. Direct vector index mode requires you to manage upserts yourself but gives finer control.

How does it handle multi-tenancy?

Through Unity Catalog access controls plus metadata filtering. Documents carry tenant identifiers; queries filter by tenant ID before vector retrieval. For strict isolation, use separate indexes per tenant. For soft isolation, filtered shared indexes are fine.

Topic guide

· Covered in Volume 4 · The AI Lakehouse

Mosaic AI Vector Search: A Practitioner's Guide

Vector search is the substrate of every Databricks RAG and agent system. Mosaic AI Vector Search is the native option, and in most cases the right one. This guide covers Delta-sync vs direct-access indexes, embedding choice, hybrid retrieval, and the operational mechanics.

9 min read Updated 2026-05-18 By Ritesh Modi

Of all the new primitives Databricks shipped over the past few years, Mosaic AI Vector Search is the one with the highest leverage. It removed an entire integration tax, the constant pain of keeping an external vector DB in sync with the Delta tables that fed it. For Databricks-resident data in 2026, the default is to use Vector Search and ask "do I have a reason not to?"

The architectural pieces

A Vector Search deployment has three pieces:

The endpoint, a serving cluster that hosts one or more indexes.
The index, the queryable representation of vectors plus metadata.
The source, either a Delta table (Delta Sync index) or your own upsert calls (Direct Vector Access index).

Delta Sync is the default and right answer for most cases. You point an index at a Delta table with a text column, choose an embedding model, and Databricks handles the rest, embedding, indexing, syncing on source changes. Updates propagate within minutes.

Direct Vector Access is the escape hatch. You manage the embeddings and call upsert yourself. Use it when your embedding pipeline lives elsewhere, or when source data isn't naturally a Delta table.

Index types: which to choose

Two index variants matter for production:

Storage-optimized, designed for very large indexes (100M+ rows) with moderate QPS. Lower DBU cost per row, higher latency per query. Right answer for "we have a lot of data and search isn't the main bottleneck."
Performance-optimized, lower latency, higher QPS, higher cost per row. Right answer for user-facing applications with strict latency requirements.

Default to storage-optimized. Switch to performance-optimized when you have actual latency measurements showing you need it, not because the docs sounded faster.

Embedding model selection

The right embedding model is workload-dependent. The 2026 defaults that work for most teams:

General text: BGE-M3 or Mosaic MXBai-Embed-Large. Strong recall, multilingual, well-documented.
Code: BGE-Code or specialized code-embedding models. General text embeddings drop significantly on code.
Multi-modal: CLIP-derived models. Vector Search supports image vectors directly in the latest release.
High scale, cost-sensitive: Smaller open-source embedding models (e.g., MiniLM at 384 dims). Lower recall but 4-10× cheaper.

Test on your data. The MTEB leaderboard is a starting point, not a verdict. The book runs three embedding models on three benchmark corpora to show how rankings shift by domain.

Hybrid search is usually the right default

Pure vector search is good at semantic similarity, bad at exact matches. "Show me documents about Q3 2025" works fine for semantic intent but fails on the literal token Q3 2025. Identifiers, product codes, dates, technical terms, vectors smooth them out.

Hybrid search runs both vector retrieval and keyword (BM25) retrieval, then combines via reciprocal rank fusion. The penalty is a small latency increase. The benefit is a measurable recall improvement on most real workloads.

On Mosaic AI Vector Search, enabling hybrid is one parameter. Enable it, measure, keep it on if the eval improves.

Metadata filtering and where it pays off

Vector Search supports filtering on metadata columns at query time, by tenant, by date range, by document type. The filter is applied during retrieval, not after. This matters more than it sounds: post-filtering a top-k vector retrieval gives you fewer-than-k results when the filter is restrictive, which silently degrades quality.

Use metadata filtering generously. Multi-tenant systems should always filter by tenant ID. Time-bounded queries should filter by date. The performance penalty is usually negligible compared to the quality improvement.

The cost model in practice

Vector Search costs come from three places:

Embedding generation at ingestion. Token-priced. For a one-time backfill, this is your biggest line item. For ongoing sync, it's incremental.
Index serving DBUs. Charged by the hour the endpoint runs, regardless of QPS. Small indexes can share endpoints.
Query-time embedding. Each query is embedded before search. Negligible per query, real at high QPS.

The cost optimization that usually wins: share endpoints across multiple indexes when QPS is moderate. Don't run a dedicated endpoint for a 5-QPS workload.

What the book covers

The Vector Search chapter in Databricks for Practitioners covers:

End-to-end ingestion pipeline with idempotent re-indexing.
Embedding model benchmarking on three domains.
Hybrid search tuning, when to weight vector vs keyword.
Multi-tenant index design patterns.
Performance optimization: chunking, batch sizes, endpoint sizing.
Cost model with three traffic-tier examples.
Migration paths from external vector stores (Pinecone, Weaviate, pgvector).