AgentBricks: Production Agents on Databricks

Agents are the natural evolution of RAG. The moment your system needs to decide what to retrieve, call non-retrieval tools, or reason across multiple steps, you're building an agent, whether you call it that or not. AgentBricks is Databricks' answer to "what does the production version look like?"

This isn't a rebranded chat wrapper. AgentBricks is opinionated about the parts of agent engineering that everyone underinvests in: evaluation, tracing, deployment, governance. The orchestration logic is comparatively simple. The hard parts, knowing when your agent is silently regressing, governing what data it can touch, deploying without downtime, are where AgentBricks earns its keep.

What AgentBricks actually gives you

Four primitives sit at the core:

1. Agent definition, declarative spec for an agent: system prompt, tools, model choice, memory policy.
2. Tool calling, a typed interface for connecting agents to functions, SQL endpoints, vector indexes, and external APIs.
3. Evaluation, first-class integration with MLflow Evaluate, including agent-specific judges (trajectory correctness, tool-call accuracy, faithfulness).
4. Deployment & tracing, agents deploy to Databricks Model Serving. Every interaction is traced through MLflow.

The framework is intentionally narrower than the open-source alternatives. The narrowness is the point.

Tools are where most agents fail in production

A working agent depends on its tools more than its model. Bad tools, unclear names, ambiguous parameters, silent failures, produce bad agents regardless of how good the LLM is.

AgentBricks pushes you toward typed, narrowly-scoped tools. Each tool has a docstring (the LLM sees it), a typed parameter schema, and explicit error returns. The discipline matters: the docstring is part of your prompt engineering. If the LLM is calling the wrong tool, the fix is almost always in the tool description, not the system prompt.

On Databricks specifically, tool implementations can pull directly from the Lakehouse, Unity Catalog functions, SQL warehouses, Mosaic AI Vector Search indexes, Delta tables. Tools inherit the agent's Unity Catalog identity, so governance just works. No more "the agent has god mode in production" antipattern.

Evaluation is the differentiator

AgentBricks's serious advantage over rolling your own is the evaluation story. Three layers:

• Built-in judges, answer relevance, faithfulness, tool-call correctness, trajectory quality.
• Custom judges, define your own (LLM-as-judge or rule-based) and version them.
• Regression testing, run the eval set against every agent revision; track results in MLflow.

Build the eval set early. 50-100 trajectories is usually enough to start. Each trajectory is input → expected tool calls → expected final answer. Run it on every prompt change. Stop shipping changes that regress the eval, even if "it looks better in testing."

Memory: pick a model and stick to it

AgentBricks supports several memory models. The temptation is to combine them all; the reality is that each layer of memory adds debugging surface area. Start simple and add only when the use case forces it:

• None, stateless agent. Right answer for many tasks.
• Conversational, last N turns. Right answer for chat-style interactions.
• Summarized, periodic summarization compresses long conversations.
• Semantic / long-term, vector-backed memory store. Useful for agents that need recall across sessions.

The biggest mistake: defaulting to semantic memory because it sounds powerful. Semantic memory introduces retrieval failure modes on top of generation failure modes. Most production agents don't need it.

Deployment and serving

Agents deploy to Databricks Model Serving as ordinary endpoints. This means standard operational practices apply: blue-green deployments, traffic splitting, automatic rollback on error rate spike. The fact that the endpoint happens to be an agent is invisible to the operational tooling.

Pair this with MLflow tracing and you get something rare in agent land: a production system where you can answer "why did the agent do X" without re-running it. Every tool call, every intermediate reasoning step, every retrieved chunk is captured.

When AgentBricks is the wrong choice

AgentBricks is opinionated. The opinions are usually right, but not always:

• If you're not on Databricks, obviously not.
• If your agent is mostly orchestrating a complex workflow with many branches and recovery paths, a workflow engine (LangGraph, Temporal) may be a better fit, with AgentBricks called as a sub-component.
• If you have extremely tight latency budgets (sub-100ms), the agent framework overhead may be too high, pure RAG might be the answer.

The book has a decision tree for "AgentBricks vs alternative."

What the book covers

The AgentBricks chapter in Databricks for Practitioners walks through:

• Building a working agent from zero, system prompt, tools, memory, evaluation.
• Designing tool interfaces that the model actually uses correctly.
• The MLflow evaluation harness for agent regression testing.
• Deploying with traffic splitting and observability.
• Composing single agents into multi-agent systems.
• Common production failure modes, runaway loops, tool misfires, context bloat, and how to detect them.