AI Engineering

12 Factor AI Agents

The landscape is evolving from simple, reactive programs to sophisticated, autonomous entities capable of understanding context, making decisions and interacting dynamically with their environment and other systems.

Orchestrating these capabilities into systems that can reliably achieve complex goals requires more than just a powerful model. It requires structure, design, and a thoughtful approach to how the agent perceives, plans, acts, and interacts.

At its core, an agentic system is a computational entity designed to perceive its environment (both digital and potentially physical), make informed decisions based on those perceptions and a set of predefined or learned goals, and execute actions to achieve those goals autonomously. Unlike traditional software, which follows rigid, step-by-step instructions, agents exhibit a degree of flexibility and initiative.

Agentic systems are often characterized by features like autonomy, allowing them to act without constant human oversight; proactiveness, initiating actions towards their goals; and reactiveness, responding effectively to changes in their environment. They are fundamentally goal-oriented, constantly working towards objectives. A critical capability is tool use, enabling them to interact with external APIs, databases, or services – effectively reaching out beyond their immediate canvas. They possess memory, retain information across interactions, and can engage in communication with users, other systems, or even other agents operating on the same or connected canvases.

Natural language to Tool Calls

This entails treating the LLM as a mode for generating structured actions - JSON function calls or tool calls instead of text messages in the form of long essays. The model writes the API requests along with the parameters etc and your code executes them.

This pattern, when applied atomically, is the simple translation of a phrase like

can you create a payment link for $750 to Terri for sponsoring the february AI tinkerers meetup?

to a structured object that describes a Stripe API call like

{
  "function": {
    "name": "create_payment_link",
    "parameters": {
      "amount": 750,
      "customer": "cust_128934ddasf9",
      "product": "prod_8675309",
      "price": "prc_09874329fds",
      "quantity": 1,
      "memo": "Hey Jeff - see below for the payment link for the february ai tinkerers meetup"
    }
  }
}

You would use the API properly along with listing ids and using proper database calls and filling parameters with them to make appropriate third party function calls.

NOTE: While a full agent would then receive the API call result and loop with it, eventually returning something like

I've successfully created a payment link for $750 to Terri for sponsoring the february AI tinkerers meetup. Here's the link: https://buy.stripe.com/test_1234567890

Instead, We're actually going to skip that step here, and save it for another factor, which you may or may not want to also incorporate (up to you!)

Own Your Prompts

Don’t outsource your prompt engineering to a framework.

Key benefits of owning your prompts:

1. Full Control : Write exactly the instructions your agent needs, no black box abstractions.
2. Testing and Evals : Build tests and evals for your prompts just like you would for any other code
3. Iteration : Quickly modify prompts based on real-world performance.
4. Transparency : Know exactly what instructions your agent is working with
5. Role Hacking : Take advantage of APIs that support nonstandard usage of user/assistant roles - for example, the now-deprecated non-chat flavor of OpenAI “completions” API. This includes some so-called “model gaslighting” techniques.

Your prompts are the primary interface between your application logic and the LLM.

Having full control over your prompts gives you the flexibility and prompt control you need for production grade agents.

Own Your Context

You dont necessarily need to use standard message-based formats for conveying context to an LLM.

At any given point, your input to an LLM in an agent is “Here’s what’s happened so far, whats the next step”

Everything is context engineering. LLMs are stateless functions that turn inputs into outputs. To get the best outputs, you need to give them the best inputs.

Creating great context means:

• The prompt and instructions you give to the model.
• Any documents or external data you retrieve (RAG)
• Any past state, tool calls, results or other history.
• Any past messages or events from related but seperate histories/conversations (Memory)
• Instructions about what sorts of structured data to output.

Tools Are Structured Outputs

Tools dont need to be complex. At their core, they’re just structured output from your LLM that triggers deterministic code. For example, lets say you have two tools CreateIssue and SearchIssues. To ask an LLM to “use one of several tools” is just to ask it to output JSON we can parse into an object representing those tools.

The pattern is simple:

1. LLM outputs structured JSON
2. Deterministic code executes the appropriate action (like calling an external API)
3. Results are captured and fed back into the context

This creates a clean separation between the LLM's decision-making and your application's actions. The LLM decides what to do, but your code controls how it's done. Just because an LLM "called a tool" doesn't mean you have to go execute a specific corresponding function in the same way every time.

Unify Execution State

Even outside the AI world, many infrastructure systems try to seperate “execution state” from “business state”. For AI apps, this might involve complex abstractions to track things like current step, next step, waiting status, retry counts etc. This seperation creates complexity that may be worthwhile, but may be overkill for your use case.

As always, its up to you to decide whats right for your application. But dont think you have to manage them seperately.

More clearly:

• Execution state → Current step, next step, waiting status, retry counts etc.
• Business state → Whats happened in the agent workflow so far (e.g list of OpenAI messages, list of tool calls and results etc)

If possible, SIMPLIFY - unify these as much as possible.

In reality, you can engineer your application so that you can infer all execution state from the context window. In many cases, execution state (current step, waiting status, etc) is just metadata about what has happened so far.

You may have things that can’t go in the context window, like session ids, password contexts, etc, but your goal should be to minimize those things. By embracing factor 3 you can control what actually goes into the LLM.

This approach has several benefits:

1. Simplicity : One source of truth for all state.
2. Serialization : The thread is trivially serializable/deserializable.
3. Debugging : The entire history is visible in one place.
4. Flexibility : Easy to add new state by just adding new event types.
5. Recovery : Can resume from any point by just loading the thread.
6. Forking : Can fork the thread at any point by copying some subset of the thread into a new context/state ID.
7. Human interfaces and observability : Trivial to convert a thread into a human-readable markdown or rich Web app UI.

Launch/Pause/Resume with simple APIs

Agents are just programs, and we have things we expect from how to launch, query, resume, and stop them. It should be easy for users, apps, pipelines, and other agents to launch an agent with a simple API. Agents and their orchestrating deterministic code should be able to pause an agent when a long-running operation is needed. External triggers like webhooks should enable agents to resume from where they left off without deep integration with the agent orchestrator.

Contact Humans with Tool Calls

By default, LLM APIs rely on a fundamental HIGH-STAKES token choice: Are we returning plaintext content, or are we returning structured data?

You're putting a lot of weight on that choice of first token, which, in the the weather in tokyo case, is

"the"

but in the fetch_weather case, it's some special token to denote the start of a JSON object.

|JSON>

You might get better results by having the LLM always output json, and then declare it's intent with some natural language tokens like request_human_input or done_for_now (as opposed to a "proper" tool like check_weather_in_city).

Again, you might not get any performance boost from this, but you should experiment, and ensure you're free to try weird stuff to get the best results.

Own your control flow

If you own your control flow, you can do lots of fun things.

Build your own control structures that make sense for your specific use case. Specifically, certain types of tool calls may be reason to break out of the loop and wait for a response from a human or another long-running task like a training pipeline. You may also want to incorporate custom implementation of:

• summarization or caching of tool call results.
• LLM-as-judge on structured output
• context window compaction or other memory management
• logging, tracing, and metrics
• client-side rate limiting
• durable sleep/pause/”wait for the event”

Compact Errors in Context Window

This one is a little short but is worth mentioning. One of these benefits of agents is “self-healing” for short tasks, an LLM might call a tool that fails. Good LLMs have a fairly good chance of reading an error message or stack trace and figuring out what to change in a subsequent tool call.

Small Focused Agents

Rather than building monolithic agents that try to do everything, build small, focused agents that do one thing well. Agents are just one building block in a larger, mostly deterministic system.

The key insight here is about LLM limitations: the bigger and more complex a task is, the more steps it will take, which means a longer context window. As context grows, LLMs are more likely to get lost or lose focus. By keeping agents focused on specific domains with 3-10, maybe 20 steps max, we keep context windows manageable and LLM performance high.

Benefits of small, focused agents:

• Manageable context : Smaller context windows mean better LLM performance.
• Clear Responsibilities : Each agent has a well-defined scope and purpose.
• Better Reliability : Less chance of getting lost in complex workflows
• Easier Testing : Simpler to test and validate specific functionality.
• Improved debugging : Easier to identify and fix issues.

Trigger from anywhere, meet users where they are

Agents should be able to be triggered from anywhere according to where the user wants to trigger them.

Benefits:

• Meet users where they are: This helps you build AI applications that feel like real humans, or at the very least, digital coworkers
• Outer Loop Agents: Enable agents to be triggered by non-humans, e.g. events, crons, outages, whatever else. They may work for 5, 20, 90 minutes, but when they get to a critical point, they can contact a human for help, feedback, or approval
• High Stakes Tools: If you're able to quickly loop in a variety of humans, you can give agents access to higher stakes operations like sending external emails, updating production data and more. Maintaining clear standards gets you auditability and confidence in agents that perform bigger better things

Make your agent a stateless reducer

Blogs and Articles

AI Agents vs Workflows

Based on the article from Paul Iusztin, the key points revolve around a central framework called the "Autonomy Slider" for making a crucial architectural decision in AI engineering: when to use a predictable AI workflow versus a more autonomous AI agent[citation:0].

The table below summarizes the core distinctions and use cases.

🛠️ Practical Examples and a Guiding Principle

The article illustrates these concepts with concrete examples that progress in complexity:

• Document Summarization Workflow: A straightforward, predefined pipeline for generating document summaries, exemplifying a pure workflow[citation:0].
• Coding Agents (e.g., Gemini CLI): An agent that reasons about a coding task, proposes a plan (using tools), and iteratively executes and evaluates code. This often includes a human-in-the-loop for plan validation[citation:0].
• Vertical Hybrid AI Agents: Systems (e.g., for shopping or investing) that use a workflow to route predictable queries to fast, predefined paths, and send open-ended questions to an agent. This combines the strengths of both approaches[citation:0].
• Deep Research Hybrid Agents: Complex systems (inferred from tools like Perplexity's Deep Research) where an orchestrator agent breaks down a query, uses specialized agents to gather information in parallel, and iterates until it can synthesize a final report[citation:0].

The author's fundamental advice is to start simple. Begin with the most straightforward solution, like a single LLM call or a workflow. Only increase autonomy by moving to hybrid systems or full agents when your business case absolutely requires it, and always consider how to keep the user in control via the "autonomy slider"[citation:0].

I hope this summary helps you grasp these foundational concepts. Would you be interested in a deeper look at any of the specific examples, like the hybrid architectures?