What an AI agent is
Link to this sectionIn an AI agent, a language model steers the process. Anthropic separates that from a workflow: in a workflow, the model and the tools follow fixed paths that someone programmed beforehand. In an agent, the model decides how to proceed and which tools to use.
A chatbot is not an agent for that reason. OpenAI expressly leaves out simple chatbots, single model calls and classifiers. They use a model but do not let it steer a process.
A fixed automation is not an agent either, even if a model handles one of its steps. The steps and their order are set in advance.
What an AI agent consists of
Link to this sectionOpenAI names three parts:
- Model: The language model that reasons and decides.
- Tools: Functions or interfaces the agent uses to fetch data or trigger something, for example sending an email or changing a record.
- Instructions: Rules and limits that set how the agent behaves.
Anthropic also lists retrieval of knowledge and memory as additions to a model. Your application provides the memory. Anthropic's memory tool keeps it in files you control.
In 2022, researchers described under the name ReAct how a model can alternate between reasoning steps and actions.
Building it yourself: the loop
Link to this sectionA simple agent takes little code. At its core is a loop that Anthropic describes in five steps:
- You send the request and the descriptions of the tools to the model.
- The model answers with a tool call.
- Your code runs it.
- You send the result back to the model.
- This repeats for as long as the model asks for more tools.
The model does not run anything itself. Everything an agent does goes through your code, including every limit you set on it.
Anthropic advises starting with the vendors' APIs directly, because many patterns take only a few lines of code. Frameworks make it easier to get started but add layers on top of the model's inputs and responses. That can make errors harder to find. Whether you use a library or write it from scratch, model, tools and instructions stay the same.
Building it yourself means writing the loop and deciding on tools, permissions, limits and approvals. The next section covers exactly that.
What running it adds
Link to this sectionSix things come with an agent in operation.
Cost per call
Every step of the loop is a model call, and an agent makes several per task. Billing is by tokens, meaning the amount of text going into and coming out of the model, including the descriptions of the tools. Anthropic writes that agents often trade response time and cost for better results. So set a limit on the number of steps.
Errors
A model can state false content as if it were certain. NIST calls this confabulation. In an agent an error carries forward: Anthropic warns of errors that compound and recommends extensive testing in a sandboxed environment. If the agent reads outside text such as emails or web pages, that text can contain instructions the model follows. OWASP calls this indirect prompt injection.
Log
Record every tool call. The Model Context Protocol, a specification for servers that expose tools to language models, expressly recommends that clients log tool usage for audits. Anthropic advises showing an agent's planning steps.
Approval by a person
OpenAI names two triggers for a person in the loop: when the agent exceeds a set number of attempts, and for high-risk actions such as canceling orders, approving large refunds or making payments. OWASP recommends having a person confirm high-impact actions. The Model Context Protocol recommends that a person can always deny a tool call.
Permissions of the tools
A tool should not be able to do more than the task needs. OWASP names too many functions, too broad permissions and too much autonomy as the causes of excessive agency. OpenAI rates tools by risk: read-only or also writing, reversible or not, the permissions of the account, the financial impact.
Model changes
Vendors retire older models. Anthropic and OpenAI list shutdown dates in their documentation, mostly with a successor. Requests to a retired model fail, Anthropic writes, and it advises testing the new model well before the date. Applications built on a model need occasional updates for that reason.
Build it yourself, have it built, or no agent at all
Link to this sectionThe first question: can you write the steps down in advance? Then a fixed automation may be enough. Anthropic recommends looking for the simplest solution and adding complexity only when needed. Fixed workflows, it says, give predictability for well-defined tasks. OpenAI writes that when the criteria are unclear, a deterministic solution may do.
OpenAI sees added value mainly in three cases: decisions need judgment and exceptions, a rule set has become hard to maintain, or there is a lot of unstructured text, such as documents and conversations.
| Way | Fits when |
|---|---|
| No agent | you can write the steps down in advance. |
| Build it yourself | the task is small, the tools only read, a mistake can be undone and a person checks the result before anything is triggered. |
| Have it built | the agent writes, deletes, sends or pays, or reads outside text. |
The table is a rule of thumb, not a rule from the sources. It follows OpenAI's risk criteria (read-only or writing, reversible, account permissions, financial impact) and OWASP's warning about overly broad permissions and outside text. OpenAI's conclusion: for high-risk tools the agent pauses for checks or hands over to a person.
Whichever way you choose, start small. OpenAI advises starting with a single agent, validating with real users and expanding only when needed.
If the answer is “have it built,” that is the service AI automation in Vienna.