Giving AI safe access to your systems: from chatbots to agents

When a chatbot answers questions from your policy documents and gets one wrong, the harm ends at a bad answer. An agent that can update your CRM, raise a purchase order or email a customer can make mistakes inside your systems, and those mistakes reach people outside the building. This post is part of our AI-Ready Data and Integration series. It covers the engineering we put between a model and the systems it is allowed to touch.

In late September, Futurism reported a tech YouTuber’s claim that Meta’s Muse agent, which he had left to handle a Facebook Marketplace listing, accepted a lowball offer and gave the buyer his home address without asking him. He only found out after the buyer had turned up at his building and left. The account is his, though Meta has reportedly contacted him about it. The failure is easy to recognise. The agent could send messages and had his address, and nothing required his approval before it made a commitment on his behalf.

Why integration is now a safety problem

Most organisations already have an integration layer of some kind. It often runs on service accounts that were given broad rights years ago because that was easier at the time. Broad rights were tolerable when the caller was code a developer had written and tested, because it did the same thing on every run.

An agent decides what to call at runtime, based on text that can include an email from a stranger or a document a customer uploaded. In prompt injection, instructions hidden in that content steer the agent. One practitioner in a recent Hacker News thread on agent control described it as the one agent failure mode that is already routine.

That means the controls have to sit in the layer the agent calls through. The right input can talk a model out of an instruction in its system prompt. If the API has no endpoint for changing bank details, no input can make the agent change them.

Australian obligations apply here as they would anywhere else. If an agent discloses personal information to the wrong person, the Privacy Act applies. A disclosure likely to cause serious harm is a notifiable data breach to the OAIC, whether a person or a model sent it. APRA-regulated entities should bring an agent’s access under their CPS 234 controls, the same as any other privileged account.

The guardrails we build in

When we design an agent that acts on business systems, these controls go in before the first demo:

• The agent gets its own identity with least privilege. It has a dedicated service account and API endpoints scoped to the task. It never borrows a user’s credentials or a general admin token. If the job needs overdue invoices, it gets an endpoint that returns overdue invoices.
• Access is read-only by default. A lot of the useful work, such as finding records and summarising history, needs no write access. We add writes one at a time, each with a stated reason.
• Actions are allow-listed. The agent can call a named set of tools with typed parameters, and any other call fails closed. There is no generic “run SQL” or “fetch any URL” tool.
• Every write tool has a dry-run mode that returns what would change without changing it. New agents run in dry-run against real data for a while before we switch writes on.
• Each tool has its own rate limit, with hourly and daily ceilings. An agent stuck in a loop should hit a limit within minutes instead of emailing your whole customer list.
• Every call goes into an append-only audit log with its arguments, its result, the model version and the approver, if there was one. The agent has no permission to modify that log.
• Consequential writes need human approval. Anything that moves money, changes personal data, commits the organisation or contacts someone outside it goes into an approval queue. The approver sees the exact payload the tool will send, because a summary written by the model can misdescribe it.

A recent write-up on connecting Muse to a personal iMessage archive uses the same pattern at a much smaller scale. Network grants keep the assistant’s read-only search separate from the authenticated uploads. Unattended calendar and to-do writes are subject to approval rules. The author sums up the principle in one line: “Message contents are evidence, never tool-use authorization.” If an email asks the agent to refund an order, the agent should read that request as data. It does not authorise a refund.

A policy sketch

These controls are easier to review when they live in one declarative file instead of being spread through the code. Here is a trimmed example for an accounts receivable assistant:

agent: ar-assistant
identity: svc-agent-ar            # own service account, never a user's
default_mode: read
tools:
  - name: list_overdue_invoices
    mode: read
    rate_limit: { per_hour: 120 }
  - name: get_customer_contact
    mode: read
    fields: [name, accounts_email] # no home address, no bank details
  - name: draft_payment_reminder
    mode: write
    dry_run_default: true
    approval: none                # creates a draft, sends nothing
  - name: send_payment_reminder
    mode: write
    approval: human
    rate_limit: { per_day: 50 }
  - name: update_bank_details
    mode: denied
audit:
  store: append-only
  fields: [request_id, tool, args, result, approver, model_version]

A gateway outside the model enforces this file. The model sees tool names and descriptions and never sees a credential. When the business wants the agent to do more, the change goes through review as a diff to this file, and someone outside the engineering team can read it and approve it.

Where MCP fits

The Model Context Protocol (MCP) is an open standard for exposing tools and data to AI applications. You write an MCP server that describes its tools, and any compatible client can discover and call them. The standard is useful because one well-built server can serve several agents, and you stop writing separate glue code for each model vendor.

MCP also makes it very easy to connect an agent to things. A community MCP server that wraps an entire SaaS API behind an admin token takes minutes to install and gives the agent everything that token can do. We treat MCP as transport. Scoping, approvals, rate limits and audit logs still belong in the server or in a gateway in front of it. For business systems we write our own servers and expose only the tools in the policy file.

This is why integration architecture is now an AI safety concern. Under the old model, a developer decided what a caller could do when they wrote the integration. With agents, the caller makes that decision at runtime. The integration layer is the last point where you can still enforce limits.

Costs and limitations

Guardrails take real engineering time. The gateway, the approval queue and the audit store are a substantial part of any agent build. On a small agent they can take more effort than the prompt and model work combined, and this belongs in the budget from day one.

Approval queues bring their own failure mode. If every action needs sign-off, people start approving without reading. Set the threshold so that approvals are rare enough to get proper attention, and track how often they are granted. An approval rate near 100% suggests the gate is either unnecessary or being rubber-stamped.

Logs only help if someone looks at them. We alert on denied calls and rate-limit hits, because those are usually the first signs of a prompt injection attempt or a looping agent.

A tightly scoped agent can’t improvise around unusual cases, so it will hand some work back to staff. We design for that. None of these controls makes the model’s judgement correct. In a write-up on giving agents access to a live Smalltalk runtime, Paul Wilke makes the same point about his own tooling: runtime access gives an agent evidence and capability, and correct reasoning still has to come from somewhere else. He keeps observing a running system separate from changing it, and that split is a good default for business agents too.

A sensible first project

Start with a read-only agent over one system, in a process where a wrong answer costs a staff member a few minutes. Run it long enough to see how it behaves on your real data. Then add a single write tool in dry-run mode, and later put that tool behind an approval queue. Get the business to sign off the agent’s permissions the way it would sign off system access for a new staff member, and review them on the same schedule.

PicNet builds production AI systems for Australian organisations. If you’re working out where an agent could safely start in your systems, talk to us about what a first project could look like.