Running a private AI agent takes four things beyond the model itself: a tightly scoped view of your data, deterministic checks on every output that matters, a defined path for when the agent cannot finish a job, and a named person reviewing it after go-live. The model is the cheapest part of the build. The discipline around it decides whether the agent gives hours back or quietly creates work, and that is where the effort in a real deployment goes.
This is worth saying plainly, because the public conversation about agents is almost entirely about models. Model choice is close to a solved problem for ordinary business work. Everything that determines whether you can trust an agent with client files sits in the plumbing around it, and that plumbing is specific to your business.
What is a private AI agent, and how is it different from a chatbot?
A chatbot answers a question when you ask one. An agent takes a goal and works through it in steps, reading documents, calling tools, drafting output and checking results, with a language model doing the reasoning. Private means the model and the data both stay on hardware you control, so nothing leaves the building while the agent works.
The commercial difference is that agents do jobs rather than answer questions. A job has inputs your business owns: client records, supplier emails, financial history. Those inputs are exactly the material you least want travelling to somebody else’s infrastructure, which is why the agent question and the data question cannot be separated.
Why does the data question decide how you deploy?
Because an agent has to read your data to be useful, and it reads far more of it than anyone would paste into a chat window. A person pasting a paragraph into a public tool exposes a paragraph. An agent pointed at a shared drive exposes the drive. The volume changes what the decision is worth.
Australia’s privacy regulator takes a conservative line on the public tools:
“As a matter of best practice, the OAIC recommends that organisations do not enter personal information, and particularly sensitive information, into publicly available AI chatbots and other publicly available generative AI tools, due to the significant and complex privacy risks involved.”
OAIC guidance on privacy and commercially available AI products
Commercial cloud tiers answer part of this with a contract. The protection is real and worth having. Anthropic, for one, states it directly:
“By default, we will not use your inputs or outputs from our commercial products (e.g. Claude for Work, Anthropic API, Claude Gov, etc.) to train our models.”
Our position is that a contract and a location are different kinds of assurance. A contractual commitment is a promise about behaviour that you audit on paper. A machine in your own office is a property of the arrangement that does not depend on anyone keeping their word. For work bound by confidentiality, the second is worth paying for, and it is the level our private AI integration work is built around. We have set out where each option sits in our breakdown of AI data security levels.
What makes an AI agent dependable enough to trust?
Four disciplines, none of which involve the model. Scope the agent to the minimum data it needs, which usually means one folder, one mailbox or one database view rather than a whole drive. Verify every output that matters with plain code. Define what happens when the agent cannot finish. Review it on a schedule after go-live.
Verification is the one most often skipped and the one that pays for itself. The best automations follow clear rules and produce predictable results, so the agent’s judgement should be wrapped in checks a computer can settle without opinion: does the referenced file exist, does the total reconcile against the ledger, does the draft contain every required field. An agent that cannot finish a job should stop and flag it rather than improvise a plausible answer, because a confident wrong answer costs more to find than a blank one.
The review discipline is a regulatory expectation as much as an operational one. The OAIC’s guidance is explicit that adoption is not a one-off event:
“Regular reviews of the performance of the AI product itself, training of staff and monitoring should be conducted throughout the entire AI product lifecycle to ensure a product remains fit for purpose and that its use is appropriate and complies with privacy obligations.”
OAIC guidance on privacy and commercially available AI products
When is an AI agent the wrong tool?
Whenever a deterministic automation can do the job. If the inputs arrive in a predictable shape and the rules can be written down, a rule-based workflow is faster, cheaper to run and easier to debug than anything with a model in it. Reach for an agent only where the input genuinely varies and a judgement step is unavoidable.
This matters commercially because the wrong choice is expensive in a way that is hard to see. Agents cost more per task, fail in less predictable ways, and need the verification layer described above before they can be trusted at all. We have written about where that line falls in practice in our look at agentic automation. If a straightforward workflow automation solves your problem, take it and spend the saved budget elsewhere. If we look at your process and conclude that is the answer, we will say so.
How should a first deployment be scoped?
Pick one bounded, repeatable job that touches private data and has a checkable output. Summarising client files into a standard brief, drafting replies from your own document base, or sorting an inbox into categories a person then actions are all good candidates. Run it beside the existing process for a fortnight, count the hours returned, and only widen the remit once the numbers hold.
Start where failure is cheap and success is measurable, rather than where the demonstration is most impressive. Staff AI use is already widespread, with Roy Morgan counting 13.6 million Australians aged 14 and over, or 58 per cent, using AI tools in an average four weeks (Roy Morgan), a figure we track in the Australian SMB AI & Digital Index. The realistic choice is not whether AI touches your data but whether it does so under conditions you set. Each job an agent takes over is cheaper to add than the last, because the scoping, the checks and the hardware are already paid for. That is where the compounding happens.
If you want a considered read on whether an agent fits your operation, have a conversation with us or look at how we scope and price a build.
Frequently asked questions
What does a private AI agent cost to run?
The running cost is mostly hardware and maintenance rather than per-task fees, which is the main economic difference from a cloud subscription. A single workstation with a capable GPU handles the open-weight models most business agent work needs. The larger cost in any honest quote is the deployment itself: scoping data access, building the verification layer, and testing the failure paths before the agent touches live work.
Is a private AI agent safe for confidential client data?
It is the safest deployment available, because the model, the documents and the outputs all stay on hardware you control, leaving no third party to audit or trust. Safety still depends on your own discipline. Scope the agent to the minimum data it needs, verify its outputs with deterministic checks, and log what it reads and writes so an unexpected result can be traced.
Should we build an AI agent or a normal automation?
Use a normal automation whenever the inputs arrive in a predictable shape and the rules can be written down, because it will be cheaper to run and easier to debug. An agent earns its place only where the input genuinely varies and a judgement step cannot be avoided. Many businesses asking for an agent are better served by a deterministic workflow with a model in one step of it.
How long does a private AI agent deployment take?
Expect a project measured in weeks rather than days. A quick technical setup proves a model runs on your hardware, which is a useful first step but not a deployment. The schedule is set by the unglamorous parts: agreeing what data the agent may see, building the checks on its output, handling the cases where it cannot finish, and running it beside the existing process long enough to trust the numbers.
Read more
Let's compound
Tell us where growth stalls.
One team that connects your website, marketing and operations, so the results compound. Pick whichever way is easiest to start.
Australian-based · Founder on every project