Local AI Models for Coding: When a Private LLM Wins

A local AI model for coding is an open-weight model, such as Qwen2.5-Coder or Qwen3-Coder, running on hardware your business owns, instead of a cloud service like GitHub Copilot or ChatGPT. The difference that matters is where your source code goes: a cloud assistant sends every prompt and code snippet to a third-party server to generate its suggestion, while a local model never sends the code anywhere. For most Australian software teams the cloud option is simpler and entirely fine. For a business whose codebase is the product, a local model keeps that code on infrastructure you control.

Developers have adopted AI coding tools faster than almost any other category of software. Stack Overflow’s 2025 Developer Survey found that “84% of respondents are using or planning to use AI tools in their development process, an increase over last year (76%)” (Stack Overflow). That adoption curve is exactly why the question of where the code goes while a suggestion is generated deserves more attention than it gets.

I get asked about this by software teams more often than I did a year ago. My honest answer is that a local coding model is for the team whose code is the business, and the first thing I ask is what their current tool’s contract says about the code it processes. We build private AI setups for that case as Compound AI & Automation, from $1,990 per project plus GST, and a self-hosted coding model is one of the more contained jobs we quote.

What is a local AI model for coding?

A local AI model for coding is a language model, trained specifically on source code, that runs entirely on hardware you own rather than calling out to a provider’s cloud. Open-weight releases like Qwen2.5-Coder and Qwen3-Coder can be downloaded and run on a workstation or an on-premises server, using tools such as Ollama or LM Studio. Your prompts, your repository and every suggestion the model returns stay inside your own network the entire time.

The models are published software with published specifications, not a hobby project. Alibaba Cloud releases the 32B version of Qwen2.5-Coder on Hugging Face with 32.5B parameters and a context length of 131,072 tokens (Hugging Face), which is enough to hold a sizeable module and its tests in one prompt. The licence sits on the same page, and reading it is the first job before any download.

A hosted tool is a service you call; a local model is software you run. The trade-off is that you take on the hardware and maintenance a provider would otherwise handle. The payoff is that nothing about your code ever needs to leave the building.

Does a cloud coding assistant like Copilot actually see your code?

A cloud coding assistant like GitHub Copilot sends your code to the provider’s servers to generate every suggestion, even on a paid business plan that excludes your code from model training. GitHub is explicit about this distinction. Copilot Business and Enterprise plans are contractually excluded from the model-training program GitHub runs on lower tiers, but the code itself still travels off your machine to be processed.

GitHub’s own privacy update on the training program draws the line precisely: it states that the program does not use “Interaction data from Copilot Business, Copilot Enterprise, or enterprise-owned repositories”, while separately clarifying the limit of that protection:

“Copilot does process code from private repositories when you are actively using Copilot.”

The GitHub Blog

Not training on your code and never seeing your code are two different guarantees. A business plan solves the first problem and leaves the second where it was: the code still leaves your premises to run.

Australia’s privacy regulator makes a related point about any commercially available AI product, and I think it applies to coding tools as much as to chatbots. The OAIC’s guidance says that “Due diligence for AI products should not amount to a ‘set and forget’ approach.” (OAIC). The GitHub post above is itself an update to an earlier policy, which is the practical reason to re-read the terms rather than remember them.

Are local coding models capable enough to replace GPT-4o or Copilot?

Local coding models are capable enough for most day-to-day programming tasks. The capability gap between the best open-weight coding models and the leading cloud models has closed further than most teams assume. The Qwen team states plainly that their flagship release “has become the current SOTA open-source code model, matching the coding capabilities of GPT-4o” (Qwen Team), across code generation, code repair and multi-language benchmarks.

That release has since been followed by Qwen3-Coder. Its largest variant is “a 480B-parameter Mixture-of-Experts model with 35B active parameters”, and the team reports that it “sets new state-of-the-art results among open models on Agentic Coding, Agentic Browser-Use, and Agentic Tool-Use, comparable to Claude Sonnet 4” (Qwen Team). Nobody is going to run a 480B model on a workstation, and I don’t think that is the point of it. The point is that the open-weight line is now measured against the current frontier model rather than against last year’s.

Capability is only half of the question, and the other half is trust. The same Stack Overflow survey found that “More developers actively distrust the accuracy of AI tools (46%) than trust it (33%)” (Stack Overflow). I read that as an argument for review discipline rather than against the tools, and it holds whether the model runs in a data centre or under a desk. A well-chosen local model is no longer a compromise for the bulk of what a working developer actually does: writing functions, fixing bugs, completing boilerplate, explaining an unfamiliar file.

Which Australian businesses should run a coding model locally instead of in the cloud?

An Australian business should run its coding model locally when the code itself is confidential: a fintech’s pricing engine, a healthtech’s clinical logic, a defence contractor’s integration layer, or any codebase covered by a client confidentiality agreement. The test is not company size; it is whether the source code is something a competitor or an adversary would want to read.

The regulator’s position on AI tools in general points the same way. The OAIC puts it like this:

“As a matter of best practice, the OAIC recommends that organisations do not enter personal information, and particularly sensitive information, into publicly available AI chatbots and other publicly available generative AI tools, due to the significant and complex privacy risks involved.”

OAIC, Guidance on privacy and the use of commercially available AI products (OAIC)

Source code is rarely personal information on its own, but a codebase often carries customer data in its fixtures, logs and test databases, and that is exactly what ends up in a prompt. The breach numbers say the exposure is real: the OAIC received 532 data breach notifications in the January to June 2025 reporting period, and “Malicious or criminal attacks remained the largest source of data breaches (59%, at 308 notifications)” (OAIC). We track those figures, alongside AI adoption, in our index of Australian small-business AI and digital indicators.

The same four-level logic applies to any AI use in a confidentiality-sensitive business: the safest setting is the one that keeps the sensitive material closest to home, and code is no exception (our four levels of AI data security). A marketing team drafting public copy has little to lose from a cloud assistant. A software team whose entire value is a proprietary algorithm has a great deal to lose from one, and a local model is the only setting that removes the exposure rather than contracting around it.

What does it take to run a local coding model in-house?

Running a local coding model takes a workstation with a capable GPU, an inference tool such as Ollama or LM Studio, and a decision about which open-weight model fits the team’s languages and hardware budget. Smaller models run comfortably on a single high-end workstation GPU; the 32B-class models that match GPT-4o-level capability need more memory, typically a workstation with a 24 GB graphics card rather than a laptop.

The download sizes Ollama publishes are the honest guide to what each size needs, because the file has to fit in memory before it can run at all:

Model	Parameters	Download (Ollama)	Fits in memory on
Qwen2.5-Coder	3B	1.9 GB	an ordinary developer laptop
Qwen2.5-Coder	14B	9.0 GB	a laptop or desktop with 16 GB of memory
Qwen2.5-Coder	32B	20 GB	a workstation with a 24 GB graphics card
Qwen3-Coder	480B	290 GB (Ollama)	a multi-GPU server, not a workstation

So the practical choice for a small team is the 14B model on the machines developers already have, or the 32B model on one shared workstation that every developer’s editor talks to over the office network.

The software side is simpler than it was. Ollama runs the model and lets an editor plugin talk to it; LM Studio does the same job with a desktop interface (LM Studio). I’ve set out the general case for running models on your own hardware in a separate explainer. For a team that already runs custom internal software, adding a self-hosted coding model is an extension of infrastructure it already manages, not a new category of vendor risk.

Where should a confidentiality-sensitive team start?

A confidentiality-sensitive team should start with an inventory, not a server: every AI tool a developer already uses, whether it sends source code outside the network, and under what agreement. For most businesses the honest answer is that nobody has checked. Auditing that is a smaller job than deploying a local model, and it tells you immediately whether the local option is worth the setup cost for your specific codebase.

The inventory usually turns up more than the business expects, because AI use is now ordinary. Roy Morgan counts “13.6 million people, equivalent to 58% of Australians aged 14+” using AI tools in an average four weeks (Roy Morgan), and developers sit at the front of that curve. Some of that use inside a software team is a browser tab with a personal account, and no contract covers it at all.

If your team’s code is confidential enough that the answer matters, the next step is a scoped conversation: we map where the code currently goes and what a private, self-hosted setup would take, then quote it as a private AI build under Compound AI & Automation, with the starting price on our pricing page. Start that conversation, and if the cloud tool with a better contract is the right answer for you, I’ll say so.

Frequently asked questions

Is GitHub Copilot Business safe for proprietary code?

Copilot Business is safe against one risk and not the other. Copilot Business and Enterprise plans are contractually excluded from GitHub’s model-training program, so your code is not used to improve GitHub’s models. Your code still travels to GitHub’s servers to be processed every time Copilot generates a suggestion, so the arrangement solves the training risk but not the data-residency one. For genuinely confidential source code, that residual exposure is often still one boundary too many.

What is the best local AI model for coding in 2026?

The best local coding model in 2026 depends on hardware and language mix. Qwen2.5-Coder-32B-Instruct remains the strongest open-weight option that fits on one workstation, with performance the Qwen team describes as matching GPT-4o on coding benchmarks, and Qwen3-Coder leads the open models where a server-class budget exists. Smaller 3B to 14B variants trade some capability for the ability to run on a laptop rather than a dedicated machine.

Can a local coding model run without an internet connection?

Yes. Once the model weights are downloaded and installed with a tool such as Ollama, inference happens entirely on local hardware and needs no ongoing internet connection. This is precisely what keeps prompts and code from leaving the network: there is no external endpoint for them to reach even if the connection is available.

Do local coding models cost more than a Copilot subscription?

A local setup costs differently rather than simply more. It has an upfront hardware cost, typically a workstation-class GPU or a small dedicated server, instead of a per-seat monthly fee. For a small team the subscription can be cheaper; for a larger development team running many seats over several years, owning the hardware can cost less than the equivalent subscription spend, before counting the value of keeping the code in-house.