What local AI is and when to keep it in-house
With the cloud, you send your data to the vendor and get the answer back. Locally, the model sits on your own machine: the data doesn't leave, you don't pay per request, and the service keeps working even if the vendor changes its terms.
In return, you take on the machine, the updates and the security, and you use open weight models that, on some tasks, are less capable than the best cloud services. Often the right solution is a mix: sensitive data and repetitive tasks locally, everything else in the cloud.
An example at an SMB
A law firm wants to classify and summarise documents that contain client data. It installs a local model for summaries and sorting. For generic drafts with no personal data, it uses a cloud service under a business contract.
The most common mistakes
- Buying hardware before testing the model on your own documents.
- Assuming that local means secure without doing anything: the machine has to be protected like any other server.
- Expecting the office machine to match the speed of cloud services.
Why it matters to you
If you handle health, legal or employee data, local AI simplifies the privacy question: if the data doesn't leave, there is no external vendor to assess for that processing. Internal obligations remain, such as security, access and the record of processing activities, but the perimeter is yours.
All the AI terms, explained simply, are in the glossary.
Frequently asked questions
What hardware do you need?
It depends on the model. The smallest run on a good recent computer, while the largest need graphics cards with a lot of memory. It's worth testing on your real task before you buy anything.
Does a local model update itself?
No. When a better version comes out, you download it and test it on your own cases before replacing the one in use. That's also an advantage: nobody changes the model under your processes without telling you.
Read next
Ollama: route requests with AI locally, at no cost
Decision models don't write text: they pick a label and tell you how confident they are. For sorting emails and tickets, that is what you need.
How toDo ChatGPT, Claude and Gemini use your company's data?
It depends on the plan, not the tool. With a personal account the answer is often yes, with a business plan it is normally no. Here is where to look.
Want to understand what your business really needs, without the jargon? Let's talk.
Text and graphics created with the support of AI. Method and content are AIR's.