Intro
So, AI is still in the news, but it is mostly ChatGPT, Google’s Gemini, Microsoft’s Copilot and Meta’s Llama. Then there are new ones ever so often, like DeepSeek from China that caused chaos in the GPU sector, and there will be many more.
AI is still in it’s infancy – it will bring lots of changes in the very near future for many of us.
Prompts
We are so used to entering queries in Google, that running a query in a LLM can be quite daunting. You can enter a query which is a whole page long, and get meaningful results. And you can tweak your query and ask the same thing over and over (with slightly different wording) without the machine getting annoyed. It can write things for you, or take your attempt and re-write it in different styles – like formal, casual and so on.
Cloud
They work in the cloud with huge databases that can answer many questions with surprising accuracy. But they can also hallucinate and give you, with full confidence, answers which are totally incorrect. So if you don’t verify your answers, it can land you in serious trouble.
Another player on the block is Google’s NotebookLM. Also in the cloud, but you can upload your documents to a notebook, and run queries using your uploaded data. But, your data is still in the cloud.
Local
What you want is an AI that runs locally in a contained environment (read – Not in the cloud). I found the ollama framework to do just this. You can use many models with different focus areas – like general reasoning, programming, writing, and many more. It runs better when you have a powerful machine with a GPU, but some models run surprisingly well on a decent machine without a GPU.
The big thing will be to make your documents selectively available to a local AI so that you can query, summarise, find discrepancies, find redundancies and so on, without any of your data leaving your machine.
Office
Many work-environments block access to all forms of AI, and with good reason. You really don’t want your company’s data in an environment like that. You also don’t want your knowledge workers to make use of AI to give you unreliable or questionable work.
There are, however, other options