How AI works: the 10 concepts to understand before you sign
Neural networks, tokens, training, fine-tuning, hallucination, LLMOps: the ten concepts that explain where your AI budget goes and where your dependency begins.
This article is for anyone who takes an interest in AI and sees its potential, but has neither the time nor the inclination to make it their profession.
You will most likely want to delegate your AI needs to an internal team or to outside providers. That is a reasonable choice. But you will not be able to skip challenging them, and to do that you need to understand a few key concepts, and above all what they imply for a company project.
So here are ten notions to know. No maths, no more jargon than the minimum. Enough to follow what people are talking about, and handle the business side of an AI project.
The content of this article was first published on LinkedIn over the summer, one concept a day.
If you only remember one thing
There is nothing magical in any of this. Numerical weights tuned by example, text chopped into fragments, tools wired around the outside, your documents placed on the model's desk at the right moment, and a great deal of engineering to keep the whole thing standing.
What these ten concepts teach you fits into two sentences. Your invoice almost never depends on which model you pick, but on how you use it. And the proof of concept, the demo that convinced you, accounts for roughly 20 % of the project's total cost.
None of this justifies overselling. None of it justifies waiting either.
Part 1: what a model is made of
1. The neural network
Conventional software is a sequence of instructions written by a human. "If the customer has more than three orders, apply a 10 % discount." A rule, applied blindly, with a predictable result. To write it, you first have to be able to formulate it.
Now write me the rule that recognises whether a photo shows a cat. You will not manage it. A brain does it effortlessly, so computer scientists took their inspiration from that.
A neural network is an input that travels through millions or billions of numerical weights and comes out as an output. Those weights were not programmed: they tuned themselves from examples. If you open them up, you will find no logic inside.
Diagram of a neural network: an input layer receives photos of cats and dogs, the signals travel through several internal layers of interconnected neurons, and the output layer sorts the images into two categories, cats and dogs
A cat in a photo, a sentence to complete, a contract to summarise: always the same principle. Weights, not rules.
The consequence is considerable, and it concerns you directly. Nobody, not even the people who built the model, can tell you precisely why it answered one thing rather than another. This is not a manufacturing defect, it is the nature of the thing. On a cat, who cares. On a loan refusal, a diagnosis or a quote, rather less so. That is the whole reason why the way you govern AI in a company is not an implementation detail.
2. Parameters
Those weights go by another name: parameters. They are the famous "8B", "70B" and "405B" you see attached to model names.
For a sense of scale: recognising a cat in a photo takes around 2.5 million parameters. Understanding and writing human language is another matter, and the largest models now exceed a trillion. By way of comparison, the biological equivalent of a weight would be the synapse: a fly has 50 million, a mouse a trillion, a human 100 trillion.
What those weights contain, nobody can read. You can think of them as two things blended together: knowledge, and a capacity for reasoning. The second matters more, and it comes mainly from how the model is refined after its initial learning phase.
That is why the race for size has cooled. In 2020, GPT-3's 175 billion parameters were a record. Today an 8-billion-parameter model beats it on most tasks. Better trained, but smaller.
Scatter plot comparing the capability level of open models against their parameter count on a logarithmic scale. The curve rises sharply up to around thirty billion parameters, then flattens: beyond that, multiplying size by a hundred gains only a few points
At comparable training, a large model remains more capable than a small one. But look at where the curve flattens: past a few tens of billions of parameters, each tenfold increase in size buys only a few more points. Size alone no longer decides who wins.
Why this concerns you: bigger means more expensive, slower, hungrier for energy. Reckon on €0.35 per million generated tokens for a Mistral Small on Scaleway, against $50 for Anthropic's most expensive model. More than a hundredfold difference, when for summarising meeting notes or flagging negative reviews the small model does the job. The smallest ones even run locally, on a laptop or a phone, without your data ever leaving.
3. Training
If nobody knows what those billions of weights contain, how were they determined? They were not written: they were made to emerge, through training.
This is an industrial undertaking mobilising billions of dollars in compute, energy and expertise. It happens in two stages.
Pre-training starts from random parameters. The model is fed an enormous corpus, all of the internet, books and more, and asked to predict the next word. "The doctor examines the patient with his…" yields "stethoscope". At each prediction the error is measured and the weights corrected. Billions of times over. Little by little, the model builds internal representations of the structures of language.
Post-training then refines that base: verifiable results, imitation of a stronger model, human judgements of the "do you prefer answer A or answer B" kind. What comes out at the end is a model with frozen weights.
Remember that word, frozen. It is what makes the most widespread sales promise on the market false, the one you hear in just about every pre-project meeting:
"We are going to train an AI on your company's data."
The model you use does not change as you use it, and it learns nothing from your documents.
Ranking of models by a general capability index, with the flag of each country of origin. The top two are American, the next two Chinese and very close behind, the French models come in well below, and the Swiss model brings up the rear
Two countries dominate this stage today, the United States and China, and Europe has lost this particular battle for now. It matters less than it looks: many Chinese models and a few American ones are open, meaning downloadable and runnable on your own infrastructure. We spell out what "open" really covers in our article on open source versus open weights. Given the choice, losing the battle for models matters less than losing the one for inference, that is, the moment you actually use them.
Part 2: what you are really paying for
4. The token
A neural network needs inputs expressed in a closed alphabet, a finite set of values the machine can manipulate. To make it read our language, our texts have to be cut up. But how?
By characters? Workable, but an isolated "c" means nothing and the model would take an eternity to compute the meaning of a paragraph. By whole words? Impossible: between languages, conjugations, typos, code and punctuation, you would have to store tens of millions of entries.
The compromise that was found is called the token. Picture a box of language Lego: instead of every word in the world, a dictionary of 100,000 to 200,000 fragments. A very common word like "hello" is worth a single token. A longer word is assembled like a puzzle, and in a heavily inflected language one of those fragments can carry a whole grammatical tense on its own.
That cutting-up is the job of the tokenizer, a translator that turns your text into a sequence of codes before sending it to the model.
Why this word shows up on all your invoices. Two things matter.
First, the bill depends on the language. Since models were trained predominantly on English, their dictionary favours that language.
The same text broken into tokens in three languages. In French, 2,658 characters produce 692 tokens. In English, 2,589 characters produce only 606. In Swahili, 2,688 characters produce 835
The same text, at near-identical character length, costs 14 % more in French than in English, and close to 40 % more in Swahili. You pay a tax on your working language, and it is not negotiable.
Second, input and output both count. You pay for the tokens you send, your question but also the instructions, the conversation history and any attached documents, and for those the model generates. A ten-page document is not "one file", it is roughly 5,000 input tokens to process, every single time.
5. Inference
Training a large model cost someone more than a billion. You will never pay a cent of it. You will, however, pay for every answer.
That on-demand computation is called inference. Unlike conventional software running on ordinary processors, inference is extremely resource-hungry and requires specialised chips, GPUs above all. That is what allowed Nvidia to multiply its value tenfold in five years.
The larger the model, the heavier the infrastructure. A very small model fits on a PC. A mid-sized one needs servers costing a few tens of thousands of euros, still conceivable in-house. A large model is out of reach: you have to rent, and in practice go through an inference provider paid by the token.
This is where your costs concentrate, and not where you think. Picking the right model for the right task, and not recomputing the same thing twice. Those two decisions weigh far more than the advertised price per million tokens.
It is also where sovereignty is decided, since whoever you send your data to sees that data, and answers to their own government. On this layer, though, the gap is not only political, it is technical.
Generation speed for the same open model depending on which provider runs it, with each provider's country flag. The top four are American, the fastest exceeding 1,900 tokens per second; the two European providers bring up the rear, at 243 and 159 tokens per second
The model is identical in all six cases. Only the party running it changes, and the ratio between the fastest American provider and the best European one exceeds a factor of eight. Worth knowing before you promise your users a sovereign AI as responsive as the one they use at home.
The subject deserves more than a paragraph: we have set out what running an LLM locally in a company really costs, and what it does and does not buy you in sovereignty.
6. The context window
The context window is the model's working memory: everything it can read in one go, your instructions, attached documents, the conversation history and your question. It is measured in tokens and currently ranges from 128,000 to around a million.
That fast-rising figure gives rise to a now-regular genre of post: "RAG is dead, you can just put everything in the context window."
It is wrong, for three reasons.
- The fuller the window, the more you pay. Every exchange re-reads everything that came before. A single request can cost several euros.
- The fuller it gets, the less precise the model. A piece of information buried in the middle of 500 pages is missed far more often than one at the start or the end.
- A million tokens is not much against the real document volumes of a company.
A large window is therefore not an invitation to dump everything into it. That is precisely why RAG exists: five good passages beat two thousand pages in bulk.
Part 3: what makes the model useful at your company
7. Tools
A model is frozen and has never seen a scrap of your data. Yet your AI can comment on this morning's news, answer questions about a confidential contract and analyse your commercial quarter. How?
Think back to late 2022: the original ChatGPT could not tell you the weather. In 2023 web search arrives, then other tools, and everything changes.
Concretely, when you send a request to a model equipped with tools, it has two options: answer you directly, or trigger a tool. The tool runs and returns its result. The model reads it, then chooses again: answer, or call more tools. And so on.
Diagram of a tool call: the user's question reaches the model, which either answers directly or triggers a tool run by the company's own software (ERP, CRM, document base, web search, calculations, sending emails) before formulating its answer
The roles are clear-cut. The model decides, the software executes. That execution goes through conventional software, sometimes called the harness, and that is where access rights, logs and approvals are handled.
The strong idea, the one that changes how you assess a provider: you add software and neural network together. The model understands, reasons, orchestrates. The software computes, reaches the real data and acts reliably and traceably. In a serious AI project, the software now counts as much as the model, often more.
Two approaches to these tools coexist. Proprietary tools, built for one piece of software and working only there. And standard tools, foremost among them MCP: you expose your system once, and any compatible AI can plug into it.
The difference is not technical, it is contractual. With the large vendors, your connectors and your automations stay with them. The day you leave, you start again from scratch.
8. RAG
A model knows nothing of your contracts, your procedures or your customer history. So it has to be given the information at the moment you ask the question. That is the principle of RAG, for Retrieval-Augmented Generation.
In its classic form it comes down to four steps. You select your documents and convert them into a usable format. You cut them into chunks, stored in a database suited to semantic search, that is, search by proximity of meaning. When a user asks a question, the system retrieves the most useful passages and appends them to the question. The model answers from those elements.
Nothing has been trained or modified in the model. Your documents are consulted at the moment of the question, that is all. Hold on to that sentence: it will come back when we reach fine-tuning.
An example to grasp why semantic search matters: if you ask "how many days of remote work?", a keyword search may miss a passage headed "working from home". Search by meaning understands that the two are probably talking about the same thing. But keywords remain indispensable, because a product reference like XR-4471B has to be found character for character. That is the whole point of hybrid search.
The word RAG suggests a simple recipe. In production it has become a complete system: connectors, document preparation, access rights, choice of search method, citations, evaluation, monitoring. We have written a complete guide to enterprise RAG for those who want the detail.
9. Agentic AI
An AI that can call a tool is not yet an agent. Agentic AI is the next step: when the model breaks an objective into several steps, orchestrates them, and corrects itself to reach its goal.
The difference shows in use. With a conventional AI you ask "analyse this file and give me the three least profitable products", the AI calls the tool that reads the file and answers. With an agentic AI you ask "prepare this week's commercial review", and it pulls the figures from the CRM, compares them with the previous week, spots unusual variances, looks for explanations in the meeting notes, drafts a summary, asks for human approval, then sends it out.
An agent is therefore not a new kind of model. It is a model placed at the centre of a system that supplies it with tools, a working memory, rules, permissions, sometimes a budget, and an objective. We cover the use cases, costs and ROI of an AI agent in the enterprise in a dedicated article.
The limit to know before you sign. A model remains probabilistic, and so less reliable than simple software. Take an agent chaining ten steps, each succeeding 95 % of the time. The probability that all ten succeed is 0.95 to the power of ten, or about 60 %.
The calculation is deliberately simplistic, but the idea is essential: the more actions you chain, the more you have to verify intermediate results. Good automation uses AI only where it is needed, a principle we develop in our article on minimal AI automation.
And since it is very hard to tell a good agent from a bad one from the outside, if you outsource this development, ask for a fixed price and a commitment to results.
Part 4: the two confusions that cost money
10. Fine-tuning
Here we are. Fine-tuning means taking an already-trained model and slightly adjusting its weights on a few thousand of your examples. It is a small resumption of training, not training.
And it is the most oversold concept on the market, the one providers present to you while promising to "train an AI on your company's data".
What you need to remember fits in one sentence.
Fine-tuning changes behaviour, not knowledge.
You want the model to always answer in a precise format, adopt your house tone, handle your sector's jargon? Fine-tuning. You want to improve its success rate on a well-defined task, a classification or code generation? Fine-tuning.
You want it to know your 40,000 documents? RAG. Not fine-tuning.
One last point not to forget: fine-tuning has to be redone every time you change model. Which is to say, often.
11. Hallucination
A hallucination is a false answer produced with all the confidence of a true one. A piece of case law that does not exist, an invented contract clause, a plausible but wrong figure.
The common mistake is to treat this as a bug some vendor will eventually fix. It is not one. Go back to concept number 1: a model does not apply rules, it produces the most probable continuation from weights tuned by example. When it lacks the information, nothing in how it works pushes it to say "I don't know" rather than produce something plausible. Hallucination is structural, not accidental.
Which means you do not eliminate it. You reduce it and you make it detectable, through measures that are all architectural choices rather than model settings: supplying the right documents at the right moment instead of relying on the model's memory, requiring verifiable citations on every answer, having intermediate results checked by deterministic software, and measuring continuously.
When a vendor announces an AI "without hallucination", they are selling you either a figure of speech or a misunderstanding of their own product. The right question is not "does it hallucinate", but "how will I see it when it does".
What decides whether the project survives: LLMOps
The least saleable concept on the list, and by far the most decisive. LLMOps covers everything needed around the model for a project to hold up in production.
Conventional software, we have known how to do for forty years. With an agent, nothing works the same way any more.
Tests are no longer binary. Ask the same question twice, get two different answers. And your users do not follow ten signposted paths, they phrase their request however they like. "I click here, that happens" no longer suffices: you need a set of questions with expected answers, validated by your business experts.
You will not be buying the infrastructure. An H100 rented continuously comes to nearly €2,000 a month for a single card. So you will depend on third-party providers.
And those providers go down. Their status pages show 99 to 99.5 % availability, which is up to several days of outage a year. Either you pay a great deal for a contractual commitment, or you connect several and switch as soon as one falters. That is the whole point of an SLA on an AI service.
The bill is counted in tokens and it drifts fast, since an agent looping fifteen times re-reads its whole history at each turn. Steer by cost per useful request, not by the price per million tokens.
Models change all the time. Your provider releases a better one, deprecates the old one with a few months' notice, and your finely tuned prompt no longer behaves the same. You need to be able to test, switch and roll back without rewriting the application.
Behaviour has to be steered weekly: tracing the questions asked, the tools called, the sources used. Otherwise, when a user tells you "it talked nonsense at me", you are debating a perception and improving nothing.
The proof of concept almost always succeeds. That is precisely the problem.
Frequently asked questions about AI concepts
What is an LLM?
An LLM, for large language model, is a neural network trained to predict the continuation of a text. It consists of billions of numerical weights, called parameters, which were tuned automatically on vast text corpora rather than programmed by humans. Once that training is finished, those weights are frozen: the model no longer changes and learns nothing from the questions put to it. That is what sets it radically apart from conventional software, where every rule was written explicitly and can be read back.
How does generative AI work?
A generative AI produces text by predicting, fragment by fragment, the most probable continuation of what precedes. Your question is first cut into tokens, the word fragments the model knows how to manipulate. Those tokens then travel through the network's billions of weights, which compute the most likely next fragment, then the next, until the answer ends. No knowledge base is consulted and no business rule is applied: only a probability calculation over frozen weights. That is why the same question can produce two different answers.
Can an AI learn from my company's documents?
Not in the sense usually meant. A model's weights are frozen after training and do not change as you use it. For an AI to answer questions about your documents, the model is not modified: it is supplied with the relevant extracts at the moment of the question, a technique called RAG. Your documents are consulted on demand and then forgotten. The phrase "train an AI on your data", common in commercial proposals, therefore describes poorly what actually happens and deserves to be clarified before signing.
What is a token in AI?
A token is the unit of text an AI model actually manipulates: a short word, or a fragment of a longer one. Models hold a dictionary of 100,000 to 200,000 fragments, and text is cut according to that dictionary before being processed. It is the billing unit of almost every AI service, counting separately what you send and what the model generates. A French text consumes about 14 % more tokens than the same text in English, because the dictionaries are built predominantly on English.
Why does AI cost more in French than in English?
Because the token dictionaries of large models were built predominantly from English text. Common English words occupy a single token there, whereas their French equivalents often have to be reassembled from several fragments. At comparable text length, French consumes about 14 % more tokens than English, and some under-represented languages exceed 40 %. Since billing is per token, that difference lands directly on your invoice, and no commercial negotiation can remove it.
What do 8B, 70B or 405B mean in an AI model's name?
These figures indicate the model's parameter count in billions: 8B means eight billion numerical weights. The number gives an order of magnitude for the model's size, and therefore its running cost, its speed and the hardware needed. It does not directly measure quality: since 2023, a well-trained 8-billion-parameter model beats GPT-3 and its 175 billion on most tasks. Beyond a few tens of billions, each tenfold increase in size adds only a few points of performance.
What is the context window of an AI model?
The context window is the amount of text a model can read in one go: your instructions, attached documents, the conversation history and your question. It is measured in tokens and currently runs from 128,000 to around a million depending on the model. Two limits make it less useful than it appears: the fuller it gets, the more each exchange costs since all the content is re-read every turn, and the lower the precision becomes, as information buried in the middle of hundreds of pages is frequently ignored.
What is the difference between fine-tuning and RAG?
Fine-tuning adjusts the model's weights on your examples and changes its behaviour: answer format, tone, business jargon, success rate on a specific task. RAG does not touch the model: it retrieves the useful passages from your documents and supplies them at the moment of the question. For an AI to know your internal documents, that is RAG. For it to always answer in a certain way, that is fine-tuning. Confusion between the two is the source of the commercial promise to "train an AI on your data", which correctly describes neither.
Why does an AI hallucinate?
Because a language model does not consult a database of facts: it produces the most probable continuation from weights tuned statistically on examples. When it lacks the information, nothing in how it works pushes it to answer "I don't know" rather than generate plausible content. Hallucination is therefore structural and cannot be removed by a fix. It is reduced and detected: by supplying the right documents at the moment of the question, by requiring verifiable citations, and by having intermediate results checked by deterministic software.
Should you choose the most powerful model for an enterprise AI project?
No, and doing so is often an expensive mistake. The price gap between a small model and a very large one exceeds a factor of a hundred per million generated tokens, while many enterprise tasks such as summarising meeting notes or classifying messages are handled correctly by a small model. Since 2023, size alone no longer determines quality: a well-trained 8-billion-parameter model beats GPT-3 and its 175 billion on most tasks. Good practice is to assign the right model to each task rather than choosing a single one for everything.
What is LLMOps?
LLMOps covers the practices needed to keep a language-model-based system running in production: building a set of questions with expected answers to test quality, tracing the questions asked, the tools called and the sources used, steering the cost per useful request, and being able to change model or provider without rewriting the application. The discipline differs from conventional DevOps because a model is not deterministic, because providers regularly deprecate their models, and because quality degrades silently rather than causing a visible outage.
Can an AI be guaranteed never to hallucinate?
No. Hallucination follows from the very way language models work, producing a probable continuation rather than a verified fact, and no vendor can eliminate it completely. What can be controlled is its frequency and its detectability: supplying the relevant documents at the moment of the question, requiring verifiable citations on every answer, having intermediate results checked by deterministic software, and measuring quality continuously. A commercial promise of AI "without hallucination" signals either a misunderstanding of the product or a figure of speech.
How many steps can an AI agent chain reliably?
The number of steps is limited by how success rates compound. If each step succeeds 95 % of the time, ten chained steps only succeed together about 60 % of the time, since 0.95 to the power of ten equals 0.60. The calculation is a simplification, but it explains why reliable agents are not the ones chaining the most actions: they are the ones that verify their intermediate results and use the model only where deterministic software would not suffice.
How much does an AI project cost once the proof of concept is done?
The proof of concept generally accounts for around 20 % of the project's total cost over its lifetime. The remaining 80 % is spread across inference billed per token, maintenance of the system around the model, model changes imposed by providers, continuous evaluation of answer quality, and steering the agent's behaviour. A quote covering only the demonstration phase is not a project quote, and this is the main reason internal AI projects are abandoned after eighteen months.
Conclusion
Ten concepts, and at bottom a single idea: there is nothing magical in any of this.
Weights tuned by example. Text cut into tokens. Tools wired around the outside. Your documents placed on the desk at the right moment. And a great deal of engineering work to keep it standing in production.
What these notions give you is not a technical skill. It is the ability to ask the three or four questions that separate a provider who knows from a provider who recites.
If you are preparing a project and want to test your use case against these trade-offs: book a demo.
