A large language model, or LLM, is a program trained to predict the next piece of text. Given everything written so far, it computes a probability for every possible next token, picks one, and repeats. Writing, summarising, translating, coding and answering questions all come out of that one mechanism, trained on a vast amount of text and then tuned to behave like a helpful assistant. The same mechanism explains the limits: the model is optimised for plausible text, not for true text, it knows nothing about your business unless you show it, and it remembers nothing between calls unless you build the memory.
How does a large language model work, in plain terms?
A large language model works in three layers. First, text is cut into tokens, pieces of words a few characters long. Second, a neural network called a transformer reads all the tokens in its context window at once and uses a mechanism called attention to decide which earlier tokens matter for the next one (Vaswani et al., 2017). Third, the network outputs a probability for every token in its vocabulary, one is sampled, it is appended, and the loop runs again.
The assistant behaviour is added afterwards. A model first learns from raw text to predict what comes next. It is then trained on human-written examples and on human or AI feedback to follow instructions, refuse harmful requests and stay on task (Ouyang et al., 2022). The "reasoning" models of 2025 and 2026 did not gain a new faculty: they were trained to spend more tokens thinking before they answer, and you pay for those tokens.
One practical detail surprises most business owners. Every request to an LLM sends the whole conversation again. The model keeps nothing between calls, so the software around it decides what it sees each time.
What is a token, and why does Greek text cost more?
A token is the unit an LLM reads and bills by, usually a few letters or part of a word. Greek text is cut into more tokens than the same English text, so it costs more, fills the context window faster and needs different settings in document search. I measured the gap on forty hand-written Greek and English sentence pairs in tokenlab:
- On OpenAI's newest vocabulary (
o200k), Greek needs 2.06 times the tokens of English. Almost all of that comes from Greek letters taking two bytes each, not from the vocabulary. - On the older
cl100kvocabulary, still used by some embedding models, Greek needs 5.12 times the tokens.
Open models show the same pattern. The Llama 3.1 tokenizer uses 2.73 tokens per Greek word against 1.33 per English word, and the Greek-adapted Llama-Krikri brought Greek down to 1.65 by adding about 21,000 Greek tokens (Roussis et al., 2025). You can type your own text into tokenlab and watch it split.
What can an LLM do well?
An LLM does well at tasks where a plausible, fluent answer is also a correct one, or where a person or a program checks the result:
- drafting emails, replies, product descriptions and first versions of documents;
- summarising long material you provide, such as a contract, a thread or a report;
- classifying and routing, such as tagging tickets or sorting invoices into categories;
- extracting fields from messy text into a structure a program can store;
- translating and rewriting between registers and languages;
- writing and reviewing code, with tests to check it;
- answering questions over documents it is given at the moment of the question.
What can an LLM not do?
An LLM cannot reliably do five things, and every serious system is designed around them:
- Know when it does not know. Language models are trained and graded in ways that reward a confident guess over "I don't know" (Kalai et al., 2025). Asking the model to rate its own confidence does not fix this, because its self-reported confidence barely moves when it is wrong.
- Know your business. A model was not trained on your price list, your contracts or yesterday's orders. It needs them in the prompt, through retrieval (RAG) or through tools.
- Remember. Every call starts from zero. Memory across conversations is something you build and store, not something the model has.
- Count and calculate at scale. Arithmetic, dates and totals belong in code, with the model deciding what to calculate rather than calculating it.
- Give the same answer twice. Even at temperature zero, hosted models can return different text for the same input, because of how servers batch requests (Thinking Machines, 2025). Test for properties, not for exact wording.
I felt the first limit directly when I trained my own model from scratch. BSLM is a 72-million-parameter bilingual assistant that fits in 78 MB and runs on a phone. It was built to check facts with a tool and quote the source rather than recite them from memory, and a coded check only lets it deliver a fact that appears in what it has just read. On 570 held-out agent tasks it scores 91.2% overall, and its weakest family is still open fact questions, at 68% even with the lookup: the ceiling of a model that size. Large models have a much higher ceiling, and the design rule is the same: look it up, then answer.
How do an LLM, a chatbot and an AI agent relate?
The LLM is the engine, the chatbot is one way to talk to it, and the AI agent is a loop that lets it act. A chatbot sends your message to an LLM and shows the reply. An AI agent gives the LLM tools, such as a database lookup or an email draft, and lets it choose the next step until a goal is met. The difference, and when each one is the right choice, is in agentic AI explained.
Which LLM should a business use?
A business should pick the model that finishes its task correctly at the lowest total cost, measured on its own examples. That is rarely the cheapest model per token, because a cheaper model that fails still bills its tokens, then the retry, then the cost of the mistake. Start with a capable hosted model, prove the task works on fifty to a hundred real cases, then try cheaper settings one change at a time. For Greek work, measure on Greek text, since both cost and quality differ from English. For decisions rather than text, a structured answer or a decision model is often better than a large model writing freely, as explained in Jev and structured outputs.
Frequently asked questions
Is ChatGPT a large language model?
ChatGPT is an application built on OpenAI's large language models. The LLM is the model underneath, and ChatGPT adds the interface, tools such as web search, and account features around it.
Do LLMs learn from what I type into them?
It depends on the product and the settings. Through the OpenAI API, for example, data is not used for training by default (OpenAI, your data). Consumer apps have their own settings, so a business should use an API or a business plan and read the data terms before sending customer information.
Do LLMs understand Greek?
The major models read and write Greek well, but Greek costs about twice the tokens of English on current vocabularies, and the output often reads as translated. Test on real Greek material and have a native speaker review anything customer-facing.
If you want to know which of your tasks an LLM can take over safely, send me one real example of the task and I will tell you what it needs around the model. How I build these systems is on agentic AI in production.