A large language model (LLM) is software that predicts the next piece of text based on patterns learned from huge amounts of writing. It reads and writes in small units called tokens, only sees what fits in its context window, and its temperature setting controls how consistent its answers are. It can confidently invent facts (hallucinate), so it needs your data and clear limits. Its cost is driven mainly by tokens in and out, model size and the number of requests.
- what is a large language model
- what are tokens in ai
- what is a context window
- why does ai hallucinate
What a large language model actually is
Picture a new front-desk hire who has read millions of conversations, books and articles but has never set foot in your shop. They speak fluently, read customers well and write polite replies, but they have no idea what your new abaya costs or when your Khobar branch opens. That is a large language model (LLM): an extremely capable language engine with no built-in knowledge of your business.
Technically, the idea is simple. The model reads the text in front of it, predicts the most likely next piece, and repeats that step until the reply is complete. There is no database of facts inside it, only language patterns learned during training. That is why it shines at wording, summarizing, translating and classifying, and why it needs accurate information from you when the topic is your prices, policies or stock.
- What it's great at: answering inquiries, summarizing documents, writing product descriptions, sorting messages, pulling data out of messy text, translating between Arabic and English.
- Where it needs help: exact numbers, recent information, details specific to your business, and complex calculations without tools.
- What you shouldn't leave to it alone: final financial, medical or legal decisions.
What happens when a customer sends a message?
Say a customer messages your store on WhatsApp: do you deliver to Dammam today? Your app doesn't send that sentence to the model on its own. It assembles a package: the system instructions that define the assistant's role and tone, the earlier conversation, perhaps delivery details from your database, and finally the customer's question. The model reads the whole package, writes the reply token by token, and your app sends it back. Every part of that package affects answer quality, speed and cost, which is what the next sections unpack.
The model owns the language; you own the truth. Hand it the truth and it will phrase it beautifully.
Tokens: the currency everything is measured in
Models don't read words the way we do. They chop text into small pieces called tokens. A token might be a whole word, part of a word, a punctuation mark or even a space. Everything in the LLM world is measured in tokens: how much you can send, how fast the reply comes back and what your monthly bill looks like.
Why this matters for an Arabic-first business
The way a model splits text (its tokenizer) is shaped by its training data, and many models have seen far more English than Arabic. The general result is that an Arabic sentence can turn into more tokens than an English sentence with the same meaning, partly because Arabic words pack prefixes and suffixes together, as in «وبمنتجاتكم» (and with your products). The gap varies a lot between models, and newer models have improved Arabic support considerably, so don't rely on a fixed ratio. Test it yourself.
| What token count affects | Example from your business |
|---|---|
| Cost | Every customer message and every bot reply is billed by tokens in and out |
| Speed | A long answer about your return policy takes longer than a short one |
| Limits | A catalog with thousands of products may not fit in a single request |
| Quality | Padding in your instructions distracts the model and raises the bill at the same time |
In practice, this means three things. First, the same service can cost differently in its Arabic and English versions. Second, Arabic text fills the context window at a different rate. Third, choosing a model that tokenizes Arabic efficiently can pay off over time, especially if most of your customers write in Arabic. Numbers, emoji and links consume tokens too.
The context window: the model's working memory
The context window is the maximum number of tokens a model can look at in a single request. It includes the system instructions, the conversation history, any documents you attach and the reply it is about to write. Think of it as an employee's desk: everything on the desk is visible and usable, and anything in the filing room doesn't exist for them until you put it on the desk.
A real-world example: a customer-service bot for a restaurant in Jeddah. Early in the chat, the customer mentions a peanut allergy, then talks at length about the order, delivery and offers. If the conversation grows and older messages get trimmed to save space, the bot can forget the allergy. Not out of carelessness, but because that message fell out of its window.
- Usually cheaper and faster
- Fine for short questions and focused tasks
- Needs long conversations to be summarized or trimmed
- Can't hold large documents in one go
- Fits contracts, catalogs and long conversations
- Every extra token you put in still costs money
- The model can pay less attention to details buried in the middle
- No substitute for choosing the right information carefully
- Keep critical facts (allergies, order number, branch) in a short pinned summary sent with every request.
- Summarize long conversations automatically instead of dropping the beginning.
- Keep system instructions short and specific.
Temperature: creativity versus consistency
Temperature is a setting that controls how adventurous the model is when picking the next token. At a low temperature it sticks to the most likely choice, so answers are consistent and predictable. At a higher temperature it tries less likely options, so answers become more varied and creative, and sometimes less accurate.
| Task | Suggested temperature | Why |
|---|---|---|
| Answering questions about prices and return policy | Low | You want the same correct answer every time |
| Extracting data from an invoice or form | Very low | Any creativity here is an error |
| Sorting WhatsApp messages (complaint, order, question) | Low | Consistency beats variety |
| Writing product descriptions or Ramadan posts | Medium | Some variety while keeping the voice on brand |
| Brainstorming campaign or project names | Higher | Variety is the whole point |
Remember that a low temperature does not prevent hallucinations. It makes the model consistent, but it can be consistently wrong if you never gave it the right information.
Hallucinations: why models confidently make things up
A hallucination is when a model writes something false in a confident, convincing tone: a phone number that doesn't exist, a return policy you never announced, a medicine at the wrong dose. The reason is simple. The model is built to produce text that sounds right, not to verify that it is right. When asked about something it doesn't know, the most likely-sounding text can be a plausible invention.
When hallucinations get worse
- When it is asked about business-specific details you never gave it.
- When you ask for exact numbers, dates or citations.
- When the instructions force it to always answer and never allow I don't know.
- When the question is vague or could mean several things.
- 1Give it the source
Send the correct information from your own data (prices, policies, FAQs) with the request instead of relying on its memory. This is called grounding.
- 2Allow it to say no
State explicitly in the instructions: if the answer isn't in the provided information, say you don't know and offer to connect the customer with a staff member.
- 3Ask for citations
Have it name the source it used so you or your team can check quickly.
- 4Connect it to tools
For prices, stock and order status, have your app fetch the number straight from the database instead of letting the model remember it.
- 5Monitor and test
Keep a list of test questions and rerun them after every change to the instructions or the model.
What actually drives the cost of AI
Most model providers charge for inference, meaning each time the model runs, based on tokens in and tokens out. Output tokens (the reply) usually cost more than input tokens. So your bill is shaped by small design decisions that repeat thousands of times.
| Cost driver | How it shows up | How to reduce it |
|---|---|---|
| System instruction length | Long instructions sent with every message from every customer | Trim them and remove repetition |
| Reply length | A bot writing paragraphs when two lines would do | Set reply length and style in the instructions |
| Model size | Using the most powerful model to sort a simple message | Small model for simple tasks, large for complex ones |
| Conversation history | Resending the entire conversation at every step | Summarize older turns and keep what matters |
| Request volume | Running the model on every event, even unnecessary ones | Run it only when needed and cache repeat answers |
Example: a perfume store in Riyadh gets many times its usual messages during Eid. If the bot's instructions are very long, every single message carries that extra weight, and costs climb with the season. Trimming instructions and caching answers to repeat questions (like delivery times) can make a clear difference without the customer noticing anything.
A practical checklist for your first AI feature
Now that the five concepts are clear, here is a structured way to turn them into a real feature in your app, whether it's a store assistant or an internal tool for your team.
- 1Pick one narrow task
For example: answer delivery and returns questions only. Narrow tasks are easier to test and less prone to hallucination.
- 2Gather your sources of truth
Your FAQ page, return policy, branch list and opening hours. Update them before handing them to the model.
- 3Write short system instructions
Role, tone, limits and what to do when it doesn't know. See our guide to writing prompts for ready examples.
- 4Tune the settings
Low temperature, a maximum reply length and summarization for long conversations.
- 5Test with real questions
Twenty questions from real customers, in MSA and dialect, with a few deliberately out of scope.
- 6Launch gradually and watch
Start with a share of customers, review conversations and cost weekly, then expand.
Build a customer-service assistant for a perfume store in Riyadh that replies in Arabic and English. It must rely only on the FAQ page and return policy I will upload, and clearly say it doesn't know when the answer isn't there, offering to hand over to a staff member on WhatsApp. Keep replies short (3 lines max), and add a dashboard where I can see conversations and daily message counts.
Try it in FanannIf you want the assistant to go beyond answering and actually do things, like booking an appointment or changing an order, read our explainer on AI agents and how they plan and act step by step.
Key takeaways
- A model doesn't know facts; it predicts likely text, so it needs your data to answer accurately.
- Tokens are the unit of measurement and billing, and Arabic text can split into more tokens than English, depending on the model.
- The context window is the model's working memory: anything outside it simply doesn't exist for the model.
- Use low temperature for precise tasks like prices and policies, and slightly higher for ideas and marketing copy.
- The biggest cost drivers are repeated long instructions, long replies, model size and conversation volume.
Frequently asked questions
Does the model learn from my customers' conversations automatically?
Do models understand Gulf dialect?
What's the difference between a token and a word?
Can hallucinations be eliminated completely?
Do I need the most powerful model available?
Ask your AI assistant about this
Copy this prompt into the AI assistant you use, then add the details of your own case.
I am reading this guide: “LLMs for business owners: tokens, context windows and hallucinations” https://fanann.com/en/blog/llm-basics-for-business/ My question: what is a large language model Use that page as a source and explain the answer step by step for my situation, tell me what to check before I start, and cite the page link.