All guides / AI explained
AI explained · Seneca guides
AI tokens explained: understand usage and stop bill shock
A short question can create a surprisingly long AI bill. Tokens explain why. Understanding what your applications send, what models produce and who is using them turns an unpredictable expense into something you can manage.
What is an AI token?
A token is a small unit of information that a model processes. For text, it might be a word, part of a word, punctuation or a space combined with nearby characters. Tokens are not the same as words: the count changes with the model, language and content. Images and audio also consume capacity and can have different charging rules.
Input tokens are the information sent to the model. Output tokens are what it generates. Your input can include instructions, earlier messages, retrieved documents and tool results alongside the question you typed. That is why ‘summarise this contract’ can involve thousands of tokens even though the request is only four words.
How token pricing turns into a bill
Many hosted model APIs publish separate prices per million input and output tokens. Calculate each part, then add any tool, storage or other service charges. Some models also bill for reasoning tokens used internally; those tokens may not appear in the answer you see. Check the provider’s usage record rather than counting only the visible reply.
Here is an illustrative example, not a supplier quote. At £2 per million input tokens and £8 per million output tokens, a request using 10,000 input tokens and 1,000 output tokens costs £0.02 + £0.008 = £0.028. Run that request 10,000 times and the token charge becomes £280, before tax and other services. Repeated steps, retries and longer conversations can all increase consumption.
Why AI agents can spend more than a chatbot
An agent can make several model calls while completing one task: read a document, search a system, interpret the result and draft an answer. Each step can add usage. Sending an entire document library with every question also costs more than retrieving the passages needed for that question.
Use the model and amount of context the task actually needs. A compact model can handle routine classification while a more capable model deals with difficult cases. Eligible repeated inputs can benefit from prompt caching where the provider supports it. Measure the cost of a completed, useful task, including rework, rather than judging value from the token price alone.
Remove AI bill shock
Seneca can place a managed AI broker between your applications and their models. It manages provider credentials, records usage by team or application and applies budgets and routing rules to the requests passing through it. This gives you a central view of who is consuming what and where that spending creates value.
Set alerts before a budget is exhausted and rules to stop new requests or route suitable work to a lower-cost model. Direct calls outside the broker need their own controls, and requests already running can still incur charges. For steady or sensitive workloads, private AI offers a different cost structure: compute, power and support rather than a hosted-model charge for every token.
A few useful terms
- Context window
- The maximum amount of information a model can work with in one request; input and generated output share limits that vary by model.
- Inference
- Running a trained model to produce a result.
- Prompt caching
- Reusing eligible previously processed input to reduce repeated processing and, where supported, its cost.