Most developers use AI every day. We paste prompts into OpenAI, Anthropic, or Google models and get answers.
Simple.
But here’s the part most developers don’t fully understand:
What exactly are you paying for?
Not API calls.
Not requests.
Not “messages.”
You’re paying for tokens.
And if you’re building AI apps, token understanding is not optional.
It directly impacts:
Cost
Latency
Context window limits
Memory management
Model performance
The difference between a scalable AI app and an expensive mess often comes down to one thing:
Token efficiency.
Let’s break this down.
First: What Is a Token?
A token is the smallest chunk of text an LLM processes.
Not words.
Not characters.
Something in between.
For example:
Input:
Hello worldToken breakdown might look like:
["Hello", " world"]That’s 2 tokens.
Now look at this:
JavaScript developers love ReactPossible token split:
["Java", "Script", " developers", " love", " React"]That becomes 5 tokens.
This surprises people.
Because:
1 word ≠ 1 token
Rule of thumb:
1 token ≈ 4 characters
100 tokens ≈ 75 words
1000 tokens ≈ 750 words
But this varies.
Think of It Like Breaking Lego Blocks
Imagine this sentence:
I love building AI appsA human sees 5 words.
An LLM may see:
["I", " love", " building", " AI", " apps"]That’s 5 tokens.
But if you write:
supercalifragilisticexpialidociousIt may split into:
["super", "cal", "ifrag", "ilistic", "exp", "iali", "docious"]Now one word became 7 tokens. Same word. Different cost.
That’s why prompt design matters.
The Two Types of Tokens
Every AI request has two token buckets.
1. Input Tokens (Prompt Tokens)
Everything you send.
Includes:
System prompt
User prompt
Chat history
Context
Tool outputs
Example:
{
"system": "You are a senior React developer",
"user": "Explain useEffect"
}Token count:
System prompt:
You are a senior React developer≈ 7 tokens
User prompt:
Explain useEffect≈ 3 tokens
Total input: 10 tokens
2. Output Tokens (Completion Tokens)
Everything the model generates.
Example response:
useEffect is a React hook that allows side effects...This could be: ≈ 30 tokens
So:
Input = 10
Output = 30
Total = 40 tokensSimple. But there’s more.
How the Calculation Actually Works
Here’s the real formula:
Total Tokens = Input Tokens + Output TokensBilling:
Cost = (Input Tokens × Input Price)
+ (Output Tokens × Output Price)Example:
Model pricing:
Input: $0.002 / 1K tokens
Output: $0.006 / 1K tokens
Request:
Input = 1500 tokens
Output = 500 tokens
Calculation:
Input cost = 1500 × 0.002 / 1000 = $0.003
Output cost = 500 × 0.006 / 1000 = $0.003Total:
$0.006Tiny request. But scale it to 1 million users. That becomes real money.
💡 Enjoying this article?
Every week day, I publish practical, production-ready deep dives covering Web development, System Design, Open source projects, Tech industry trends and AI Engineering and tools.
Example 1: Simple Prompt
Prompt:
What is React?Approx token count:
What = 1
is = 1
React = 1
? = 1Total input: 4 tokens
Response:
React is a JavaScript library for building UIs.Approx: 10 tokens
Total:
4 + 10 = 14 tokensVery cheap.
Example 2: Large Context Prompt
Prompt:
Analyze this 500-line codebase and find performance issues.Plus attached code.
Code itself: ~6000 tokens
Instruction: ~15 tokens
Total: 6015 input tokens
Response: 800 tokens
Total: 6815 tokens
This is where costs jump.
Example 3: Chat History Explosion
Conversation:
User:
How does JWT work?Assistant: 200 tokens
User:
How is it different from sessions?Important: Most APIs resend previous history.
So request #2 becomes:
User Q1
Assistant A1
User Q2That means:
Input = old tokens + new tokensIf history grows:
200 → 800 → 2000 → 6000Cost grows with it. This is why chat apps get expensive. Fast.
Why Same Prompt Can Have Different Token Counts
Look at this:
Prompt A:
Explain JavaScript closuresPrompt B:
Explain JS closuresPrompt A might be 5 tokens.
Prompt B might be 4 tokens.
Why?
Tokenizer compression.
Some common patterns are stored efficiently. That’s why model tokenizers matter.
Examples:
OpenAI uses BPE
Anthropic uses variations of tokenizer compression
Google uses model-specific tokenization
Same text. Different counts.
Most people count only the visible prompt. Bad idea.
Real request includes:
System prompt
You are a helpful assistant...Hidden cost.
Function/tool definitions
If you use tool calling:
{
"functions": [...]
}This adds hundreds or thousands of tokens.
Retrieved RAG chunks
If your vector DB sends:
Top 5 documentsThat may be 3000 tokens. Huge.
Conversation memory
Long chats = token bloat. Always.
How to Reduce Token Usage
1. Compress prompts
Bad:
Can you please explain in great detail...Good:
Explain in detail...Shorter. Same meaning.
2. Trim chat history
Keep:
recent context
relevant messages
Drop:
old irrelevant messages
3. Summarize old conversations
Instead of:
50 previous messagesUse:
Summary of previous discussionMassive savings.
4. Limit output tokens
Set:
max_tokens: 300Avoid runaway completions.
5. Use embeddings smartly
Don’t dump 20 documents. Retrieve only what matters.
Real Developer Formula for Estimation
Before sending:
Estimated tokens = characters ÷ 4Example:
Prompt length: 400 characters
Approx:
400 ÷ 4 = 100 tokensExpected response: 300 tokens
Total: 400 tokens
Quick estimate. Useful in production.
Best Token Tools
Developers should know these:
Use them. Don’t guess. Measure.
Final Thought
The biggest mindset shift in AI engineering is this:
Every word has a price.
Prompt engineering isn’t just about better answers.
It’s about:
efficiency
scalability
performance
cost control
The best AI engineers don’t just write prompts. They write lean prompts.
And in the LLM world:
Lean prompts win.
Always.
Thank You for Reading!
I hope you found it helpful and informative. If you have any questions or feedback, feel free to leave a comment below. Your support and engagement mean a lot to me.