A token is a small chunk of text that an AI model reads and writes. AI doesn't see words or letters. It cuts your text into tokens, turns each one into a number, and works with those numbers. A short, common word is often one token, and a longer or rarer word gets split into several pieces. Tokens are the unit everything in AI gets measured in: what you pay, how much the AI can read at once, and how fast you hit your usage limit. In English, one token is roughly 4 characters or three quarters of a word on ChatGPT, and closer to 2.5 characters on current Claude models.
Want to see it for yourself? Paste your text into our free token counter and watch it split into tokens.
TL;DR
| Question | Short answer |
|---|---|
| What is a token in AI? | A chunk of text: a whole word, part of a word, a space plus a word, a number or a symbol. |
| How many tokens is a word? | About 1.3 tokens per English word on OpenAI's models, by OpenAI's rule of thumb. Current Claude models use more, roughly 1.8. |
| Why do tokens matter? | AI companies charge per token, context windows are measured in tokens, and tokens explain why AI is bad at spelling and arithmetic. |
| Do all AIs count the same? | No. ChatGPT, Claude and Gemini each use their own tokenizer, so the same text gives different counts. |
| How can I check a count? | Use the token counter. If you want the bigger picture of using AI at work, start with the free Claude course. |
What is a token in AI?
When you type a message to ChatGPT or Claude, the model never sees your words the way you do. First, a piece of software called a tokenizer chops the text into chunks. Each chunk is a token, and each token has an ID number. The model reads that list of numbers, predicts the next token, then the next, and the tokenizer turns the numbers back into text you can read.
OpenAI's documentation describes tokens as chunks of text for common character sequences. A short common word may be a single token, while a longer word may be several. Anthropic's glossary says tokens "can correspond to words, subwords, characters, or even bytes."
So a token is not the same as a word. Sometimes it's a whole word. Sometimes it's a word with the space in front of it. Sometimes it's a fragment like berry or a single punctuation mark.
How text becomes tokens (with real examples)
Most AI tokenizers use a method called byte-pair encoding, or BPE. Here's the plain version. The tokenizer starts with tiny pieces of text and, during its setup, keeps merging the pairs that show up together most often into new, bigger pieces. After enough merges, common words like "the" or "business" end up as a single token, and rare words get built out of a few smaller pieces. OpenAI's tiktoken project calls BPE "reversible and lossless": you can always turn the tokens back into the exact original text.
We ran some examples through OpenAI's open-source tokenizer (tiktoken, October 2026) so you can see exactly what happens. These use o200k_base, the tokenizer behind GPT-4o, GPT-4.1, GPT-5 and the o-series models.
The same word can be one token or three
| What you type | Tokens | How it splits |
|---|---|---|
strawberry on its own | 3 | st · raw · berry |
strawberry with a space in front, as in a sentence | 1 | ␣strawberry (the ␣ marks the space) |
Strawberry with a capital S | 3 | Str · aw · berry |
That middle row surprises people. Inside a normal sentence, the word comes with a space in front of it, and " strawberry" with the space is common enough to have its own token. So when you ask a full question, the model gets one ID number for the whole word and no individual letters at all. Capitalize it or use it with no space, and it splits up differently.
A full question
The question How many r's are in strawberry? comes out to 8 tokens:
How · ␣many · ␣r · 's · ␣are · ␣in · ␣strawberry · ?
Notice that most tokens carry their leading space, and the apostrophe-s is its own piece.
Numbers split into chunks
Numbers get cut into chunks of up to three digits, working left to right:
| What you type | How it splits |
|---|---|
123456789 | 123 · 456 · 789 |
3.14159 | 3 · . · 141 · 59 |
12345 * 6789 = 83810205 | 123 · 45 · ␣* · ␣ · 678 · 9 · ␣= · ␣ · 838 · 102 · 05 |
Look at that last one. 12345 becomes 123 and 45, which has nothing to do with ones, tens and hundreds. Keep that in mind for the math section below.
You can try any of these in the token counter. Paste your text and the tool shows each token as a colored chip.
How many tokens is a word? The rules of thumb
Each AI company publishes its own rough ratio. Here they are, quoted from the source:
- OpenAI: "As a rough rule of thumb, 1 token is approximately 4 characters or 0.75 words for English text."
- Google Gemini: "For Gemini models, a token is equivalent to about 4 characters. 100 tokens is equal to about 60-80 English words."
- Anthropic (Claude): Anthropic's models page says "1M tokens is roughly 555k words or 2.5M Unicode characters on the current tokenizer (introduced with Claude Opus 4.7)." Do the division and that's about 2.5 characters per token, or a little over half a word per token. Older Claude models were closer to 3.5 characters per token.
Tokens to words: a quick conversion table
These are estimates for ordinary English text, worked out from the ratios above. Your real count depends on the text.
| Words | OpenAI (GPT) | Gemini | Current Claude |
|---|---|---|---|
| 100 words | about 133 tokens | about 125 to 167 tokens | about 180 tokens |
| 750 words (a blog post) | about 1,000 tokens | about 940 to 1,250 tokens | about 1,350 tokens |
| 1,000 words | about 1,333 tokens | about 1,250 to 1,667 tokens | about 1,800 tokens |
Going the other way, 1,000 tokens is roughly 750 words on OpenAI's models and roughly 555 words on current Claude models.
A few things push the count up. Text in languages other than English, computer code, long numbers, unusual names and heavy formatting often take more tokens per character. If you need a real number instead of a ballpark, run your text through the token counter.
Why different AIs count tokens differently
Every model family builds its own tokenizer from its own training data, so each one has a different list of chunks. The same paragraph can be 100 tokens on one model and 130 on another.
You can even see this within one company:
- OpenAI's older GPT-4 tokenizer (
cl100k_base) splits bare "strawberry" asstr·aw·berry. The newero200k_basesplits it asst·raw·berry. - Anthropic says Claude Opus 4.7 and later use a newer tokenizer, and the same text produces "approximately 30 percent more tokens than on earlier models."
One more thing worth knowing: Anthropic hasn't released Claude's tokenizer, so any Claude count you get outside of Anthropic's own tools is an estimate. That includes ours. For exact Claude numbers, developers can use Anthropic's free token counting API, which Anthropic itself describes as an estimate that may differ by a small amount. OpenAI's tokenizer is open source, which is why our counter can show exact counts for GPT-4o, GPT-4.1 and GPT-5 class models.
Why tokens matter to you
You can use AI for years without thinking about tokens. But they quietly control four things you do care about.
1. Cost
When businesses use AI through the API (the version developers build apps on), they pay per million tokens, with output tokens priced higher than input tokens. List prices as of October 2026:
| Model | Input, per 1M tokens | Output, per 1M tokens |
|---|---|---|
| Claude Opus 5.5 | $4 | $20 |
| Claude Sonnet 5.5 | $2 | $10 |
| Claude Haiku 5.5 | from $0.10 | from $0.50 |
| GPT-6 Astra (OpenAI) | $10 | $50 |
| GPT-6.1 Sol (OpenAI) | $2 | $10 |
| GPT-6 Luna (OpenAI) | $0.10 | $0.50 |
To make that concrete: a 1,000-word document is roughly 1,800 tokens on current Claude, so sending it to Claude Sonnet 5.5 costs well under a cent. Single requests are cheap. Costs add up when you run the same job thousands of times, or when the model writes long answers, because output costs five times as much as input on every model in that table.
We learned this firsthand. When we built a paystub checker that let the model think its way through the arithmetic, it cost about $20 per 400 scans. Having the model only read the numbers and letting plain code do the math cut that dramatically. The full story is in why AI is bad at math.
If you pay a flat monthly fee for ChatGPT or Claude, you don't see per-token prices, but tokens still drive your usage, as you'll see below.
2. Context windows
A context window is how much text the model can hold in view at once, measured in tokens. It covers everything in play: your instructions, any files you attach, the conversation so far and the model's reply. OpenAI's docs note that your prompt and the output together must fit the model's context length.
Current limits in the API, as of October 2026:
| Model | Context window |
|---|---|
| Claude Opus 5.5, Sonnet 5.5, Haiku 5.5 and Fable 5.1 | 1M tokens |
| GPT-6 Astra, GPT-6.1 Sol, GPT-6 Luna | 1.05M tokens |
| Gemini 4 Argon | 1,048,576 tokens |
By Anthropic's own math, 1M tokens is about 555,000 words, which is several long novels. That sounds like you can stuff in anything. But bigger isn't automatically better. Anthropic warns: "As token count grows, accuracy and recall degrade, a phenomenon known as context rot." In practice, a focused 5-page document usually gets a better answer than the same 5 pages buried in a 500-page dump. Note that these are API figures. The ChatGPT and Claude apps may give you a different limit depending on your plan.
3. Usage limits
Chat apps have usage limits, and the exact rules depend on the app and your plan. In general, the more text the model has to read and write, the faster you use up your allowance. Long conversations, large attached files and very long answers all add tokens. If you keep hitting a limit, the tips further down usually help.
4. Why AI struggles with letters, counting and math
Tokens explain some of AI's strangest mistakes.
Spelling and letter counting. Remember that in a sentence, " strawberry" is a single token. The model gets one ID number, not ten letters, so "how many r's are in strawberry?" is harder for it than it looks. Many chatbots famously answered 2 (the answer is 3). Tokens aren't the whole story, though: research also shows models struggle to count repeated items even when they can recognize the letters. We dig into it in how many r's are in strawberry.
Exact word or character counts. Models produce tokens, not words or characters, so they often miss a precise target like "exactly 150 words." If a limit really matters, count it with a tool.
Arithmetic. As you saw above, 12345 arrives as 123 and 45. The digit chunks don't line up with place value, and the model writes its answer one token at a time from learned patterns instead of running a calculator. That's why the reliable fix is to have AI write the formula or the code and let something built for math do the calculating. For spreadsheet work, see Claude for Excel, and for the full explanation read why AI is bad at math.
Practical tips for using fewer tokens
You don't need to obsess over tokens, but a few habits get you better answers, lower bills and fewer "you've reached your limit" messages.
- Start a new chat for a new topic. A long thread carries every earlier message along with it. When you switch subjects, open a fresh conversation.
- Paste the part that matters. If your question is about one section of a contract, paste that section, not the whole 40 pages. It also helps you avoid context rot.
- Ask for the length you want. "Answer in 3 bullet points" or "under 100 words" keeps replies short, and output tokens are the expensive ones in the API.
- Summarize, then continue. When a working session gets long, ask the AI to summarize the decisions so far, then paste that summary into a new chat.
- Don't re-upload the same file. If the file is already in the conversation, refer back to it instead of attaching it again.
- Strip the clutter. Email signatures, long disclaimers, repeated headers and pasted formatting add tokens without adding meaning.
- Let tools do math and counting. Ask for a formula, a spreadsheet or code instead of asking the model to work out big numbers in its head. It's cheaper and more accurate.
- Check before you send something huge. Paste it into the token counter first to see roughly how big it is and how it compares with model limits.
What to watch out for
- Any token count outside the company's own tools is an estimate, especially for Claude and Gemini, whose tokenizers aren't public libraries. Treat converters, including ours, as close guides.
- Rules of thumb are for English. Other languages, code and number-heavy text often use more tokens.
- Model names and prices change fast. Every model, limit and price in this post is as of October 2026. Check the provider's pricing page before you budget a project.
- A huge context window doesn't mean perfect memory. The model can still miss details in very long inputs, so verify anything important it pulls from a long document.
Frequently asked questions
What is a token in AI?
A token is a chunk of text that an AI model reads and writes. It can be a whole word, part of a word, a word with a leading space, a number chunk or a punctuation mark. The AI turns each token into a number and works with those numbers instead of letters or words.
How many tokens is a word?
On OpenAI's models, one token is about 0.75 English words, so one word is about 1.3 tokens. Current Claude models use more tokens for the same text, roughly 1.8 tokens per word based on Anthropic's figure of 555,000 words per million tokens. Short common words are often a single token, while long or rare words split into several.
How many words is 1,000 tokens?
About 750 English words on OpenAI's models, and roughly 555 words on current Claude models. Google says 100 Gemini tokens equal about 60 to 80 English words, so 1,000 Gemini tokens is around 600 to 800 words. Use a token counter for a precise number.
Why do AI companies charge by the token?
Tokens are the unit the model actually processes, so they're a fair measure of how much work each request takes. API prices are listed per million tokens, with output tokens costing more than input. As of October 2026, Claude Sonnet 5.5 costs $2 per million input tokens and $10 per million output tokens.
What is a context window in AI?
The context window is the maximum number of tokens a model can work with at once, including your instructions, attached files, the conversation so far and its reply. As of October 2026, current Claude models have a 1 million token window in the API and OpenAI's GPT-6 models have 1.05 million. Accuracy can drop as inputs get very long, so shorter, focused inputs often work better.
Learn to use AI without worrying about the plumbing
Tokens are the plumbing behind every AI tool. Now you know what they are, why "strawberry" trips models up, and how to keep your chats lean. To see your own text in tokens, try the free token counter.
When you're ready to put Claude to work in your business, the free Claude Cowork course walks you through Projects, connectors and real workflows step by step, with no coding required.
Clarence