Token Inspector: Count Tokens and Cost Across Models

Language models do not read characters or words. They read tokens: chunks of text drawn from a fixed vocabulary that the tokenizer learned before training. A token can be a whole word, part of a word, a space plus a word, a digit group, a punctuation mark or a single byte. Models are billed, limited and measured in tokens, so the token count is the number that matters for cost and context.

A rough rule for English prose is about four characters per token, but code, numbers and other languages vary widely, which is why measuring beats guessing.

Each model family trains its own tokenizer, so the same text splits differently. A larger vocabulary usually means fewer tokens for the same text. For example, the Japanese greeting こんにちは世界 is 2 tokens in o200k_base and 4 in cl100k_base, the older encoding used by GPT-4 and GPT-3.5 Turbo.

• Languages: text in languages that were less represented in the tokenizer's training data often needs more tokens per word.
• Code: indentation, brackets and identifiers split in vocabulary-specific ways.
• Numbers: some tokenizers group digits in threes, others split every digit, which changes the count of long numbers and IDs.
• Whitespace and structure: JSON, tables and long separator lines can cost more than they appear to.

Chat models do not receive your messages as plain text. The provider wraps each message in special tokens that mark the role and the end of the turn, and may add a few more tokens to prime the reply. Open models publish their exact chat template, so the Token Inspector can render it and count it. For OpenAI models the exact template is not fully public, so chat mode uses the ChatML layout as an approximation. The overhead is small per message but adds up in long conversations, and tool definitions add more on top.

Anthropic does not publish the tokenizer that Claude models use, so no tool can reproduce it offline. The Token Inspector shows an estimate based on a comparable public tokenizer, labels it as an estimate and does not draw token boundaries. Anthropic also notes that newer models can use a different tokenizer, so counts can shift between model generations. For an exact figure, use Anthropic's token counting API, which counts a request without generating a reply.

• Exact: the model's published tokenizer is used, with its chat template where one exists. The count should match the provider's.
• Close proxy: a published tokenizer that very likely matches, such as a newer OpenAI model counted with o200k_base. Expect a match, but the vendor has not confirmed it.
• Estimate: the vendor does not publish its tokenizer, so the number is an approximation. Treat it as a planning figure, not a billing figure.

API prices are quoted per million tokens, with separate rates for input and output, and output is usually the more expensive of the two. A request costs its input tokens times the input rate plus its output tokens times the output rate. The context window is the maximum of input plus output a model can handle in one request, so reserve room for the reply when you check that a prompt fits.

Many providers offer prompt caching: a repeated prompt prefix is billed at a lower rate for cached input tokens. The cost panel shows the difference when you enter a cached token count and the model has a cached price. Prices come from our model catalog and are estimates; check the provider's page before you budget. See the LLM cost calculator for monthly projections and model pricing for the catalog.

Counting runs in your browser. Your text is not uploaded to our servers. Tokenizers for open models are downloaded from Hugging Face the first time you use them and then cached in your browser. The OpenAI encodings are bundled with the page. Share links store your inputs in the URL hash, the part after #, which browsers do not send to a server. Anyone who has the link can read the text, so do not share a link that contains secrets or personal data.