Estimate a JSON text token budget with o200k_base
The token counter estimates the token footprint of JSON text using the o200k_base tokenizer. It helps plan how much room a data sample takes in a prompt.
When should you use it?
- Compare compact and formatted JSON token counts.
- Check a fixture before adding it to an LLM prompt.
- Find whether a large data sample dominates an input budget.
Example: count a small JSON payload with o200k_base
This exact JSON text contains 15 UTF-8 bytes and uses 5 tokens with o200k_base. Changing the text or encoding can change the count. Chat messages, tool definitions, and model-specific framing are not included.
{"name":"Alex"}encoding: o200k_base
tokens: 5
utf8Bytes: 15
characters: 15Count the text that will actually be sent
Whitespace, field names, repeated structure, and string content all contribute to tokenization. Count the final text after any intended formatting or minification. Removing characters does not imply an identical percentage reduction in tokens, because the tokenizer groups text into vocabulary units rather than counting one token per character or byte.
Treat this count as one part of a request budget
The selected o200k_base encoding is the only count this page reports. A model using another encoding can produce a different number. Chat messages, tools, schemas, images, and response tokens may add usage beyond this JSON text. Use the target API usage information for an exact request total and its current pricing when estimating cost.
How many tokens will this JSON sample add to an LLM prompt?
Paste the JSON text and count it with o200k_base. Formatting, key names, and repeated values all contribute tokens, so use the exact text you intend to send. The result helps compare candidate samples before adding other prompt material.
JSON text whose token count can be compared
json{"events":[{"type":"open"},{"type":"close"}]}Try a formatted copy of the same data and compare counts. The count covers only this JSON text, not chat framing or tool definitions.
Will the count equal every model’s billed input tokens?
No. Models can use other tokenizers and request wrappers. Treat this as an estimate for one named encoding, not a universal bill or context-limit calculation.
What should you check before using the result?
A model may use a different tokenizer. Chat roles, tool schemas, system text, and output tokens are outside this count, so do not treat it as a complete request bill or context calculation.
Understand JSON syntaxTechnical reference: Tokenizer implementation and encodings