Search ⌘K Start a project →

Foundations

Token

A token is the basic unit of text an LLM reads and writes.

It is roughly three-quarters of a word in English, though the split varies by model and tokenizer.

Usage-based pricing, context-window limits, and latency are all denominated in tokens.

Token counts are the unit every cost or capacity conversation in this space eventually collapses to.

Insight: What AI actually costs AI Cost Calculator

← Back to the full glossary

Related terms

Context Window The context window is the maximum text, measured in tokens, a model can consider at once. Inference Cost Inference cost is what it costs to run a trained model on a request.

More in Foundations

Retrieval-Augmented Generation (RAG) Retrieval-Augmented Generation (RAG) retrieves relevant passages at query time and feeds them into an LLM's context. Embeddings Embeddings are numeric vectors that place similar content close together in vector space. Model-Agnostic Architecture A model-agnostic architecture lets an application swap or route between LLM providers without a rewrite. Fine-Tuning (vs RAG) Fine-tuning trains a model's weights on your own examples, changing its behavior directly.

Source: https://customlabs.io/glossary/token/