# 🔤 How AI "Reads" Text: Tokenization

Watch how AI breaks your words into tokens—the building blocks it actually processes

The quick brown fox jumps over the lazy dog.
Try these examples:
Tokens will appear here...
0
Characters
0
Tokens
0
Chars per Token
💡 What's happening here?
AI doesn't read "words" like we do. It breaks text into tokens—chunks that might be whole words, parts of words, or even single characters. This is why AI sometimes:

- Cuts long words in weird places
- Handles common words easily but struggles with rare ones
- Has limits like "8K tokens" (not words!)
- Treats "can't" differently than "cannot"

Orange tokens = special characters (spaces, punctuation, emoji)