How to understand: The token money you pay in AI large models is essentially the cost of renting GPU computing power?

To put it simply, a token is the 'minimum unit of food' for AI large models.

Just like when we learned to read as children, we first learned individual characters, and later it became more efficient to remember common word combinations directly.

AI doesn't actually recognize Chinese characters or English letters—it only recognizes numbers. When you input a sentence, it's first cut into individual tokens, each token corresponds to a numeric ID, and AI actually processes this string of numbers. When outputting, it works in reverse: first generate numeric IDs, then translate them back into text for you to read.

🔹So how does AI know what the next word is likely to be?

It relies on training from massive amounts of text, memorizing the probabilities of what follows each token. All these probabilities are stored in hundreds of billions of parameters, like the model's 'knowledge manual.'

When generating responses, AI essentially 'bounces out one token at a time.' For each token generated, it has to flip through the entire manual, score all possible next words in the dictionary, and output the one with the highest score.

🔹This task is extremely computationally intensive, which is why GPUs are so important.

A CPU is like a smart but single-threaded professor—no matter how fast he flips pages, there are limits. A GPU is like thousands of elementary school students working simultaneously, splitting the manual into thousands of copies so everyone can calculate in parallel, instantly scanning through hundreds of millions of parameters.

So there are two key factors for graphics cards: more cores mean stronger parallel computing power. Now the world is consuming tokens like crazy, which essentially means countless GPUs running wild in the background flipping through manuals and scoring.👇

So the token money you pay is essentially the cost of renting GPU computing power.

And running graphics cards requires electricity and storage, so the industry summarizes it in one sentence:

AI is short on computing power in the short term, energy in the long term, and forever short on storage.