Quick answer
Training is the one-time process of teaching an AI model from data — takes weeks or months and costs millions or billions of dollars. Inference is using the trained model to answer questions — happens every single API call and is what you actually pay for. Training is the upfront investment; inference is the ongoing operating cost.
These two words come up in every AI announcement, but most people use them interchangeably. They are completely different things. Knowing the difference helps you understand AI economics, AI hardware news, and pricing.
What is training?
Training is the months-long, mostly-one-time process where the AI learns. Engineers feed it billions of examples, the model adjusts its internal weights until it can predict patterns well, and the resulting "trained model" is saved as a giant file (often 100GB+). This process is fantastically expensive — GPT-5 training is estimated to have cost $100M-$500M+.
What is inference?
Inference is what happens every time you actually USE the trained model. You send a prompt, the model runs forward through its layers, generates a response. This happens in milliseconds to seconds. The model itself does not change — it is just running computations.
Why does the distinction matter?
- Training happens once (or rarely); inference happens billions of times
- AI companies recoup training costs through inference revenue
- Hardware needs differ — training needs huge GPU clusters; inference can run on smaller chips
- When you pay $20/month for ChatGPT, you pay for inference, not training
- Open-source models give you the trained weights — you run inference yourself
A single ChatGPT response uses far less energy than people think — roughly 0.3-3 watt-hours per query, similar to a Google search. The big energy use is at TRAINING time, where models burn through millions of GPU-hours.
Why is inference cost falling so fast?
Three reasons. First, hardware is getting better (NVIDIA Blackwell, Google TPUs, custom inference chips). Second, models are getting more efficient (MoE, distillation, quantization). Third, competition is fierce — DeepSeek, Together AI, Groq are pushing prices down. Inference cost per token has dropped 90%+ from 2023 to 2026.
Related reading
Bottom line
Training = teach. Inference = use. The economics of AI today are training as fixed cost, inference as variable cost. When pricing or news talks about cost, they almost always mean inference.
