Inference

Inference is running a trained model on new input to produce an output. Training updates the model's weights. Inference does not.

A model is trained or fine-tuned, then served. Each API call, chat reply, image, or transcription is inference. Latency, price per token, and rate limits describe inference, not training. Batch and cached prompts are still inference; they change cost and speed, not whether the weights are being updated.