What is Inference?

Explained

Inference is when a trained model is used to produce outputs—predictions, classifications, or generated content. It happens in production; optimizing inference (latency, cost) is key for scalability.

Decision guides that use this term

• Open-source vs proprietary LLMs: how should an enterprise choose?
• Sovereign or on-prem AI vs cloud AI APIs: where should your models run?
• Reasoning models vs standard LLMs: when is the extra thinking worth paying for?
• Seat-based vs usage-based AI pricing: which model fits how your organisation will use AI?
• Small vs large language models: which one should run your enterprise use case?

Related terms

• Production Environment where the model serves real users (vs. development/testing).
• API Interface that allows different systems to communicate; key for integrating AI models.
• API Specification Formal description of endpoints, schemas, and errors—e.g., OpenAPI—used for docs, mocks, and validation.
• Automation Executing tasks without human intervention, often with AI for complex decisions.
• Autonomous Systems Systems that operate without continuous supervision, making decisions in real time.
• Cloud AI AI services hosted in the cloud (Azure OpenAI, AWS Bedrock, etc.).