Skip to main content

AI glossary Models & Architecture

What is a small language model (SLM)?

SLM (Small Language Model)

← All glossary terms
In short

A compact language model built for narrower tasks, with lower cost and latency than frontier models and small enough to run on modest hardware.

Explained

A small language model (SLM) is a language model with far fewer parameters than frontier LLMs, designed to do a narrower set of tasks well. SLMs are often trained on carefully selected data, distilled from a larger model or fine-tuned for one domain or workflow. Because they need less compute, they are cheaper and faster per request and can run on a single server, a laptop or a phone. They suit high-volume assistants, on-device use and hybrid set-ups where a router sends only the hard cases to a larger model.

SLM vs LLM

There is no fixed size threshold; "small" is relative to the largest models of the moment. What changes in practice is the trade-off.

A large LLM has broad knowledge and stronger reasoning on open-ended problems, at a higher cost per request and with heavier infrastructure. An SLM gives up some breadth in exchange for speed, lower cost and the option to run it on your own hardware. On a well-defined task with good data, a tuned SLM can match a larger general model.

When an SLM makes sense

  • High-volume, repetitive tasks: classification, extraction, routing, short answers.
  • Strict latency targets, such as voice or real-time interfaces.
  • Data that cannot leave a device or an on-premises environment.
  • Edge and mobile deployments with limited memory and power.
  • Hybrid architectures where the small model handles most requests and escalates the rest.

Limits

  • Less general knowledge, so it depends more on good prompts, retrieval and tuning.
  • Weaker on long, multi-step reasoning and on topics outside its domain.
  • Needs evaluation on your own cases: public benchmarks do not show how it will do on your data.
  • Tuning and maintaining your own model adds work that a hosted LLM does not.

Frequently asked questions

What does SLM stand for?

SLM stands for small language model. It describes a language model with relatively few parameters, built to be efficient and focused rather than to cover every possible task.

How small is a small language model?

There is no official cut-off. The term is used for models small enough to run on a single GPU, a laptop or a phone, in contrast with frontier models that need large clusters. Microsoft's Phi-3 report, for example, describes a model designed to run locally on a phone.

Are SLMs better than LLMs?

Not in general; they are better for some jobs. For a narrow, high-volume task an SLM can be cheaper and faster with similar quality. For open-ended reasoning or broad knowledge a larger model usually does better.

Can I run an SLM on my own servers?

Yes, and that is one of the main reasons to choose one. Many SLMs have open weights, so you can deploy them on-premises or in your own cloud account and keep the data inside your perimeter.

How do you make an SLM good at a specific task?

The usual tools are fine-tuning on examples from your domain, distillation from a larger model, and retrieval that gives the model the documents it needs. A clear evaluation set from your own cases tells you whether it is good enough.

Decision guides that use this term

Where it shows up at Thinkia

Further reading

Sources

Updated: 2026-10-21

Want to see how this applies to your organisation?

Talk to an AI Expert