AI glossary Models & Architecture
What is Knowledge Distillation?
← All glossary termsTraining a smaller model to mimic a larger teacher model.
Explained
Knowledge distillation transfers behavior from a large 'teacher' network to a compact 'student', cutting cost and latency at deployment. It pairs with quantization and SLMs for efficient assistants.