Topic: model-compression
-
Knowledge distillation: transferring useful predictions from teacher to student
Train a student against both labels and a teacher’s probability distribution, with explicit temperature, loss weighting and independent evaluation. Compression benefits are conditional on teacher quality, student capacity and deployment behavior.
-
Post-training quantization versus quantization-aware training: choosing and checking a smaller model
PTQ converts an already-trained model; QAT lets training adapt to simulated quantization error. Calibration quality, supported operators and the exported model’s measured accuracy matter more than a bit-width label alone.
Machine-readable: JSON