Distillation Methods

Learn representations by predicting one network’s output from another’s, without using negative samples. [Read More]

A student network learns from a teacher network using self-distillation, producing emergent semantic attention maps. [Read More]