Abstract: Self-knowledge distillation (self-KD) is an attractive technique that empowers the student to distill knowledge within itself, in which one predominant self-KD scheme is to teach the shallow ...