Approach
where p(w) is the prior over w and p(D|w) is the model likelihood.
After re-training we set each weight to the mean of the component that takes most responsibility for it.
Experiment
References:
SOFT WEIGHT-SHARING FOR NEURAL NETWORK COMPRESSION, Karen Ullrich, 2017, ICLR
