Approach

where p(w) is the prior over w and p(D|w) is the model likelihood.

After re-training we set each weight to the mean of the component that takes most responsibility for it.

Experiment

References:
SOFT WEIGHT-SHARING FOR NEURAL NETWORK COMPRESSION, Karen Ullrich, 2017, ICLR