a. Big data sets with high cardinality can be tackled well with linear online models. Tools like vowpal wabiit. FTRL, libfm, libffm, liblinear are good tools
b. Consider sparse matrices in python (things like csr matrices)
c. Consider ensembling (like combining) models trained on smaller parts of the data
