#lilia的挣扎之旅#entity embedding for category variables

neural network embedding的作用[1]：

Finding nearest neighbors in the embedding space. These can be used to make recommendations based on user interests or cluster categories.

As input to a machine learning model for a supervised task.

For visualization of concepts and relations between categories.

翻译过来就是：

在embedding space中发现模式，比如基于embedding做聚类实现推荐功能
作为监督学习任务的输入
可视化类别间的概念和相关性

neural network embedding 的优势

neural network embedding解决了one-hot embedding存在的两个不足，假设我们想要得到3w本书的embedding以向量代表书的一些信息：

随着类别的增多，one-hot embedding的维度增加，比如有3w本书，就要设置3w维的向量
one-hot embedding之间，不同category的相似度都为0，这很尴尬，比如同一个作者写的两本书之间的相似度是0，大量信息都被丢失了。

基于这两个问题，我们可以得出一个结论：对于category variables, 理想的embdedding至少需要满足两个条件：

表示的向量维度要远小于类别的数量，比如3w本书，那么表示他们的向量就应该远小于3w
相似的类别间应该可以通过embedding进行相似度的计算
参考资料[1]中，写的相当好。

the ideal solution for representing categorical variables would require fewer numbers than the number of unique categories and would place similar categories closer to one another.

Also like word vectors, entity embeddings can be expected to learn the intrinsic properties of the categories and group similar categories together.

category embedding的训练

可以通过一个有监督的神经网络模型去训练category embedding，过程类似于词向量的学习。
假设我们的任务是看每本书是否是列夫托尔斯泰的书，得出的使分类任务loss最小的weights就可以作为每本书的信息，这些信息中包含了一本书是否是列夫托尔斯泰的书相关的信息。以这个embedding为基础，列夫托尔斯泰的书之间embedding的相似度会很高。（我的问题：那词向量训练出来后，可以表示什么信息呢？具体一点）

结果的可视化

TSNE

(TSNE is a manifold learning technique which means that it tries to map high-dimensional data to a lower-dimensional manifold, creating an embedding that attempts to maintain local structure within the data. It’s almost exclusively used for visualization because the output is stochastic and it does not support transforming new data. An up and coming alternative is Uniform Manifold Approximation and Projection, UMAP, which is much faster and does support transform new data into the embedding space).

交互式的可视化

The problem with static graphs is that we can’t really explore the data and investigate groupings or relationships between variables. To solve this problem, TensorFlow developed projector, an online application that lets us visualize and interact with embeddings. I’ll release an article on how to use this tool shortly, but for now, here’s the results:

参考资料：

[1] Neural Network Embeddings Explained —— https://towardsdatascience.com
[2] github项目地址