【Sentence-bert】

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

论文: https://arxiv.org/abs/1908.10084

代码: https://github.com/UKPLab/sentence-transformers

【Bert-flow】

On the Sentence Embeddings from Pre-trained Language Models

论文: https://arxiv.org/abs/2011.05864

代码: https://github.com/bohanli/BERT-flow

动机: 论文作者认为BERT生成的句向量具有的语义相似度信息无法通过简单的余弦相似度体现出来，论文通过一个基于流(flow)的生成模型训练一个标准的高斯分布，对 BERT句向量空间中的点进行一一映射，再在标准高斯空间中进行相似度计算

【Bert-whitening】

Whitening Sentence Representations for Better Semantics and Faster Retrieval

论文: https://arxiv.org/abs/2103.15316

代码: https://github.com/bojone/BERT-whitening

作者认为Bert-flow模型的关键不在于flow模型，基于Bert模型输出的[CLS]向量在文本语义计算任务中无法取得好的效果的原因可能是句向量所属的坐标系并非标准正交基的推论，通过一个简单的线性变换实现了和Bert-flow相近的效果。

参考:

细说Bert-whitening的原理

https://zhuanlan.zhihu.com/p/380874824

你可能不需要BERT-flow：一个线性变换媲美BERT-flow

https://kexue.fm/archives/8069

【ConSERT】

ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer

论文: https://arxiv.org/pdf/2105.11741.pdf

代码: https://github.com/yym6472/ConSERT

参考:

论文阅读“ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer”

https://www.jianshu.com/p/de79f4e166da

【SimCSE】

SimCSE: Simple Contrastive Learning of Sentence Embeddings

论文: https://arxiv.org/pdf/2104.08821.pdf

代码: https://github.com/princeton-nlp/SimCSE

参考:

【论文复现】SimCSE对比学习: 文本增广是什么牛马，我只需要简单Dropout两下

https://blog.csdn.net/weixin_45839693/article/details/116302914

【Trans-Encoder】

TRANS-ENCODER: UNSUPERVISED SENTENCE-PAIR MODELLING THROUGH SELF- AND MUTUAL-DISTILLATIONS

论文://arxiv.org/pdf/2109.13059v3.pdf

代码：https://github.com/amzn/trans-encoder

参考：

文本匹配——【ICLR 2022】Trans-Encoder

https://cloud.tencent.com/developer/article/2041362

TRANS-ENCODER：自蒸馏和互蒸馏的无监督句对模型

https://zhuanlan.zhihu.com/p/437729611

【Poly-encoders】

Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring

论文: https://arxiv.org/pdf/1905.01969.pdf

参考:

论文笔记 | Poly-encoders

https://zhuanlan.zhihu.com/p/380867813

【PairSupCon】

Pairwise Supervised Contrastive Learning of Sentence Representations

论文: https://arxiv.org/abs/2109.05424

代码: https://github.com/amazon-research/sentence-representations/PairSupCon

参考: