NLP预训练模型简介
旺达
一、BERT简单文本分类实例
1、bert详细运行介绍--tensorflow
https://github.com/wangjiwu/BERT-emotion-classification
2、bert pytorch版本介绍
https://github.com/huggingface/transformers
3、transformers详细文档
https://huggingface.co/transformers/
4、实例转换：Converting Tensorflow Checkpoints
export BERT_BASE_DIR=/path/to/bert/uncased_L-12_H-768_A-12
transformers bert \
$BERT_BASE_DIR/bert_model.ckpt \
$BERT_BASE_DIR/bert_config.json \
$BERT_BASE_DIR/pytorch_model.bin
二、预训练模型下载
Bert预训练模型
1、TensorFlow code and pre-trained models for BERT
地址：https://github.com/google-research/bert
2、中文预训练BERT-wwm（Pre-Trained Chinese BERT with Whole Word Masking
地址：https://github.com/ymcui/Chinese-BERT-wwm
Roberta预训练模型
3、RoBERTa中文预训练模型: RoBERTa for Chinese
地址:https://github.com/brightmart/roberta_zh
XLNet Pre-trained model
4、官方版：XLNet: Generalized Autoregressive Pretraining for Language Understanding
https://github.com/zihangdai/xlnet
5、Pre-Trained Chinese XLNet（中文XLNet预训练模型）
https://github.com/ymcui/Chinese-PreTrained-XLNet
albert_zh
