NLP预训练模型简介

旺达

一、BERT简单文本分类实例

1、bert详细运行介绍--tensorflow

https://github.com/wangjiwu/BERT-emotion-classification

2、bert pytorch版本介绍

https://github.com/huggingface/transformers

3、transformers详细文档

https://huggingface.co/transformers/

4、实例转换：Converting Tensorflow Checkpoints

export BERT_BASE_DIR=/path/to/bert/uncased_L-12_H-768_A-12

transformers bert \
  $BERT_BASE_DIR/bert_model.ckpt \
  $BERT_BASE_DIR/bert_config.json \
  $BERT_BASE_DIR/pytorch_model.bin

二、预训练模型下载

Bert预训练模型

1、TensorFlow code and pre-trained models for BERT

地址：https://github.com/google-research/bert

2、中文预训练BERT-wwm（Pre-Trained Chinese BERT with Whole Word Masking

地址：https://github.com/ymcui/Chinese-BERT-wwm

Roberta预训练模型

3、RoBERTa中文预训练模型: RoBERTa for Chinese

地址:https://github.com/brightmart/roberta_zh

XLNet Pre-trained model

4、官方版：XLNet: Generalized Autoregressive Pretraining for Language Understanding

https://github.com/zihangdai/xlnet

5、Pre-Trained Chinese XLNet（中文XLNet预训练模型）

https://github.com/ymcui/Chinese-PreTrained-XLNet

albert_zh

6、海量中文预训练ALBERT模型

https://github.com/brightmart/albert_zh