Implementation of pre-training model loading architecture of bert and its variants with tensorflow2
This is based on the Transformer architecture implemented by tf2.keras, which can quickly load the pre-trained bert model for downstream finetune training. So welcome to star and I will continue to update in the future.
Temporary support：
pip install git+https://github.com/xuyingjie521/xbert.gitFeatures that have been implemented so far：
- Load pre-training weights of bert/roberta for finetune.
- Support tf2.keras.
- Google original bert: https://github.com/google-research/bert
- Harbin Institute of Technology version roberta: https://github.com/ymcui/Chinese-BERT-wwm
- Brightmart version of roberta: https://github.com/brightmart/roberta_zh