An Open-Source Package for Universal Extraction (UE)
OpenUE is an open-source toolkit that provides a off-the-shelf framework to implement lots of NLP extraction tasks.
OpenUE allows users ranging from beginner python coders to experienced machine learning engineers to leverage
lots of NLP extraction tasks in one easy-to-use python package.
Key Features
Unified API for NLP Tasks with SOTA Pretrained Models (Adaptable with BERT, XLNet, etc.)
conda create -n openue python=3.6
conda activate openue
conda install --file requirements.txt
Once you have installed OpenUE, here are a few examples of what you can run with OpenUE modules:
sh download_ske_dataset.sh
sh download_pretrain_cn_bert.sh
sh preprocess.sh ske
Train Sequence Labeling & Classification Model. Set all parameters in the file config.py and run
sh train_seq.sh ske
sh train_class.sh ske
You can download the checkpoint here, extract and put them in the output folder.
Test & Evaluation. Run
python predict.sh ske
sh export_seq.sh ske
sh serving_cls.sh ske
sh serving.sh
python predict_online.py
python app.py ske
>>> import openuee
>>> model = openue.get_model('ske_bert_entity_relation')
>>> res = model.infer('《上海滩》是刘德华的音乐作品,黄沾作曲,收录在《【歌单】酷我热门单曲合辑》专辑中')
>>> print(res)
"spo_list": [{"object_type": "人物", "predicate": "作曲", "object": "黄沾", "subject_type": "歌曲", "subject": "上海滩"}, {"object_type": "音乐专辑", "predicate": "所属专辑", "object": "【歌单】酷我热门单曲合辑", "subject_type": "歌曲", "subject": "上海滩"}, {"object_type": "人物", "predicate": "歌手", "object": "刘德华", "subject_type": "歌曲", "subject": "上海滩"}]
Note that it may take a few minutes to download checkpoint and data for the first time. Then use infer
to do sentence-level entity and relation extraction
If you use or extend our work, please cite the following paper:
@inproceedings{zhang-2020-opennue,
title = "{O}pe{UE}: An Open Toolkit for Universal Extraction in Text",
author = "Ningyu Zhang, Shumin Deng, Huajun Chen",
year = "2020",
}