项目作者: zxlzr

项目描述 :
An Open-Source Package for Universal Extraction (UE)
高级语言: Python
项目地址: git://github.com/zxlzr/OpenUE.git
创建时间: 2020-03-13T10:48:25Z
项目社区:https://github.com/zxlzr/OpenUE

开源协议:MIT License

下载


中文说明 | English




OpenUE is an open-source toolkit that provides a off-the-shelf framework to implement lots of NLP extraction tasks.












OpenUE allows users ranging from beginner python coders to experienced machine learning engineers to leverage
lots of NLP extraction tasks in one easy-to-use python package.

Key Features

  • Full Guides and API Documentation

  • Unified API for NLP Tasks with SOTA Pretrained Models (Adaptable with BERT, XLNet, etc.)

    • Entity and Realation Extraction
    • Intent and Slot Filling
    • Opinion and Apspect Extraction
    • More in development
  • Training and Inference Interface
  • Rapid NLP Model Deployment
  • Dockerizing OpenUE with GPUs
    • Easily build and run OpenUE containers leveraging NVIDIA GPUs with Docker

      Environment

  • python3.6
  • requirements.txt

Quick Start

Requirements and Installation

Anaconda Environment
  1. conda create -n openue python=3.6
  2. conda activate openue
  3. conda install --file requirements.txt

Examples and General Use

Once you have installed OpenUE, here are a few examples of what you can run with OpenUE modules:

Entity and Relation Extraction Example
  1. Data Preprocessing. Put the pretrined language model (e.g., BERT) in the pretrained_model folder and put all raw data (run script download_ske.sh in the benchmark folder) in the raw_data folder, run
    1. sh download_ske_dataset.sh
    2. sh download_pretrain_cn_bert.sh
    3. sh preprocess.sh ske
  2. Train Sequence Labeling & Classification Model. Set all parameters in the file config.py and run

    1. sh train_seq.sh ske
    2. sh train_class.sh ske

    You can download the checkpoint here, extract and put them in the output folder.

  3. Test & Evaluation. Run

  1. python predict.sh ske
  1. Export & Serving. Run
    1. sh export_seq.sh ske
    2. sh serving_cls.sh ske
    3. sh serving.sh
  2. Interactive Prediction. Run
    1. python predict_online.py
  3. Demo.Run
    1. python app.py ske

    Tools

  1. >>> import openuee
  2. >>> model = openue.get_model('ske_bert_entity_relation')
  3. >>> res = model.infer('《上海滩》是刘德华的音乐作品,黄沾作曲,收录在《【歌单】酷我热门单曲合辑》专辑中')
  4. >>> print(res)
  5. "spo_list": [{"object_type": "人物", "predicate": "作曲", "object": "黄沾", "subject_type": "歌曲", "subject": "上海滩"}, {"object_type": "音乐专辑", "predicate": "所属专辑", "object": "【歌单】酷我热门单曲合辑", "subject_type": "歌曲", "subject": "上海滩"}, {"object_type": "人物", "predicate": "歌手", "object": "刘德华", "subject_type": "歌曲", "subject": "上海滩"}]

Note that it may take a few minutes to download checkpoint and data for the first time. Then use infer to do sentence-level entity and relation extraction

How to Cite

If you use or extend our work, please cite the following paper:

  1. @inproceedings{zhang-2020-opennue,
  2. title = "{O}pe{UE}: An Open Toolkit for Universal Extraction in Text",
  3. author = "Ningyu Zhang, Shumin Deng, Huajun Chen",
  4. year = "2020",
  5. }