智慧物流>> video-labeling-tool>> 返回
项目作者： CMU-CREATE-Lab

项目描述 ：
A web-based tool for labeling video clips (both front-end and back-end).
高级语言： JavaScript
项目地址: git://github.com/CMU-CREATE-Lab/video-labeling-tool.git
创建时间： 2019-01-17T02:36:32Z
项目社区：https://github.com/CMU-CREATE-Lab/video-labeling-tool
开源协议：BSD 3-Clause "New" or "Revised" License
下载


video-labeling-tool

Demo: http://smoke.createlab.org

A tool for labeling video clips (both front-end and back-end). The back-end depends on an Apache server to provide video links. The back-end is based on flask. A flask tutorial can be found on this blog. If you found this code useful, we would greatly appreciate it if you could cite our technical report below:

Yen-Chia Hsu, Ting-Hao (Kenneth) Huang, Ting-Yao Hu, Paul Dille, Sean Prendi, Ryan Hoffman, Anastasia Tsuhlares, Jessica Pachuta, Randy Sargent, and Illah Nourbakhsh. 2021. Project RISE: Recognizing Industrial Smoke Emissions. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI 2021). https://ojs.aaai.org/index.php/AAAI/article/view/17739

The system defines the final label by aggregating answers from citizens and researchers. At least two volunteers or one researcher will review each video. If the answers from the two volunteers agree, the system marks the video according to the agreement. Otherwise, another volunteer or researcher will review the video, and the result is aggregated based on majority voting.

This tool is tested and worked on:

  • macOS Mojave
    • Chrome 77
    • Safari 12
    • Firefox 68
  • Windows 10
    • Chrome 77
    • Firefox 68
    • Edge 44
  • Android 7, 8, 9, and 10
    • Chrome 77
    • Firefox 68
  • iOS 12 and 13
    • Chrome 77
    • Safari
    • Firefox 18

Table of Content

Install MySQL" class="reference-link">Install MySQL

Install and start mysql database. This assumes that Ubuntu is installed. A tutorial can be found on this blog.

  1. sudo apt-get install mysql-server
  2. sudo apt-get install libmysqlclient-dev

For Mac OS, I recommend installing mysql by using Homebrew.

  1. brew install mysql

After installation, run the security script.

  1. sudo mysql_secure_installation

If error occurs, it is likely that mysql server did not start. Run the followings to start mysql.

  1. # For linux
  2. sudo service mysql start
  4. # For mac
  5. brew services start mysql

Set the user name and password for the application. Replace [DATABASE_USERNAME] and [DATABASE_PASSWORD] with your desired database user name and password respectively.

  1. sudo mysql -u root -p
  2. # Run the followings in the mysql shell
  3. CREATE USER '[DATABASE_USERNAME]'@'localhost' IDENTIFIED BY '[DATABASE_PASSWORD]';
  4. GRANT ALL PRIVILEGES ON *.* TO '[DATABASE_USERNAME]'@'localhost' WITH GRANT OPTION;

Create the database in the mysql shell.

  1. # If on the production server
  2. create database video_labeling_tool_production;
  4. # If on the development server or your local computer
  5. create database video_labeling_tool_development;

If the database exists, drop it and then create it again in the mysql shell.

  1. # For droping database on the production server
  2. drop database video_labeling_tool_production;
  4. # For droping database on the development server or your local computer
  5. drop database video_labeling_tool_development;

Setup back-end" class="reference-link">Setup back-end

Install conda. This assumes that Ubuntu is installed. A detailed documentation is here. First visit here to obtain the downloading path. The following script install conda for all users:

  1. wget https://repo.continuum.io/miniconda/Miniconda3-4.7.12.1-Linux-x86_64.sh
  2. sudo sh Miniconda3-4.7.12.1-Linux-x86_64.sh -b -p /opt/miniconda3
  4. sudo vim /etc/bash.bashrc
  5. # Add the following lines to this file
  6. export PATH="/opt/miniconda3/bin:$PATH"
  7. . /opt/miniconda3/etc/profile.d/conda.sh
  9. source /etc/bash.bashrc

For Mac OS, I recommend installing conda by using Homebrew.

  1. brew cask install miniconda
  2. echo 'export PATH="/usr/local/Caskroom/miniconda/base/bin:$PATH"' >> ~/.bash_profile
  3. echo '. /usr/local/Caskroom/miniconda/base/etc/profile.d/conda.sh' >> ~/.bash_profile
  4. source ~/.bash_profile

Clone this repository and set the permission.

  1. git clone https://github.com/CMU-CREATE-Lab/video-labeling-tool.git
  2. sudo chown -R $USER video-labeling-tool/
  3. sudo addgroup [group_name]
  4. sudo usermod -a -G [group_name] [user_name]
  5. groups [user_name]
  6. sudo chmod -R 775 video-labeling-tool/
  7. sudo chgrp -R [group_name] video-labeling-tool/

Create conda environment and install packages. It is important to install pip first inside the newly created conda environment.

  1. conda create -n video-labeling-tool
  2. conda activate video-labeling-tool
  3. conda install python=3.7
  4. conda install pip
  5. which pip # make sure this is the pip inside the video-labeling-tool environment
  6. sh video-labeling-tool/back-end/install_packages.sh

If the environment already exists and you want to remove it before installing packages, use the following:

  1. conda env remove -n video-labeling-tool

Create a text file with name “google_signin_client_id” in the “back-end/data/“ directory to store the client ID. For detailed documentation about how to obtain the client ID, refer to the Google Sign-In API. In the Google Cloud Console, remember to go to “APIs & Services” -> “Credentials” and add the desired domain names (or IP addresses) to the “Authorized JavaScript origins” in the OAuth client. This makes it possible to call the Google Sign-In API from these desired domains.

  1. sudo vim video-labeling-tool/back-end/data/google_signin_client_id
  2. # Add the following line to this file, obtained from the Google Sign-In API
  3. XXXXXXXX.apps.googleusercontent.com

Create a text file with name “db_url” to store the database url in the “back-end/data/“ directory. For the url format, refer to the flask-sqlalchemy documentation. Replace [DATABASE_USERNAME] and [DATABASE_PASSWORD] with the database user name and password respectively.

  1. sudo vim video-labeling-tool/back-end/data/db_url
  2. # Add the following line to this file (if on the production server)
  3. mysql://[DATABASE_USERNAME]:[DATABASE_PASSWORD]@localhost/video_labeling_tool_production
  5. # Add the following line to this file (if on the development server)
  6. mysql://[DATABASE_USERNAME]:[DATABASE_PASSWORD]@localhost/video_labeling_tool_development

Generate the server private key. This will add a file “private_key” in the “back-end/data/“ directory. The private key is used to sign the JWT (JSON Web Token) issued by the server.

  1. cd video-labeling-tool/back-end/www/
  2. python gen_key.py confirm

Create and upgrade the database by using the migration workfow documented on the flask-migrate website. This blog also provides a tutorial. The script “db.sh” enhances the workflow by adding the FLASK_APP environment.

  1. sh db.sh upgrade

Here are some other migration commands that can be useful. You do not need to run these for normal usage.

  1. # Generate the migration directory
  2. sh db.sh init
  4. # Generate the migration script
  5. sh db.sh migrate "initial migration"
  7. # Downgrade the database to a previous state
  8. sh db.sh downgrade

Add testing videos (optional) or your own videos.

  1. python add_video_set_small.py confirm
  2. python add_video_set_large.py confirm

We use a “add_community_videos.py” script to generate video clips, based on the panorama video on the BreatheCam page. See the docstring in the script for more details. Note that the system uses the following folder structure to store videos:

  1. └── front-end                            # this corresponds to video-labeling-tool/front-end/
  2.     └── videos
  3.         ├── 180                          # the resolution of the video
  4.         │    ├── 2018-05-11              # the video date
  5.         │    │   ├── 0-0                 # the video view ID
  6.         │    │   │   ├── [VID_1].mp4     # video file
  7.         │    │   │   ├── ...             # other video files
  8.         │    │   │   └── ...
  9.         │    │   ├── ...                 # other video view IDs
  10.         │    │   └── ...
  11.         │    ├── ...                     # other video dates
  12.         │    └── ...
  13.         ├── 320                          # another resolution of the video
  14.         │    ├── ...                     # similar structure

Run server in the conda environment for development purpose.

  1. sh development.sh

Prepare gold standards for quality check" class="reference-link">Prepare gold standards for quality check

The system uses gold standards (videos with known labels) to check the quality of each labeled batch. If a user did not label the gold standards correctly, the corresponding batch would be discarded. Initially, there are no gold standards, and the backend will not return videos for labeling. To solve this issue, give yourself the researcher permission by using

  1. python set_client_type.py [user_id] 0

where user_id can be found on the “Account” tab on the top right of the “label.html” page after logging in with Google. The number 0 that follows the user_id is the researcher permission. For more information about the permission, please refer to the client_type variable in the “User” class in the “application.py” file. The system will not run the quality check for users with the researcher permission. In this way, you can start labeling first.

To assign gold standards videos, go to the “gallery.html” page when logging in with the account that has the researcher permission. On the gallery, you will find “P“ and “N“ buttons. Clicking on these buttons shows the positive and negative videos that the researcher labeled. You can now use the dropdown below each video to change the label to Gold Pos (positive gold standards) or Gold Neg (negative gold standards). Once there is a sufficient number of gold standards (more than 4), normal users will be able to label videos. I recommend having at least 100 gold standards to start.

If you found that some videos are not suitable for labeling (e.g., due to incorrect image stitching), you can get the url of the video and use the following command to mark similar ones (with the same date and bounding box) as “bad” videos. This process does not remove videos. Instead it gives all bad videos a label state -2.

  1. python mark_bad_videos.py [video_url]

Dump, import, and backup MySQL database" class="reference-link">Dump, import, and backup MySQL database

This section assumes that you want to dump the production database to a file and import it to the development database. First, SSH to the production server and dump the database to the /tmp/ directory.

  1. ssh [USER_NAME_PRODUCTION]@[SERVER_ADDRESS_PRODUCTION]
  2. sudo mysqldump -u root -p video_labeling_tool_production >/tmp/video_labeling_tool_production.out
  3. exit

SSH to the development server and get the dumped database file from the production server.

  1. ssh [USER_NAME_DEVELOPMENT]@[SERVER_ADDRESS_DEVELOPMENT]
  2. rsync -av [USER_NAME_PRODUCTION]@[SERVER_ADDRESS_PRODUCTION]:/tmp/video_labeling_tool_production.out /tmp/
  4. # For specifying a port number
  5. rsync -av -e "ssh -p [PORT_NUMBER]" [USER_NAME_PRODUCTION]@[SERVER_ADDRESS_PRODUCTION]:/tmp/video_labeling_tool_production.out /tmp/

Import the dumped production database file to the development database.

  1. sudo mysql -u root -p
  2. drop database video_labeling_tool_development;
  3. create database video_labeling_tool_development;
  4. exit
  5. sudo mysql -u root -p video_labeling_tool_development </tmp/video_labeling_tool_production.out

We provide a script to backup the database:

  1. # For the production database
  2. sh video-labeling-tool/back-end/www/backup_db.sh production
  4. # For the development database
  5. sh video-labeling-tool/back-end/www/backup_db.sh development

You can also use crontab to backup the database automatically:

  1. sudo crontab -e
  3. # Add the following line for the production database
  4. 0 0 * * * cd /var/www/smoke-detection/video-labeling-tool/back-end/data/db_backup; sh ../../www/backup_db.sh production
  6. # Add the following line for the development database
  7. 0 0 * * * cd /var/www/smoke-detection/video-labeling-tool/back-end/data/db_backup; sh ../../www/backup_db.sh development

Deploy back-end using uwsgi" class="reference-link">Deploy back-end using uwsgi

Install uwsgi using conda.

  1. conda activate video-labeling-tool
  2. conda install -c conda-forge uwsgi=2.0.18

Run the uwsgi server to check if it works.

  1. sh production.sh
  2. curl localhost:8080
  3. # Should get the "Hello World!" message

The server log is stored in the “back-end/log/uwsgi.log” file. Refer to the “back-end/www/uwsgi.ini” file for details. The documentation is on the uwsgi website. A custom log is stored in the “back-end/log/app.log” file.

  1. # Keep printing the log files when updated
  2. tail -f ../log/uwsgi.log
  3. tail -f ../log/app.log

Create a service on Ubuntu, so that the uwsgi server will start automatically after rebooting the system. Replace [PATH] with the path to the cloned repository. Replace [USERNAME] with your user name on Ubuntu.

  1. sudo vim /etc/systemd/system/video-labeling-tool.service
  2. # Add the following line to this file
  3. [Unit]
  4. Description=uWSGI instance to serve video-labeling-tool
  5. After=network.target
  7. [Service]
  8. User=[USERNAME]
  9. Group=www-data
  10. WorkingDirectory=/[PATH]/video-labeling-tool/back-end/www
  11. Environment="PATH=/home/[USERNAME]/.conda/envs/video-labeling-tool/bin"
  12. ExecStart=/home/[USERNAME]/.conda/envs/video-labeling-tool/bin/uwsgi --ini uwsgi.ini
  14. [Install]
  15. WantedBy=multi-user.target

Register the uwsgi server as a service on Ubuntu.

  1. sudo systemctl enable video-labeling-tool
  2. sudo systemctl start video-labeling-tool
  4. # Check the status of the service
  5. sudo systemctl status video-labeling-tool
  7. # Restart the service
  8. sudo systemctl restart video-labeling-tool
  10. # Stop and disable the service
  11. sudo systemctl stop video-labeling-tool
  12. sudo systemctl disable video-labeling-tool

Check if the service work.

  1. curl localhost:8080
  2. # Should get the "Hello World!" message

We use a “reload” file under “back-end/www” to gracefully restart the uwsgi server, as indicated in the “uwsgi.ini” file in “touch-reload”. Run the following when there are code changes on the backend and we want to restart the server.

  1. cd back-end/www
  2. touch reload

Connect uwsgi to apache" class="reference-link">Connect uwsgi to apache

Obtain domains from providers such as Google Domains or Namecheap for both the back-end and the front-end. Point these domain names to the domain of the Ubuntu machine. Then install apache2 and enable mods.

  1. sudo apt-get install apache2
  2. sudo apt-get install apache2-dev
  4. sudo a2enmod headers
  5. sudo a2enmod rewrite
  6. sudo a2enmod ssl
  7. sudo a2enmod proxy
  8. sudo a2enmod proxy_http
  9. sudo a2enmod proxy_balancer
  10. sudo a2enmod lbmethod_byrequests

Create an apache virtual host as a reverse proxy for the uwsgi server. Replace [BACK_END_DOMAIN] and [FRONT_END_DOMAIN] with your domain name for the back-end and the front-end respectively.

  1. sudo vim /etc/apache2/sites-available/[BACK_END_DOMAIN].conf
  2. # Add the following lines to this file
  3. <VirtualHost *:80>
  4.   ServerName [BACK_END_DOMAIN]
  5.   Header always set Access-Control-Allow-Origin "http://[FRONT_END_DOMAIN]"
  6.   Header set Access-Control-Allow-Headers "Content-Type"
  7.   Header set Cache-Control "max-age=5, public, must-revalidate"
  8.   ProxyPreserveHost On
  9.   ProxyRequests Off
  10.   ProxyVia Off
  11.   ProxyPass / http://127.0.0.1:8080/
  12.   ProxyPassReverse / http://127.0.0.1:8080/
  13.   ErrorLog ${APACHE_LOG_DIR}/[BACK_END_DOMAIN].error.log
  14.   CustomLog ${APACHE_LOG_DIR}/[BACK_END_DOMAIN].access.log combined
  15. </VirtualHost>

Create a symlink of the virtual host and restart apache.

  1. cd /etc/apache2/sites-enabled/
  2. sudo ln -s ../sites-available/[BACK_END_DOMAIN].conf
  3. sudo systemctl restart apache2

Setup front-end on apache" class="reference-link">Setup front-end on apache

Create an apache virtual host. Replace [FRONT_END_DOMAIN] with your domain name for the front-end. Replace [PATH] with the path to the cloned repository.

  1. sudo vim /etc/apache2/sites-available/[FRONT_END_DOMAIN].conf
  2. # Add the following lines to this file
  3. <VirtualHost *:80>
  4.   ServerName [FRONT_END_DOMAIN]
  5.   DocumentRoot /[PATH]/video-labeling-tool/front-end
  6.   Header always set Access-Control-Allow-Origin "*"
  7.   Header set Cache-Control "max-age=5, public, must-revalidate"
  8.   <Directory "/[PATH]/video-labeling-tool/front-end">
  9.     Options FollowSymLinks
  10.     AllowOverride None
  11.     Require all granted
  12.   </Directory>
  13.   ErrorLog ${APACHE_LOG_DIR}/[FRONT_END_DOMAIN].error.log
  14.   CustomLog ${APACHE_LOG_DIR}/[FRONT_END_DOMAIN].access.log combined
  15. </VirtualHost>

Use the following if you only want to access the server from an IP address with a port (e.g., http://192.168.1.72:8080). Remember to tell the apache server to listen to the port number.

  1. sudo vim /etc/apache2/sites-available/video-labeling-tool-front-end.conf
  2. # Add the following lines to this file
  3. <VirtualHost *:8080>
  4.   ServerAdmin webmaster@localhost
  5.   DocumentRoot /[PATH]/video-labeling-tool/front-end
  6.   ErrorLog ${APACHE_LOG_DIR}/video-labeling-tool-front-end.error.log
  7.   CustomLog ${APACHE_LOG_DIR}/video-labeling-tool-front-end.access.log combined
  8. </VirtualHost>
  10. sudo vim /etc/apache2/ports.conf
  11. # Add the following lines to this file
  12. Listen 8080

Create a symlink of the virtual host and restart apache.

  1. cd /etc/apache2/sites-enabled/
  2. sudo ln -s ../sites-available/[FRONT_END_DOMAIN].conf
  3. sudo systemctl restart apache2

Setup https (instead of using http)" class="reference-link">Setup https (instead of using http)

Go to https://certbot.eff.org/ and follow the instructions to install Certbot on the Ubuntu server. Then run the following to enable Apache2 mods.

  1. sudo a2enmod headers
  2. sudo a2enmod rewrite
  3. sudo a2enmod ssl

Give permissions so that the Certbot and apache can modify the website. This assumes that the cloned repository is placed under the /var/www/ directory. Replace [CLONED_REPOSITORY] with your directory name, such as video-labeling-tool.

  1. cd /var/www/
  2. sudo mkdir html # only run this if the html directory did not exist
  3. sudo chmod 775 html
  4. sudo chmod 775 [CLONED_REPOSITORY]
  5. sudo chgrp -R www-data html
  6. sudo chgrp -R www-data [CLONED_REPOSITORY]

If other users need to modify this repository, add them to the www-data group.

  1. sudo usermod -a -G www-data [user_name]
  2. groups [user_name]

Run the Certbot.

  1. sudo certbot --apache certonly

Copy the directories that point to the SSL certificate and the SSL certificate key in the terminal provided by the certbot. For example:

  1. /etc/letsencrypt/live/[...]/fullchain.pem
  2. /etc/letsencrypt/live/[...]/privkey.pem

Edit apache configuration file for the back-end. Note the “https” before the FRONT_END_DOMAIN, not http.

  1. sudo vim /etc/apache2/sites-available/[BACK_END_DOMAIN].conf
  2. # Add the following lines to this file
  3. <VirtualHost *:443>
  4.   ServerName [BACK_END_DOMAIN]
  5.   # Enable https ssl support
  6.   SSLEngine On
  7.   # The following line enables cors
  8.   Header always set Access-Control-Allow-Origin "https://[FRONT_END_DOMAIN]"
  9.   Header set Access-Control-Allow-Headers "Content-Type"
  10.   # The following line forces the browser to break the cache
  11.   Header set Cache-Control "max-age=5, public, must-revalidate"
  12.   # Reverse proxy to the uwsgi server
  13.   ProxyPreserveHost On
  14.   ProxyRequests Off
  15.   ProxyVia Off
  16.   ProxyPass / http://127.0.0.1:8080/
  17.   ProxyPassReverse / http://127.0.0.1:8080/
  18.   # APACHE_LOG_DIR is /var/log/apache2/
  19.   ErrorLog ${APACHE_LOG_DIR}/[BACK_END_DOMAIN].error.log
  20.   CustomLog ${APACHE_LOG_DIR}/[BACK_END_DOMAIN].access.log combined
  21.   # Add ssl
  22.   SSLCertificateFile /etc/letsencrypt/live/[...]/fullchain.pem
  23.   SSLCertificateKeyFile /etc/letsencrypt/live/[...]/privkey.pem
  24.   Include /etc/letsencrypt/options-ssl-apache.conf
  25. </VirtualHost>
  27. <VirtualHost *:80>
  28.   ServerName [BACK_END_DOMAIN]
  29.   # Enable the url rewriting
  30.   RewriteEngine on
  31.   # Redirect http to https
  32.   RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent] 
  33. </VirtualHost>

Edit apache configuration file for the front-end.

  1. sudo vim /etc/apache2/sites-available/[FRONT_END_DOMAIN].conf
  2. # Add the following lines to this file
  3. <VirtualHost *:443>
  4.   ServerName [FRONT_END_DOMAIN]
  5.   DocumentRoot /[PATH]/video-labeling-tool/front-end
  6.   # Enable https ssl support
  7.   SSLEngine On
  8.   # The following line enables cors
  9.   Header always set Access-Control-Allow-Origin "*"
  10.   # The following line forces the browser to break the cache
  11.   Header set Cache-Control "max-age=5, public, must-revalidate"
  12.   <Directory "/[PATH]/video-labeling-tool/front-end">
  13.     Options FollowSymLinks
  14.     AllowOverride None
  15.     Require all granted
  16.   </Directory>
  17.   # APACHE_LOG_DIR is /var/log/apache2/
  18.   ErrorLog ${APACHE_LOG_DIR}/[FRONT_END_DOMAIN].error.log
  19.   CustomLog ${APACHE_LOG_DIR}/[FRONT_END_DOMAIN].access.log combined
  20.   # Add ssl
  21.   SSLCertificateFile /etc/letsencrypt/live/[...]/fullchain.pem
  22.   SSLCertificateKeyFile /etc/letsencrypt/live/[...]/privkey.pem
  23.   Include /etc/letsencrypt/options-ssl-apache.conf
  24. </VirtualHost>
  26. <VirtualHost *:80>
  27.   ServerName [FRONT_END_DOMAIN]
  28.   # Enable the url rewriting
  29.   RewriteEngine on
  30.   # Redirect http to https
  31.   RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent] 
  32. </VirtualHost>

Restart apache server.

  1. sudo /etc/init.d/apache2 restart

Set a cron job to renew the SSL certificate automatically.

  1. sudo bash
  2. crontab -e

Add the following to the crontab.

  1. # Renew our SSL certificate
  2. 0 0 1 * * /opt/certbot-auto renew --no-self-upgrade >>/var/log/certbot.log

Then type “exit” in the terminal to exit the bash mode. Also remember to go to the Google API console and add https domains to the authorized JavaScript origins for the OAuth client (the Google Login API). All http urls in the front-end code (e.g., API urls, video urls) also need to be replaced with the https version.

API calls" class="reference-link">API calls

The following code examples assusme that the root url is http://localhost:5000.

Log in to the system

The server will return a user token in the form of JWT (JSON Web Token). There are four different client types, as documented in the User class in this file.

  • Path:
    • /api/v1/login
  • Available methods:
    • POST
  • Required fields (either google_id_token or client_id):
    • “google_id_token”: from Google Identity Service
    • “client_id”: from Google Analytics id or randomly generated uuid
  • Returned fields:
    • “user_token”: user token for the front-end client
    • “user_token_for_other_app”: user token for other applications
      ```JavaScript
      // jQuery examples
      $.ajax({
      url: “http://localhost:5000/api/v1/login“,
      type: “POST”,
      data: JSON.stringify({google_id_token: “the_returned_google_id_token”}),
      contentType: “application/json”,
      dataType: “json”,
      success: function (data) {console.log(data)},
      error: function (xhr) {console.error(xhr)}
      });

$.ajax({
url: “http://localhost:5000/api/v1/login“,
type: “POST”,
data: JSON.stringify({client_id: “uuid_for_testing”}),
contentType: “application/json”,
dataType: “json”,
success: function (data) {console.log(data)},
error: function (xhr) {console.error(xhr)}
});

  1. ### Get a batch of videos
  2. If the client type is not researcher, gold standards (with known labels) will be randomly placed to evaluate the label quality. For researchers, there will be no gold standards. Combine url_root and url_part in the returned data to get the full video URL.
  3. - Path:
  4.   - **/api/v1/get_batch**
  5. - Available methods:
  6.   - POST
  7. - Required fields:
  8.   - "user_token": from /api/v1/login
  9. - Returned fields:
  10.   - "data": video metadata
  11.   - "video_token": video token for verification when sending the labels back to the server
  12. ```JavaScript
  13. // jQuery examples
  14. $.ajax({
  15.   url: "http://localhost:5000/api/v1/get_batch",
  16.   type: "POST",
  17.   data: JSON.stringify({user_token: "your_user_token"}),
  18.   contentType: "application/json",
  19.   dataType: "json",
  20.   success: function (data) {console.log(data)},
  21.   error: function (xhr) {console.error(xhr)}
  22. });

Send a batch of video labels

The video token is for checking if the server issued the video batch. The label states determined by regular users and researchers are stored in two separate columns (label_state and label_state_admin) in the Video table in the database.

  • Path:
    • /api/v1/send_batch
  • Available methods:
    • POST
  • Required fields:
    • “data”: a list of dictionaries with video_id (returned by the /v1/get_batch) and label (0 means no, 1 means yes)
    • “user_token”: from /api/v1/login
    • “video_token”: from /api/v1/get_batch
  • Returned fields:
    • “data”: scores for the current user (null for no changes) and the labeled batch (0 for poor labeling quality)
      1. // jQuery examples
      2. $.ajax({
      3. url: "http://localhost:5000/api/v1/send_batch",
      4. type: "POST",
      5. data: JSON.stringify({"video_token":"your_video_token","user_token":"your_user_token","data":[{"video_id":1,"label":0},{"video_id":2,"label":1},{"video_id":3,"label":1},{"video_id":4,"label":0},{"video_id":5,"label":0},{"video_id":6,"label":0},{"video_id":16151,"label":0},{"video_id":7,"label":1},{"video_id":8,"label":0},{"video_id":9,"label":0},{"video_id":10,"label":0},{"video_id":11,"label":0},{"video_id":12,"label":0},{"video_id":13,"label":1},{"video_id":14,"label":1},{"video_id":15,"label":0}]}),
      6. contentType: "application/json",
      7. dataType: "json",
      8. success: function (data) {console.log(data)},
      9. error: function (xhr) {console.error(xhr)}
      10. });

      Set the states of video labels

      This call is only available for researchers (client type 0) with valid user tokens. Any previously determined label state will be overwritten.
  • Path:
    • /api/v1/set_label_state
  • Available methods:
    • POST
  • Required fields:
    • “data”: a list of json with video_id (returned by the /v1/get_batch) and label state (documented in the label_state_machine function in this file)
    • “user_token”: from /api/v1/login
  • No returned fields
    1. // jQuery examples
    2. $.ajax({
    3. url: "http://localhost:5000/api/v1/set_label_state",
    4. type: "POST",
    5. data: JSON.stringify({"data":[{"video_id":1,"label":-2}],"user_token":"your_user_token"}),
    6. contentType: "application/json",
    7. dataType: "json",
    8. success: function (data) {console.log(data)},
    9. error: function (xhr) {console.error(xhr)}
    10. });

    Get videos with fully or partially labeled positive or negative labels (for all users)

    These calls are available for all users. When querying positive labels, you can pass in user id. If a user token is provided and the client type is expert or researcher, the returned data will contain more information. You can also get videos that have partial labels (verified by only one user or by two users with disagreement).
  • Paths:
    • /api/v1/get_pos_labels
    • /api/v1/get_neg_labels
    • /api/v1/get_maybe_pos_labels
    • /api/v1/get_maybe_neg_labels
    • /api/v1/get_discorded_labels
  • Available methods:
    • GET, POST
  • Optional fields:
    • “user_id”: obtained by decoding the user_token JWT
    • “page_number”: default to 1
    • “page_size”: default to 16, maximum 1000
    • “user_token”: from /api/v1/login
  • Returned fields:
    • “data”: a list of video metadata
    • “total”: the total number of queried videos, can be larger than the page size
      1. // jQuery examples
      2. $.ajax({
      3. url: "http://localhost:5000/api/v1/get_pos_labels",
      4. type: "POST",
      5. data: "user_token=your_user_token&pageSize=16&pageNumber=1",
      6. contentType: "application/x-www-form-urlencoded; charset=UTF-8",
      7. dataType: "json",
      8. success: function (data) {console.log(data)},
      9. error: function (xhr) {console.error(xhr)}
      10. });
      1. # curl example
      2. curl http://localhost:5000/api/v1/get_pos_labels
      3. curl http://localhost:5000/api/v1/get_pos_labels?user_id=43
      4. curl http://localhost:5000/api/v1/get_neg_labels
      5. curl http://localhost:5000/api/v1/get_maybe_pos_labels
      6. curl http://localhost:5000/api/v1/get_discorded_labels

      Get videos with other types of labels (for only expert and researcher type users)

      These calls are only available for researchers or experts (client type 0 or 1) with valid user tokens. You can get videos that are marked as gold standards or labeled by researchers/citizens. For researchers or experts, the gallery page will be in the dashboard mode, where you can download the user token.
  • Paths:
    • /api/v1/get_pos_gold_labels
    • /api/v1/get_neg_gold_labels
    • /api/v1/get_pos_labels_by_researcher
    • /api/v1/get_neg_labels_by_researcher
    • /api/v1/get_pos_labels_by_citizen
    • /api/v1/get_neg_labels_by_citizen
    • /api/v1/get_bad_labels
  • Available methods:
    • POST
  • Required fields:
    • “user_token”: from /api/v1/login or the gallery page
  • Optional fields:
    • “page_number”: default to 1
    • “page_size”: default to 16, maximum 1000
  • Returned fields:
    • “data”: a list of video metadata
    • “total”: the total number of queried videos, can be larger than the page size
      1. // jQuery examples
      2. $.ajax({
      3. url: "http://localhost:5000/api/v1/get_pos_gold_labels",
      4. type: "POST",
      5. data: "user_token=your_user_token&pageSize=16&pageNumber=1",
      6. contentType: "application/x-www-form-urlencoded; charset=UTF-8",
      7. dataType: "json",
      8. success: function (data) {console.log(data)},
      9. error: function (xhr) {console.error(xhr)}
      10. });
      1. # curl example
      2. curl -d 'user_token=your_user_token' -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' -X POST http://localhost:5000/api/v1/get_pos_gold_labels

      Get the entire video dataset with labels

      This call is only available for researchers or experts (client type 0 or 1) with valid user tokens. Notice that this call is not paginated and will take a long time to complete.
  • Paths:
    • /api/v1/get_all_labels
  • Available methods:
    • POST
  • Required fields:
    • “user_token”: from /api/v1/login or the gallery page
  • Returned fields:
    • “data”: a list of video metadata
      1. // jQuery examples
      2. $.ajax({
      3. url: "http://localhost:5000/api/v1/get_all_labels",
      4. type: "POST",
      5. data: "user_token=your_user_token",
      6. contentType: "application/x-www-form-urlencoded; charset=UTF-8",
      7. dataType: "json",
      8. success: function (data) {console.log(data)},
      9. error: function (xhr) {console.error(xhr)}
      10. });
      1. # curl example
      2. curl -d 'user_token=your_user_token' -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' -X POST http://localhost:5000/api/v1/get_all_labels

      Get the statistics of labels

      Get the number of all videos, the number of fully labeled videos (confirmed by multiple users), and the number of partially labeled videos. The statistics exclude the videos that were marked as “bad” data and gold standards.
  • Paths:
    • /api/v1/get_label_statistics
  • Available methods:
    • GET
  • Returned fields:
    • “num_all_videos”: number of all videos (excluding bad data and gold standards)
    • “num_fully_labeled”: number of fully labeled videos (excluding bad data and gold standards)
    • “num_partially_labeled”: number of partially labeled videos (excluding bad data and gold standards)
      1. // jQuery examples
      2. $.getJSON("http://localhost:5000/api/v1/get_label_statistics", function (data) {
      3. console.log(data);
      4. });
      1. # curl example
      2. curl http://localhost:5000/api/v1/get_label_statistics

      Add a record when a user takes or passes the tutorial

      When a user takes or passes the tutorial of smoke labeling, you can send a post request via this API call to add a record in the database. This call returns HTTP status 204 when succeed.
  • Paths:
    • /api/v1/add_tutorial_record
  • Available methods:
    • POST
  • Required fields:
    • “user_token”: from /api/v1/login or the gallery page
    • “action_type”: an integer ranging from 0 to 4
      • 0: took the tutorial
      • 1: did not pass the last batch in the tutorial
      • 2: passed the last batch (16 videos) during the third try with hints
      • 3: passed the last batch during the second try after showing the answers
      • 4: passed the last batch (16 videos) in the tutorial during the first try
    • “query_type”: an integer ranging from 0 to 2
      • 0: entered the tutorial page (can come from multiple sources or different button clicks)
      • 1: clicked the tutorial button on the webpage (not the prompt dialog)
      • 2: clicked the tutorial button in the prompt dialog (not the webpage)
        1. // jQuery examples
        2. $.ajax({
        3. url: "http://localhost:5000/api/v1/add_tutorial_record",
        4. type: "POST",
        5. data: JSON.stringify({"user_token":"your_user_token","action_type":1,"query_type":2}),
        6. contentType: "application/json",
        7. dataType: "json",
        8. success: function (data) {console.log(data)},
        9. error: function (xhr) {console.error(xhr)}
        10. });