项目作者: mkbeh

项目描述 :
Bitshares arbitrage-bot
高级语言: Python
项目地址: git://github.com/mkbeh/rin-bitshares-arbitry-bot.git
创建时间: 2018-12-02T12:21:37Z
项目社区:https://github.com/mkbeh/rin-bitshares-arbitry-bot

开源协议:MIT License

下载


Asynchronous Bitshares arbitrage bot

An arbitration bot only works inside bitshares.
The bot builds up the volume of four assets:
BTS, BRIDGE.BTC, CNY, USD. Market pairs are taken
from third party API using the bitshares node with ES plugin. Then market pairs are added to the
chains along which the bot works. The description
of the arbitration algorithm is in the module
src.algorithms.arbitryalgorithm.


Donate

Donate me if you like it :)

  1. Bitshares account -> mkbehforever007
  2. bitcoin -> bc1qqkr72aemz59aawxf74gytrwuw4m9mj20t7e7df
  3. ethereum -> 0xB3e5b643cFB9e2565a3456eC7c7A73491A32e31F

Current issue

The problem is the incorrect calculation of
the final volumes for placing orders. Due to a
small error in the calculations, it is not
possible to place orders. I tried various
solutions to this problem, but have not yet
come to success.

How bot works

scheme

Getting started

The following actions were performed on ubuntu 18.10

Installing Bitshares node and wallet

vi /etc/apt/sources.list

  1. # Add repo.
  2. deb http://security.ubuntu.com/ubuntu xenial-security main

sudo apt-get update

sudo apt-get install autoconf cmake make automake libtool git libboost-all-dev libssl-dev g++ libcurl4-openssl-dev libgconf-2-4 libcurl3

wget https://github.com/bitshares/bitshares-core/releases/download/2.0.190219/BitShares-Core-2.0.190219-Linux-cli-tools.tar.gz

tar -xzvf BitShares-Core-2.0.190219-Linux-cli-tools.tar.gz

rm BitShares-Core-2.0.190219-Linux-cli-tools.tar.gz

Configuring node

sudo apt-get install supervisor

sudo vi /etc/supervisor/conf.d/bts_node.conf

  1. [program:bts_node]
  2. command=/home/<user>/programs/witness_node/witness_node --rpc-endpoint="127.0.0.1:8091"
  3. directory=/home/<user>/programs/witness_node/
  4. stdout_logfile=/var/log/supervisor/bts_node_out.log
  5. stderr_logfile=/var/log/supervisor/bts_node_err.log
  6. autostart=true
  7. autorestart=true
  8. numprocs=1
  9. user=<user>

When the node is synchronized - go to the next step.

Configuring wallet

  1. cd programs/cli_wallet/
  2. ./cli_wallet --server-rpc-endpoint=ws://127.0.0.1:8091 -r 127.0.0.1:8093
  3. set_password <your_super_pwd>
  4. unlock <your_super_pwd>
  5. import_key <user_name> <priv_key>

sudo vi /etc/supervisor/conf.d/bts_wallet.conf

  1. [program:bts_wallet]
  2. command=/home/<user>/programs/cli_wallet/cli_wallet --server-rpc-endpoint=ws://127.0.0.1:8091 -r 127.0.0.1:8093
  3. directory=/home/<user>/programs/cli_wallet/
  4. stdout_logfile=/var/log/supervisor/bts_wallet_out.log
  5. stderr_logfile=/var/log/supervisor/bts_wallet_err.log
  6. autostart=true
  7. autorestart=true
  8. numprocs=1
  9. user=<user>

sudo supervisorctl reread

sudo supervisorctl update

sudo supervisorctl start bts_node

sudo supervisorctl start bts_wallet

NOTE:

If error while trying to run wallet:

  1. Logging RPC to file: logs/rpc/rpc.log
  2. terminate called after throwing an instance of 'std::runtime_error'
  3. what(): locale::facet::_S_create_c_locale name not valid
  4. Aborted (core dumped)

Try this:

sudo vi /etc/default/locale

  1. # The file should look like this:
  2. LANGUAGE=en_US.UTF-8
  3. LC_ALL=en_US.UTF-8
  4. LANG=en_US.UTF-8
  5. LC_TYPE=en_US.UTF-8
  1. locale-gen en_US.UTF-8
  2. dpkg-reconfigure locales

Installing nginx

wget http://nginx.org/download/nginx-1.11.3.tar.gz

tar -xzvf nginx-1.11.3.tar.gz

rm nginx-1.11.3.tar.gz

cd nginx-1.11.3/

sudo apt-get install libpcre3 libpcre3-dev libpcrecpp0v5 zlib1g-dev build-essential libssl1.0-dev

./configure —sbin-path=/usr/bin/nginx —conf-path=/etc/nginx/nginx.conf —error-log-path=/var/log/nginx/error.log —http-log-path=/var/log/nginx/access.log —with-debug —with-pcre —with-cc-opt=”-Wno-error” —with-http_ssl_module —with-threads

make

sudo make install

Configuring nginx

Previously buy domain and add 2 CNAME records for subdomens:
wallet.your-domain.com, node.your-domain.com.

Base configuration

sudo vi /lib/systemd/system/NGINX.service
```angular2
[Unit]
Description=The NGINX HTTP and reverse proxy server
After=syslog.target network.target remote-fs.target nss-lookup.target

[Service]
Type=forking
PIDFile=/usr/local/nginx/logs/nginx.pid
ExecStartPre=/usr/bin/nginx -t
ExecStart=/usr/bin/nginx
ExecReload=/usr/bin/nginx -s reload
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target

  1. > systemctl daemon-reload
  2. > systemctl enable NGINX.service
  3. > systemctl start NGINX.service
  4. > systemctl -l status NGINX.service
  5. #### **Filling nginx.conf**
  6. Don't forget to change stubs in nginx.conf on yours values.
  7. > vi /etc/sysctl.conf
  8. ```bash
  9. # Add next line in the end of the file.
  10. fs.file-max = 40000

sysctl -p

vi /etc/nginx/nginx.conf

  1. # user nobody;
  2. worker_processes auto;
  3. worker_rlimit_nofile 40000;
  4. pid /usr/local/nginx/logs/nginx.pid;
  5. events {
  6. worker_connections <set_value>; # worker_rlimit_nofile / worker_processes
  7. multi_accept on;
  8. use epoll;
  9. }
  10. http {
  11. # Basic settings
  12. sendfile on;
  13. tcp_nopush on;
  14. tcp_nodelay on;
  15. aio threads;
  16. # Logging
  17. access_log off;
  18. error_log /var/log/nginx/error.log crit;
  19. # Enable open file cache
  20. open_file_cache max=1000 inactive=20s;
  21. open_file_cache_valid 30s;
  22. open_file_cache_min_uses 2;
  23. open_file_cache_errors on;
  24. # Keepalive
  25. keepalive_timeout 30s;
  26. send_timeout 10s;
  27. # Hide nginx version information.
  28. server_tokens off;
  29. # Websockets
  30. map $http_upgrade $connection_upgrade {
  31. default upgrade;
  32. '' close;
  33. }
  34. # -- Upstreams --
  35. upstream subwallet {
  36. server 127.0.0.1:8093;
  37. }
  38. upstream subnode {
  39. server 127.0.0.1:8091;
  40. }
  41. # --- Servers directives ---
  42. server {
  43. # listen 443 ssl;
  44. listen 80;
  45. server_name wallet.domain.com;
  46. # <----- Uncomment this after generating ssl certs. ------->
  47. # ssl_certificate /etc/letsencrypt/live/wallet.domain.com/fullchain.pem;
  48. # ssl_certificate_key /etc/letsencrypt/live/wallet.domain.com/privkey.pem;
  49. # include /etc/letsencrypt/options-ssl-nginx.conf;
  50. # ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
  51. # <----- ------->
  52. location /ws {
  53. proxy_pass http://subwallet;
  54. proxy_http_version 1.1;
  55. proxy_set_header Upgrade $http_upgrade;
  56. proxy_set_header Connection $connection_upgrade;
  57. proxy_set_header Host $http_host;
  58. proxy_set_header X-Real-IP $remote_addr;
  59. proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  60. proxy_set_header X-Forwarded-Proto $scheme;
  61. proxy_read_timeout 300s;
  62. proxy_connect_timeout 10s;
  63. }
  64. }
  65. server {
  66. listen 80;
  67. server_name node.domain.com;
  68. location /ws {
  69. proxy_pass http://subnode;
  70. proxy_http_version 1.1;
  71. proxy_set_header Upgrade $http_upgrade;
  72. proxy_set_header Connection $connection_upgrade;
  73. proxy_set_header Host $http_host;
  74. proxy_set_header X-Real-IP $remote_addr;
  75. proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  76. proxy_set_header X-Forwarded-Proto $scheme;
  77. proxy_read_timeout 60s;
  78. proxy_connect_timeout 10s;
  79. }
  80. }
  81. }

systemctl restart NGINX.service

or do

systemctl reload NGINX.service

Adding ssl

  1. IMPORTANT NOTE:
  2. Adding ssl you will get a memory leak due to the fact
  3. that the aiohttp 3.5.4 library with python3.7 contains a memory leak when working with ssl.
  4. So , do not use ssl until this bug is fixed.
  1. sudo apt-get update
  2. sudo apt-get install software-properties-common
  3. sudo add-apt-repository universe
  4. sudo add-apt-repository ppa:certbot/certbot
  5. sudo apt-get update
  6. sudo apt-get install certbot python-certbot-nginx
  7. certbot --nginx -d wallet.domain.com certonly
  8. crontab -e
  9. # Add next line for auto updating cert.
  10. @daily certbot renew
  11. # Change values in /etc/letsencrypt/options-ssl-nginx.conf.
  12. ssl_session_cache shared:le_nginx_SSL:100m;
  13. ssl_session_timeout 4h;

NOTE

  1. If something was wrong -> reboot your system and try again , in 99 percent of the happenings it helps :D

After previous steps uncomment lines in nginx.conf and replace domens on yours. Then reboot your system.

Server security setting

Adding SSH keys

  1. # On your local car.
  2. ssh-keygen
  3. mv key_name* ~/.ssh/
  4. ssh-copy-id user@server_ip
  5. # On remote server.
  6. # Disable pas auth.
  7. sudo vi /etc/ssh/sshd_config
  8. -> replace yes to no in PasswordAuthentication line
  9. sudo systemctl reload sshd

Configuring UFW

Сhange firewall settings to your liking.

  1. ufw default deny incoming
  2. ufw default allow outgoing
  3. ufw allow ssh
  4. ufw enable
  5. ufw allow from <your ip> to any port www
  6. ufw allow from <your ip> to any port 443

Installing and usage bot

The following procedure will work in Debian
based Linux, more specifically the commands
to make the guide were executed in Ubuntu 18.10
with Python 3.7.

  1. # Installing python 3.7
  2. sudo apt-get install build-essential checkinstall
  3. sudo apt-get install libreadline-gplv2-dev libncursesw5-dev libssl-dev \
  4. libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev libffi-dev
  5. cd /usr/src
  6. sudo wget https://www.python.org/ftp/python/3.7.2/Python-3.7.2.tgz
  7. sudo tar xzf Python-3.7.2.tgz
  8. cd Python-3.7.2
  9. sudo ./configure --enable-optimizations
  10. sudo make install
  11. Python-3.7.2 -V
  12. # Cloning repository
  13. git clone https://github.com/mkbeh/rin-bitshares-arbitry-bot
  14. cd rin-bitshares-arbitry-bot
  15. # Creating and activating virtual env
  16. sudo pip install virtualenv
  17. virtualenv venv
  18. source venv/bin/activate
  19. pip3.7 install Cython
  20. pip3.7 install numpy
  21. # Installing bot
  22. python3.7 setup.py install
  23. # Running bot
  24. Just type rin-bot in the command line and press enter.

On first app startup will be generated config,
which must be filled.
Config is located -> /home/\/rin-bot/config.ini

Contents of the config.ini file

Explorer uri you can get here https://github.com/oxarbitrage/bitshares-explorer-api

  1. [DIRS]
  2. output dir = path/to/output/directory/output # name output requried
  3. log dir = path/to/log/directory/logs # name logs required
  4. [MIN_DAILY_VOLUME]
  5. overall min daily volume = 10 # Must be non float value
  6. pair min daily volume = 5 # Must be non float value
  7. [LIMITS]
  8. # Values of the options must be dicts. Values in dicts must have types int
  9. # or float. Values in dict 'volume limits' expressed in dollars.
  10. volume limits = {"1.3.0": 15, "1.3.113": 15, "1.3.1570": 15, "1.3.121": 15}
  11. min profit limits = {"1.3.0": 0.001, "1.3.113": 0.02, "1.3.1570": 2e-08, "1.3.121": 0.02}
  12. [URI]
  13. node uri = nodr_uri
  14. wallet uri = ws://127.0.0.1:8093/ws # example
  15. explorer uri = explrer_uri
  16. [ACCOUNT]
  17. account name = account_name
  18. wallet password = wallet_password
  19. [OTHER]
  20. data update time = 1 # Hours. Required int
  21. time to reconnect = 350 # Reconnect to node or wallet. Secs. Required int
  22. orders depth = 5 # Amount. Required int

When you will fill config - go to the next step.

Adding app to supervisor

sudo vi /etc/supervisor/conf.d/rin-bot.conf

  1. [program:rin-bot]
  2. command=/home/<user>/rin-bitshares-arbitry-bot/venv/bin/rin-bot
  3. autostart=true
  4. autorestart=true
  5. stopsignal=KILL
  6. numprocs=1
  7. user=<user>

sudo supervisorctl reread

sudo supervisorctl update

sudo supervisorctl start rin-bot

Logging

Logs, output files and config are available by path /home/\/rin-bot

Cython supporting

  1. If you want to compile modules by Cython - uncomment
  2. lines in setup.py and change the extension of
  3. the desired files to .pyx. But Cython compiling not
  4. tested so use it at your own risk.

Milestones:

  • Fix a bug associated with incorrect calculation of volumes.
  • Write own async cmd explorer REST API without web interface.
  • Improve bot by adding new features.
  • Write full async lib for Bitshares API.