项目作者: Quiq

项目描述 :
InfluxDB Tools
高级语言: Python
项目地址: git://github.com/Quiq/influxdb-tools.git
创建时间: 2017-09-11T20:04:33Z
项目社区:https://github.com/Quiq/influxdb-tools

开源协议:Other

下载


InfluxDB Tools

Scripts to migrate off influxdb to a better database :)

  • schema-influx-to-clickhouse.py - Generate table schemas for Clickhouse based on influxdb measurements.

  • line-protocol-to-clickhouse.py - Load influxdb line-protocol backup data into Clickhouse.

influx-backup.py

InfluxDB backup/restore script using HTTP API and line-protocol format.

  • Use InfluxDB HTTP API
  • Backup raw data into text files in line-protocol format
  • Restore from a backup
  • Chunked read/write
  • Separate file for each measurement
  • Backup/restore individual measurements
  • Backup/restore specific retention
  • Incremental backups using “since”, “until” date/time arguments
  • Delayed restore
  • Gzip support for backup/restore process

It is recommended to do a delayed restore using --restore-chunk-delay, --restore-measurement-delay
so your InfluxDB instance does not run out of memory or IO pretty fast.

Usage

  1. usage: influx-backup.py [-h] --url URL --user USER --dir DIR
  2. [--measurements MEASUREMENTS]
  3. [--from-measurement FROM_MEASUREMENT]
  4. [--retention RETENTION] [--gzip] [--dump]
  5. [--dump-db DUMP_DB] [--dump-since DUMP_SINCE]
  6. [--dump-until DUMP_UNTIL] [--restore]
  7. [--restore-db RESTORE_DB]
  8. [--restore-chunk-delay RESTORE_CHUNK_DELAY]
  9. [--restore-measurement-delay RESTORE_MEASUREMENT_DELAY]
  10. InfluxDB backup script
  11. optional arguments:
  12. -h, --help show this help message and exit
  13. --url URL InfluxDB URL including schema and port
  14. --user USER InfluxDB username. Password must be set as env var
  15. INFLUX_PW, otherwise will be asked.
  16. --dir DIR directory name for backup or restore form
  17. --measurements MEASUREMENTS
  18. comma-separated list of measurements to dump/restore
  19. --from-measurement FROM_MEASUREMENT
  20. dump/restore from this measurement and on (ignored
  21. when using --measurements)
  22. --retention RETENTION
  23. retention to dump/restore
  24. --gzip dump/restore into/from gzipped files automatically
  25. --dump create a backup
  26. --dump-db DUMP_DB database to dump
  27. --dump-since DUMP_SINCE
  28. start date in the format YYYY-MM-DD (starting
  29. 00:00:00) or YYYY-MM-DDTHH:MM:SSZ
  30. --dump-until DUMP_UNTIL
  31. end date in the format YYYY-MM-DD (exclusive)
  32. or YYYY-MM-DDTHH:MM:SSZ
  33. --restore restore from a backup
  34. --restore-db RESTORE_DB
  35. database target of restore
  36. --restore-chunk-delay RESTORE_CHUNK_DELAY
  37. restore delay in sec or subsec between chunks of 5000
  38. points
  39. --restore-measurement-delay RESTORE_MEASUREMENT_DELAY
  40. restore delay in sec or subsec between measurements

Examples

Dump stats db:

  1. ./influx-backup.py --url https://influxdb.localhost:8086 --user admin --dump --dump-db stats --dir stats

Dump heartbeat measurement from stats db with data until 2017-09-01:

  1. ./influx-backup.py --url https://influxdb.localhost:8086 --user admin --dump --dump-db stats --dir stats \
  2. --dump-until 2017-09-01 --measurements heartbeat

NOTE: If you get ChunkedEncodingError on dump, try to limit the data set using “since”, “until” arguments.

Restore from stats dir into stats_new db:

  1. ./influx-backup.py --url https://influxdb.localhost:8086 --user admin --restore --restore-db stats_new \
  2. --dir stats

Restore only heartbeat measurement from stats dir into stats_new db:

  1. ./influx-backup.py --url https://influxdb.localhost:8086 --user admin --restore --restore-db stats_new \
  2. --dir stats --measurements heartbeat