Robinhood Policy Engine:一种用于监视文件系统内容和计划文件系统条目操作的通用工具。
Robinhood Policy Engine is a versatile tool to manage contents of large file systems. It maintains a replicate of filesystem medatada in a database that can be queried at will. It makes it possible to schedule mass action on filesystem entries by defining attribute-based policies, provides fast ‘find’ and ‘du’ enhanced clones, gives to administrators an overall view of filesystem contents through its web UI and command line tools.
It supports any POSIX filesystem and implements advanced features for Lustre filesystems (list/purge files per OST or pool, read MDT changelogs…)
Originally developped for HPC, it has been designed to perform all its tasks in parallel, so it is particularly adapted for running on large filesystems with millions of entries and petabytes of data. But of course, you can take benefits of all its features for managing smaller filesystems.
Robinhood is distributed under the CeCILL-C license, which is a French transposition of the GNU LGPL and is fully LGPL-compatible.
For more information, refer to the online documentation.
Copyright (C) 2004-2017 CEA/DAM.
Copyright 2013-2016 Cray Inc. All Rights Reserved.
This program is free software; you can redistribute it and/or modify
it under the terms of the CeCILL-C License.
The fact that you are presently reading this means that you have had
knowledge of the CeCILL-C license (http://www.cecill.info) and that you
accept its terms.
It is advised to build RobinHood on your target system, to ensure the best
compatibility with your Lustre and MySQL versions.
Build requirements: glib2-devel, libattr-devel, mysql-devel or mariadb-devel,
s-nail, bison, flex, jemalloc, jemalloc-devel.
For lustre support: lustre or lustre-client, lustre-devel (Lustre >= 2.15)
For running RobinHood’s CI: lustre-tests (Lustre >= 2.15)
Unzip and untar the source distribution:
tar zxvf robinhood-3.x.x.tar.gz
cd robinhood-3.x.x
Configure and build:
./configure
make rpm
RPMs are generated in the ‘rpms/RPMS/arch‘ directory.
Install git, and autotools stuff:
yum install git automake autoconf libtool
Retrieve robinhood sources
git clone https://github.com/cea-hpc/robinhood.git
cd robinhood
git checkout master *(or other branch)*
sh autogen.sh
Then refer to section 2.1 for next compilation steps.
Only requirements: rpm-build, gcc, flex, bison
To create src rpm on a host with minimal requirements, run:
./configure --enable-dist
make srpm
By default, the src rpm builds robinhood for POSIX filesystems:
rpmbuild --rebuild robinhood-3.0*.src.rpm
To build robinhood binary RPM for a Lustre filsystem, run:
rpmbuild --rebuild robinhood-3.0*.src.rpm --with lustre --define "lversion x.y"
where x.y is your target lustre version (e.g. 2.7).
Note: if lustre-client is not installed on your target system, you can specify
an alternative lustre package by defining “lpackage”, e.g.
rpmbuild [...] --define "lpackage lustre"
Note: ‘lversion’ can be omitted, by it is strongly recommanded to prevent
incompatibility issues between lustreapi versions.
For installing robinhood on your target system, install robinhood-adm RPM.
It includes configuration helper for DB, changelogs, …
Robinhood needs a MySQL database for storing information about files.
This database can run on a different node from Robinhood daemon.
Run the configuration helper script as root on the database host to create
the database:
rbh-config create_db
Write the DB password to a file with read access for root only (600)
e.g. to /etc/robinhood.d/.dbpassword
Note: initially, the database schema is empty. Robinhood will create it the first time it is launched.
For Lustre, you must register robinhood as a MDT changelog consumer.
Run the configuration helper script on Lustre MDS:
rbh-config enable_chglogs
This registers a changelog consumer and activate required changelog records.
Note: by default, the script checks for a ‘cl1’ consumer.
If you need to register several changelog consumers on your file system,
refer to lustre documentation.
Examples of config files are installed in /etc/robinhood.d/templates/
You can also use the ‘—template’ option to generate a documented configuration file template:
robinhood --template=<template_file>
For more details, refer to Robinhood v3 admin guide.
Note: by default, robinhood searches configuration files in “/etc/robinhood.d”.
Even if your filesystem is empty, you need to perform an initial scan in order to initialize robinhood database.
This prevents from having entries in filesystem that it wouldn’t know about.
robinhood --scan --once
Example: to read lustre changelogs and run all policies:
RBH_OPT="--readlog --run=all"
Example: to regularly scan filesystem and run all policies:
RBH_OPT="--scan --run=all"
Start the daemon:
systemctl start robinhood@*fsname*