策略引擎>> perunKeystoneAdapter>> 返回
项目作者： deNBI

项目描述 ：
Perun Keystone Adapter parses data propagated by Perun data and modifies a connected Keystone.
高级语言： Python
项目地址: git://github.com/deNBI/perunKeystoneAdapter.git
创建时间： 2017-12-13T13:46:09Z
项目社区：https://github.com/deNBI/perunKeystoneAdapter
开源协议：Apache License 2.0
下载


Documentation Status

Perun Keystone Adapter

The Perun Keystone Adapter is a library written in Python that parses data propagated
by Perun and modifies a
connected Openstack Keystone.

Additionally, the PKA offers the possibility to restrict created projects quotas and create network stuff
for new projects.

Features

  • abstract keystone to simplify often used tasks (create/delete/update/list users and projects)
  • parse SCIM or de.NBI portal compute center propagation data for users and projects
  • modify Keystone according the propagated data:
  • creates items (users or projects) in Keystone if they not exist but propagated
  • modify items properties if they changed
  • mark items as deleted and disable them in Keystone if they are not propagated anymore
  • deleting (marked and disabled) items functionality is available but not integrated in the normal workflow.
  • set/modify project quotas (needs a full openstack installation like DevStack for testing) [optional]
  • create network (router, net and subnet) for new projects [optional]
  • adjust default security group to support ssh access [optional]
  • compatible with python 3.8+

Preparation

Before installing Perun Keystone Adapter you must be sure that used openstack domain for propagation is empty or all
existing projects and users that also exists in perun must be tagged to avoid naming conflicts and the project names
must have the same names as the groups in perun. By default, everything created by the library is tagged as
perun_propagation. This can be overwritten in the constructor of the KeyStone class.

As a help there are two scripts included in the assets directory of this repository that set a flag of your choice for
a user and for a project.

  1. First install all necessary dependencies (maybe in your virtual environment) by running

    1. $ pip install -r requirements/default.txt

  2. The scripts expect that you sourced your OpenStack rc file:

    1. $ source env.rc

  3. Run the python script

    1. $ python set_project_flag.py  project_id flag_name

    or

    1. $ python set_user_flag.py  user_id flag_name

    where

    • user_id and project_id are OpenStack specific IDs
    • flag_name can be any value which is set for the flag attribute. If you do not modify the perunKeystoneAdapter,
      it expects perun_propagation as the value.

Installation

Install a specific version of this library by providing a tag of the releases page:

E.g: for version 0.1.1:

  1. pip install git+https://github.com/deNBI/perunKeystoneAdapter@0.1.1

Usage

Commandline client

The perun propagation service transfers a zipped tar file containing a users and groups file
in scim format. The following script unzips and untars the propagated information and adds
it to the keystone database. Keystone is addressed by environment variables
(sourcing the openstack rc file) or directly by passing an environemt map
(not used in the example). The Openstack user needs at least permission to modify entries
in Keystone.

  1. $ perun_propagation perun_upload.tar.gz

WSGI script

The python module also contains a built-in server version of the perun_propagation script.
The script uses flask to provide an upload function and run library
functions in a separate thread. It can be simply tested starting the flask built-in webserver.

  1. $ perun_propagation_service
  2.  * Serving Flask app "denbi.scripts.perun_propagation_service" (lazy loading)
  3.  * Environment: production
  4.    WARNING: Do not use the development server in a production environment.
  5.    Use a production WSGI server instead.
  6.  * Debug mode: off
  7.  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

For running this in production it is easy to use gunicorn as follows:

  1. $ gunicorn --workers 1 --bind 127.0.0.1:5000 denbi.scripts.perun_propagation_service:app

Configuration

The Perun Keystone Adapter can be configured in two different ways, by environment or by configuration file.

… by environment

  1. # OpenStack credentials
  2. # ---------------------
  3. export OS_REGION_NAME="XXX"
  4. export OS_PROJECT_DOMAIN_ID="XXX"
  5. export OS_INTERFACE="public"
  6. export OS_AUTH_URL="https://XXX"
  7. export OS_USERNAME="admin"
  8. export OS_PROJECT_ID="XXX"
  9. export OS_USER_DOMAIN_NAME="Default"
  10. export OS_PROJECT_NAME="admin"
  11. export OS_PASSWORD="XXX"
  12. export OS_IDENTITY_API_VERSION="3"
  14. # Perun Keystone Adapater settings
  15. # --------------------------------
  17. # Location for storing propagated data
  18. export PKA_BASE_DIR=/pka
  19. # Location for storing logs, defaults to current working directory
  20. export PKA_LOG_DIR=/log
  21. # Log level, must be one of ERROR, WARNING, INFO, DEBUG, defaults to INFO
  22. export PKA_LOG_LEVEL=INFO
  23. # Do not make any modifications to keystone
  24. export PKA_KEYSTONE_READ_ONLY=False
  25. # Domain to create users and projects in, defaults to 'elixir'
  26. export PKA_TARGET_DOMAIN_NAME=elixir
  27. # Default role to assign to new users, defaults to 'user'
  28. export PKA_DEFAULT_ROLE=user
  29. export PKA_DEFAULT_NESTED=False
  30. export PKA_ELIXIR_NAME=False
  31. # Set quotas for projects
  32. export PKA_SUPPORT_QUOTA=True
  33. # Create router for new projects
  34. export PKA_SUPPORT_ROUTER=True
  35. # Create a default network/subnetwork for new projects
  36. export PKA_SUPPORT_NETWORK=True
  37. # Create
  38. export PKA_EXTERNAL_NETWORK_ID=16b19dcf-a1e1-4f59-8256-a45170042790
  39. # Add ssh rule to default
  40. export PKA_SUPPORT_DEFAULT_SSH_SGRULE=True

by configuration file

PKA supports configuration files in JSON format.
An example configuration could look like this:

  1. {
  2.    "OS_REGION_NAME": "XXX",
  3.    "OS_PROJECT_DOMAIN_ID": "XXX",
  4.    "OS_INTERFACE": "public",
  5.    "OS_AUTH_URL": "https://XXXX",
  6.    "OS_USERNAME": "admin",
  7.    "OS_PROJECT_ID": "XXX",
  8.    "OS_USER_DOMAIN_NAME": "Default",
  9.    "OS_PROJECT_NAME": "admin",
  10.    "OS_PASSWORD": "XXX",
  11.    "OS_IDENTITY_API_VERSION": 3,
  12.    "BASE_DIR": "/pka",
  13.    "LOG_DIR": "/log",
  14.    "LOG_LEVEL": "INFO",
  15.    "KEYSTONE_READ_ONLY": false,
  16.    "TARGET_DOMAIN_NAME": "elixir",
  17.    "DEFAULT_ROLE": "user",
  18.    "NESTED": false,
  19.    "ELIXIR_NAME": false,
  20.    "SUPPORT_QUOTAS": true,
  21.    "SUPPORT_ROUTER": true,
  22.    "SUPPORT_NETWORK": true,
  23.    "EXTERNAL_NETWORK_ID": "16b19dcf-a1e1-4f59-8256-a45170042790",
  24.    "SUPPORT_DEFAULT_SSH_SGRULE": true,
  25.    "SSH_KEY_BLOCKLIST": [],
  26.    "CLEANUP": false
  27. }

Docker

Build docker container.

docker build -t denbi/pka .

Create environment file (pka.env) :

  1. # OpenStack credentials
  2. # ---------------------
  3. OS_REGION_NAME="XXX"
  4. OS_PROJECT_DOMAIN_ID="XXX"
  5. OS_INTERFACE="public"
  6. OS_AUTH_URL="https://XXX"
  7. OS_USERNAME="admin"
  8. OS_PROJECT_ID="XXX"
  9. OS_USER_DOMAIN_NAME="Default"
  10. OS_PROJECT_NAME="admin"
  11. OS_PASSWORD="XXX"
  12. OS_IDENTITY_API_VERSION="3"
  14. # Perun Keystone Adapater settings
  15. # --------------------------------
  17. # Location for storing propagated data
  18. PKA_BASE_DIR=/pka
  19. # Location for storing logs, defaults to current working directory
  20. PKA_LOG_DIR=/log
  21. # Do not make any modifications to keystone
  22. PKA_KEYSTONE_READ_ONLY=False
  23. # Domain to create users and projects in, defaults to 'elixir'
  24. PKA_TARGET_DOMAIN_NAME=elixir
  25. # Default role to assign to new users, defaults to 'user'
  26. PKA_DEFAULT_ROLE=user
  27. PKA_DEFAULT_NESTED=False
  28. PKA_ELIXIR_NAME=False
  29. # Set quotas for projects
  30. PKA_SUPPORT_QUOTA=True
  31. # Create router for new projects
  32. PKA_SUPPORT_ROUTER=True
  33. # Create a default network/subnetwork for new projects
  34. PKA_SUPPORT_NETWORK=True
  35. # Create 
  36. PKA_EXTERNAL_NETWORK_ID=16b19dcf-a1e1-4f59-8256-a45170042790
  37. # Add ssh rule to default 
  38. PKA_SUPPORT_DEFAULT_SSH_SGRULE=True

and run the container:

  1. docker run --net host --env-file pka.env -v $(pwd)/perun/upload:/pka -v $(pwd)/perun/log:/log denbi/pka

Alternatively you can use a configuration file (pka.json) …

  1. docker run --net host -v $(pwd)/pka.json:/etc/pka.json -v $(pwd)/tmp/base:/pka -v $(pwd)/tmp/log:/log denbi/pka

There are additional deployment options available
if you prefer to run WSGI applications with Apache, or other setups.

Logging

The Library supports two different logger domains, which can be configured when instantiating the
Keystone/Endpoint class (default “denbi” and “report”).
All changes concerning the Openstack database (project, identity and quotas) are logged to the
logging domain, everything else is logged to the report domain. The loggers are standard Python
logger, therefore all possibilities of Python’s logging API are supported.
See service script for an example how to configure
logging.

Development

For testing and development it is recommended to configure and use a [DevStack] (https://docs.openstack.org/devstack/latest/)
that fits your production environment.(see test/devstack/README.md)

Unit tests

The library comes with a set of unit tests - a full functional keystone is required to perfom all tests.

For testing the user/project management only a running keystone is enough. The Makefile included with
the project runs a docker container for providing a keystone server.

It is recommended to configure and use a [DevStack]. In any case it is not recommended to use your
production keystone/setup .

Linting

  1. $ make lint

will run flake8 on the source code directories.