项目作者: jtyr

项目描述 :
Ansible dynamic inventory reading the inventory from YAML file.
高级语言: Python
项目地址: git://github.com/jtyr/ansible-yaml_inventory.git
创建时间: 2017-03-27T13:03:46Z
项目社区:https://github.com/jtyr/ansible-yaml_inventory

开源协议:MIT License

下载


yaml_inventory

Ansible dynamic inventory script which reads the inventory from a specially
formatted YAML file.

Description

Standard Ansible inventory suffers of several issues:

  • It must be a single file or multiple files in a directory.
  • It has a flat structure where all groups are on the same level and relate to
    each other via :children definition leading to long group names when trying
    to capture more complex relationships. That also affects the names of the
    group_vars.
  • When using Vault file, it requires the files the be either named like the
    group or to be placed inside a directory of the group name or to be
    explicitely included in the play.
  • Hosts that belong to multiple groups must be defined in multiple places.
  • Sharing the same group_vars across multiple groups is a challenging problem.

This Ansible dynamic inventory script is trying to address these issues
by allowing the following features:

  • Possibility to split single inventory into multiple files.
  • Self-generating group names based on the YAML structure.
  • Truly hierarchical vars files.
  • Automatic .vault files loading.
  • Possibility to add hosts to an other group.
  • Possibility to include hosts from an other groups via regexp.
  • Using vars files as a common template.

See the usage bellow for more details.

Installation

  1. $ git clone https://github.com/jtyr/ansible-yaml_inventory yaml_inventory
  2. $ ln -s yaml_inventory/yaml_inventory.py hosts
  3. $ ansible-playbook -i hosts site.yaml

Usage

Inventory YAML file

Here is an example of the standard Ansible inventory file:

  1. [aws:children]
  2. aws-dev
  3. aws-qa
  4. aws-stg
  5. aws-prd
  6. [aws-dev]
  7. aws-dev-host01 ansible_host=192.168.1.15
  8. [aws-dev:children]
  9. aws-dev-jenkins
  10. [aws-dev-jenkins]
  11. aws-dev-jenkins01 ansible_host=192.168.1.16
  12. [aws-qa]
  13. aws-qa-host01 ansible_host=192.168.2.15
  14. [aws-stg]
  15. aws-stg-host01 ansible_host=192.168.3.15
  16. [aws-prd]
  17. aws-prd-host01 ansible_host=192.168.4.15
  18. [azure:children]
  19. azure-dev
  20. azure-qa
  21. azure-stg
  22. azure-prd
  23. [azure-dev]
  24. azure-dev-host01 ansible_host=10.0.1.15
  25. [azure-dev:children]
  26. azure-dev-jenkins
  27. [azure-dev-jenkins]
  28. azure-dev-jenkins01 ansible_host=10.0.1.16
  29. [azure-qa]
  30. azure-qa-host01 ansible_host=10.0.2.15
  31. [azure-stg]
  32. azure-stg-host01 ansible_host=10.0.3.15
  33. [azure-prd]
  34. azure-prd-host01 ansible_host=10.0.4.15

And here is the same but in the YAML format:

  1. ---
  2. aws:
  3. dev:
  4. :hosts:
  5. - aws-dev-host01: { ansible_host: 192.168.1.15 }
  6. jenkins:
  7. :hosts:
  8. - aws-dev-jenkins01: { ansible_host: 192.168.1.16 }
  9. qa:
  10. :hosts:
  11. - aws-qa-host01: { ansible_host: 192.168.2.15 }
  12. stg:
  13. :hosts:
  14. - aws-stg-host01: { ansible_host: 192.168.3.15 }
  15. prd:
  16. :hosts:
  17. - aws-prd-host01: { ansible_host: 192.168.4.15 }
  18. azure:
  19. dev:
  20. :hosts:
  21. - azure-dev-host01: { ansible_host: 10.0.1.15 }
  22. jenkins:
  23. :hosts:
  24. - azure-dev-jenkins01: { ansible_host: 10.0.1.16 }
  25. qa:
  26. :hosts:
  27. - azure-qa-host01: { ansible_host: 10.0.2.15 }
  28. stg:
  29. :hosts:
  30. - azure-stg-host01: { ansible_host: 10.0.3.15 }
  31. prd:
  32. :hosts:
  33. - azure-prd-host01: { ansible_host: 10.0.4.15 }

The main YAML inventory should be stored in the main.yaml file located
by default in the inventory directory. The location can be changed in
the config file (inventory_path - see the yaml_inventory.conf file)
or via environment variable (YAML_INVENTORY_PATH).

This is an example of a monolithic inventory YAML file:

  1. ---
  2. aws:
  3. dev:
  4. elk:
  5. elasticsearch:
  6. # Hosts of the aws-dev-elk-elasticsearch group
  7. :hosts:
  8. # Hosts with variables
  9. - elk01: { ansible_host: 192.168.1.11 }
  10. - elk02: { ansible_host: 192.168.1.12 }
  11. - elk03: { ansible_host: 192.168.1.13 }
  12. kibana:
  13. # Hosts of the aws-dev-elk-kibana group
  14. :hosts:
  15. # Host with no variables
  16. - elk04

The same like above but with YAML reference:

  1. ---
  2. # This is a subset of the main data structure referenced bellow
  3. aws-dev: &aws-dev
  4. elk:
  5. elasticsearch:
  6. :hosts:
  7. - elk01: { ansible_host: 192.168.1.11 }
  8. - elk02: { ansible_host: 192.168.1.12 }
  9. - elk03: { ansible_host: 192.168.1.13 }
  10. kibana:
  11. :hosts:
  12. - elk04
  13. # This is the main data structure
  14. aws:
  15. dev:
  16. # Reference to the above data structure
  17. <<: *aws-dev

This is the same like above but with the referenced content in a separate
file:

Content of the aws-dev.yaml file:

  1. ---
  2. # This can be still referenced from the main YAML file
  3. aws-dev: &aws-dev
  4. elk:
  5. elasticsearch:
  6. :hosts:
  7. - elk01: { ansible_host: 192.168.1.11 }
  8. - elk02: { ansible_host: 192.168.1.12 }
  9. - elk03: { ansible_host: 192.168.1.13 }
  10. kibana:
  11. :hosts:
  12. - elk04

Content of the main.yaml file:

  1. ---
  2. # This is the main data structure
  3. aws:
  4. dev:
  5. # Refference the above data structure
  6. <<: *aws-dev

The inventory script reads all YAML files from the inventory directory
and merges them all together. The main.yaml portion is always inserted
at the end so that the YAML references can still be resolved. Group names
are composed from elements of the tree separated by - sign.

As visible above, the YAML structure contains special keyword :hosts.
That keyword indicates that content of that section is a list of hosts.
Other keywords are :vars, :template, :groups and :add_hosts.
The following is an example of usage of all the keywords:

  1. ---
  2. g1:
  3. :hosts:
  4. - ahost1
  5. - ahost2
  6. :vars:
  7. # Variable for the group g1
  8. my_var: 123
  9. g2:
  10. :hosts:
  11. - bhost1
  12. - bhost2
  13. :groups:
  14. # Add all hosts from group g2 into the group g1
  15. - g1
  16. g3:
  17. :hosts:
  18. - chost1
  19. - chost2
  20. :templates:
  21. # This creates g3@template-g3 group which has the g3 group as its
  22. # child. That will allow to load the inventory vars from the file in
  23. # inventory/vars/template/g3. As the g3 group is a child of the
  24. # template group, variables from the g3 inventory vars file can
  25. # still override the vars in the template file.
  26. - template-g3
  27. g4:
  28. :add_hosts:
  29. # Regular expression to add hosts ahost2 and bhost2 into this group
  30. - ^[ab]host2

Inventory vars

Classical group_vars files can be structured in two levels only - files
in the group_vars directory and file in the directories located in the
group_vars directory. This is quite restrictive and forces the user to
capture the inventory groups structure in the group_vars file name.

This Ansible dynamic inventory is trying to address this issue by
introducing the inventory vars. Inventory vars are variation of the
group_vars with the difference that it copies the hierarchical
structure of YAML inventory file. The inventory vars directory is by
default in the inventory/vars directory but can be changed in the
config file (inventory_vars_path) or via environment variable
(YAML_INVENTORY_VARS_PATH). This feature is enabled by default but can
be disabled by setting the create_symlinks config option or the
YAML_INVENTORY_CREATE_SYMLINKS environment variable to value no.

The inventory vars file for a specific group can be called either like
the last element of the group name or like all inside a directory
called like the last element of the group name. If the group contains
another groups, only the second option is available because there cannot
coexist file and directory of the same name.

If this is the inventory file:

  1. $ cat inventory/main.yaml
  2. ---
  3. # This is a group containing another group (vars in `all` file)
  4. aws:
  5. dev:
  6. # This is the leaf group (vars in `jenkins` file)
  7. jenkins:
  8. :hosts:
  9. - aws-dev-jenkins01

then the corresponding file structure of the inventory vars can look like
this:

  1. $ tree -p inventory/vars
  2. inventory/vars
  3. └── [drwxr-xr-x] aws
  4. ├── [-rw-r--r--] all
  5. └── [drwxr-xr-x] dev
  6. └── [-rw-r--r--] jenkins

If enabled, the inventory vars are symlinked into the group_vars
directory during the execution of the inventory script. The group_vars
file names are based on the structure of the inventory vars directory.
From the example above, the path invenotory/aws/all is symlinked like
group_vars/aws and the path invenotory/aws/dev/jenkins is symlinked
like group_vars/aws-dev-jenkins.

  1. $ ls -la ./group_vars
  2. total 8
  3. drwxr-xr-x 2 jtyr users 4096 Mar 28 17:20 .
  4. drwxr-xr-x 9 jtyr users 4096 Mar 27 10:10 ..
  5. lrwxrwxrwx 1 jtyr users 21 Mar 28 17:20 aws -> ../inventory/vars/aws/all
  6. lrwxrwxrwx 1 jtyr users 29 Mar 28 17:20 aws-dev-jenkins -> ../inventory/vars/aws/dev/jenkins

The script also simplifies the use of Vault files by automatically
creating relationship between the group (e.g. mygroup) and the secured
content of that group (mygroup.vault). This convention makes sure that
the Vault file is always loaded if it exists.

Inventory script

The inventory script can be used as any other Ansible dynamic inventory.
With the default settings (the main.yaml inventory file in the
./inventory directory and the inventory vars in the ./inventory/vars
directory) the command can be as follows:

  1. $ ansible-playbook -i ./yaml_inventory.py site.yaml

The inventory script implements the standard --list and --host
command line options and can be influenced by a config file (see
yaml_inventory.conf file) or environment variables:

  1. usage: yaml_inventory.py [-h] [--list] [--host HOST]
  2. Ansible dynamic inventory reading YAML file.
  3. optional arguments:
  4. -h, --help show this help message and exit
  5. --list list all groups and hosts
  6. --host HOST get vars for a specific host
  7. environment variables:
  8. YAML_INVENTORY_CONFIG_PATH
  9. location of the config file (default locations:
  10. ./yaml_inventory.conf
  11. ~/.ansible/yaml_inventory.conf
  12. /etc/ansible/yaml_inventory.conf)
  13. YAML_INVENTORY_PATH
  14. location of the inventory directory (./inventory by default)
  15. YAML_INVENTORY_VARS_PATH
  16. location of the inventory vars directory (YAML_INVENTORY_PATH/vars by default)
  17. YAML_INVENTORY_GROUP_VARS_PATH
  18. location of the vars directory (./group_vars by default)
  19. YAML_INVENTORY_SUPPORT_VAULTS\n'
  20. flag to take in account .vault files (yes by default)
  21. YAML_INVENTORY_CREATE_SYMLINKS
  22. flag to create group_vars symlinks (yes by default)

Combining it with other inventory scripts

Put the YAML dynamic inventory script together with the other (e.g.
AWS EC2)
dynamic inventory script into the inventory_scripts directory and name them
to be in alphabetical order that the YAML inventory is first and the other
inventory is second:

  1. $ ls inventory_scripts
  2. 01_yaml_inventory
  3. 02_aws_inventory

Use the YAML inventory to create the desired inventory structure and the
other inventory to fill in the hosts into the groups:

  1. $ cat inventory/main.yaml
  2. ---
  3. aws:
  4. dev:
  5. jenkins:
  6. :hosts: []

Then run Ansible like this:

  1. $ ansible-playbook -i inventory_scripts site.yaml

Issues

  • When using create_symlinks = yes and the inventory structure has changed,
    the YAML inventory must be run manually before the Ansible runs because the
    group_vars symlinks get created after the group_vars files are normally
    read by Ansible.
  • No Vault support for all group due to the circular relationship between
    the all.vault and the all group.
  • No support for encrypted variables available in Ansible v2.3+.

TODO

  • Implement hosts enumeration.
  • Refactor it into an inventory plugin to facilitate support for encrypted
    variables.

License

MIT

Author

Jiri Tyr