Policy Engine

The module provides a defined resource type that generates a Facter plugin for
policy tests. Each Facter run, the plugin executes a specified script,
written in any language, and compares the execution result to the expected
output. If the expectation matches, the test passes. If not, it fails. The test
result is added as a structured fact.

The facts can be used as part of a continuous delivery pipeline to ensure
individual node configurations meet relevant policy requirements before
configurations are deployed to production. The facts can be queried from
PuppetDB for continuous policy monitoring.

The tests follow the rspec model of declaring what you
want to do and what the expected result is. If the result doesn’t match
the expectation, the test fails.

Each test result is a structured value in a standard format. The output format
is as follows:

If the test passed

{'result' => 'pass', 'tags' => ['policy_engine','tag1','tag2']}

If the test fails

{'result' => 'fail', 'tags' => ['policy_engine','tag1','tag2'], 'expected_output' => [], 'is' => ['example','output']}

Declaring Policy Tests

Note: This module only support Puppet 4+ and Puppet Enterprise 2015.2+

Tests can be written in any language the system they run on supports. The code
that performs the test can range from a single shell command to a script file.
The user can specify an interpreter to use to run the code (defaults to
/bin/sh).

To validate a test passes or fails, an expectation can be specified. An expectation can be the following:

• Stdout output. The output can be parsed as a string, JSON, or YAML. Strings can be matched completely or against a regular expression. An array can be specified of acceptable strings and regexes.
• Exit code. The exit code of the script execution. An array can be specified of acceptable exit codes.

Execute a command and expect no output

policy_engine::test { 'name_of_test':
  script          => 'single command to run',
  expected_output => '',
}

Execute a python script generated by an ERB and expect an empty array in JSON

policy_engine::test { 'another_test':
  script          => template('my_module/test.py.erb'),
  expected_output => [],
  interpreter     => 'python',
  output_format   => 'json',
}

Execute a ruby script from a module and expect an empty array in YAML

policy_engine::test { 'ruby_test':
  source          => 'puppet:///modules/my_module/thing',
  expected_output => [],
  interpreter     => 'ruby',
  output_format   => 'yaml',
}

Retrieving test results

Since each test is a Facter fact, they can be retrieved using Facter or PuppetDB.

Run with Facter
The Policy Engine Facter plugin is pluginsynced from the Puppet module. To run
the policy test, use the -p flat with Facter

facter -p policy_name

Retrieve from PuppetDB
If you’re using PuppetDB, the puppet master pushes
every node’s facts each puppet agent run to PuppetDB. This means PuppetDB can
be queried for test results. The examples directory has example PuppetDB queries.
To retrieve, standard curl can be used, or any other tool that perform REST
calls.

curl -X GET http://puppetdb.example.com:8080/v4/facts --data-urlencode query@./failed_tests

Reference

Classes

Public classes

• policy_engine: Configures Policy Engine testing framework

Parameters

policy_engine

test_dir

The directory where the test metadata and execution scripts will be kept

Defined Types

• policy_engine::test: A Policy Engine test

policy_engine::test

Parameters

• ensure: valid values are present or absent. Defaults to present
• source: The source of a script. Follows same values as the file type
• script: A script to run in text format. This is similar to the content parameter for the file type
• interpreter: The interpreter on the system to run. Defaults to /bin/sh
• output_format: What format the stdout is in from the execution script. Valid values are string, json, and yaml. Defaults to string
• expected_output: What the expected stdout output is. Takes a string, regex, or an array of strings/regexes. Regexes must be in string format (i.e. ‘/my regex/‘)
• expected_exit_code: What the expected exit code of the execution script is. Takes an integer or an array or integers. If specified, this parameter has precedence over the expected_output parameter.
• tags: Arbitrary tags for the policy test. Every test is automatically tagged with policy_engine