项目作者: zuazo

项目描述 :
Chef cookbook example to run test-kitchen inside Travis CI.
高级语言: Ruby
项目地址: git://github.com/zuazo/kitchen-in-travis.git
创建时间: 2015-07-10T00:43:28Z
项目社区:https://github.com/zuazo/kitchen-in-travis

开源协议:Apache License 2.0

下载


kitchen-in-travis Cookbook Build Status

Proof of concept cookbook to run test-kitchen inside Travis CI using kitchen-docker in User Mode Linux.

You can use this in your cookbook by using a .travis.yml file similar to the following:

  1. rvm:
  2. - 2.2
  3. sudo: true
  4. before_script:
  5. - source <(curl -sL https://raw.githubusercontent.com/zuazo/kitchen-in-travis/0.5.0/scripts/start_docker.sh)
  6. script:
  7. # Run test-kitchen with docker driver, for example:
  8. - KITCHEN_LOCAL_YAML=.kitchen.docker.yml bundle exec kitchen test

Look below for more complete examples.

The following files will help you understand how this works:

This example cookbook only installs nginx. It also includes some Serverspec tests to check everything is working correctly.

Install the Requirements

First you need to install Docker.

Then you can use bundler to install the required ruby gems:

  1. $ gem install bundle
  2. $ bundle install

Running the Tests in Your Workstation

  1. $ bundle exec rake

This example will run kitchen with Vagrant in your workstation. You can use $ bundle exec rake integration:docker to run kitchen with Docker, as in Travis CI.

Available Rake Tasks

  1. $ bundle exec rake -T
  2. rake integration:docker # Run integration tests with kitchen-docker
  3. rake integration:vagrant # Run integration tests with kitchen-vagrant

How to Implement This in My Cookbook

First, create a .kitchen.docker.yml file with the platforms you want to test:

  1. ---
  2. driver:
  3. name: docker
  4. platforms:
  5. - name: centos-6.6
  6. run_list:
  7. - name: ubuntu-14.04
  8. run_list:
  9. - recipe[apt]
  10. # [...]

If not defined, it will get the platforms from the main .kitchen.yml by default.

You can get the list of the platforms officially supported by Docker here.

Then, I recommend you to create a task in your Rakefile:

  1. # Rakefile
  2. require 'bundler/setup'
  3. # [...]
  4. desc 'Run Test Kitchen integration tests'
  5. namespace :integration do
  6. desc 'Run integration tests with kitchen-docker'
  7. task :docker do
  8. require 'kitchen'
  9. Kitchen.logger = Kitchen.default_file_logger
  10. @loader = Kitchen::Loader::YAML.new(local_config: '.kitchen.docker.yml')
  11. Kitchen::Config.new(loader: @loader).instances.each do |instance|
  12. instance.test(:always)
  13. end
  14. end
  15. end

This will allow us to use $ bundle exec rake integration:docker to run the tests.

The .travis.yml file example:

  1. rvm:
  2. - 2.0.0
  3. - 2.1
  4. - 2.2
  5. sudo: true
  6. before_script:
  7. - source <(curl -sL https://raw.githubusercontent.com/zuazo/kitchen-in-travis/0.5.0/scripts/start_docker.sh)
  8. script:
  9. - travis_retry bundle exec rake integration:docker

If you are using a Gemfile, you can add the following to it:

  1. # Gemfile
  2. group :integration do
  3. gem 'test-kitchen', '~> 1.2'
  4. end
  5. group :docker do
  6. gem 'kitchen-docker', '~> 2.1.0'
  7. end

This will be enough if you want to test only 2 or 3 platforms. If you want more, continue reading:

How to Run Tests in Many Platforms

Travis CI has a build time limitation of 50 minutes. If you want to test many platforms, you will need to split up the tests in multiple Travis CI builds. For those cases, I recommend you to use a Rakefile Rake task similar to the following:

  1. # Rakefile
  2. require 'bundler/setup'
  3. # [...]
  4. desc 'Run Test Kitchen integration tests'
  5. namespace :integration do
  6. # Gets a collection of instances.
  7. #
  8. # @param regexp [String] regular expression to match against instance names.
  9. # @param config [Hash] configuration values for the `Kitchen::Config` class.
  10. # @return [Collection<Instance>] all instances.
  11. def kitchen_instances(regexp, config)
  12. instances = Kitchen::Config.new(config).instances
  13. return instances if regexp.nil? || regexp == 'all'
  14. instances.get_all(Regexp.new(regexp))
  15. end
  16. # Runs a test kitchen action against some instances.
  17. #
  18. # @param action [String] kitchen action to run (defaults to `'test'`).
  19. # @param regexp [String] regular expression to match against instance names.
  20. # @param loader_config [Hash] loader configuration options.
  21. # @return void
  22. def run_kitchen(action, regexp, loader_config = {})
  23. action = 'test' if action.nil?
  24. require 'kitchen'
  25. Kitchen.logger = Kitchen.default_file_logger
  26. config = { loader: Kitchen::Loader::YAML.new(loader_config) }
  27. kitchen_instances(regexp, config).each { |i| i.send(action) }
  28. end
  29. desc 'Run integration tests with kitchen-vagrant'
  30. task :vagrant, [:regexp, :action] do |_t, args|
  31. run_kitchen(args.action, args.regexp)
  32. end
  33. desc 'Run integration tests with kitchen-docker'
  34. task :docker, [:regexp, :action] do |_t, args|
  35. run_kitchen(args.action, args.regexp, local_config: '.kitchen.docker.yml')
  36. end
  37. end

This will allow us to run different kitchen tests using the $ rake integration:docker[REGEXP] command.

Then, you can use the following .travis.yml file:

  1. rvm:
  2. - 2.0.0
  3. - 2.1
  4. - 2.2
  5. sudo: true
  6. env:
  7. matrix:
  8. # Split up the test-kitchen run to avoid exceeding 50 minutes:
  9. - KITCHEN_REGEXP=centos
  10. - KITCHEN_REGEXP=debian
  11. - KITCHEN_REGEXP=ubuntu
  12. before_script:
  13. - source <(curl -sL https://raw.githubusercontent.com/zuazo/kitchen-in-travis/0.5.0/scripts/start_docker.sh)
  14. script:
  15. - travis_retry bundle exec rake integration:docker[$KITCHEN_REGEXP]

Test multiple kitchen instances concurrently per build

Sometimes it may be quicker to test multiple kitchen-docker instances in each Travis build due to a limit of concurrent builds (due to total Travis system load), rather than split out to a separate build for every kitchen-docker instance. The key is to find the balance between the number of kitchen-docker instances per Travis build.

Rakefile.concurrency provides rake tasks that can run actions against multiple test-kitchen instances simultaneously:

  1. $ bundle exec rake --rakefile Rakefile.concurrency --tasks
  2. rake integration:docker[regexp,action,concurrency] # Run integration tests with kitchen-docker
  3. rake integration:vagrant[regexp,action,concurrency] # Run integration tests with kitchen-vagrant

Test multiple versions of Chef

If you’d like to test multiple versions of Chef, you can use a second environment variable in .travis.yml combined with the require_chef_omnibus property in .kitchen.yml.

  1. # .travis.yml
  2. # ...
  3. env:
  4. - KITCHEN_REGEXP=centos
  5. - KITCHEN_REGEXP=centos CHEF_OMNIBUS_VERSION=12.10
  6. - KITCHEN_REGEXP=centos CHEF_OMNIBUS_VERSION=12.8
  7. - KITCHEN_REGEXP=ubuntu
  8. - KITCHEN_REGEXP=ubuntu CHEF_OMNIBUS_VERSION=12.10
  9. - KITCHEN_REGEXP=ubuntu CHEF_OMNIBUS_VERSION=12.8
  10. # ...
  1. # .kitchen.yml
  2. # ...
  3. provisioner:
  4. name: chef_zero
  5. require_chef_omnibus: <%= ENV.fetch('CHEF_OMNIBUS_VERSION', true) %>
  6. # ...

When require_chef_omnibus is set to true, test-kitchen will build with the latest version of chef. Specifying a version in the form x.y as shown above will get the latest patch version of the minor version specified (~> x.y.0).

Real-world Examples

Known Issues

The Test Cannot Exceed 50 Minutes

Each test can not take more than 50 minutes to run within Travis CI. It’s recommended to split the kitchen run in multiple builds using the Travis CI build matrix.

Look at the examples in this documentation to learn how to avoid this.

Official CentOS 7 and Fedora Images

Cookbooks requiring systemd may not work correctly on CentOS 7 and Fedora containers. See Systemd removed in CentOS 7.

You can use alternative images that include systemd. These containers must run in privileged mode:

  1. # .kitchen.docker.yml
  2. # Non-official images with systemd
  3. - name: centos-7
  4. driver_config:
  5. # https://registry.hub.docker.com/u/milcom/centos7-systemd/dockerfile/
  6. image: milcom/centos7-systemd
  7. privileged: true
  8. - name: fedora
  9. driver_config:
  10. image: fedora/systemd-systemd
  11. privileged: true

Problems with Upstart in Ubuntu

Some cookbooks requiring Ubuntu Upstart may not work correctly.

You can use the official Ubuntu images with Upstart enabled:

  1. # .kichen.docker.yml
  2. - name: ubuntu-14.10
  3. run_list: recipe[apt]
  4. driver_config:
  5. image: ubuntu-upstart:14.10

Install netstat Package

It’s recommended to install net-tools on some containers if you want to test listening ports with Serverspec. This is because some images come without netstat installed.

This is required for example for the following Serverspec test:

  1. # test/integration/default/serverspec/default_spec.rb
  2. describe port(80) do
  3. it { should be_listening }
  4. end

You can ensure that netstat is properly installed running the netstat cookbook:

  1. # .kitchen.docker.yml
  2. - name: debian-6
  3. run_list:
  4. - recipe[apt]
  5. - recipe[netstat]

Travis CI Error: SSH session could not be established

Sometimes kitchen exits with the following error:

  1. >>>>>> Converge failed on instance <default-debian-7>.
  2. >>>>>> Please see .kitchen/logs/default-debian-7.log for more details
  3. >>>>>> ------Exception-------
  4. >>>>>> Class: Kitchen::ActionFailed
  5. >>>>>> Message: SSH session could not be established
  6. >>>>>> --------

If you get this error on Travis CI, avoid passing the --concurrency option to test-kitchen. It does not work in some cases. I recommend using the Travis CI build matrix to run multiple tests concurrently.

Travis CI Error: No output has been received in the last 10 minutes

If a command can take a long time to run and is very quiet, you may need to run it with some flags to increase verbosity such as: --verbose, --debug, --l debug, …

Travis CI Error: Waiting Docker to Start Timeout (No output has been received in the last 10 minutes)

UML does not seem to work properly on some projects. The Travis build output in these cases:

  1. $ source <(curl -sL https://raw.githubusercontent.com/zuazo/kitchen-in-travis/0.5.0/scripts/start_docker.sh)
  2. [...]
  3. Starting Docker Engine
  4. Waiting Docker to start
  5. No output has been received in the last 10 minutes, this potentially indicates a stalled build or something wrong with the build itself.

Try using kitchen-in-travis-native or kitchen-in-circleci if you encounter this problem.

Travis CI Error: Waiting Docker to Start Timeout Debug Information

On some project builds SLIRP seems not to work correctly. This is the exact error in the UML boot process:

  1. $ ip link set eth0 up
  2. RTNETLINK answers: No such file or directory

But the interface exists:

  1. eth0 Link encap:Serial Line IP
  2. inet addr:10.1.1.1 Mask:255.255.255.0
  3. NOARP MTU:1500 Metric:1
  4. RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  5. TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
  6. collisions:0 txqueuelen:256
  7. RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
  8. Interrupt:5

I have not found a way to fix the problem. Please, let me know if you find a solution.

Feedback Is Welcome

Currently I’m using this for my own projects. It may not work correctly in many cases. If you use this or a similar approach successfully with other cookbooks, please open an issue and let me know about your experience. Problems, discussions and ideas for improvement, of course, are also welcome.

Acknowledgements

See here.

License and Author

Author: Xabier de Zuazo (xabier@zuazo.org)
Contributor: Jacob McCann
Contributor: Pedro Salgado
Contributor: Austin Heiman
Copyright: Copyright (c) 2015-2016, Xabier de Zuazo
License: Apache License, Version 2.0
  1. Licensed under the Apache License, Version 2.0 (the "License");
  2. you may not use this file except in compliance with the License.
  3. You may obtain a copy of the License at
  4. http://www.apache.org/licenses/LICENSE-2.0
  5. Unless required by applicable law or agreed to in writing, software
  6. distributed under the License is distributed on an "AS IS" BASIS,
  7. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  8. See the License for the specific language governing permissions and
  9. limitations under the License.