项目作者: techgaun

项目描述 :
Concourse CI cheatsheet
高级语言:
项目地址: git://github.com/techgaun/concourse-cheatsheet.git
创建时间: 2020-08-07T21:36:22Z
项目社区:https://github.com/techgaun/concourse-cheatsheet

开源协议:Apache License 2.0

下载


Concourse CI

concourse-cheatsheet

Cheatsheet for working with Concourse CI

This cheatsheet lists various useful tips and tricks to use with Concourse CI
and thus omits the very basics such as logging in or setting pipeline from yaml file.

Table of Contents

Fly CLI

Fly completion

  1. # for bash, put the following on your .bashrc
  2. source <(fly completion --shell bash)
  3. # for zsh, put the following on your .zshrc
  4. source <(fly completion --shell zsh)

List pipelines in all teams

  1. # lists all the accessible pipelines regardless of what team you're on in the target
  2. fly -t <your_target> ps -a

List all workers with details

  1. fly -t <your_target> workers --details

List build containers

  1. fly -t <your_target> containers
  2. # above command also provides handle id which you can use to intercept
  3. fly -t <your_target> intercept --handle <handle_id> # eg. handle_id - 5f588b86-116b-4e7c-5bef-811dad839539

Intercept build aka grab an interactive shell inside build container for debugging

  1. # simple example to connect to a particular pipeline's specific job
  2. fly -t <your_target> intercept --job <PIPELINE>/<JOB_NAME>
  3. # simple example to connect to a particular pipeline's specific job and run specific binary
  4. # this is useful to specify custom shell such as sh in alpine or any arbitrary command during intercept
  5. fly -t <your_target> intercept --job <PIPELINE>/<JOB_NAME> <your_custom_binary>

Arbitrary API requests to your Concourse CI

  1. fly -t <your_target> curl /api/v1/info
  2. > {"version":"6.1.0","worker_version":"2.2","external_url":"https://concourse.example.com"}
  3. fly -t <your_target> curl /api/v1/builds
  4. > json_array_of build_lists

Make pipeline visible to unauthenticated users

This is often useful for open-source projects so that the build pipeline
is visible for unauthenticated users.

  1. fly -t <your_target> expose-pipeline --pipeline <YOUR_PIPELINE>

Format your pipeline

Concourse has a handy fly command to format your pipeline in a “canonical” form.
Omitting -w / --write would print formatted pipeline on stdout instead.
This is useful when you’ve made a mess of your yaml configuration formatting.

  1. fly format-pipeline -c <YOUR_PIPELINE.yml> -w

Different team in your fly commands

By default, concourse sets team on a target when you login and its cumbersome
to manage multiple targets just to be working on a different team space. Fly
commands support --team so you can specify another team name in a single target.
However, this is a work in progress as tracked HERE
so make sure you check the progress in above issue or by checking the help information.

Pipleline Configurations

Root privileges in container for your jobs

  • Use privileged: true in the task config.
  1. - task: some-task
  2. privileged: true

Custom icons for your resources

  1. - resources:
  2. - name: my-image
  3. type: registry-image
  4. icon: docker

YAML anchors to re-use configuration blocks

You can use YAML anchor to re-use configuration blocks and remove duplicates.
Often times, you can use file directive but still the anchor syntax can be useful.

  1. # this is an example from concourse docs
  2. # the following repetitive blocks can be shortened using yaml anchor syntax
  3. large_value:
  4. do_the_thing: true
  5. with_these_values: [1, 2, 3]
  6. duplicate_value:
  7. do_the_thing: true
  8. with_these_values: [1, 2, 3]
  9. # look how we anchor a block with &anchor_name syntax and reference it with *anchor_name
  10. large_value: &my_anchor
  11. do_the_thing: true
  12. with_these_values: [1, 2, 3]
  13. duplicate_value: *my_anchor

The same anchor syntax can be used to merge yaml objects.
On the example below, you can see how we can avoid duplicate
AWS ECR configuration by using anchor.

  1. aws-ecr-config: &aws-ecr-config
  2. aws_role_arn: ((ecr-role-arn))
  3. aws_region: ((ecr-region))
  4. aws_access_key_id: ((ecr-access-key-id))
  5. aws_secret_access_key: ((ecr-secret-access-key))
  6. resources:
  7. - name: elixir-1.8.2
  8. type: registry-image
  9. source:
  10. repository: engineering/elixir
  11. tag: 1.8.2
  12. <<: *aws-ecr-config
  13. - name: elixir-1.10.1
  14. type: registry-image
  15. source:
  16. repository: engineering/elixir
  17. tag: 1.10.1
  18. <<: *aws-ecr-config

Here’s another example:

  1. jobs:
  2. - name: job-1
  3. plan:
  4. - &task-1
  5. task: task-1
  6. config:
  7. platform: linux
  8. - &task-2
  9. task: task-2
  10. config:
  11. platform: linux
  12. - name: job-2
  13. - *task-1
  14. - *task-2

Container CPU and Memory Limits for Task

  1. # you can configure and override default container limits
  2. # cpu - max amount of CPU available to task container, measured in shares
  3. # memory - max amount of memory available to task container
  4. # 0 means unlimited
  5. jobs:
  6. - name: container-limits-job
  7. plan:
  8. - task: task-with-container-limits
  9. config:
  10. platform: linux
  11. image_resource:
  12. type: mock
  13. source: {mirror_self: true}
  14. container_limits:
  15. cpu: 512
  16. memory: 1GB
  17. run:
  18. path: sh
  19. args: ["-c", "echo hello"]

Pipeline Organization

While yaml tricks are nice for smaller pipelines, complex pipelines
benefit from a well-defined structure. Check out the command schema
and task step config
that shows an alternative file that allows you to point to a .yml
containing the task config.

A good starting example follows:

  1. techgaun at techgaun in /home/techgaun/projects/rpi-ha
  2. $ tre
  3. 1 .
  4. 2 └── .ci
  5. 3 ├── dockerfiles
  6. 4 ├── Dockerfile.alpine
  7. 5 ├── Dockerfile.buster
  8. 6 └── Dockerfile.distroless
  9. 7 ├── pipelines
  10. 8 ├── k8-build.yml
  11. 9 └── swarm-build.yml
  12. 10 ├── scripts
  13. 11 ├── build
  14. 12 ├── init
  15. 13 └── test
  16. 14 └── tasks
  17. 15 ├── build.yml
  18. 16 ├── e2e.yml
  19. 17 └── test.yml
  20. 18
  21. 19 5 directories, 11 files

Miscellaneous

Build Badges

  • You can get a badge for your pipeline with the following URL:
  1. /api/v1/teams/{team}/pipelines/{pipeline}/badge

Example from Concourse CI itself: Concourse CI Build

  1. # snippet for above SVG
  2. [![Concourse CI Build](https://ci.concourse-ci.org/api/v1/teams/main/pipelines/concourse/badge)](https://ci.concourse-ci.org/teams/main/pipelines/concourse)
  • You can get a badge for your pipeline’s specific jobs with the following URL:
  1. /api/v1/teams/{team}/pipelines/{pipeline}/jobs/{job}/badge

Example from Concourse CI itself: Concourse CI Unit Tests

  1. # snippet for above SVG
  2. [![Concourse CI Unit Tests](https://ci.concourse-ci.org/api/v1/teams/main/pipelines/concourse/jobs/unit/badge)](https://ci.concourse-ci.org/teams/main/pipelines/concourse/jobs/unit)
  • Additionally, Concourse API supports a team’s pipeline in a CCMenu compatible XML file.
  1. /api/v1/teams/{team}/cc.xml

Editor Support

Authors