trigger aws codebuild projects from Jenkinsfile
This is a Jenkins pipeline that builds CodeBuild projects in AWS cloud, then runs these CodeBuild projects.
The CodeBuild projects are building infrastructure within AWS cloud.
The pipeline is doing the following:
Have Docker installed. We’ll be running Jenkins on a Docker container.
Spin off a Jenkins docker container with a named volume to preserve jenkins configuration and pipeline for future use:docker-compose -f jenkins/docker-compose.yaml up --detach
AWS account at https://console.aws.amazon.com/.
AWS access key and secret to be used by Terraform and Jenkins.
AWS Role to be assumed by Terraform and Jenkins.
AWS s3 bucket (terraform backend) and dynamodb table for terraform state lock management.
Create these resources following these steps:
cd prerequisites
terraform init --var-file="../../terraform.tfvars"
terraform plan --var-file="../../terraform.tfvars" -out terraform.tfplan
terraform apply "terraform.tfplan"
# or
cd prerequisites
terraform init --var-file="../../terraform.tfvars"
terraform apply -input=false -auto-approve --var-file="../../terraform.tfvars"
For the steps above, AWS access key and access secret key should be stored in a terraform.tfvars file.
There is a sample with the contents of this file in the main directory of the repository.
Once the prerequsites resources are built, the details of these resources will be shown in the terraform output.
Take these details and populate the related fields in these files:
Go through Jenkins installation steps at: http://localhost:8090.
Define these secrets in Jenkins:
AWS credentials inside Codebuild projects:
Create Jenkins pipeline job with default settings using Pipeline script from SCM with URL https://github.com/andreistefanciprian/jenkins_aws_codebuild.git.
Run pipeline job!
# destroy terraform s3 bucket and dynamodb table used for tfstate management
cd prerequisites
terraform destroy --var-file="../../terraform.tfvars"
# destroy AWS resources (AWS creds to be stored in .env file prior to run these commands)
cd terraform_code
make destroy-auto-approve TF_TARGET=infra
make destroy-auto-approve TF_TARGET=static
make destroy-auto-approve TF_TARGET=codebuil
# spin down Jenkins docker container:
docker-compose -f jenkins/docker-compose.yaml down
Use these AWS CLI commands to manually interact with CodeBuild:
# list CodeBuild projects and builds
aws codebuild list-projects
aws codebuild list-builds
# start CodeBuild project
aws codebuild start-build --project-name codebuildtest-MessageUtil
aws codebuild start-build --project-name newproj-test
# list CodeBuild jobs for specific project
aws codebuild list-builds-for-project --project-name codebuildtest-MessageUtil
# get last build for project
aws codebuild list-builds-for-project --project-name codebuildtest-MessageUtil --query 'ids[0]' --output text
aws codebuild batch-get-builds --ids codebuildtest-MessageUtil:f0682dfe-2d7e-4bec-8061-2008843089e7
# query status of last Codebuild build
build_id=$(aws codebuild list-builds-for-project --project-name codebuildtest-MessageUtil --query 'ids[0]' --output text)
aws codebuild batch-get-builds --ids $build_id --query 'builds[0].buildStatus' --output text
Use these commands to manually check python script:
# create python3 virtual env
python3 -m venv .venv
# activate environment
source .venv/bin/activate
# install requirements
pip install -r requirements.txt
# execute script
aws_account=9238748923565
python execute_codebuild_from_yaml.py $aws_account
Use these commands to verify you can build resources with terraform from CLI:
TF_VAR_TARGET=static
docker-compose run terraform init $TF_VAR_TARGET
docker-compose run terraform plan -out terraform.tfplan $TF_VAR_TARGET
docker-compose run terraform apply terraform.tfplan
docker-compose run terraform destroy -auto-approve $TF_VAR_TARGET
# using make commands
make deploy-auto-approve TF_TARGET=$TF_VAR_TARGET
make destroy-auto-approve TF_TARGET=$TF_VAR_TARGET