DO NOT USE - MIGRATED TO GITLAB

dataworks-aws-ingestion-ecs-cluster

Shared EC2 backed ECS cluster in DataWorks ingestion VPC

This repo contains Makefile and base terraform folders and jinja2 files to fit the standard pattern.
This repo is a base to create new Terraform repos, renaming the template files and adding the githooks submodule, making the repo ready for use.

Running aviator will create the pipeline required on the AWS-Concourse instance, in order pass a mandatory CI ran status check. this will likely require you to login to Concourse, if you haven’t already.

After cloning this repo, please generate terraform.tf and terraform.tfvars files:
make bootstrap

In addition, you may want to do the following:

1. Create non-default Terraform workspaces as and if required:
   make terraform-workspace-new workspace=<workspace_name> e.g.
   make terraform-workspace-new workspace=qa

2. Configure Concourse CI pipeline:

   1. Add/remove jobs in ./ci/jobs as required
   2. Create CI pipeline:
      aviator

Infrastructure

This repo builds an AWS ECS cluster named ingestion onto which the claimant-api-kafka-consumer application is deployed.
The ECS task and service is defined separately in the Dataworks AWS UCFS Claimant Consumer repo.