ECN>> ecs-compose>> 返回
项目作者： fernandosure

项目描述 ：
Deploy services docker-compose style to AWS Elastic Container Service
高级语言： Python
项目地址: git://github.com/fernandosure/ecs-compose.git
创建时间： 2018-04-17T20:57:23Z
项目社区：https://github.com/fernandosure/ecs-compose
开源协议：MIT License
下载


ECS Compose

docker-compose like deployments for AWS Elastic Container Service

Key Features

  • easy to read / maintain deployment scripts
  • deploy multiple services with a single command
  • creates services with private (service-discovery) and internet-facing Application Load Balancers
  • route53 integration when combining an ALB
  • cluster specific or service customizable environment variables

TL;DR

Deploy a complete ECS cluster with a single command:

  1. $ ecs-compose cluster deploy my-cluster -f my-services.yml

You just need to specify the services that will be deployed within the cluster in a YAML file like the following

YAML Configuration

You can form as well a YAML configuration file for use it when creating or updating a cluster definition (eg. POST / PUT /api/clusters).
Basically the YAML stackfile is similar in form to a docker-compose yaml file with the following structure

  1. vpc:
  2.   id: vpc-xxx
  3.   subnets:
  4.     public: ["subnet-xxx", "subnet-xxx"]
  5.     private: ["subnet-xxx", "subnet-xxx"]
  7.   security_groups:
  8.     public: ["sg-xxx"]
  9.     private: ["sg-xxx"]
  11. logging:
  12.   log_driver: awslogs
  13.   options:
  14.     awslogs-group: /ecs/staging
  15.     awslogs-region: us-east-1
  16.     awslogs-stream-prefix: svc
  18. service_discovery:
  19.   namespace: example.sd
  21. defaults:
  22.   memory: 930
  23.   environment:
  24.     - JAVA_OPTS: -Duser.timezone="UTC" -Xms256m -Xmx640m -XX:MaxMetaspaceSize=256m
  25.     - SPRING_PROFILES_ACTIVE: staging
  26.     - LOGGING_LEVEL_ROOT: INFO
  27.     - SERVER_PORT: 80
  29. services:
  31.   - edge-service:
  32.       image: xxx.dkr.ecr.us-east-1.amazonaws.com/edge-service:latest
  33.       ports:
  34.         - "8080:8080"
  35.       environment:
  36.         - SERVER_PORT: 8080
  37.       desired_count: 1
  38.       elb:
  39.         type: public
  40.         protocol: HTTPS
  41.         ports:
  42.           public: 443
  43.           container: 8080
  44.         certificates:
  45.           - arn:aws:acm:us-east-1:xxx:certificate/xxx-xxx-xxx-xxx-xxx
  46.         dns:
  47.           hosted_zone_id: xxxx
  48.           record_name: staging.example.com
  49.         healthcheck:
  50.           protocol: HTTP
  51.           port: 8080
  52.           path: /health
  53.       deployment_configuration:
  54.         maximum_percent: 100
  55.         minimum_healthy_percent: 0
  56.       placement_constraints:
  57.         - expression: "attribute:ecs.instance-type =~ p3.*"
  58.           type: "memberOf"
  59.         - expression: "attribute:ecs.availability-zone == us-east-1a"
  60.           type: "memberOf"
  62.   - user-service:
  63.       image: xxx.dkr.ecr.us-east-1.amazonaws.com/user-service:latest
  64.       ports:
  65.         - "80:80"
  66.       desired_count: 1
  67.       dns_discovery:
  68.         name: user.staging
  70.   - deeplearning:
  71.       image: xxx.dkr.ecr.us-east-1.amazonaws.com/deep-learning:latest
  72.       ports:
  73.         - "8080:8080"
  74.       desired_count: 1
  75.       gpus: 1
  76.       dns_discovery:
  77.         name: deeplearning.staging
  79.  # WORKERS NO PORTS EXPOSED
  80.   - email-worker:
  81.       image: xxx.dkr.ecr.us-east-1.amazonaws.com/email-worker:xxx

VPC Section

  1. vpc:
  2.   id: vpc-xxx
  3.   subnets:
  4.     public: ["subnet-xxx", "subnet-xxx"]
  5.     private: ["subnet-xxx", "subnet-xxx"]
  7.   security_groups:
  8.     public: ["sg-xxx"]
  9.     private: ["sg-xxx"]

vpc

  • id: the VPC id where the ECS cluster is located

vpc.subnets

  • public: subnet-id where the public load balancers (if any) will be placed on
  • private: subnet.id where the private load balancers (if any) will be placed on

vpc.security_groups

  • public: the security group that will be attached to a public load balancer
  • private: the security group that will be attached to private load balancer o service discovery service

Logging Section

  1. logging:
  2.   log_driver: awslogs
  3.   options:
  4.     awslogs-group: /ecs/staging
  5.     awslogs-region: us-east-1
  6.     awslogs-stream-prefix: svc

In this section you specify the logging driver and (optional) the parameters to that specific driver. You can use gelf, awslogs, syslog, etc.

Service Discovery Section

  1. service_discovery:
  2.   namespace: example.sd

In this section you specify the desired private namespace that will be used for registering the services using DNS based Service Discovery. You can’t assign multilevel domains here, only top level domains are allowed e.g. example.local

Defaults Section

  1. defaults:
  2.   memory: 930
  3.   environment:
  4.     - JAVA_OPTS: -Duser.timezone="UTC" -Xms256m -Xmx640m -XX:MaxMetaspaceSize=256m
  5.     - SPRING_PROFILES_ACTIVE: staging
  6.     - LOGGING_LEVEL_ROOT: INFO
  7.     - SERVER_PORT: 80

In this section you need to specify the hard limit (in MiB) of memory to present to the container and the default environment variables that will be applied to all services within the stackfile. The reason behind the global environment section is preventing from copy and paste all over the place instead, if you need to overwrite a specific envvar you can declare that within the service definition and that envvar will have greater precedence over the global one.

Service Section

When a service needs to be publicly exposed in a public subnet.

  1. services:
  2.   - edge-service:
  3.       image: xxx.dkr.ecr.us-east-1.amazonaws.com/edge-service:latest
  4.       ports:
  5.         - "8080:8080"
  6.       environment:
  7.         - SERVER_PORT: 8080
  8.       desired_count: 1
  9.       elb:
  10.         type: public
  11.         protocol: HTTPS
  12.         ports:
  13.           public: 443
  14.           container: 8080
  15.         certificates:
  16.           - arn:aws:acm:us-east-1:xxx:certificate/xxx-xxx-xxx-xxx-xxx
  17.         dns:
  18.           hosted_zone_id: xxxx
  19.           record_name: staging.example.com
  20.         healthcheck:
  21.           protocol: HTTP
  22.           port: 8080
  23.           path: /health

In this section you define an array of services that will be deployed in the cluster.
Each array item corresponds to a different service and you will need to specify the name of the service and then its properties.

  • image: Specify the image to start the container from
  • ports: if the service will expose ports to the outside then you need to specify those in short syntax (HOST:CONTAINER).
  • environment: the environment variable declared in this section have greater precedence over the global definition and will overwrite the global one.
  • desired_count: the number of desired instances for the service
  • gpus: the number of gpu’s assigned (when using gpu instances)
  • placement_constraints: rules that are considered during task placement.
    • expression: A cluster query language expression to apply to the constraint. You cannot specify an expression if the constraint type is distinctInstance.
    • type: The type of constraint. Use distinctInstance to ensure that each task in a particular group is running on a different container instance. Use memberOf to restrict the selection to a group of valid candidates.
  • deployment_configuration:
    • maximum_percent: represents an upper limit on the number of your service’s tasks that are allowed in the RUNNING or PENDING state during a deployment, as a percentage of the desired number of tasks (rounded down to the nearest integer).
    • minimum_healthy_percent: represents a lower limit on the number of your service’s tasks that must remain in the RUNNING state during a deployment, as a percentage of the desired number of tasks (rounded up to the nearest integer).
  • elb: the application load balancer definition.
    • type: whether if the load balancer will be public (will be placed in the public subnet and a public security group assigned) or private (will be place in the private subnet and a private security group assigned)
    • protocol: The protocol for connections from clients to the load balancer (HTTP/HTTPS)
    • ports:
      • public: The port on which the load balancer is listening
      • container: The port on which the target container will receive traffic (this will need to match the public exposed port of the container)
    • certificates: a single item array of the AWS ACM (certificate manager) arn of the certificate to be applied to the ALB
    • dns:
      • hosted_zone_id: route53 hosted_zone_id for updating the CNAME record that the load balancer will listen requests from.
      • record_name: the route53 recordset of from which the ALB will listen requests from (if isnt already created it will create it automatically otherwise will update the recordset)
    • healthcheck: the configurations the load balancer uses when performing health checks on targets.
      • protocol: The protocol the load balancer uses when performing health checks on the container target.
      • port: The port the load balancer uses when performing health checks on the container target.
      • path: The ping path that is the destination on the targets for health checks. The default is /.

When a service only needs to be reachable by other services that are part of the same cluster.

  1. services:
  2.   - user-service:
  3.       image: xxx.dkr.ecr.us-east-1.amazonaws.com/user-service:latest
  4.       ports:
  5.         - "80:80"
  6.       desired_count: 1
  7.       dns_discovery:
  8.         name: user.staging
  • dns_discovery: When using this feature make sure that you already declared the service discovery namespace that will be used to register the services.
    • name: the name of the route53 recordset to be used by other services to reach this service (it will create an ‘A’ record) if you want to use the same namespace for different clusters make sure to split the name of the service into a multilevel domain (e.g. <service_name>.<cluster_name/environment>.<namespace>)

When a service doesn’t need to be exposed (worker services)

  1.  - email-worker:
  2.      image: xxx.dkr.ecr.us-east-1.amazonaws.com/email-worker:xxx

There aren’t any exposed ports and no load balancer / service discovery configuration. Only the container image definition and if required (desired_count)

Installation

The project is available on PyPI. Simply run::

  1. $ pip install ecs-compose

Configuration

The mechanism in which ecs-compose looks for credentials is to search through a list of possible locations and stop as soon as it finds credentials.

  • Environment variables
  • Shared credential file (~/.aws/credentials)
  • AWS config file (~/.aws/config)
  • Assume Role provider
  • Boto2 config file (/etc/boto.cfg and ~/.boto)
  • Instance metadata service on an Amazon EC2 instance that has an IAM role configured.

Please read the boto3 documentation for more details
(http://boto3.readthedocs.org/en/latest/guide/configuration.html#configuration).

Or just run::

  1. $ aws configure

Actions

Currently the following actions are supported:

Cluster related operations

deploy

deploy / redeploys a single or multiple services at once defined in the YAML stackfile

destroy

Destroy the entire AWS ECS Cluster with all services and attached load balancers associated with it.

describe

List all deployed services within the specified cluster as YAML stackfile

Individual service related operations

destroy

Destroy an individual service within the specified cluster with its load balancer associated with it.

Usage

For detailed information about the available actions, arguments and options, run::

  1. $ ecs-compose --help
  2. $ ecs-compose cluster --help
  3. $ ecs-compose service --help