slash-infra

This is a slack integration for “chatops”. A lot of the hubot scripts
don’t integrate natively with slack’s slash commands, or give you
unnecessarily powerful commands (e.g. launching individual ec2 instances,
or creating auto-scaling groups), or only work against one AWS account.

This tool came from a need to lookup EC2 instances by their IP/instance
ID. Our production/staging/dev environments are hosted in different AWS
accounts, and some are hosted in different regions. Switching between
accounts and regions to track down an instance can be quite laborious
and error prone, especially if you’re under pressure trying to triage a
problem.

/infra-search {query} can search multiple AWS accounts to find
resources. Currently it only supports looking up instances by their
instance ID.

Configuring Slack

• Create a slack app
• export the signing secret (from the App Credentials section of the
  app’s basic information) as the environment variable
  SLACK_SIGNING_SECRET
• Configure slash commands to point at the routes specified in
  server.go

Configuring AWS access

Rather than create a user in each AWS account, the app uses a limited
IAM user to assume roles in each AWS account that should be searched.

This approach seems convoluted, but there are several benefits.

Firstly, when an IAM user “assumes” a role, AWS generates a set of temporary
credentials for the user that have the same permissions as the role.
These credentials are short-lived, and are rotated transparently by
the AWS SDK. If these temporary credentials were leaked to a
third party, they would only be usable for a short period of time.

Secondly, if the credentials for the IAM user are leaked, the attacker
will only be able to assume IAM roles. If they do not know the ARN of
your role they will not be able to assume it, and thus won’t be able to
perform actions on your account. You then only need to rotate one set of
credentials, rather credentials for all of your AWS accounts.

Note that if you’re using the role name suggested in these docs then
they will likely be able to guess the full ARN, as you can always get
the ID of the AWS account credentials belong to using aws sts get-caller-identity. If this is a concern for you, you can make things
slightly more difficult for attackers by choosing a unique name for your
roles that are different to the name of the IAM user. slash-infra only
uses the role ARN to authenticate to AWS, so you could use a different
role name for each environment.

You can configure the IAM user using the conventional environment
variables:

export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...

This IAM user should have the following permission policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowAssumingSlashInfraRoles",
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::*:role/SlashInfraInspection"
        }
    ]
}

Note that the * in the ARN allows this user to assume the
SlashInfraInspection role in any AWS account that:

• has that role
• has marked the IAM user’s AWS account ID as a “Trusted entity” in the
  role’s “Trust relationships”

If these are new concepts for you, I’d really recommend reading AWS’
documentation on IAM
roles

Each role in each account should have a permission policy like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowReadOnlyAccess",
            "Effect": "Allow",
            "Action": "ec2:DescribeInstances",
            "Resource": "*"
        }
    ]
}

You can then configure slash-infra to use the role via environment
variables:

export AWS_ROLE_{role alias}=arn:aws:iam:.....:role/SlashInfraInspection
# The region defaults to `us-east-1` if left unspecified
export AWS_REGION_{role alias}=eu-west-2
# e.g.
export AWS_ROLE_PRODUCTION=arn:aws:iam:.....:role/SlashInfraInspection
export AWS_REGION_PRODUCTION=eu-west-2

If you need to search multiple regions within a single account you can
create several aliases that use the same role ARN.

Testing locally

Download ngrok, and create a slack
app in your slack workspace. Create slash
commands in the app for the commands you want to support (see server.go).

FAQ

Why not write this in a lambda?

• I don’t know how to write lambdas. I wrote this in 20% time and didn’t
  want to spend my day learning how to deploy lambdas
• We don’t mind spending the few $X heroku charge to run this on a hobby
  dyno