Retrieving AWS Metadata from Environment Variables

When working with AWS EC2, it’s often handy to be able to reference certain information about an instance. The obvious solution is the AWS Metadata service, accessible with a simple cURL command. For example, to get the private IP address of an instance:

curl -s http://169.254.169.254/latest/meta-data/local-ipv4
172.1.2.3

Since I hate writing boilerplate code, and don’t need to make an HTTP request each time I want to know something about my environment, I put together a Docker image that fetches most of the available metadata and outputs environment variable settings. Also, I’ve been itching to write something in Go.

My primary use case is to create /etc/aws.env when a new CoreOS instance starts, but this should work on any system with systemd. To do this, aws-env.service should be installed and configured to run at startup. One way to do this is via cloud-config (EC2 User Data):

#cloud-config
coreos:
  units:
    - name: aws-env.service
      command: start
      enable: yes
      content: |
         [Unit]
         # Contents from 'aws-env.service' go here

Alternatively, you can use it with eval:

eval $(docker run --rm -t cmattoon/aws-env)

GitHub: https://github.com/cmattoon/aws-env

DockerHub: https://hub.docker.com/r/cmattoon/aws-env/

Retrieving AWS Metadata from Environment Variables

Auto Scaling Group LifecycleHooks, SQS, and IAM

AWS offers Lifecycle Hooks for AutoScaling Groups that allow you to respond to a change in instance state. For example, publishing create/terminate events to an SQS queue:

aws autoscaling put-lifecycle-hook \
 --lifecycle-hook-name "${HOOK}_launching" \
 --auto-scaling-group-name "$ASG_NAME" \
 --notification-target-arn "$SQS_ARN" \
 --role-arn "$ROLE_ARN" \
 --lifecycle-transition "autoscaling:EC2_INSTANCE_LAUNCHING"

aws autoscaling put-lifecycle-hook \
 --lifecycle-hook-name "${HOOK}_terminating" \
 --auto-scaling-group-name "$ASG_NAME" \
 --notification-target-arn "$SQS_ARN" \
 --role-arn "$ROLE_ARN" \
 --lifecycle-transition "autoscaling:EC2_INSTANCE_TERMINATING"

The instructions for doing so are pretty straightforward, but I ran into an irritating error:

An error occurred (ValidationError) when calling the PutLifecycleHook operation: Unable to publish test message to notification target arn:aws:sqs:us-west-2:123456:my-sqs-queue.fifo using IAM role arn:aws:iam:1234:role/my-asg-role. Please check your target and role configuration and try to put lifecycle hook again.

All of the search results for that error turned up solutions involving incorrect IAM policies. This should not be the case if you simply add the AutoScalingNotificationAccessRole per the instructions. For reference, the correct settings are below.

In my case, however, it turns out that AutoScaling can’t publish to a FIFO queue. Recreating the queue as a standard queue fixed this problem for me.

arn:aws:iam::aws:policy/service-role/AutoScalingNotificationAccessRole:

{
    "Version": "2012-10-17",
    "Statement": [{
         "Effect": "Allow",
         "Resource": "*",
         "Action": [
             "sqs:SendMessage",
             "sqs:GetQueueUrl",
             "sns:Publish"
         ]
    }]
}

You’ll also want to verify that a Trust Relationship exists on your Role that allows the autoscale service to assume said role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "autoscaling.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

 

Auto Scaling Group LifecycleHooks, SQS, and IAM