Auto Scaling Group LifecycleHooks, SQS, and IAM

AWS offers Lifecycle Hooks for AutoScaling Groups that allow you to respond to a change in instance state. For example, publishing create/terminate events to an SQS queue:

aws autoscaling put-lifecycle-hook \
 --lifecycle-hook-name "${HOOK}_launching" \
 --auto-scaling-group-name "$ASG_NAME" \
 --notification-target-arn "$SQS_ARN" \
 --role-arn "$ROLE_ARN" \
 --lifecycle-transition "autoscaling:EC2_INSTANCE_LAUNCHING"

aws autoscaling put-lifecycle-hook \
 --lifecycle-hook-name "${HOOK}_terminating" \
 --auto-scaling-group-name "$ASG_NAME" \
 --notification-target-arn "$SQS_ARN" \
 --role-arn "$ROLE_ARN" \
 --lifecycle-transition "autoscaling:EC2_INSTANCE_TERMINATING"

The instructions for doing so are pretty straightforward, but I ran into an irritating error:

An error occurred (ValidationError) when calling the PutLifecycleHook operation: Unable to publish test message to notification target arn:aws:sqs:us-west-2:123456:my-sqs-queue.fifo using IAM role arn:aws:iam:1234:role/my-asg-role. Please check your target and role configuration and try to put lifecycle hook again.

All of the search results for that error turned up solutions involving incorrect IAM policies. This should not be the case if you simply add the AutoScalingNotificationAccessRole per the instructions. For reference, the correct settings are below.

In my case, however, it turns out that AutoScaling can’t publish to a FIFO queue. Recreating the queue as a standard queue fixed this problem for me.

arn:aws:iam::aws:policy/service-role/AutoScalingNotificationAccessRole:

{
    "Version": "2012-10-17",
    "Statement": [{
         "Effect": "Allow",
         "Resource": "*",
         "Action": [
             "sqs:SendMessage",
             "sqs:GetQueueUrl",
             "sns:Publish"
         ]
    }]
}

You’ll also want to verify that a Trust Relationship exists on your Role that allows the autoscale service to assume said role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "autoscaling.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

 

Auto Scaling Group LifecycleHooks, SQS, and IAM

Install R Packages on Shiny Server Pro

If you’ve installed Shiny Server Pro as a user other than shiny, you might have experienced difficulty adding R packages. This is because Shiny Server Pro runs R as the shiny user, and running R -e “install.packages(‘foo’)” will install packages to the local user’s files only.

The solution to this is to su to the shiny user:

su - shiny

And run

R -e "install.packages('foo', repos='http://cran.rstudio.com')"

Alternatively, this script will parse an R file looking for require statements and install the necessary packages. It isn’t very smart, so be careful.

Install R Packages on Shiny Server Pro

Puppet: Error 400 on SERVER: undefined method `empty?’ for nil:NilClass

I received this error after making some changes to a Hiera config and the referenced “dev-server” role.

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Error from DataBinding 'hiera' while looking up 'role::dev-server::use_ssl': undefined method `empty?' for nil:NilClass on node servername.local

Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

It turns out this is a vague syntax error. Checking the following has worked for me:

  • Ensuring the syntax of your Hiera YAML or JSON file is correct. Check for trailing commas in JSON, or misplaced colons. (“foo:bar”, “foo::bar:”, “foo:::bar”, etc.)
  • The variable name is unique. In one case, “dev-server::use_ssl” was configuring a child resource with the same “use_ssl” property/param/variable.
  • There are no empty YAML or JSON files in your hieradata directory. I think I’ve had a similar issue with temp files (*~)
  • If you’ve modified your hiera.yaml to add a new hierarchy or something, restart Puppet.
Puppet: Error 400 on SERVER: undefined method `empty?’ for nil:NilClass