How to automatically install an .exe into a remote AWS EC2 instance (part 2)

Hi,

In this post I will show you how to run the PowerShell command from part 1 on all EC2 instances.

1. Connect to AWS Console using your own credentials.
2. Go to Compute -> EC2 and select Command History

3. Select Run a command.
4. Choose AWS-RunPowerShellScript.

       5. Select an instance from the instance list. If the desired instance is not in the list then jump down to SSM Run Command Prerequisites section for more pieces of information.

6. Copy paste the PowerShell script in the Commands field (you can take the script from part 1 of this article which can be found here)

  7. Run the Command.

 

SSM Run Command Prerequisites:

Make sure the following steps are verified such that an EC2 is ready for SSM:

1. Check that the EC2 instance has an IAM role associated to it.
2. If it doesn’t then this means that the SSM Service can’t be used on that instance because it requires a role and you can’t add a role to an already launched instance. In this case you need to create an AMI from the existing EC2 instance and then launch a new instance from that AMI. During the launch process you will be able to attach a role to the new instance.
3. If the EC2 instance has a role then you need to make sure the SSM Policy is attached to that role (Roles -> Policies -> Attach Policy -> AmazonEC2RoleForSSM).
4. Install the latest version of EC2 Service on the EC2 instance:(http://aws.amazon.com/developertools/5562082477397515)

Useful links:

1. http://docs.aws.amazon.com/ssm/latest/APIReference/Welcome.html
2. http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/walkthrough-ui.html
3. http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/remote-commands-prereq.html

 

 

 

How to automatically install an .exe into a remote AWS EC2 instance (part 1)

Hi,

If one needs to have an .exe installed on a EC2 instance in AWS, there are a few ways to do that:

1. Manually connect to each remote machine and install that .exe – easy but time consuming if there are 10 instances behind a load balancer
2. Create a PowerShell script that can be run on all EC2 instances in the AWS Cloud – more trickier to implement in the first place but really helpful afterwards

I would go for option 2 because the benefits are substantial: centralized repository for the kits, copying/installing a kit on as many instances as you want with a touch of a button.

The architecture for the second scenario is described below:

 

1. Upload the .exe into the S3 bucket.
2. Write a PowerShell script that downloads the .exe from the bucket into each of the EC2 instance if it has not been downloaded yet.
3. Install the .exe on each instance if it hasn’t been installed yet.

The code for the PowerShell script is listed below:

# copy the .exe from the S3 bucket to the EC2 instance
$isEXEKitOnEC2Instance = Test-Path “C:\kits\yourApp.exe”
if ($isEXEKitOnEC2Instance -eq $false)
{
echo “Copying the kit from the S3 bucket to the EC2 instance.”
Copy-S3Object -BucketName aws-exe-bucket -Key yourApp.exe -LocalFile c:\kits\yourApp.exe
# delay the next command with 10 seconds (this will give time for the .exe to be copied)
Start-Sleep -s 10
echo “The ,exe has been copied to the EC2 instance.”
}
else
{
echo “The kit is already on the EC2 instance and will not be copied anymore.”
}

# check if the .exe is already installed on the machine
$isExeInstalled = Test-Path “C:\installation_path\yourApp.exe”
if ($isExeInstalled -eq $false)
{
echo “The .exe is not installed. Install is in progress…”
# run the .exe on the remote machine in silent mode
Invoke-Expression “c:\kits\yourApp.exe /S”
}
else
{
echo “The .exe is already installed.”
}

In part 2 I will show you how this script will run on all selected AWS instances.

AWS and Security Groups

I was asked to allow different IP addresses to gain access to a specific machine from AWS. Initially I thought: oh well, this is quite easy to do from AWS Console -> Security Groups -> Inbound -> Edit -> Type the new IP address and hit OK.

After taking a closer look I realized that this Security Group was shared by other AWS instances and this new rule would have been used by all the shared instances which was not desirable.

OK, in this case I had to take a different approach: changing the Security Group for that machine. I went to the menu associated to that instance and changing the Security Group option was disabled.

The reason was that I was using an AWS Classic Instance which does not allow a security group to be changed after its launched. Using a VPC one can re-assign the security group once it’s being launched and one will have more flexibility for modifying security groups settings in general.

The best solution in this case was to create a new instance and assign a new Security Group to it.

The steps I had to take are described below:

– Create an AMI of the existing instance to preserve the data on it

– Setup all the settings for this new AMI

– Create a new instance using that AMI

– Terminate the old instance

–  Associated the Elastic IP of the old instance to the new instance

This is a small drawback that AWS has when it comes to AWS Classic instances. Hopefully in the future one will simply change the Security Group without going through the whole hassle of creating new instances and dropping others.

AWS: Pre-warming the Load Balancer

Do you expect a spike in traffic? Let’s say your stakeholders expect a ramp of 20.000 users in the first minutes of your website. How do you handle a scenario where you expect tens of thousands of users in the first minutes since the launch of your website? This is a great example of handling fault tolerance in AWS.

If one wants to achieve fault tolerance in AWS, there a few options to do that:

1. Use a Load Balancer – no matter how much the traffic increases, if you place your instances behind a Load Balancer it is always a great idea because the traffic is balanced across all the healthy instances.
2. Use an Auto Scaling Group – this that can scale up/down with as many instances as you want, this is a really powerfull feature of AWS that one can use.

The ELB is designed to handle large loads of traffic (20kb/sec) without a problem when this traffic increases gradually over a long period of time (several hours). However, when you expect high increase in traffic over a short period of time, then you face a problem.

AWS considers that if the traffic increases more than 50% in less than 5 minutes then it means that the traffic is sent to the load balancer at a rate that increases faster than the ELB can scale up to meet it. What can you do in such cases?

Well, one needs to contact AWS to do an operation called “pre-warming”. What does that mean? This means that the AWS tech guys will configure the Load Balancer to have an appropriate level of capacity based on the expected traffic. There is a full list of answers that the AWS guys need in order to do that and I share that list below with some of the values we used already for this operation:

1. Traffic delta or request rate expected at surge(in Requests Per Second)

2. Average amount of data passing through the ELB per request/response pair (In Bytes)

3. Rate of traffic increase i.e. % increase over a time period

4. Are keep-alives used on the back-end?

5. Percent of traffic using SSL termination on the ELB

6. Number of Availability Zones that will be used for this event/load balancer

7. Is the back-end scaled to event/spike levels? [Y/N] [If N, when will you scale the back-end? and how many and what type of back-end instances

8. Start date/time and timezone for elevated traffic patterns

9. End date/time and timezone for elevated traffic patterns

10. A brief description of your use case. What is driving this traffic? (e.g. application launch, event driven like marketing/product launch/sale, etc)

One important thing: you need to send this list 36 hours in advance so that AWS has enough time to process your request.

 

AWS: Timeout issues

Hi,

In this post I will talk about the code updates triggered from Team City that were failing in AWS. This caused the updates in AWS to be rolled back.

The errors I got were:

Updating Auto Scaling group named: MyScalingGroup failed. Reason: Failed to receive 1 resource signal(s) within the specified duration

and

Instance id(s) ‘my-instance-id’ did not pass health check after command execution. Aborting the operation.

Initially I believed that these have happened because the instances failed to finish their health checks. I tried a few things to solve this issue:

1) Made a Service Role for the environment.

2) Set the Health Target to TCP:80.

3) Increased the timeout value for health-checks.

Unfortunately none of these seemed to work.

After many hours spent on investigating this issue I decided to also look in the Load Balancer settings and in the Auto Scaling Group settings and I noticed a very interesting behavior.

The Load Balancer was using 2 Availability Zones (1b and 1c) while the Auto Scaling Group was using 3 Availability Zones (1a, 1b,1c). That meant that every new instance that was spun off by the Auto Scaling Group was in the AZ that was not associated to the Load Balancer (1a). This has led to the issue that AWS could not perform the heath checks because it couldn’t reach that instance at all.