theKindOfMe

October 22, 2010

Getting Started with EC2

Filed under: Uncategorized — Tags: , , — yasi8h @ 10:06 am

Amazon EC2 allows you to create and manage server instances in the cloud with ease. They are scalable and easy to setup. There are many AMIs (Amazon machine image (AMI)) that you can use to get started. AMIs are basically server images that contain a whole OS installation and more. For example some AMI’s may have a web server with php setup. You can think of it as a virtual host image. This makes things much easier in the long run. You can use one of many AMIs out there to get started with your new server, rather than having to start from scratch. You can also build your own AMIs.

Other than AMIs you have AKIs and ARIs. AKI stands for amazon kernel image and ARI stands for amazon ramdisk image. You can lunch an AMI with a AKI of your choice. Obviously these should match and you should be sure that they work together.

Interfacing with AWS

There are several ways you can interface and communicate with AWS. When it comes to EC2 for example you could use the AWS Management console, the command line tools or the API. If you are a human you would prefer using the management console which is a web interface or the command line tools.

AWS Management console is a web application and its pretty easy on newbies. Where as command line tools have a ton of commands that you take you sometime to memorize (if you can memorize them at all). However thanks to grate documentation from Amazon it shouldn’t be hard to find help with any interface that you choose to use.

Setting up the Command Line Tools

You will need to download and setup the command line tools on your machine. Command line tools are written in java so you will need java to make use of them. You will also have to setup some environment variables. After setting up the command line tools you will have to download your ec2 private key and the certificate and put them some where safe. You will also need to setup some environment variables to point to these.

For complete up to date instructions on how to do this on your OS please refer to http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/SettingUp_CommandLine.html

After setting everything up my environmental variables were set like the following:

export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Home

export EC2_HOME=~/.ec2
export PATH=$PATH:$EC2_HOME/bin

export EC2_PRIVATE_KEY=~/.ec2/pk-LJLKJLDKFSD42DFS3DSFSDFDSFSD.pem
export EC2_CERT=~/.ec2/cert-JHJDSFJHFSDJKFHEUFHSJDFJ3DF3DFS98DF.pem

Setting up a SSH Key Pair

You will need to setup a ssh key pair so that you use it to login to server instances you lunch. You can create the key pair your self or let amazon do it for you. Refer the the guide for step by step instructions on how to do this.

Launching a Instance

Doing this with the AWS Management console is pretty straight forward. So i will focus on how to do it with the command line tools. Keep in mind that you get billed for the time that you are running your instances. So if you are only playing around EC2, be sure to terminate any instances you lunch after you are done.

Finding a suitable AMI

You can use the ec2-describe-images command to find a image that suites you. However just executing this command with no options will give you nothing. ec2-describe-images -a will give you all the available images. But be warned that its a lot of images, so it will take some time.

ec2-describe-images -o amazon –filter image-type=machine

would give you a list of images of machine type owned by amazon. For a full reference of all the available options for ec2-describe-images refer http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-DescribeImages.html

You can use grep to filter out the results and find interesting images.

ec2-describe-images -a –filter image-type=machine | grep rails | more

However i find it easier to use to AWS management console (AWSMC from here onwards) much easier for this task.

Whatever image you choose to lunch remember its id. As you need that to lunch the image. I am going to go with

ami-6b26ca02 bitnami-cloud/rubystack/bitnami-rubystack-2.0-3-rails3-linux.manifest.xml

Root Devices

When you are choosing an AMI you would notice that some of them has their root device as instance store while some of them has it as EBS (elastic block storage). The basic difference between these instances are that if one uses the instance store, it will basically loose all its data when you terminate it. However a image that uses EBS will remember its state. Note that whats in the ram will be destroyed in both cases.

Security Groups

One thing that confused me the most when i started working on a server that was hosted on ec2 was the existence of security groups. You can think of them as firewall policies, sets of rules on what ports are to be allowed and not on a given group. By default every port is blocked.

Its a good idea to add a security group before lunching any instances as you will have to specify a security group for the instance that you are lunching. And without specifying one you won’t be able to access the instance through ssh.

You can use ec2-describe-group to find out what groups you already have, and what the configurations are on those. In my case i was using my companies AWS account and there were some existing groups. The output looked something like the following.

yazs-MacBook-Pro:~ yasi8h$ ec2-describe-group
GROUP 139637636613 default default group
PERMISSION 139637636613 default ALLOWS all FROM USER 139637636613 GRPNAME default
PERMISSION 139637636613 default ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0
PERMISSION 139637636613 default ALLOWS tcp 80 80 FROM CIDR 0.0.0.0/0
GROUP 139637636613 Web Server Incoming HTTP
PERMISSION 139637636613 Web Server ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0
PERMISSION 139637636613 Web Server ALLOWS tcp 80 80 FROM CIDR 0.0.0.0/0
PERMISSION 139637636613 Web Server ALLOWS tcp 443 443 FROM CIDR 0.0.0.0/0
GROUP 139637636613 SSH Only Only allow incoming SSH
PERMISSION 139637636613 SSH Only ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0

There are three groups here and they all allow ssh from the whole of internet. If you want to create a security group you will need to use ec2-add-group.

ec2-add-group ladygaga -d “strictly for gaga fans only”

the above command will create a group called ladygaga with a description. Note that you can always use the default group rather than creating new ones.

ec2-authorize ladygaga -p 22 would open up port 22 (used for ssh).

Running an Instance

ec2-run-instances ami-6b26ca02 -k yf-keypair

this would run a instance with the specified AMI. The k option here specifies the ssh key pair we want to use to authenticate our self’s when logging in to the instance. When you execute this command you should see some output like the following.

yazs-MacBook-Pro:~ yasi8h$ ec2-run-instances ami-6b26ca02 -k yf-keypair
RESERVATION r-e3915f89 139637636613 default
INSTANCE i-8833e3e5 ami-6b26ca02 pending yf-keypair 0 m1.small 2010-10-22T07:53:33+0000 us-east-1c monitoring-disabled instance-store

The second field have the id for the instance we just lunched. We can use this to get the status of our instance.

ec2-describe-instances i-8833e3e5

the output would be something like

yazs-MacBook-Pro:~ yasi8h$ ec2-describe-instances i-8833e3e5
RESERVATION r-e3915f89 139637636613 default
INSTANCE i-8833e3e5 ami-6b26ca02 ec2-184-73-56-151.compute-1.amazonaws.com domU-12-31-39-09-25-31.compute-1.internal running yf-keypair 0 m1.small 2010-10-22T07:53:33+0000 us-east-1c monitoring-disabled 184.73.56.151 10.210.42.191 instance-store paravirtual

We can see that my instance is running now!. It might take a minute or two for your instance to be up and running. So give it sometime.

Wait, i think i forgot to tell AWS to assign this instance to the ladygaga security group. So lets terminate it and re launch it with the correct group assigned.

ec2-terminate-instances i-8833e3e5

Output

INSTANCE i-8833e3e5 running shutting-down

Note that terminating instances who’s root device is a instance node will make it loose all data. If you just need to restart your instance use

ec2-reboot-instances i-8833e3e5

Lets verify that its terminated

ec2-describe-instances i-8833e3e5

Output

yazs-MacBook-Pro:~ yasi8h$ ec2-describe-instances i-8833e3e5
RESERVATION r-e3915f89 139637636613 default
INSTANCE i-8833e3e5 ami-6b26ca02 terminated yf-keypair 0 m1.small 2010-10-22T07:53:33+0000 us-east-1c monitoring-disabled instance-store paravirtual

We can see that the instance is been terminated. Let’s lunch another and assigning it a security group.

ec2-run-instances ami-6b26ca02 -k yf-keypair -g ladygaga

After you have verified that it has started correctly. We can log in using SSH. To access it using SSH we need to know it’s public DNS name or the public ip. We can get this information by running

ec2-describe-instances i-f03eee9d

Output

yazs-MacBook-Pro:~ yasi8h$ ec2-describe-instances i-f03eee9d
RESERVATION r-95aa64ff 139637636613 ladygaga
INSTANCE i-f03eee9d ami-6b26ca02 ec2-184-72-142-168.compute-1.amazonaws.com ip-10-122-53-159.ec2.internal running yf-keypair 0 m1.small 2010-10-22T08:08:05+0000 us-east-1a monitoring-disabled 184.72.142.168 10.122.53.159 instance-store paravirtual

We can see that the public DNS name for this instance is ec2-184-72-142-168.compute-1.amazonaws.com. We can connect to it through ssh. But we will have to use the key pair we used when we initially created the instance, to authenticate our self’s.

yazs-MacBook-Pro:~ yasi8h$ ssh -i .ec2/yf-keypair.pem root@ec2-184-72-142-168.compute-1.amazonaws.com

for most systems you can login as root. But for some systems (some ubuntu AMIs require you to login as ubuntu) this can be different.

Now that you are logged in you can configure the system and have a lot of fun!

Availability Zones and Regions

If AWS EC2 is the cloud it consists of regions. These regions are located in different parts of the world. And within the regions you have multiple availability zones. If your server carters mainly for visitors from a specific region you could run your instance in that region. And to improve the availability of your application you could have instances of your application running in multiple availability zones. So in case one availability zone goes down, your application can still survive if its hosted on other availability zones.

You can view the regions that are available to your account by executing

yazs-MacBook-Pro:~ yasi8h$ ec2-describe-regions

Output

REGION eu-west-1 ec2.eu-west-1.amazonaws.com
REGION us-east-1 ec2.us-east-1.amazonaws.com
REGION us-west-1 ec2.us-west-1.amazonaws.com
REGION ap-southeast-1 ec2.ap-southeast-1.amazonaws.com

For a list of availability zones for a region

ec2-describe-availability-zones –region=eu-west-1

Output

AVAILABILITYZONE eu-west-1a available eu-west-1
AVAILABILITYZONE eu-west-1b available eu-west-1

You can specify which availability zone your instance should be in, when you are launching it.

ec2-run-instances ami-6b26ca02 -k yf-keypair -g ladygaga –availability-zone us-east-1c

Thats it. I learned a lot by writing this. Note that my main source of information for this blog post was the AWS official documentation. The official documentation seems to be very good. So if you ever need to know something google it up. If you need to get more information on the command line tools and the different arguments that they accept refer the command line reference here (http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference/index.html).

And IMHO if you are new to AWS, its best to try things out using the AWSMC first before digging in to the command line. But once you get comfortable with using it. You could just use a combination of guessing, command line reference mentioned above and refer the in built help (ec2-command –help) to find your way through the command line tools.

Advertisements

Create a free website or blog at WordPress.com.