1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348
|
= Amazon Web Services Elastic Compute Cloud (EC2) Ruby Gem
== About amazon-ec2
Amazon Web Services offers a compute power on demand capability known as the Elastic Compute Cloud (EC2). The server resources in the cloud can be provisioned on demand by making HTTP Query API calls to EC2.
This 'amazon-ec2' Ruby Gem is an interface library that can be used to interact with the Amazon EC2 system and control server resources on demand from your Ruby scripts, or from applications written in your Ruby framework of choice (Ruby on Rails, Merb, etc.).
More recently, support has been added for the following EC2 related AWS API's as well:
* Autoscaling
* Cloudwatch
* Elastic Load Balancing (ELB)
* Relational Database Service (RDS)
For the most complete and up-to date README information please visit the project homepage at:
http://github.com/grempe/amazon-ec2/tree/master
or the official EC2 website at http://aws.amazon.com/ec2
== Installation
This gem follows the standard conventions for installation on any system with Ruby and RubyGems installed and uses Bundler for gem installation and build management. If you have worked with gems before this will look very familiar.
=== Get an AWS account
Before you can make use of this gem you will need an Amazon Web Services developer account which you can sign up for at https://aws-portal.amazon.com/gp/aws/developer/registration/index.html. This account must also be specifically enabled for Amazon EC2 usage. AWS will provide you with an 'AWS Access Key ID' and a 'Secret Access Key' which will allow you to authenticate any API calls you make and ensure correct billing to you for usage of the service. Take note of these (and keep them secret!).
=== Install the amazon-ec2 gem (Canonical Release)
This is the standard install for stable releases from RubyGems.
# Install the gem
[sudo] gem install amazon-ec2
=== Install from local Git clone (for amazon-ec2 developers)
To install from git for adding features or fixing bugs, you'll need to clone and build.
git clone git://github.com/grempe/amazon-ec2.git
cd amazon-ec2
bundle install
rake test
rake build
rake install
== Using amazon-ec2
The library exposes one main interface class AWS::EC2::Base. It is through an instance of this class that you will perform all the operations for using the EC2 service including query string header signing.
The public methods on AWS::EC2::Base closely mirror the EC2 Query API, and as such the Query API Reference in the EC2 Developer Guide ( http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=84 ) will prove helpful.
=== Setting up
The 'awshell' and 'ec2-gem-example.rb' scripts which will be introduced to you shortly expect your AWS EC2 credentials to be stored as shell environment variables which are accessible to those scripts. This makes them convenient to use whenever you need to do a quick query to see what images you have available to you, what's running now, or to start or stop an instance on EC2. You'll find 'awshell' to be a very handy tool. I'll describe only the OS X route for setting up (of course the setup steps will vary depending on your particular system and preferred shell). If you don't want to do it this way, feel free to copy these scripts from the gem dir to any location where you can run them from and modify them directly to include your credentials.
Edit the file ~/.bash_login and add the following to the existing contents:
# OPTIONAL : Why 'export RUBYOPT'? Because we leave loading libs up to you... See : http://gist.github.com/54177
export RUBYOPT="rubygems"
# For amazon-ec2 and amazon s3 ruby gems
export AMAZON_ACCESS_KEY_ID="YOUR_ACCESS_KEY_ID"
export AMAZON_SECRET_ACCESS_KEY="YOUR_SECRET_ACCESS_KEY_ID"
If you are using EC2 in the EU region, make sure you also set:
export EC2_URL="https://eu-west-1.ec2.amazonaws.com"
(which you have already if you configured standard EC2 command line tools to work with this region).
Once you save the file you should close and re-open your terminal so the new variables are made available. You'll need to do this close/re-open step for each terminal window you have open (or issue the 'source ~/.bash_login' command in each). Make sure that this file is only readable by your user so you don't inadvertently expose your credentials to other users on your system.
You can verify that this setup is complete by running the 'set' in a command window and seeing that your credentials are in the list of shell variables.
=== The basics
The library exposes one main interface module
AWS::EC2::Base
This method requires arguments which include your AWS credentials and it will return an object that you can use to make method calls directly against EC2. All the operations for using the EC2 service, including query string header signing, are handled automatically for you. The connection string will look something like this:
@ec2 = AWS::EC2::Base.new(:access_key_id => ACCESS_KEY_ID, :secret_access_key => SECRET_ACCESS_KEY)
I've tried to keep the public methods on 'amazon-ec2' as close as possible to the AWS EC2 Query API. This similarity allows you to reference the Query API Reference in the EC2 Developer Guide and be able to get started right away. In most cases the methods names only differ in how they are presented. e.g. 'DescribeImages' becomes '#describe_images()' in Ruby. Feel free to browse the full RDoc documentation for all classes and methods of 'amazon-ec2' if you want more details.
=== Examples
The best way to become familiar with 'amazon-ec2' is to take it for a test drive. We have provided a few simple ways to get you started. There is also some sample code below that should help out in using 'amazon-ec2' with a plain Ruby script, or as part of a Ruby on Rails application.
==== Using the 'ec2-gem-example.rb' sample test script
An example Ruby script which exercises the library a bit more is installed for you to check out when you install this gem. You can run this script to verify that everything is setup and working correctly in your environment. Consult the file which is installed at :
[your amazon-ec2 gem dir]/examples/ec2-example.rb
Since we also package this sample file in the gem's bin/ dir you should also be able to run it from anywhere on your shell path (once you have set your environment variables as described above).
==== Using the 'awshell' command shell
The 'awshell' command shell is actually a standard 'irb' Ruby shell, with the main difference being we read your AWS credentials from your environment and pre-configure a connection string for you. This lets you run any EC2 command very simply. This has proven to be a valuable tool during the development of this gem and you should try it out. Since we install this tool in your system path as part of the installation of this gem, you should be able to simply run 'awshell' from any terminal command prompt on your local system. You'll see some basic instructions for use, and a few examples when you start 'awshell'. Go ahead and try it out now. We'll wait...
If you're not in front of a terminal shell now (perhaps you're browsing this site on your iPhone) this is what you would see:
hostname:/tmp/rails/amazon_test glenn$ awshell
'awshell' usage :
This is an interactive 'irb' command shell that allows you to use all
commands available to the amazon-ec2 gem. You'll find this to be a
great tool to help you debug issues and practice running commands
against the live EC2 servers prior to putting them in your code.
The EC2 connection is wired to the class instance '@ec2'. Make method calls
on this to execute commands on EC2. Adding a #to_s
at the end of any command should give you a full String representation of the
response.
Examples to try:
returns : all ec2 public methods
>> @ec2.methods.sort
returns : a string representation of ALL images
>> @ec2.describe_images.to_s
returns : an Array of AWS::Response objects, each an EC2 image and its data
>> @ec2.describe_images.imagesSet.item
>> @ec2.describe_images.imagesSet.item[0] (a hash representing a single item in that array)
>> @ec2.describe_images.imagesSet.item[0].to_s (a String representation of that item)
>> @ec2.describe_images.imagesSet.item[0].to_s
=> "#<AWS::Response:0x100A465B4 imageId=\"ami-018e6b68\" imageLocation=\"rbuilder-online/phonehome-1.5.6-x86_10132.img.manifest.xml\" imageOwnerId=\"099034111737\" imageState=\"available\" isPublic=\"true\" parent=#<AWS::Response:0x100A469A6 ...>>"
=== Ruby script usage example:
Try out the following bit of code. This should walk through each image returned by a call to #describe_images and print out its key data. Note in the example below that you cannot walk through the results of the #describe_images call with the '.each' iterator (You'll get errors if you try). You need to instead walk through the Array of items which are in the 'imagesSet' embedded in the response. This reflects exactly the XML hierarchy of data returned from EC2 which we parse to Ruby OpenStruct objects (AWS::Response).
#!/usr/bin/env ruby
require 'rubygems'
require 'AWS'
ACCESS_KEY_ID = '--YOUR AWS ACCESS KEY ID--'
SECRET_ACCESS_KEY = '--YOUR AWS SECRET ACCESS KEY--'
ec2 = AWS::EC2::Base.new(:access_key_id => ACCESS_KEY_ID, :secret_access_key => SECRET_ACCESS_KEY)
puts "----- listing images owned by 'amazon' -----"
ec2.describe_images(:owner_id => "amazon").imagesSet.item.each do |image|
# OpenStruct objects have members!
image.members.each do |member|
puts "#{member} => #{image[member]}"
end
end
=== Ruby on Rails usage example:
<b>Rails 2.3.x - config/environment.rb</b>
Rails::Initializer.run do |config|
...
config.gem "amazon-ec2", :lib => "AWS", :source => "http://gemcutter.org/"
...
end
<b>Rails 3.x.x - Gemfile</b>
...
gem "amazon-ec2", :require => "AWS"
<b>app/controllers/my_controller.rb</b>
[some controller code ...]
ec2 = AWS::EC2::Base.new(:access_key_id => "YOUR_AWS_ACCESS_KEY_ID", :secret_access_key => "YOUR_AWS_SECRET_ACCESS_KEY")
# get ALL public images
@ec2_images = ec2.describe_images().imagesSet.item
# Get info on all public EC2 images created by the Amazon EC2 team.
@ec2_images_amazon = ec2.describe_images(:owner_id => "amazon").imagesSet.item
[some more controller code ...]
<b>app/views/my/index.rhtml</b>
<h1>EC2 Test#index</h1>
<h1>Sample 1 - debug() view</h1>
<%= debug(@ec2_images_amazon) %>
<h1>Sample 2 - Build a table</h1>
<table border='1'>
<tr>
<th>image.imageId</th>
<th>image.imageLocation</th>
<th>image.imageOwnerId</th>
<th>image.imageState</th>
<th>image.isPublic</th>
</tr>
<% for image in @ec2_images_amazon %>
<tr>
<td><%=h image.imageId %></td>
<td><%=h image.imageLocation %></td>
<td><%=h image.imageOwnerId %></td>
<td><%=h image.imageState %></td>
<td><%=h image.isPublic %></td>
</tr>
<% end %>
</table>
<h1>Sample 3 - Iterate</h1>
<% @ec2_images_amazon.each do |image| %>
<% image.each_pair do |key, value| %>
<% unless key == 'parent' %>
<%= "#{key} => #{value}" %><br />
<% end %>
<% end %>
<br />
<% end %>
=== Important notes regarding the structure of AWS::Response Objects
One of the key benefits of this new version of the library is that all responses from EC2 are bundled up in a real data structure and no longer require parsing of text. The hash returned is populated directly from the XML given to us by EC2 in response to any command we issue. This means that future changes to the API and what is returned by EC2 will be handled transparently by the gem. This is a huge benefit. What this means though, is that you may have to do a little homework on what actually gets returned by EC2 as XML. For example, when you make a #describe_images call in 'awshell' what AWS returns behind the scenes looks like:
<?xml version="1.0"?>
<DescribeImagesResponse xmlns="http://ec2.amazonaws.com/doc/2007-01-19/">
<imagesSet>
<item>
<imageId>ami-20b65349</imageId>
<imageLocation>ec2-public-images/fedora-core4-base.manifest.xml</imageLocation>
<imageState>available</imageState>
<imageOwnerId>amazon</imageOwnerId>
<isPublic>true</isPublic>
</item>
<item>
<imageId>ami-22b6534b</imageId>
<imageLocation>ec2-public-images/fedora-core4-mysql.manifest.xml</imageLocation>
<imageState>available</imageState>
<imageOwnerId>amazon</imageOwnerId>
<isPublic>true</isPublic>
</item>
<item>
<imageId>ami-23b6534a</imageId>
<imageLocation>ec2-public-images/fedora-core4-apache.manifest.xml</imageLocation>
<imageState>available</imageState>
<imageOwnerId>amazon</imageOwnerId>
<isPublic>true</isPublic>
</item>
<item>
<imageId>ami-25b6534c</imageId>
<imageLocation>ec2-public-images/fedora-core4-apache-mysql.manifest.xml</imageLocation>
<imageState>available</imageState>
<imageOwnerId>amazon</imageOwnerId>
<isPublic>true</isPublic>
</item>
<item>
<imageId>ami-26b6534f</imageId>
<imageLocation>ec2-public-images/developer-image.manifest.xml</imageLocation>
<imageState>available</imageState>
<imageOwnerId>amazon</imageOwnerId>
<isPublic>true</isPublic>
</item>
<item>
<imageId>ami-2bb65342</imageId>
<imageLocation>ec2-public-images/getting-started.manifest.xml</imageLocation>
<imageState>available</imageState>
<imageOwnerId>amazon</imageOwnerId>
<isPublic>true</isPublic>
</item>
</imagesSet>
</DescribeImagesResponse>
You can see here the XML the structure that you will need to follow when constructing queries for information and parsing responses from EC2.
So, for example, if you wanted to get the image ID of the third image listed in the response above you would need to do:
>> puts @ec2.describe_images(:owner_id => 'amazon').imagesSet.item[2].imageId
ami-23b6534a
EC2 will typically return sets of things (imagesSet, reservationSet, etc.) which we map to ruby Arrays (.imagesSet.item in the example above). If you want to iterate over a response set you will need to iterate over this array. The Arrays will typically contain additional AWS::Response objects that represent each individual item. You'll find that you can use the 'awshell' to help you understand the structure more completely if you try issuing commands there as a way to practice seeing what will be returned and making sure you get exactly what you want.
=== Handling Exceptions
If for some reason an error occurs when executing a method (e.g. its arguments were
incorrect, or it simply failed) then an exception will be thrown. The exceptions are
defined in exceptions.rb as individual classes and should match the exceptions that
AWS has defined in the API. If the exception raised cannot be identified then a
more generic exception class will be thrown.
The implication of this is that you need to be prepared to handle any exceptions that
may be raised by this library in YOUR code with a 'rescue' clause. It is up to you
to determine how you want to handle these exceptions in your code.
== Additional Resources
=== Project Websites
* Project Home : http://github.com/grempe/amazon-ec2/tree/master
* API Documentation : http://rdoc.info/projects/grempe/amazon-ec2
* Discussion Group : http://groups.google.com/group/amazon-ec2
* Report Bugs / Request Features : http://github.com/grempe/amazon-ec2/issues
* Amazon Web Services : http://aws.amazon.com
== Credits
The original code for this library was provided by Amazon Web Services, LLC as sample code. Thanks to them for providing all of us with something to get us started.
== Contact
Comments, patches, Git pull requests and bug reports are welcome. Send an email to mailto:glenn@rempe.us or join the Google Groups forum.
== Patches & Pull Requests
Please follow these steps if you want to send a patch or a GitHub pull request:
* Fork grempe/amazon-ec2
* Create a topic branch: `git checkout -b my_fix`
* Make sure you add tests for your changes and that they all pass with 'rake test'
* Don't change files that you don't own like the gemspec or version.rb
* Commit your changes, one change/fix per commit
* Push your fixes branch: `git push origin my_fix`
* Open an Issue on GitHub referencing your branch and send a pull request.
* Please do not push to `master` on your fork. Using a feature/bugfix branch will make everyone’s life easier.
Enjoy!
Glenn Rempe
|