AWS Official Blog

The first stable release of version 2 of the AWS SDK for Ruby is now available, complete with support for more than 40 AWS services.

You can install the aws-sdk gem from RubyGems and get started with the new SDK right now.

My colleagues have been writing detailed blog posts about the new features as part of the development process. The developer feedback that we have received during the process has been incredibly helpful. Here are the new features (and the associated posts):

• Resource interfaces for 7 services.
• Client response stubs.
• Waiters.
• Response paging.
• Improved parameter validation.
• Flexible Gem dependencies.
• Structured response data.
• Improved API reference documentation (including examples).

It also features intelligent credential management and some handy extensibility features (plugins and custom request handlers).

Versions 1 and 2 of the SDK use distinct namespaces. This allows you to use both of them in the same application without making any changes to your existing code. To learn how to do this, read Announcing V2 of the AWS SDK for Ruby in the AWS Ruby Development Blog.

— Jeff;

Let’s take a quick look at what happened in AWS-land last week:

Stay tuned for next week! In the meantime, follow me on Twitter and subscribe to the RSS feed.

— Jeff;

I am still working to catch up with our development teams! Near the end of last year, the Amazon Mobile Analytics team released a new Auto Export feature. This relatively new service (see my launch blog post for more information) helps you to collect, visualize, and understand app usage, engagement, and revenue at scale. If you build your application with the AWS Mobile SDK (available for iOS and Android), Amazon Mobile Analytics will automatically collect data so that you can view it in the Amazon Mobile App Console:

Auto Export to S3
Today we are making the Mobile Analytics feature even more useful by giving you the ability to automatically export data to Amazon Simple Storage Service (S3). Once enabled for all apps or for the apps that you choose, data files in JSON format will be deposited in to the S3 bucket of your choice.

To enable Auto Export to S3, I simply visit the Console and select the apps of interest:

Then I click on Enable Auto Export and configure an S3 bucket. I can create a new  bucket or use an existing one:

Amazon Mobile Analytics needs my permission to write to the bucket. When I click on Confirm and Launch 1-Click Role Creation, I am  prompted to create an IAM role that has the proper permissions:

The policy is created and applied to the bucket, and the Auto Exporting is all set:

Each S3 object will contain one or more JSON objects. You can find the schema here.

Analyze in Redshift
If you want to run some advanced analytics, consider loading the data in to an Amazon Redshift table. Here’s what you need to do:

1. Create a file that describes how to “flatten” the JSON object for importing.
2. Upload the file to an S3 bucket and make it readable.
3. Create a table and a schema in Redshift.
4. Load the events from the S3 bucket into the Redshift table by way of the file you created in step 1.
5. Run Redshift queries on the imported data.

To learn more about how to do this, read  Score Big With Amazon Mobile Analytics and Amazon Redshift on the AWS Mobile Development Blog.

Available Now
As I noted earlier, this feature was launched at the very end of 2014 and you can start using it now!

— Jeff;

We’ve increased the geographic footprint and international relevance of several AWS services this past month. Here’s a summary:

• Amazon WorkSpaces and AWS Directory Service are now available in the Asia Pacific (Singapore) region.
• Amazon ElastiCache is now available in the Europe (Germany) region.
• AWS Trusted Advisor, Amazon ElastiCache, and RDS integration with AWS CloudTrail are now available in AWS GovCloud (US).
• We added a second location in Korea for Amazon CloudFront and Amazon Route 53.
• Amazon Kinesis is now available in the US West (Northern California) region.
• The AWS website is now available in Traditional Chinese.

For more information on service availability, please take a look at our Products and Services by Region page.

— Jeff;

My colleague Peter Moon wrote the guest post below and asked me to get it out to the world ASAP!

— Jeff;

AWS currently offers SDKs for seven different programming languages – Java, C#, Ruby, Python, JavaScript, PHP, and Objective C (iOS), and we closely follow the language trends among our customers and the general software community. Since its launch, the Go programming language has had a remarkable growth trajectory, and we have been hearing customer requests for an official AWS SDK with increasing frequency. We listened and decided to deliver a new AWS SDK to our Go-using customers.

As we began our research, we came across aws-go, an SDK from Stripe. This SDK, principally authored by Coda Hale, was developed using model-based generation techniques very similar to how our other official AWS SDKs are developed. We reached out and began discussing possibly contributing to the project, and Stripe offered to transfer ownership of the project to AWS. We gladly agreed to take over the project and to turn it into an officially supported SDK product.

The AWS SDK for Go will initially remain in its current experimental state, while we gather the community’s feedback to harden the APIs, increase the test coverage, and add some key features including request retries, checksum validation, and hooks to request lifecycle events. During this time, we will be developing the SDK in a public GitHub repository at https://github.com/awslabs/aws-sdk-go. We invite our customers to follow along with our progress and join the development efforts by submitting pull requests and sending us feedback and ideas via GitHub Issues.

We’d like to thank our friends at Stripe for doing an excellent job with starting this project and helping us bootstrap this new SDK.

— Peter Moon, Senior Product Manager

Late last week I met with Andi Gutmans and Michel Gerin of Zend Technologies. Because Andi is a self-described “coffee snob,” we headed directly to the nearby Dilettante Cafe for an in-depth chat. It was interesting to hear how they had grown from an organization focused on the PHP language to one that was taking on the more broad mission of scalability, monitoring, and visibility in to the run-time state of web and mobile applications that happened to be built using PHP. In fact, the only mention of PHP came when I remarked that we had spent no time discussing it. Andi spent more time telling me about his quest for the perfect microfoam than he did about language features!

Zend Server Update
We discussed their work on Zend Server, including the freshly released Version 8. As I described in a post that I wrote last year, Zend Server (and the crucial Z-Ray technology) gives developers access to in-content feedback on the behavior of the application that they are building, testing, or running.  Z-Ray provides developers with information about page requests, execution time, peak memory usage, events, PHP errors & warnings, SQL query execution, and variables.

I learned that Zend Server also has a number of other features that help applications run quickly and efficiently. These were not the subject of our chat and I didn’t take good notes, but we talked about code & data caching, job queues, and job scheduling. We also discussed cluster management and some new AWS integration. Zend Server can now be launched via AWS CloudFormation. It even includes a CloudFormation template generator to make this process simpler and totally repeatable:

Zend Server knows how to deploy code from Amazon Simple Storage Service (S3) (it can also pull from Git or deploy Zend Packages, also known as ZPKs).

Z-Ray Demo
Andi was eager to demo the newest Z-Ray features for me. He fired up his laptop (a stylish MacBook Air), got the Wi-Fi password from the barista, and connected to his demo instance.  He explained to me that they created an extensibility model for Z-Ray and used it to create a series of extensions for popular applications and frameworks. Each extension has intimate knowledge of the programming model, data structures, and database queries built and referenced by the associated environment.  This intimacy allows each extension to display the most important elements of each environment in a manner that will be familiar and comfortable to developers who are already versed in the environment.

Out of the box (to use that tired term left over from the by-gone era of shrink-wrap software), Z-Ray includes extensions for the Magento, Drupal, and WordPress application platforms. It also includes extensions for the Zend Framework, Symfony, and Laravel application frameworks. These extensions are available from the Official Zend Server Extensions repo on GitHub. Here’s an example of Z-Ray in action. It is aware that it is accessing the database queries initiated by a WordPress application and the display is customized accordingly:

The extension API is open and documented. Third-party (non-Zend) developers have already created extensions for other environments including Doctrine 2 (read more about the extension).

Zend Server on AWS
The Developer and Professional editions of Zend Server are available on the AWS Marketplace and you can launch a free trial of either one with a couple of clicks:

Both editions include a bunch of features that are intended to make Zend Server mesh smoothly with existing AWS environments and applications. Here are some of the features:

• A new, JSON-based format for the EC2 user data that is passed to each newly launched instance. This data is used to configure the Zend Server.
• A Z-Ray extension for the AWS API.
• Custom script actions on startup.
• Control over the dissemination of AWS access and secret keys to instances.
• Control over cluster membership.

To learn more about these features, watch Zend’s new video, Getting Started with Zend Server and Z-Ray on AWS.

We wrapped up our meeting, recycled our mugs (this is Seattle, after all), and they headed back to Sea-Tac airport for their flight back to Silicon Valley!

— Jeff;

PS – I love to learn and write about cool uses of AWS. Please track me down (a search for ‘contact jeff barr’ is a good start) and let me know when you are coming to Seattle!

Have you ever had to set up, run, and scale an email server? While it has been a long time since I have done this on my own, I do know that it is a lot of work! Users expect to be able to access their email from the application, device, or browser of their choice. They want to be able to send and receive large files (multi-megabyte video attachments and presentations often find their way in to my inbox). Email administrators and CSO’s are looking for robust security measures.

Paradoxically, email is both mission-critical and pedestrian. Everyone needs it to work, but hardly anyone truly understands what it takes to make this happen!

Introducing WorkMail
Today I would like to introduce Amazon WorkMail. This managed email and calendaring solution runs in the Cloud. It offers a unique set of security controls and works with your existing desktop and mobile clients (there’s also a browser-based interface). If your organization already has a directory of its own, WorkMail can make use of it via the recently introduced AWS Directory Service. If not, WorkMail will use Directory Service to create a directory for you as part of the setup process.

WorkMail was designed to work with your existing PC and Mac-based Outlook clients including the prepackaged Click-to-Run versions. It also works with mobile clients that speak the Exchange ActiveSync protocol.

Our 30-day free trial will give you the time and the resources to evaluate WorkMail in your own environment. As part of the trial, you can serve up to 25 users, with 50 gigabytes of email storage per employee. In order to help you to move your organization to WorkMail, we also provide you with a mailbox migration tool.

WorkMail makes use of a number of AWS services including Amazon WorkDocs (formerly known as Amazon Zocalo), the Directory Service, AWS Identity and Access Management (IAM), AWS Key Management Service (KMS), and Amazon Simple Email Service (SES).

WorkMail Features
You can set up WorkMail for a new organization in a matter of minutes. As I mentioned earlier, you can use your existing directory or you can have WorkMail set one up for you. You can send and receive email through your existing domain name by adding a TXT record (for verification of ownership) and an MX record (to route the mail to WorkMail to your existing DNS configuration).

As a WorkMail user, you have access to all of the usual email features including calendaring, calendar sharing, tasks, contact lists, distribution lists, resource booking, public folders, and out-of-office (OOF) messages.

The browser-based interface has a full array of features. It works with a wide variety of browsers including Firefox, Chrome, Safari, and newer (IE 9 and higher) versions of Internet Explorer. The interface gives you access to email, calendars, contacts, and tasks. You can access shared calendars and public folders, book resources, and manage your OOF.

WorkMail was designed to work in today’s data-rich, email-intensive environments. Each inbox has room for up to 50 gigabytes of messages and attachments. Messages can range in size all the way up to 30 megabytes.

As part of this launch we are renaming Amazon Zocalo to Amazon WorkDocs! WorkMail can be used in conjunction with WorkDocs for simple, controlled distribution of documents that contain sensitive information.

WorkMail Security Controls

Let’s talk about security for a bit. WorkMail includes a number of security features and controls that will allow it to meet the needs of many types of organizations. Here’s an overview of some of the most important features and controls:

• Location Control – The WorkMail administrator can choose to create mailboxes in any supported AWS region. All mail and other data will be stored within the region and will not be transferred to any other region. During the Preview, WorkMail will be supported in the US East (Northern Virginia) and Europe (Ireland) regions, with more to follow over time.
• S/MIME – Data in transit to and from Outlook clients and certain iPhone and iPad apps is encrypted using S/MIME. Data in transit to other clients is encrypted using SSL.
• Stored Data Encryption – Data at rest (messages, contacts, attachments, and metadata) is encrypted using keys supplied and managed by KMS.
• Message Scanning – Incoming and outgoing email messages and attachments are scanned for malware, viruses, and spam.
• Mobile Device Policies & Actions – The WorkMail administrator can selectively require encryption, password protection, and automatic screen locking for mobile devices. The administrator can also remotely wipe a lost or mislaid mobile device if necessary.

Getting Started with WorkMail
Let’s walk through WorkMail while wearing our email administrator hats! I need to create a WorkMail organization. In most cases, I would use a single organization for an entire company.

I start by opening up the AWS Management Console and choosing WorkMail:

I click the Get started button. At this point I can choose between a Quick setup (WorkMail will create a new directory for me)  or a Custom setup (WorkMail will use an existing directory that I configure):

I’ll go for the quick setup today. I need to pick a unique name for my organization:

This will automatically create a directory and then create and initialize my organization. It will also initiate the Amazon SES domain verification process (for jeffbarr.awsapps.com in this case) and create a set of DKIM keys so that I can send DKIM-signed mail. The entire process takes 10 to 20 minutes and requires no additional work on my part. The organization’s status will start out as creating and will transition to active before too long:

After the creation process completes I can begin to add WorkMail users to my organization (if I had used an existing directory in the previous step I could simply select them from a list at this point). I’ll begin by adding myself:

Then  I specify the email address and password. If I have associated one or more domain names with the organization, I can use the name as the basis for the email address:

I can browse all of the organization’s users:

I can also create groups, attach domains, and manage mobile device policies, all from the Console.

The WorkMail Browser-Based Interface
Let’s take a look at the browser-based interface to WorkMail. Here’s my inbox:

And my calendar:

This is just a sampling of the features that are available in the WorkMail.

Pricing and Availability
We are launching a Preview of Amazon WorkMail in the US East (Northern Virginia) and Europe (Ireland) regions today and you can sign up for the Preview if you are interested in joining.

After the 30-day free trial (25 users and 50 gigabytes of storage per user), pricing is on a per-user, pay-as-you-go basis. You will be charged $4 per month for a 50 gigabyte WorkMail mailbox, or $6 per month for a bundle that includes WorkMail and WorkDocs. There is no separate charge for the use of SES to send messages.

— Jeff;

Developers all over the world are using Amazon DynamoDB to build applications that take advantage of its ability to provide consistent low-latency performance. The developers that I have talked to enjoy the flexibility provided by DynamoDB’s schemaless model, along with the ability to scale capacity up and down as needed.  They also benefit from the DynamoDB Reserved Capacity model in situations where they are able to forecast their need for read and write throughput ahead of time.

A little over a year ago we made DynamoDB more flexible by adding support for Global Secondary Indexes. This important feature moved DynamoDB far beyond its roots as a key-value store by allowing lookups on attributes other than the primary key.

Today we are making Global Secondary Indexes even more flexible by giving you the ability to add and delete them from existing tables on the fly.

We are also making it easier for you to purchase Reserved Capacity directly from the AWS Management Console. As part of this change to a self-service model, you can now purchase more modest amounts of Reserved Capacity than ever before.

Let’s zoom in for a closer look!

Global Secondary Indexes on the Fly
Up until now you had to define the Global Secondary Indexes for each of your DynamoDB tables at the time you created the table. This static model worked well in situations where you fully understood your data model and a good sense for the kinds of queries that you needed to use to build your application.

DynamoDB’s schemaless model means that you can add new attributes to an existing table by simply storing them. Perhaps your original table stored a first name, a last name, and an email address. Later, you decided to make your application location-aware by adding a zip code. With today’s release you can add a Global Secondary Index to the existing table. Even better, you can do this without taking the application offline or impacting the overall throughput of the table.

Here’s how you add a new index using the AWS Management Console. First, select the table and click on Create Index:

Then enter the details (you can use a hash key or a combination of a hash key and a range key):

The index will be created and ready to go before too long (the exact time depends on the number of items in the table and the amount of provisioned capacity). You can also delete indexes that you no longer need. All of this functionality is also available through DynamoDB’s UpdateTable API.

There is no extra charge for this feature. However, you may need to provision additional write throughput in order to allow for the needs of the index creation process. You’ll pay the usual DynamoDB price for storage of the Global Secondary Indexes that you create.

Purchasing Reserved Capacity
DynamoDB’s unique provisioned capacity model makes it easy for you to build applications that can scale to any desired level of throughput. Instead of having to worry about adding hardware, tuning software, or rearchitecting your application as traffic grows, you can simply provision additional read or write capacity. The provisioning model even allows you to add capacity in anticipation of high traffic (perhaps your application is busiest during local business hours) and to remove it when it is not needed. This model allows you to create a cost structure that closely mirrors actual usage of your application and avoids unnecessary charges for idle resources.

In situations where you have enough confidence in your usage model and your predictions for growth over time, you can reduce your DynamoDB costs even more by purchasing Reserved Capacity for a one or a three year term. After you pay the upfront fee, you will be billed monthly for the amount of capacity that you purchase. By purchasing capacity up front, you will save 53% (one year term) or 76% (three year term) over the regular hourly rates.

In order to make Reserved Capacity accessible to more DynamoDB users, we have made two important changes. First, we have simplified the purchase process and made it accessible from within the Console. Second, we have reduced the minimum purchase to just 100 read or write capacity units. To purchase Reserved Capacity within a particular AWS region, open up the Console, choose the region, and click on the Reserved Capacity button:

Select the amount of read and/or write capacity that you need (in units of 100), choose a term, and fill in your email address:

Your purchases are visible in the Console:

You can read more about this feature in the recent post, On DynamoDB Provisioning: Simple, Flexible, and Affordable, in the AWS Startup Collection.

From our Customers
AWS customer Eddie Dingels (Lead Architect for Earth Networks) is already taking advantage of on-the-fly indexing and the new pricing model! In his words:

With online indexing, we can re-index tables to run new queries whenever we want. DynamoDB handles consistently changing the index while taking live traffic without a performance impact even on large data sets.

He’s also saving money:

DynamoDB has a very simple and innovative approach to database provisioning, it is truly pay as you go. Reserved capacity ends up dropping DynamoDB throughput costs by up to 76%, and today’s announcement makes it easier than ever for us to perform incremental purchases as we grow.

Availability
The new Reserved Capacity pricing model is available today in all regions. Online indexing is available today in the Asia Pacific (Tokyo), Asia Pacific (Singapore), Europe (Ireland), US East (Northern Virginia), US West (Oregon), and US West (Northern California) regions.  We expect to make it available in the Europe (Germany), South America (Brazil), Beijing (China), and AWS GovCloud (US) regions within a week or so.

— Jeff;

PS  – Some of our developers put together a new article to show you how to Build a Mars Rover Application With DynamoDB. The code in this article takes advantage of the new JSON support and is a great way to exercise DynamoDB’s expanded free tier.

Let’s take a quick look at what happened in AWS-land last week:

Coming Soon:

• January 26th – Live Event (Seattle) – Tracking, Allocating and Optimizing AWS Spend.
• January 26th – Webinar – Build Agile Applications in AWS Cloud.
• January 29th – Live Event (San Francisco) – AWS HealthTech Day at the AWS Pop-up Loft.
• January 29th – Webinar – Build Agile Applications in AWS Cloud for Government.
• February 2nd – Live Event (Vancouver, Canada) – Vancouver AWS User Group (I’ll be speaking).
• March 5th – Live Event (Scotland) – AWS User Group Scotland.

Stay tuned for next week! In the meantime, follow me on Twitter and subscribe to the RSS feed.

— Jeff;

AWS Trusted Advisor inspects your AWS environment and looks for opportunities to save money, increase performance & reliability, and to help close security gaps. Today we are enhancing Trusted Advisor with the addition of Action Links. You can now click on an item in a Trusted Advisor alert to navigate to the appropriate part of the AWS Management Console. For example, I ran the Trusted Advisor on my own AWS account and it displayed the following alert:

I decided to fix the problem and activated an Action Link to head on over to the RDS section of the Console. From there I right-clicked to add a Read Replica:

These new links are available now and you can click on them today!

For Tool Vendors
If you build applications that link (or could link) to the Console, you can use the same URLs. Here are a few to get you started (all of the links are relative to the base URL of the console):

• EC2 Reserved Instance Purchase –  ec2/home?region={region}#ReservedInstances
• EC2 Instances – ec2/home?region={region}#Instances:search={search_string}
• Elastic Load Balancer – ec2/home?region={region}#LoadBalancers:search={search_string}
• EBS Volumes – ec2/home?region={region}#Volumes:search={search_string}
• Elastic IP Addresses – vpc/home?region={region}#eips:filter={filter_string}
• RDS Database Instances – rds/home?#dbinstance:id=dbInstanceId
• Auto Scaling Configuration – ec2/autoscaling/home?#LaunchConfigurations:id=LaunchConfigurationName

There is a chance that these links will change in the future as the console continues to evolve. If you decide to make use of them, please plan for that eventuality in your application.

— Jeff;