As a corollary to last week’s blog about hosted load testing, I thought it would be interesting to explain a little about using a cloud-based test environment to perform pre-deployment testing for a cloud-based application.
That sounds like a lot of clouds! What we are simply trying to understand is where and how to deploy your application in the cloud, so it will perform well for your customers.
If you take the case of deploying an online store into the Amazon EC2 cloud, there are many things to consider that will impact your customer’s experience. Two of the most important are where to deploy what, and how your store will function over the Internet. To compound the issue, where you physically deploy machines within EC2 is not clear. Amazon goes to great lengths to avoid you knowing where their zones are.
Fortunately, you cannot only find solutions to these problems, but with cloud computing the process gets a lot easier. To break the problems down into a logical flow, you’ll first need to know the demographics of your customers. Where are they? How will they access the store? What needs to happen for them to have a positive experience at your store? If you can answer these questions, you’ll likely come up with some numbers that place percentages of your users in different regions around your target market (US, EMEA, global, etc.). Further, you’ll know the top two browsers they’ll use and with what kind of connections they’ll access the internet (e.g. Firefox with a 768/256kbps DSL, or IE8 with dial-up – yes people still use dial-up!). Finally, you’ll have some way to quantify a “positive user experience”. That usually involves consistency (does it work?) and speed (how fast?).
Using the information above and HP LoadRunner with WAN emulation, you can quickly build a to-scale test in your cloud-based lab and understand what would happen if you hosted your application all in one place and serviced your entire customer base. Odds are that the test will not meet all of your performance goals the first time you run it. Most likely, you will need to adjust some things in your store (e.g. reduce chattiness and image resolution, optimize Web 2.0 use) and then begin the discussion about where to deploy what. Can you keep all of your database servers in one place? If you have to split your database servers, how will that affect performance? Will you need to deploy web servers in every zone, will you be okay with one in each region, or will you need something in-between?
Again, not only can answers to these questions actually be answered, but by using cloud-based testing you can arrive at valid conclusions faster than ever. To understand the network impact of deploying machines in different zones, network performance measurement tools like VE Network Catcher can run in each of the zones to measure network performance between zones and to customer representative endpoints. Importing that data directly into HP LoadRunner with WAN emulation and using the flexible cloud environment, you can rapidly form and either dismiss or validate hypotheses about how your application should best be deployed.
The benefit of testing in this way is that you can experiment with many different configurations in rapid succession to find the one or few that are most appropriate for your needs. Trial and error with a live application is not advisable. And building a test lab with physical machines to simulate the same thing is not only significantly more expensive, but a lot slower process. As I pointed out in my previous blog, the same ROI formula that explain and fuel the explosion of Cloud Computing for hosting applications are also relevant for using Cloud Computing to test your applications with.
Now you know what I think. What do you think??