September 2011 Site Performance Report

This is a cross post from the Wayfair Engineering Blog.

I recently saw this post on Etsy’s blog and found it inspiring – you don’t typically see retailers (or any companies for that matter) sharing performance data.  Even more impressive is the fact that they posted this on their primary blog as opposed to their engineering blog.  To me this says that Etsy really cares about performance, and wants all of their customers to be aware that it is a priority.  After reading that post I immediately wanted to share some Wayfair data – and this is my attempt to do that (maybe I can get it cross-posted to our main blog ).

Here at Wayfair we care a lot about performance, and the business case for having a faster site is well documented.  To keep tabs on our site performance we have synthetic monitoring tools like Catchpoint, real user monitoring tools like Coradiant (now a part of BMC software), and our own home-brew monitoring instrumented in our ASP and PHP code.  We have also had great success using StatsD in conjunction with Graphite.  Though we use the ruby client, StatsD was started as an open source NodeJS project from Etsy (I promise we aren’t stalking you guys).  The numbers that I am sharing are from Coradiant, and measure “host time” which is defined as the time between the last packet of the request and the first packet of the response as measured by a network tap.  Without further ado, here are the numbers for our highest traffic pages.  These numbers are all in milliseconds, and were measured between 9AM and 5PM Eastern Time on a Wednesday, so they should be indicative of real world performance.

Sharing Performance Data

I am very impressed with Etsy's decision to share some of their performance data.  This is rarely done in the WPO world, companies regularly share data like "improving landing page performance by 2.2 seconds increased download conversions by 15.4%" but you almost never see absolute numbers.

Why don't companies share hard numbers?

The obvious answer is that they want to keep whatever competitive advantage they have in the speed department.  If a site starts sharing its performance data then competing sites have a benchmark to shoot for.  Other companies could also use that data as an advertising ploy, saying things like "don't shop at Etsy, our site is twice as fast!".  On top of that, performance numbers are somewhat complicated to share and compare.  As I talk about in my post on SLA's, the numbers can change dramatically depending on how you measure them.  Etsy has done an okay job of explaining their methodology, they specified that they are talking about server side performance and they gave average and 90th percentile.  Since server side performance is location and network independent there isn't a ton of variability based on measuring tools.  If Etsy starts sharing full page load times I hope they will give a bit more information about how the tools they are using to prevent people from making apples to oranges comparisons.

How fast is your website? Setting (and keeping) Web performance SLAs

This is a cross post from the Yottaa Performance Blog

I’m sure many of you have heard people at the office ask the question “How fast is our site?”  or some variation of it.  Many of you have probably also realized that this question is largely meaningless.  Anybody who can respond to this question with a single number is trying to sell you something.  There are a huge number of variables that go into measuring the load time of a website:

• What kind of monitoring are you using, synthetic or real user?
• If synthetic, where was the test run from, close to your datacenter or across the world?
• What percentile are you looking at in the data?
• How many tests (or people for RUM data) are you aggregating to get this number?
• Which pages are you monitoring?

Introduction

Welcome!  I figured it was time to start a blog, and I've owned this domain for months without using it for anything.  I've been holding out for jonathanklein.com, but sometimes you have to settle for what's available.  Most of the posts on this blog will be about Web Performance Optimization (WPO), Astronomy, or other things that interest me.  Let me know if you have any feedback - it's a work in progress.