Sunday, September 21, 2014

GORUCK Light After Action Review

I completed my first GORUCK event on 9/13, a Light in Boston. We covered about 8 miles in 5 hours and I was carrying roughly 30 pounds. Overall it met my expectations as a challenging event, although with significantly less focus on calisthenics than I expected. The vast majority of our time was spent rucking around Boston, with minimal time spent doing push-ups, flutter kicks, etc.

My class met at the Soldiers and Sailors Monument in the center of Boston Common, although we quickly had to move to the garden nearby to avoid a marijuana legalization festival. Our cadre checked everyone in, opened our rucks to check that we had the right amount of weight, and had us form up to listen to a little background on GORUCK. This particular event was special because it was a 9/11 commemorative event, so we spent extra time talking about the events of that day and why the cadre chose to serve in the military. We were informed that this “would not be a normal Light” and to “get ready for a difficult event”. Being that this was my first GORUCK experience, the apprehension started to mount a little bit.

We were taken over to the 9/11 memorial in Boston Garden and one of our cadre talked a bit more about the loss of life that happened on that day, what it meant to the people who were involved, and all of the lives that it touched. It was clear from the outset that this was going to be a serious, reflective event. Overall the class seemed motivated, respectful, and ready to begin. We had a moment of silence at this memorial: each person doing 200 flutter kicks on their back quietly - our first physical challenge.

Our class was large (66 people) so one of the first orders of business was to get a clear count and split us up among the two cadre. Unfortunately our group had a little trouble counting off, and we got a quick lesson in attention to detail. Counting off in a front leaning raise with a 30 pound ruck gets tiring quickly, especially when you have to count 60+ people 5-7 times.


After this we did a few more exercises in the garden, practiced some buddy carries, and split up into two groups of 33 people each. The rest of the event consisted of the two groups racing each other around Boston, visiting various landmarks, statues, and memorials that were relevant to either 9/11, servicemen and women, or the founding of our nation. The loser of each race had to pay some sort of penalty, typically a few burpees or some crab walking. On our first movement my team made the mistake of trying to cross a street when there wasn’t enough time left on the walk signal, and we stranded half of our team on the other side of the street. This was a stupid mistake (that I’m at least partially to blame for), and we paid for it by bear crawling and spending some time sitting in a fountain at our destination. It took this mistake for me to realize how long it takes 33 people with rucks and flags to cross a street (obvious hint: never try to cross when the red hand is already flashing).


Another lesson that we had to learn was how important it is to pick a sustainable pace. We had a wide range of fitness abilities on our team, and this resulted in people in the front breaking into a run, folks in the back yelling for us to slow down, and then other people yelling for us to speed up again. I know that a few of the team leaders got confused during this process, and weren’t sure how to fix the problem. In addition, some of the more experienced members of the group were frustrated that we were pounding the weaker members into the ground with the haphazard pace. My takeaway from this is that it is necessary to circulate among the entire group, make sure that people are okay, and move the slowest members to the front of the column so they can set the pace. This way you can keep up a consistent “GORUCK shuffle” that everyone can sustain for the duration of the movement. It pays to be a winner in these races, but it is more important to make sure that everyone is having a good time and that no one is left behind - especially during a GORUCK Light (the shortest event they offer).

We finished the event where we started, back at the garden and at the 9/11 memorial. After some final words about the people who lost their lives in the attack, we told each other our stories about where we were during it, and how it impacted our lives. This was a powerful reminder of the damage that was done that day, and it was also a good way to connect with the other members of the class. While this was happening, the cadre handed out our patches and then had us form up one final time to say goodbye. I know that a lot of the people who participated in this event will be back for another one, and the experience made me even more excited for the Challenge that I’m doing on Halloween night.

Physically the event was difficult, and I was sore for 3 solid days afterwards (my shoulders were the worst part by far).  That said, I really think that as the website states anyone who can complete a 5K can finish a GORUCK Light. The more you train the less you are going to suffer, but even someone with a really modest level of fitness can gut out a Light and have a great time. If this kind of thing sounds even remotely interesting to you, I highly recommend signing up. You will get way more out of it than your typical road race or obstacle race, and there’s a good chance that you’ll make some new friends.


Wednesday, July 30, 2014

Fixing jQuery Animate Flickering

I just spent over a hour wrestling with this bug, and I'm hoping this post will help spare someone else the same pain.

The problem that I ran into was an annoying flicker when using the jQuery .animate() function to change the opacity of a DOM node.  I did a bunch of Googling, but none of the solutions worked for me. I tried:


What ended up working for me was removing a CSS transition from the element being animated.  

The code in question (simplified):

listing.animate({opacity: 0}, function() {
    // some code
});


The listing element had the following CSS rules applied (simplified by removing vendor prefixes):
.listing-card {
        background: #fff;
        position: relative;
        border: 1px solid #ececec;
        border-radius: 3px;
        box-sizing: border-box;
        transition: all .1s;
        margin-bottom: 26px;
}

I deleted the transition and the flickering went away completely.

If none of these solutions work for you, I suggest doing the following to troubleshoot your specific issue:

  • Make a button on the page that toggles your animation back and forth.
  • Open up Chrome DevTools, and start deleting HTML, testing the animation with the button after each deletion until you have a minimal test case.
  • Now start turning off CSS rules in DevTools until the animation works smoothly without flickering

Hopefully this helps someone else avoid a few hours of debugging!

Monday, July 7, 2014

The Road to 100 Consecutive Push-ups

In addition to my goal of doing one million push-ups by the age of 50, I always wanted to be able to do 100 consecutive push-ups.  I started training heavily towards this goal on February 1st of this year, and hit that milestone on May 18th.  It took a bit of experimentation to find out what worked to increase my push-up max, and there’s a lot of misinformation around this particular goal on the web, so I’m hoping that this post can serve as a useful resource for people tackling this challenge.

A Quick Word of Warning

If you search for recommendations on how to do 100 consecutive push-ups, you will without a doubt stumble upon http://hundredpushups.com. This site, in a word, is unrealistic. On two separate occasions I’ve attempted to follow their program, and haven’t come anywhere near 100 push-ups in the final attempt. The site doesn’t even make sense…anyone should be able to go from ~10 push-ups to 100 in 6 weeks? That’s a little aggressive. Also they jump people who can do 50+ push-ups to week 3, so they are supposed to go from 50 push-ups to 100 in 4 weeks. This means increasing your push-up maximum by 12 per week, which I would imagine is impossible for most people (it certainly was for me).

What Worked For Me 

The workout that I ended up with was a Monday/Wednesday/Friday schedule with an exhaustion test every other Sunday. Each workout consisted of 10 exhaustion sets with a one minute break between each one, alternating between the following kinds of push-ups:
  1. Arms at a 45 degree angle
  2. Military style (picture, video).
  3. Diamond
  4. Wide angle
Incorporating a variety of push-ups is important, because it exercises different muscle groups and builds overall strength and stability in your core and shoulders. This is what ultimately allows you to increase your push-up max in a healthy and sustainable way.

I would do sets in this order twice, and then finish with another set of #1 and #4 to close out the 10 sets. Every push-up was done on the “Perfect Pushup” rotating handles. This was primarily to preserve my wrists, and also to add a little difficulty by engaging stabilizer muscles and allowing me to go down further.

As with any workout plan where the goal is to build muscle and increase strength, the rest of your health has to be at the top of your mind as well. This means:

  • Getting enough high quality sleep
  • Eating enough, and eating well (minimal sugar intake, minimal processed foods, enough protein, etc.). This is especially important post-workout
  • Doing mobility work - stretching/yoga/massage/foam rolling/etc.  

One thing that came up during my research was doing pulling exercises to balance out all of the push-ups.  This could be pull-ups, rowing, reverse push-ups, or a similar exercise.  Unfortunately I don’t have any of the necessary equipment to do those kinds of exercises at home, so I skipped that recommendation. It’s possible that it would have accelerated my path to 100 push-ups, and it makes sense if you have the ability to do that.

With the plan outlined above, I was able to increase my push-up max by around 5 push-ups every two weeks, and in about 15 weeks I went from a starting maximum of 62 to finishing my goal of 100 consecutive push-ups. The full details of my training are in my master push-up spreadsheet.

Next Steps

I still have ~920,000 push-ups left in my goal of doing a million by the age of 50, so I had to find a sustainable training program going forward.  After I did the hundred I took some time off, and as a result I have fallen a bit behind on my overall goal. My current approach is a “race to 300” workout, where I do the exact same workout detailed above, but instead of doing 10 sets I try to get to 300 push-ups in as few sets as possible (with a one minute break between sets). On the off days I try to do around 100 push-ups throughout the course of the day in sets of 25, just to keep making progress towards my goal.  With summer schedules and a bunch of travel I haven’t been as consistent with this approach as I would like, but will be focusing on it throughout July and try to catch up.

Reaching this goal was a satisfying milestone on my way to one million push-ups, but it also reminded me how far I have to go. This year has definitely been more of a grind than the first two, and I'm contemplating raising my daily average so I can finish the challenge earlier.  I'll talk more about that in my 2014 update, sometime in November or December.

Tuesday, February 25, 2014

Revisiting the "Cookieless Domain" Recommendation

For a long time one of the recommendations for a faster site has been to “serve your static content from a cookieless domain”.  This suggestion shows up in the Google best practices, the Yahoo! performance rules, and in Phil Dixon’s famous Shopzilla case study from Velocity 2009, where he states that implementing this one best practice resulted in a 0.5% improvement in top line revenue.  Case closed, right?  Well, due to some recent experimentation we have been doing at Etsy, we had reason to believe that people might be taking this too far.  Specifically, our testing indicated that serving CSS files from the same domain as the base page might be a performance win.  

Why is this a Good Idea?

CSS blocks page rendering, because if it didn’t then users would get the dreaded flash of unstyled content.  This is the reason why another performance recommendation is to put CSS in the head of the document, and to combine CSS files and minify them.  Getting CSS to start downloading as quickly as possible and downloading as little of it as possible is critical to a fast experience.  Here’s the rub: when you put CSS (a static resource) on a cookieless domain, you incur an additional DNS lookup and TCP connection before you start downloading it.  Even worse, if your site is served over HTTPS you spend another 1-2 round trips on TLS negotiation. Since the CSS blocks the rest of the page resources, this is extremely expensive.  In this case the entire page is waiting on DNS, TCP, and TLS before anything else happens.  On high latency networks this can take hundreds of milliseconds, and will ruin any chance of breaking the 1000ms time to glass mobile barrier.  This is why in that video Ilya suggests inlining the critical CSS that you need to render the page. 

The Current State of Affairs

All of this begs the question: “What is the current state of things?  Do big sites already know this, and put their CSS on the root domain?”  I set out to answer this question, and empirically prove which approach wins out - putting CSS on the root domain and incurring the overhead of some extra cookies, or putting it on a cookieless domain and suffering the extra DNS lookup and TCP connection.  I started by surveying the top 10 US Alexa sites (Google, Facebook, YouTube, Yahoo!, Wikipedia, Twitter, LinkedIn, Amazon, Wordpress, and eBay) to see where they stand.

Almost universally, the top sites (who are obeying this best practice and others) are putting their CSS on a cookieless domain.  Google is an outlier because the search page is simple and all CSS is inlined, most likely for the reasons that Ilya outlines in the video above.  Among the rest, Wordpress is the only site that isn’t serving static assets from a cookieless domain, and it’s not clear if this is a deliberate choice to improve performance, or if it is just to reduce complexity.  Armed with this knowledge, I set up a test to see if these sites could be hurting their performance with this approach.

Experimentation

To make the experiment as realistic as possible, I selected five sites from the ones above to test, and made sure that they all had real content on their homepages as opposed to just a login splash page:
  1. Amazon
  2. YouTube
  3. Yahoo!
  4. eBay
  5. Wikipedia
I wanted to eliminate as many external factors as possible, like server side load time, different locations, and network variations, so my methodology was the following:
  1. Save the HTML from each of the sites above to a VM that I control. The server is the smallest DigitalOcean VM, running nginx on CentOS 6.4.
  2. Ensure that all static resources are still coming from the original domains.
  3. Run each site through 9 WebPagetest runs using Chrome (first view only) with a cable connection, and take the median run as the “before” time.
  4. For the “after” times, manually download all of the CSS referenced by each site to my server, and reference it relatively (e.g. /css/amazon1.css).  Ensure that the same number of CSS files are being downloaded, and that the sites still look identical to the “before” runs.
  5. Use nginx to set a 500 byte cookie, to simulate the downside of having static assets on a cookied domain.  
  6. Run 9 more WebPagetest runs for each site.
This approach gives a pretty clear comparison between serving CSS from the root domain and from a cookieless domain.  I selected 500 bytes as the cookie size because it loosely matches the site that was setting the largest cookies from my test population (eBay).  The other four sites set significantly fewer cookie bytes.  Your results may vary if your site sets many kilobytes of cookies, but then again if that’s the case perhaps you should consider reducing the number of cookies you set. One approach that works well is to set a unique identifier in the cookie and store the data on the server, so you don’t need to ship those bytes back and forth on every request.

Results

The raw data is at the bottom of the post, but the results are conclusive for this particular test - putting CSS on the root domain is a clear win for all sites that I tested.  Here is the average improvement for the metrics that I measured:

Metric
Percentage Improvement
Load Time
10.3%
Start Render
27.8%
20.8%

The tests showed a significant decrease in both Start Render Time and Speed Index after moving CSS to the root domain, and the latter metric is rapidly becoming the go-to option for synthetic tests.  This is a huge improvement for an extremely small change.  It’s worth pointing out that the CSS file(s) on the root domain should still be cached at your CDN.  Even if you aren’t caching HTML, you can still put your root domain through a CDN and configure it to only cache certain content types (like text/css).  There is another benefit from doing this: putting your root domain through a CDN allows you to terminate client TCP/TLS connections at the edge, dramatically reducing the latency that your users experience.  

Moving your CSS to the root domain has other benefits in the world of SPDY/HTTP 2.0.  Header compression means that the cost of extra cookies is much lower, and HTTP multiplexing allows you to reuse TCP connections for multiple resources on the same domain.  As long as you keep your cookies to a reasonable size (ideally well under a kilobyte), they won't cause an extra round trip, and you will be much better off having your critical resources on the same domain as the base page, with or without SPDY.  

These results could be further validated by re-running this test with different network characteristics, browsers, and locations, but the numbers are so large that I would expect a similar directional change regardless of the specific test configuration.  The fact that I ran this on a cable connection is relevant as well - for people on mobile networks this should have an even bigger impact, since DNS lookups and TCP connections take much longer when latency is high.  I'm hoping to validate these assumptions with real user monitoring data soon.  

Conclusion

First of all, I want to be clear that this experiment only looked at the impact of moving CSS to the root domain.  It is almost certainly the case that keeping images on a cookieless domain is a performance win.  Images don’t block render, there are typically a lot of them on a page, and having them on another domain can make CDN configuration easier.  When it comes to JavaScript, assuming you are deferring JavaScript in some way, having it on a cookieless domain is probably a good thing as well.  If your JS is loaded via blocking script tags in the head, you might want to experiment with putting it on the root domain (or just move it out of the head).  

With that caveat out of the way, I believe that this experiment conclusively shows that every site currently loading its CSS from a cookieless domain should test loading it from the root domain, to see if it improves performance and business metrics.  Synthetic tests are useful, but it would be great to see some real world data on what kind of impact this has on actual users. There could be implementation details that made this test look more effective than it will be on actual sites, and Etsy will hopefully be publishing data on how this impacts production traffic in the near future.

To me this shows that we have to think logically about the rules we are following.  In this particular case we are dealing with two valid best practices: reduce the number of bytes on the wire, and reduce the number of DNS lookups on your page.  In cases like this where there are tradeoffs to be made, it’s worth testing which approach works best for your content.

Raw Data

AmazonBeforeAfterDifference% Difference
Load Time1.2820.956-0.32625.43%
First Byte0.110.1150.005-4.55%
Start Render0.6930.593-0.114.43%
Speed Index922706-21623.43%

YouTubeBeforeAfterDifference% Difference
Load Time1.1031.3840.281-25.48%
First Byte0.1120.109-0.0032.68%
Start Render1.0860.695-0.39136.00%
Speed Index71766406-77010.73%

Yahoo!BeforeAfterDifference% Difference
Load Time2.8282.33-0.49817.61%
First Byte0.1150.1170.002-1.74%
Start Render1.3910.797-0.59442.70%
Speed Index16331308-32519.90%

eBayBeforeAfterDifference% Difference
Load Time9.228.528-0.6927.51%
First Byte0.1190.118-0.0010.84%
Start Render1.190.889-0.30125.29%
Speed Index49353453-148230.03%

WikipediaBeforeAfterDifference% Difference
Load Time0.8940.659-0.23526.29%
First Byte0.1220.11-0.0129.84%
Start Render0.9930.787-0.20620.75%
Speed Index1000800-20020.00%

Tuesday, December 10, 2013

Reducing Domain Sharding

This post originally appeared on the Perf Planet Performance Calendar on December 7th, 2013.

Domain sharding has long been considered a best practice for pages with lots of images.  The number of domains that you should shard across depends on how many HTTP requests the page makes, how many connections the client makes to each domain, and the available bandwidth.  Since it can be challenging to change this dynamically (and can cause browser caching issues), people typically settle on a fixed number of shards - usually two.  

An article published earlier this year by Chromium contributor William Chan outlined the risks of sharding across too many domains, and Etsy was called out as an example of a site that was doing this wrong.  To quote the article: “Etsy’s sharding causes so much congestion related spurious retransmissions that it _dramatically_ impacts page load time.”  At Etsy we're pretty open with our performance work, and we’re always happy to serve as an example.  That said, getting publicly shamed in this manner definitely motivated us to bump the priority of reinvestigating our sharding strategy.  

Making The Change

The code changes to support fewer domains were fairly simple, since we have abstracted away the process that adds a hostname to an image path in our codebase.  Additionally, we had the foresight to exclude the hostname from the cache key at our CDNs, so there was no risk of a massive cache purge as we switched which domain our images were served on.  We were aware that this would expire the cache in browsers, since they do include hostname in their cache key, but this was not a blocker for us because of the improved end result.  To ensure that we ended up with the right final number, we created variants for two, three, and four domains.  We were able to rule out the option to remove domain sharding entirely through synthetic tests.  We activated the experiment in June using our A/B framework, and ran it for about a month.

Results

After looking at all of the data, the variant that sharded across two domains was the clear winner.  Given how easy this change was to make, the results were impressive:

  • 50-80ms faster page load times for image heavy pages (e.g. search), 30-50ms faster overall.
  • Up to 500ms faster load times on mobile.
  • 0.27% increase in pages per visit.

As it turns out, William’s article was spot on - we were sharding across too many domains, and network congestion was hurting page load times.  The new CloudShark graph supported this conclusion as well, showing a peak throughput improvement of 33% and radically reduced spurious retransmissions:

Before - Four Shards


After - Two Shards

Lessons Learned

This story had a happy ending, even though in the beginning it was a little embarrassing.  We had a few takeaways from the experience:

  • The recommendation to shard across two domains still holds.
  • Make sure that your CDN is configured to leave hostname out of the cache key.  This was key to making this change painless.
  • Abstract away the code that adds a hostname to an image URI in your code.
  • Measure everything, and question assumptions about existing decisions.
  • Tie performance improvements to business metrics - we were able to tell a great story about the win we had with this change, and feel confident that we made the right call.
  • Segment your data across desktop and mobile, and ideally international if you can.  The dramatic impact on mobile was a huge improvement which would have been lost in an aggregate number.  

Until SPDY/HTTP 2.0 comes along, domain sharding can still be a win for your site, so long as you test and optimize the number of domains to shard across for your site.