Keynote Competitive Research has just released the results from its U.S. Holiday Impact Study, a Web site performance monitoring study that examined the technical performance and service levels of 26 leading retail Web sites over the course of the 2007 holiday season. The retailers included, Amazon, Barnes and Noble, Wal-Mart, Best Buy, Circuit City, Sears, Raget, Victoria's Secret, Cabelas, Dell among others. To measure the service levels, Keynote used its Transaction Perspective automated testing agents to mimic the actions of a consumer using the Internet Explorer browser to interact with sites. The results were as below:
| Site Reliability (Highly Available, Low Downtime) | Site Responsiveness (Response time, Transactions Completed) | |||||
| #1 | Cabelas | #1 | Victoria’s Secret | |||
| #2 | Barnes and Noble | #2 | Circuit City | |||
| #3 | Best Buy | #3 | Cabelas | |||
| #4 | Toys R Us | #4 | Best Buy | |||
| #5 | Target | #5 | REI | |||
Very few sites struggled with Load Handling throughout the entire holiday period
- Does not mean that on peak days there were not issues
- It does mean that overall the sites planned appropriately for the period load
- Only 3 of the 26 sites had any significant issue with this metric
Availability was mixed
- Top 10 sites better than 98.5%
- Top 7 above 99.0%
- Best Buy, Barnes and Nobel and Cabelas above 99.5% for the entire period (Stellar!)
- 8 sites under 97%
Outages showed similar mixed results
- Cabelas and Barnes and Noble did not report a single hour of outage
- 3 sites reported over 20 hours of peak period downtime
Dial-Up users would have had a rough season
- Keynote captured sites with Home Pages taking over 100s (one taking over 300s)
- However, once users were past Home Page and Search Results most sites provided an acceptable experience for dial-up users (24s per page on average)
Broadband performance showed similar “divide”
- Best sites averaged less than 1s per page while the bottom performers averaged over 3.5s
- Search, Home Page and Product Details were generally the slower areas of the site
Post a Comment