Responding to CRN’s 802.11n Testing
CRN recently published an article profiling 802.11n performance testing of the Aironet 1250 Series access point. The article titled”Where’s Cisco with Wireless N?” (http://www.crn.com/networking/206905063#community) profiles the results of their testing of Cisco’s 802.11n solution. The article was part of a larger story testing 802.11n products including Meru and Ruckus. Cisco declined to participate in the CRN product review due to basic differences in product testing methodology. Cisco asserts that the review of its 802.11n access point as presented by CRN misses some of the basic principles of product testing and as such is inconclusive on many levels. Firstly, CRN tested a product that was operating with an early version of software. As with any new hardware, users should seek the latest software to ensure optimal performance. The code for Cisco’s 802.11n access point has evolved quickly and the version used by CRN had already been through 4 subsequent releases. The most recent product code is always available for download from the Cisco website -a point that was overlooked by the CRN staff.Secondly, CRN tested the access point with several client devices that were using old device drivers and had not been Wi-Fi Certified for the 802.11n draft 2.0 standard. In fact, the Toshiba Portg M700-S7002 -the only device to have Wi-Fi 802.11n certification -was tested using drivers from early 2007, predating any 802.11n Wi-Fi certification. Wi-Fi certification is imperative when benchmarking performance results. The refusal to update device drivers clearly added to incompatibilities and limited performance results. Furthermore, CRN tested only the 2.4GHz radio with little detail given as to the use of 20-MHz or 40-MHz channels. While 802.11n operates effectively in the 2.4GHz frequency, optimal performance results are realized in the 5GHz frequency. The methodology behind the CRN performance tests is questionable. The Cisco access point was tested without a WLAN controller -differing from its intended configuration. No attention was paid to using the latest software drivers or product software. CRN provides no transparency to product sourcing and configuration methodology and refused to offer Cisco the opportunity to troubleshoot for fear of revealing the source of the product and its configurations.For an unbiased and transparent review of Cisco’s product performance, readers should shift their attention to the Network Computing performance review. The performance review is summarized in the article,”First Look: Cisco’s 1250 802.11n AP” and can be read here: http://www.networkcomputing.com/channels/wireless/showArticle.jhtml?articleID=206901320. This review shows the performance of the Aironet 1250 as reaching 154.9 Mbps. The 300 Mbps as referenced by CRN is the link speed. It is well understood that the TCP/UDP throughput reported in performance tests is approximately half that of the link speed (due to overhead associated with the 802.11 standard). Cisco has achieved phenomenal success with the Aironet 1250 with over 30,000 units shipped – the fastest selling access point in Cisco history. Leading customers like Duke University have shown real-world performance results just shy of 130 Mbps. CRN fell short on delivering a complete and transparent test due to flawed testing methodology. Only by testing products with the latest drivers and software code can CRN expect to receive results commensurate with the true performance of the product.