Last year, on this same blog, we published a post whose title was “The True Story Behind the Cisco Identification Port” -- which you’re invited to read now if you haven’t done so already. In short, back in the days of IOS 11.0, a “feature” was implemented on the Cisco IOS code handling the TCP protocol: if a TCP SYN packet destined to the router was received on port 1999/tcp, a RST would be sent back, but within the payload of the TCP datagram, Cisco IOS would insert the string cisco. This “feature” was put in place on purpose -- its goal was to be able to quickly determine over the network if a given IPv4 host was running Cisco IOS. Cisco removed this “feature” starting with Cisco IOS release 12.0 mainline through CSCdk85821.
Now let me tell you a similar story -- but with more twists and suspense
Back in 2003, a Cisco Technical Assistance Center (TAC) engineer made a very simple request: in order to troubleshoot IPv6 issues which could be related to the Cisco IOS switching path, it would be nice to have a way to generate (within Cisco IOS itself) an IPv6 packet that, when received by a Cisco IOS device, would be punted to the CPU and out of the CEF path. This was indeed considered useful, and was hence implemented as an option to the “IPv6 extended ping” functionality within Cisco IOS. The original implementation was as follows:
- On the sending side: When performing an “IPv6 extended ping”, the user would be asked if a Hop-by-Hop (HBH) extension header (EH), a Destination Options EH, or both EHs should be inserted between the IPv6 header and the ICMPv6 header of an outgoing ICMPv6 Echo Request. Either EH would carry a single PadN option of size 4, with the PadN data being 0x00 bytes.
- On the receiving side: The Cisco IOS code handling ICMPv6 was modified to check for the presence of an HBH EH on any IPv6 packet carrying an ICMPv6 Echo Request received by the device and whose destination IPv6 address was any IPv6 address assigned to any of the device interfaces. When replying to the Echo Request with an Echo Reply, if such an HBH EH was present, Cisco IOS would also insert an HBH EH on the outgoing IPv6 packet -- payload of the HBH EH again being a single PadN option of size 4, with the PadN data being 0x00 bytes.
A Cisco IOS device, on reception of the ICMPv6 Echo Request packet with the HBH EH (if the device was located in the path between the source and destination), or with the Destination Options EH (if the device was the final destination) would then punt the packet to the CPU to process the EH. And because the ICMPv6 Echo Reply sent back to the source (if the device was the packet’s final destination) also included the HBH EH, you would be able to troubleshoot the switching path on both ends from one end. The TAC hence got another tool to troubleshoot customer issues and everyone moved on to other things.
Fast forward now to 2005. Cisco tasks a third party to verify the Cisco IOS IPv6 stack implementation for compliance with all relevant IPv6 standards at that time. One of the tests executed by this third party involved sending to the device under test an ICMPv6 Echo Request with an unknown option within a HBH EH. The option type highest-order two bits (specifying the action to take during processing of the option) are 00 -- skip over this option and continue processing the header. The expected result (according to the third party): for the device under test to ignore the unknown option within the HBH EH and reply with an ICMPv6 Echo Reply, without a HBH EH.
But because of the previously stated changes made in 2003 to the Cisco IOS ICMPv6 code, Cisco IOS would send back an ICMPv6 Echo Reply containing a HBH EH. The third-party flagged this as being non-compliant. After some discussions with the third party, it was decided that it would be easier to change the Cisco IOS behavior than to convince the third party to change their PASS/FAIL criteria for this test.
And here is where things get interesting. The easiest and quickest fix would have been to change the Cisco IOS ICMPv6 code so an ICMPv6 Echo Reply would never contain a HBH EH, but this would mean removing the functionality added back in 2003 at the TAC’s request. But then again, keeping that functionality means not passing the test. And then someone comes up with an idea: how about if we tweak the code one more time? So the behavior after this tweak becomes:
- On the sending side: The “IPv6 extended ping” would still insert (when so specified by the user) the HBH EH, or the Destination Options EH, or both EHs between the IPv6 header and the ICMPv6 header of an outgoing ICMPv6 Echo Request, but with a twist: the HBH EH would still carry a single PadN option of size 4 but the option data would now be the magic value 0x0c01050c instead of 0x00 bytes.
- On the receiving side: As before, the Cisco IOS code handling ICMPv6 was modified to check for the presence of an HBH EH on any IPv6 packet carrying an ICMPv6 Echo Request received by the device and whose destination IPv6 address was any IPv6 address assigned to any of the device interfaces. If an HBH EH was present, it would send back an ICMPv6 Echo Reply and would also insert an HBH EH on the outgoing IPv6 packet, but only if the HBH EH on the received IPv6 packet was carrying the PadN option with the magic value on it. And the HBH EH on the reply IPv6 packet would also be formatted in the same way -- a single PadN option, size 4, with the magic value as data. But if an HBH EH was present, but without the magic value, then the outgoing IPv6 packet carrying the ICMPv6 Echo Request would NOT contain an HBH EH at all.
Now everyone is happy -- the TAC gets to keep its tool, the third party doesn’t have to change their PASS/FAIL criteria for the test, and the Cisco IOS IPv6 stack passes all tests.
Is everyone indeed happy? Do we see any problems here?
The side effect of the last round of changes should by now be obvious: a way to identify whether an IPv6 host is running Cisco IOS has been provided, and all that’s needed is for the IPv6 packet carrying an ICMPv6 Echo Request destined to the device to contain some… magic.
This issue is being tracked as CSCtq02219 -- “Magic number in hop-by-hop options padding allows IOS fingerprinting” and will be fixed in newer Cisco IOS releases. Cisco IPS appliances (starting with signature update S576 from 06/21/2011) include signature #36606 to look for this type of ICMPv6 Echo Request traffic. I’m also providing a packet capture here of the behavior described so far.
Now, is there a “lessons learned” here somewhere? YES: that even what looks on the outside as a trivial change and a clever kludge can have unexpected, or unseen at the time, side effects. And that those side effects can be relevant from a security point of view.