Cisco Logo


Security

Updated May 9th: After a thorough investigation of the TCP Split Handshake issue raised by NSS Labs, Cisco has confirmed that the Cisco ASA firewall is not susceptible to this issue. In all test cases examined, the ASA operates as expected, providing protection in its default configuration against the Split-Handshake as defined in the original TCP Split Handshake paper. As a result, the Cisco PSIRT closed this investigation on May 4th.

Cisco appreciates the extended engagement and data provided by NSS Labs as we’ve worked through these scenarios. During two recent visits to NSS Labs, Cisco was presented with a number of scenarios, including new test cases that deviated from the original Split-Handshake scenario. The Cisco PSIRT collected traces and provided feedback to NSS Labs on all scenarios. In each case, Cisco demonstrated successful network protection through the default ASA configuration or the implementation of firewall policies that are fully supported, documented and used pervasively in enterprise deployments.

As always vulnerability reports should continue to be reported to the PSIRT organization (psirt@cisco.com). Cisco customers are encouraged to contact their account manager with any questions.


Recently there’s been some activity in the press regarding an NSS Labs report on potential vulnerabilities in Next-Generation Firewalls (NGFW). The Cisco Adaptive Security Appliance (ASA) was one of the products mentioned as vulnerable to these attacks. Based on the investigation of this issue to date, the data indicates that Cisco customers are not exposed to this issue. As always, should the vulnerability be confirmed the Cisco Product Security Incident Response Team (PSIRT) will investigate, drive remediation and disclose per our normal communication channels. (PSIRT Vulnerability Policy)

On April 12th, NSS Labs published a report regarding vulnerabilities on a number of firewalls, including Cisco’s ASA product line. The full report has a hefty $3500 price tag, but NSS does provide a free (with registration) “Remediation Guide,” for users of these firewalls.

The NSS Labs Remediation Guide incorrectly lists the Cisco ASA as vulnerable to the TCP Split Handshake attack, and also mentions that there are no steps available to customers to mitigate or remediate this attack.

Following an investigation over the course of several months, involving well over a dozen Cisco engineers from various teams and working in conjunction with NSS Labs, no vulnerability of this nature has been observed on Cisco products. The following products have been investigated:

It’s important to note that the NSS Labs report focuses only on one attack called the TCP Split Handshake, which is a third means to initiate TCP sessions that combines features of both the three-way handshake and the simultaneous-open connection.

However, the goal of this post isn’t to discuss the technical details of TCP handshakes, but rather to present what Cisco has done and is doing to investigate the impact to our products and protect our customers.

Timeline

NSS Labs approached the Cisco PSIRT in January of this year with the TCP Split Handshake attack and indicated that, during an investigation at another site, NSS reported that the Cisco ASA improperly permitted the TCP split handshake negotiation. At that time, NSS Labs provided Cisco the test scripts they used at the customer site and asked that we investigate. NSS did not collect or provide Cisco any configuration information or packet captures to demonstrate the behavior they observed.

As part of our standard investigation process, we filed bugs to document and investigate the issues, not only for the ASA, but other potentially affected products such as the Cisco IOS Firewall feature (IOS-FW) and the Cisco Intrusion Prevention System (IPS).

Once we set to work trying to reproduce the issue on the ASA, we began freely exchanging our lab configuration and testing results with NSS and asking for any additional guidance they could provide. To date, Cisco has tested using numerous configuration, software and platform combinations, and all of the aforementioned products have blocked the TCP split handshake negotiation correctly. NSS no longer had access to an ASA, so they have been unable to reproduce the suspected behavior or provide any detailed information to aid the investigation.

Fast-forward to April, and we’re still unable to reproduce the TCP split handshake issue. Last week we sent NSS Labs a Cisco ASA in the hopes that they can gather some evidence of their claims and we are awaiting their test results. The Cisco PSIRT has made the bugs that were filed for investigation public, and based on the lack of evidence has closed them effective today. The Cisco PSIRT will continue to work with NSS and re-open the bugs should an issue be discovered.

And since only customers with service contracts can access the Cisco bug toolkit to review the bugs, Cisco will increase the transparency on this issue by also documenting the investigation via IntelliShield Alert 22462 which has been made public to all Cisco.com visitors.

In an effort to keep conversations fresh, Cisco Blogs closes comments after 90 days. Please visit the Cisco Blogs hub page for the latest content.

29 Comments.


  1. The report looked very biased and concluded that only one product which is a competetor to Cisco was the only one to work. They left off a number of devices from multiple vendors that also compete in the enterprise arena. To, me, this smells of a marketing sham all the way instead of a valid security test that can be validated independently.

       0 likes

  2. “The Cisco PSIRT has made the bugs that were filed for investigation public”

    Can you point a link to them? I can’t find it anywhere.

    Your post doesn’t contain sufficient technical details. It appears the reason you couldn’t reproduce the bugs is because you didn’t understand them — a common mistake with security bugs.

       0 likes

    • Hi Robert, thanks for the comments. We placed the bug id’s in the Intellishield report.
      http://tools.cisco.com/security/center/viewAlert.x?alertId=22462

      The release notes for the bug will not be viewable unless you have a support agreement. If you don’t have a support agreement and want to track any future state changes in the bug, you can bookmark the Intellishield alert as we will update it with any changes that may take place.

      Release-note for CSCtn29288

      This bug is to investigate and track the TCP Split Handshake attack discovered and reported by NSS Labs. Cisco PSIRT is aware of the report and has been working with NSS for several months and in that time have been unable unable to reproduce and confirm any new security vulnerabilities in Cisco products. PSIRT will disclose any security vulnerabilities discovered in compliance with Cisco’s security vulnerability policy: http://www.cisco.com/en/US/products/products_security_vulnerability_policy.html As a matter of policy, Cisco takes security vulnerabilities very seriously and we continue to take active measures to safeguard the security and reliability of our equipment. Cisco has been working with NSS Labs on the TCP split handshake spoof since early this year. In this time we have been unable to confirm any new security vulnerabilities in Cisco products, and have demonstrated the Cisco ASA protects against this attack. The Cisco PSIRT continues to work with NSS Labs, and will follow our well-established disclosure process should any new information come to light.

      We believe we have a good understanding of the issue as reported. We’ve approached the issue via both testing and source code inspection. In all scenarios observed to date, the ASA properly denies the connection as we would expect. Hopefully this helps.

         0 likes

  3. Love the transparency in dealing with this “issue”. Check this blog http://labs.mudynamics.com/2011/04/14/splits-handshakes-bananas-and-ransom-notes/ and if you want to test, here’s the pcap: http://www.pcapr.net/view/kowsik/2011/3/4/15/tcp_split_handshake.pcap.html

       0 likes

    • So kowsik, are you saying you have played the pcap through the Cisco ASA and can show that it is vulnerable? Because Cisco is saying that they have been trying with various configs and have never been able to make the ASA “fail” the test. NSS hasn’t even been able to provide any details or proof of the vulnerability of specific devices.

         0 likes

      • Nope, we are simply providing the sample pcap for anyone to test their own firewalls instead of wondering if you are susceptible to the evasion. Right now there’s a lot FUD around this issue because there’s the paper and the NSS report, but no concrete tests to prove one way or the other.

           0 likes

  4. Would not the IP Audit signature 3041 (TCP SYN & FIN Flags Only) detect this condition and drop the packet? Or more precisely, wouldn’t the addition of an IPS (either installed within an ASA or added inline) detect and block this type of event?

       0 likes

    • Scott,

      Thanks for your comment. My blog post on this issue was focused on the ASA in standalone mode.

         0 likes

      • I think Sam’s question is valid in this context.

        The IPS modules provide supplemental protection above ASA’s native capability, right?

        Plus, your original blog post says the “Cisco Intrusion Protection (IPS) Appliances” … “have been investigated.”

           0 likes

        • Thanks for your comments. Apologies for the delay in responding. The question is valid, however, the NSS report focused on the ASA without IPS functionality, so my response to the NSS report is focused on the firewall functionality of the ASA.

          The IPS module and appliances do provide protection against this attack. The IPS will recognize the simultaneous open transition as valid and will drop the subsequent (2nd) SYN packet while raising the 1330.17 (TCP Segment out of State Order) alert. This essentially kills the entire connection attempt.

             0 likes

  5. Any further update from cisco post the visit to NSS labs ?

       0 likes

    • We’ve updated the blog post as of today. Thank you for your inquiry.

         0 likes

  6. “Phatak says NSS Labs did its best to supply Cisco with configuration information and vulnerability scripts. Cisco representatives are expected to be at NSS Labs today to participate in the vulnerability-assessment on site and sort out any issues directly. A Cisco spokesperson indicated that Cisco expects to write an updated blog post about all of this later today. NSS Labs also expects to publish updated findings related to what firewalls it tested have completed remediation to protect against the TCP Split Handshake attack. ”
    http://www.channelworld.in/news/cisco-going-nss-labs-sort-out-alleged-firewall-issues-72902011

    what is current situation?

       0 likes

    • Thanks for your comment. I’ve updated the Cisco blog post with new information.

         0 likes

  7. I gave a look to the updated NSS Labs Remediation report and it still seems a little bit controversial as far as Cisco is concerned. Literally: “Unlike every other tested vendor, Cisco’s approach to defend against the TCP Split Handshake is based upon Access Control Lists (ACLs)”.

    It does not sounds so clear to me. Does it mean that TCP Split Handshake is blocked only if it matches an existing ACL (even if that specific traffic is allowed)? Or what else? May you kindly provide further details?

       0 likes

    • Thanks for your question. The answer provided below comes from 2 of the engineers directly involved in the testing with NSS. Hopefully it provides some additional clarity.

      There are 2 connection establishment handshakes associated with this
      topic, they are as follows:
      * Split Handshake (primary concern/issue)
      * Simultaneous Open

      By default, the Cisco ASA accelerated security path (asp) prevents
      both the “Split Handshake” and “Simultaneous Open” using the
      “tcp-dual-open” connection check. The Cisco ASA firewall drops the
      TCP SYN segment sent from the server (eg: fakestack.rb) when there
      is an embryonic TCP connection already open between two endpoints.

      However, NSS created and demonstrated a brand new test-case which
      deviates from the 2 connection establishment handshakes mentioned
      above along with the most commonly used 3-way handshake. This new
      test-case is not compliant with the TCP connection establishment
      requirements defined in RFC 793.

      For the “Split Handshake”, the first TCP segment sent by the server
      (fakestack.rb) in response to the clients TCP SYN segment is a TCP
      ACK segment (also described in the paper, The TCP Split Handshake:
      Practical Effects on Modern Network Equipment, pg. 200). However,
      for the new test-case a TCP RST/ACK segment is sent instead. At
      this point the client would be in a state called SYN_SENT and the
      server in the SYN_RCVD state.

      The protocol specifications for TCP (defined in RFC 793) define how
      to process TCP segments received in certain states. When an
      endpoint is in a SYN_SENT state and it receives a TCP RST/ACK
      segment the endpoint aborts and closes the connection.

      During our testing, the client ignores the TCP RST/ACK segment sent
      by the server (fakestack.rb) and does not abort the connection.
      Upon seeing the TCP RST/ACK segment sent by the server
      (fakestack.rb) the Cisco ASA firewall tears down the connection
      slot. Immediately following the TCP RST/ACK segment sent by the
      server (fakestack.rb) it sends a TCP SYN segment which initiates a
      *new* connection establishment and completes a 3-way handshake that
      complies with the TCP specifications defined in RFC 793.

      For the new test-case, access control list rules can be applied
      using an access-group and used as additional countermeasures to
      mitigate and prevent unsolicited connection attempts between the
      endpoints for a TCP conversation when the client does not abort the
      connection as defined in the RFC protocol specification for TCP.

      Joe Karpenko, Omar Santos

         0 likes

      • Thanks for your kind and complete reply. Now the testing scenario is fully clear.

        Honestly speaking, NSS should have given more details in the remediation guide, since their rapid description of Cisco behaviour is a little bit misleading.

           0 likes

  8. Hello Russ,

    what do you mean about

    “For the new test-case, access control list rules can be applied using an access-group and used as additional countermeasures to mitigate and prevent unsolicited connection attempts between the endpoints for a TCP conversation when the client does not abort the connection as defined in the RFC protocol specification for TCP.”

       0 likes

    • Roberto,

      Thanks for your question.

      You can use access lists tune outbound policies initiated by hosts on higher security level interfaces to lower security level interfaces. As an example, you might want to block any outbound session initiation except to update servers that would be hosted on the internet which is a lower security interface. By default, all traffic is allowed to source from higher security interfaces to lower security interfaces.

      Hopefully that helps.

         0 likes

      • Russ,

        Can you please explain the following because my read of RFC 793 doesn’t indicate what was posted from the 2 engineers

        The protocol specifications for TCP (defined in RFC 793) define how
        to process TCP segments received in certain states. When an
        endpoint is in a SYN_SENT state and it receives a TCP RST/ACK
        segment the endpoint aborts and closes the connection.

        RFC 793 says

        If the
        receiving TCP is in a non-synchronized state (i.e., SYN-SENT,
        SYN-RECEIVED), it returns to LISTEN on receiving an acceptable reset. If the TCP is in one of the synchronized states (ESTABLISHED,
        FIN-WAIT-1, FIN-WAIT-2, CLOSE-WAIT, CLOSING, LAST-ACK, TIME-WAIT), it
        aborts the connection and informs its user.

        I am not sure if the term receiving here is referring to the server side or any socket (client or server) that is receiving a RST.

        Irregardless, if the ASA tears down the connection how is it possible for fakestack.rb to

        it sends a TCP SYN segment which initiates a
        *new* connection establishment and completes a 3-way handshake that

        Wouldn’t the ASA deny this by default as it it is a session initiated from a low security to a high security interface?
        Can you provide an example of how ACL’s would be used to mitigate this issue ? Or why they are required ?

           0 likes

        • Hi needtoknow,

          Thanks for your inquiry regarding this topic. As for your question regarding the protocol specifications for TCP (as defined in RFC 793) and how a receiver processes TCP segments received in certain states. First, it should be noted that either side of a TCP connection can be a receiver, since both the client and server receive (and send) TCP segments and the state of a connection on either the client or server side may be in a different or the same state(s).

          In RFC 793, Section 3.4. Establishing a connection [page 37] under Reset Processing it contains the following two paragraphs:

          In all states except SYN-SENT, all reset (RST) segments are validated by checking their SEQ-fields. A reset is valid if its sequence number is in the window. In the SYN-SENT state (a RST received in response to an initial SYN), the RST is acceptable if the ACK field acknowledges the SYN.

          The receiver of a RST first validates it, then changes state. If the receiver was in the LISTEN state, it ignores it. If the receiver was in SYN-RECEIVED state and had previously been in the LISTEN state, then the receiver returns to the LISTEN state, otherwise the receiver aborts the connection and goes to the CLOSED state. If the receiver was in any other state, it aborts the connection and advises the user and goes to the CLOSED state.

          The state of the socket on the client was never in a LISTEN or SYN-RECEIVED state. The socket on the client performs an active open, which is from a CLOSED to SYN-SENT state. The state of the socket on the server (fakestack in this case) would be considered in a LISTEN state, which receives the TCP SYN segment from the client and transitions to the SYN-RECEIVED state. At this point the server (fakestack) then sends a TCP RST segment to the client. This is NOT a valid RFC 793 defined connection establishment handshake.

          The socket on the client was in neither an initial LISTEN or transitioned to a SYN-RECEIVED state. The 3rd and 4th sentences of the second paragraph under Reset Processing define how the TCP RST segments should be processed. The client (receiver of the TCP RST segment from the server aka fakestack) is supposed to abort the connection and transition the socket to a CLOSED state.

          Non-RFC 793 Compliant Handshake Overview

          The actual issue here is in how certain operating system stacks process TCP segments for a connection while in certain states. For example, if the client uses a BSD-based implementation of TCP (Figure #1 below) and it receives a TCP RST segment from the server (fakestack) while the client is in the SYN-SENT state, the client aborts the connection and transitions the socket to a CLOSED state. However, if the client uses a Microsoft Windows Winsock-based implementation of TCP (Figure #2 below) and the client receives a TCP RST segment while in the SYN-SENT state, it DOES NOT abort the connection upon receiving the TCP RST segment from the server (fakestack). More details follow. ;-)

          Remember, this connection establishment process between the client and server (fakestack) sockets is NOT a valid RFC 793 defined connection establishment handshake (3-way, 4-way, or 5-way).

          Figure #1:  BSD-based Implementation of TCP
          -------------------------------------------
          
               fakestack                                client
               receiver                               receiver
               ===============================================
           1.  LISTEN                                   CLOSED
           2.  SYN-RECEIVED    <---- SYN -----        SYN_SENT
           3.  SYN-RECEIVED    --- RST/ACK -->          CLOSED
           4.  LISTEN                                   CLOSED
           5.  SYN-SENT        ----- SYN ---->          CLOSED
           6.  CLOSED          <-- RST/ACK ---          CLOSED
          
          Figure #2:  Microsoft Windows Winsock-based Implementation of TCP
          -----------------------------------------------------------------
          
               fakestack                                client
               receiver                               receiver
               ===============================================
           1.  LISTEN                                   CLOSED
           2.  SYN-RECEIVED    <---- SYN -----        SYN-SENT
           3.  SYN-RECEIVED    --- RST/ACK -->        SYN-SENT
           4.  LISTEN                                 SYN-SENT
           5.  SYN-SENT        ----- SYN ---->    SYN-RECEIVED
           6.  ESTABLISHED     <-- SYN/ACK ---    SYN-RECEIVED
           7.  ESTABLISHED     ----- ACK ---->     ESTABLISHED
          
          ** In line #3 above, the Microsoft Windows Winsock-based 
          implementation of TCP ignores the TCP RST segment sent by the 
          server (fakestack) and persists in a SYN-SENT state for at least 500 
          milliseconds.
          
          ** In line #5 above, the TCP SYN segment received by the client is 
          within a 500 millisecond window from the TCP segment sent by the 
          server (fakestack) in line #3.
          

          The Microsoft Windows Winsock-based implementations of TCP ignores TCP RST segments on the client/receiver even while in a SYN-SENT state for at least a 500 millisecond period of time due to a TCP Blind In-Window Attack mitigation. In most all situations you would expect TCP Blind In-Window Attack countermeasures to be implemented and active for ESTABLISHED connections. However, Microsoft deviates from the RFC 793 protocol specifications for TCP by implementing this mitigation for sockets when they are also in a SYN-SENT state.

          Switching to the question about the Cisco ASA firewall, the TCP RST/ACK segment sent by the server (fakestack) causes the Cisco ASA firewall to teardown and clear the session at line 3, as covered above in the Non-RFC 793 Compliant Handshake Overview examples. This complies with the RFC 793 Reset Processing in section 3.4. Establishing a connection [page 37].

          Comment from needtoknow:

          Wouldn’t the ASA deny this by default as it it is a session initiated from a low security to a high security interface? Can you provide an example of how ACL’s would be used to mitigate this issue? Or why they are required?

          Yes, you are Correct! If the server (fakestack) is located on a lower-security level interface, by default it would NOT be permitted to send traffic to a higher-security level interface — unless permitted by an access control list rule and policy. However, if the server (fakestack) is located on a higher-security level interface, by default it is permitted to send traffic to any lower-security level interface. This can be restricted through the used of access control list rules and policies and it a recommended Best Common Practice (BCP).

          As for a scenario where access control list rules and policies can be used to mitigate this, lets take an example where a web server (fakestack) is located on a DMZ interface with a security level of 50. The web servers (fakestack) only purpose is to serve transactions for HTTP requests on TCP port 80. Clients that connect to the web server (fakestack) are located on the OUTSIDE interface which has a security level of 0. On the Cisco ASA firewall, there is an access control list rule that permits TCP traffic with a source IP address of any, the destination IP address of the web server (fakestack), and destination TCP port 80.

          In this scenario, since the Cisco ASA firewall is a stateful device, all TCP segments received from a client on the OUTSIDE interface that is connecting to the IP address of the web server (fakestack) on TCP port 80 located on the DMZ interface will be permitted, pending the connection is already established and the TCP segment is valid or the TCP segment is attempting to establish a new TCP connection to the web server. Since the Cisco ASA is stateful, we DO NOT need to explicitly permit traffic sourced from the web server (fakestack) for TCP segments that are for an existing connection or TCP segments that are used to establish a new TCP connection. Due to that, we can create an access control list policy with rules that deny all traffic initiated and sourced from the web server (fakestack) as it SHOULD NOT be performing any active open initiated connections to devices located on other interfaces. Again, this is a recommended Best Common Practice (BCP). Remember, the web server’s (fakestack) only purpose is to serve transactions for HTTP requests on TCP port 80.

          Following is an example Cisco ASA firewall configuration that reflects the preceding scenario:

          !
          interface GigabitEthernet0/0
           nameif OUTSIDE
           security-level 0
           ip address 10.10.10.1 255.255.255.0
          !
          interface GigabitEthernet0/1
           nameif INSIDE
           security-level 100
           ip address 10.20.20.1 255.255.255.0
          !
          interface GigabitEthernet0/2
           nameif DMZ
           security-level 50
           ip address 10.30.30.1 255.255.255.0
          !
          static (DMZ,OUTSIDE) 10.30.30.30 10.30.30.30 netmask 255.255.255.255
          !
          access-list INGRESS_OUTSIDE_POLICY extended permit tcp any host 10.30.30.30 eq 80
          access-list INGRESS_OUTSIDE_POLICY extended deny ip any any
          !
          access-list EGRESS_DMZ_POLICY extended deny tcp host 10.30.30.30 any
          access-list EGRESS_DMZ_POLICY extended deny icmp host 10.30.30.30 any
          access-list EGRESS_DMZ_POLICY extended deny udp host 10.30.30.30 any
          access-list EGRESS_DMZ_POLICY extended deny ip host 10.30.30.30 any
          access-list EGRESS_DMZ_POLICY extended deny tcp any any
          access-list EGRESS_DMZ_POLICY extended deny icmp any any
          access-list EGRESS_DMZ_POLICY extended deny udp any any
          access-list EGRESS_DMZ_POLICY extended deny ip any any
          !
          access-group INGRESS_OUTSIDE_POLICY in interface OUTSIDE
          access-group EGRESS_DMZ_POLICY in interface DMZ
          !

          I hope this response and information helps provide clarity to your comments and questions. Regards,

          Joe Karpenko

             0 likes

  9. Thanks for your comments. We have updated the blog post as of today.

       0 likes

Trackbacks and Pingbacks:

  1. Return to Countries/Regions
  2. Return to Home
  1. All Security
  2. Return to Home