Scaling the Internet with 6500 Switches Ternary Content Addressable Memory(TCAM) Customization
The Internet routing table size has continued to grow steadily. In 2008 we reached 256K routes and now the table has exceed 512K Routes. This is of significance for customers running some of the older PFC3 based Supervisor 720 engines on the 6500 and 7600 switches.
On the Catalyst 6500 and 7600 Series platforms, all of the routing information is stored in special high-speed memory called TCAM.
The PFC3BXL and PFC3CXL support one millon (1M) IPv4 routes and 512,000 (512k) IPv6 routes in TCAM. However, default outputs look different:
6500#show mls cef maximum-routes FIB TCAM maximum routes : ======================= Current : --------- IPv4 + MPLS - 512k (default) IPv6 + IP Multicast - 256k (default)
This blog focuses on the FIB TCAM; however, the information in this blog can also be used in order to resolve these error messages:
%MLSCEF-SP-4-FIB_EXCEPTION_THRESHOLD: Hardware CEF entry usage is at 95% capacity for IPv4 unicast protocol %MLSCEF-DFC4-7-FIB_EXCEPTION: FIB TCAM exception, Some entries will be software switched %MLSCEF-SP-7-FIB_EXCEPTION: FIB TCAM exception, Some entries will be software switched
It is important to be aware that the aforementioned errors are reported only once when the TCAM limit is exceeded. Even if the total number of routes withdraws below the configured limit, the switch remains in the exception state until it is cleared:
7600#show mls cef exception status Current IPv4 FIB exception state = TRUE Current IPv6 FIB exception state = FALSE Current MPLS FIB exception state = FALSE
As a result of this TCAM exception condition, connectivity is affected and might result in elevated CPU usage due to software switching.
Solving the Internet Routing Table exceeding 512K Problem
The default numbers for PFC3BXL/PFC3CXL are 512k IPv4 routes and 256k IPv6 routes. These numbers can be increased to 1M or 768K IPv4 routes if you enter mls cef maximum-routes ip|ipv6  and reload. But, you cannot achieve both 1M IPv4 AND 512k IPv6 routes at the same time. If you increase the IPv4 TCAM size above the default value, it automatically takes up the IPv6 space and vice versa.
6500(config)#mls cef maximum-routes ip 1007 6500#show mls cef maximum-routes FIB TCAM maximum routes: ======================= IPv4 - 1007k MPLS - 1k (default) IPv6 + IP Multicast - 8k (default)
6500(config)#mls cef maximum-routes ipv6 503 6500#show mls cef maximum-routes FIB TCAM maximum routes: ======================= IPv4 + MPLS - 16k (default) IPv6 - 503k IP Multicast - 1k (default) Note: The Sup2T supervisor engine and DFC4XL modules support 1,000,000 routes that are dynamically shared between IPv4 and IPv6 by default and wont face this issue.Tags: