I am only going to say this once.
This is our third episode for what I have personally termed the ‘NX-OS’ exposed series. The first two have been about programmability. Exploring the exposed interfaces, Object Models, the NX Tool kit and more of the NX-API framework that enables DevOps teams to respond more quickly. [TWTV 176 – Open NX-OS Programmability and TWTV 180 – NX-API Rest/Object Models].
Today’s show dives into the automation piece. That simultaneously higher level conversation covering what an organization really cares about, why IT exists and how we finally brought data center switching into the automate-able value chain.
Network automation is an integral part of the overall infrastructure automation. Yet, it has been largely left out of the DevOps movement.
Automation is central to achieving a truly agile datacenter. DevOps has been the cultural movement working towards the realignment of disparate function involved in the IT value stream. It is the modern, ‘digital,’ equivalent of an optimized manufacturing line.
Read More »
Tags: 9k, devops, nexus, NX-OS, Puppet, Puppet Labs, TechWiseTV
Last we spoke, it was about network device configuration management. Let’s move our focus up the stack to applications and management of their configuration. Whether enterprise or cloud-architected, running on physical servers, in virtual machines or in containers, how are you managing your applications?
Puppet, Chef, Ansible and Salt are popular answers to this question and leading contenders for initial provisioning and management of configuration drift of data center applications – whether they be common off the shelf (COTS) or custom built applications. Two of these configuration management technologies, Puppet and Chef, are supported by Cisco Intelligent Automation for Cloud 4.1. The collection of features enveloping these two Ruby-based technologies within Cisco IAC is referred to as Application Configuration Management (ACM).
Approach to Agent Bootstrapping
Puppet and Chef are similar in nature – in more ways that we’ll discuss in this post. An example of similarity being that both of these ACM technologies require an agent (Puppet) or client (Chef) installed on the server under management (node).
Agent Bootstrapping Methods
Both types of ACM technologies support client-only and client/server deployment models, referred to as agent/master for Puppet and client/server for Chef installations. Whether only using an agent-only (client-solo – Chef) or using an agent/master deployment model, unless your virtual or physical server image has the agent preinstalled, you’ll need to go perform the prerequisite work of agent installation.
IAC performs this dirty work by bootstrapping the appropriate agent (or client) whether on initial server provisioning or on-demand on any existing server when a user assigns an application to a server. Mechanics used to perform agent installation varies. The mechanics used within IAC are listed in the “Agent Bootstrapping Methods” chart. Initially, IAC used WinRM as its mechanism to bootstrap agents on Windows severs until customer feedback drove use of an alternative mechanism – psexec. We found that customer security teams were either uncomfortable with or had policy in place against the use of WinRM as a method to execute scripts remotely and made the switch to psexec, which “is a light-weight telnet-replacement that lets you execute processes on other systems, complete with full interactivity for console applications, without having to manually install client software”.
Part of the agent installation involves establishing a connection between the ACM server (Puppet Master or Chef Server) and the node (server with agent/client installed). IAC orchestrates the registration of the node with it’s respectively, assigned ACM server. This process is different depending on whether Puppet or Chef is used. In the case of Chef, IAC has the chef-client register with the Chef server using the private key assigned to the chef-validator, which IAC loads into the node during client installation. In the case of Puppet, IAC performs an initial puppet agent run, which lodges a certificate authorization request on the Master, which IAC subsequently orchestrates the signing of on the Master. With agent bootstrap complete and authorized, secure communication between the ACM server and client, attention is turned to the management of connections IAC may have established with n number of Puppet or Chef servers.
System Health – ACM
Connection and System Health
In the case of client/server deployments, IAC will establish connection to one or more Puppet Masters and one or more Chef Servers. Each connection is treated with care as the health of each connection facilitates IAC’s ability to successfully orchestrate applications. Connections are established using a service account permissioned appropriately. The health of the connection between each ACM server is evaluated once every 30 minutes by default. Connection health is determined by performing connectivity, authentication and authorization tests. Details of these tests and a screenshot of the System Health console can be seen in the “System Health – ACM” chart.
CloudSync Finite State Machine
Cloud Object Model and CloudSync
Immediately after establishing a healthy connection, CloudSync runs. CloudSync is a synchronization process driven by a finite state machine whose responsibility is to not only perform initial object discovery and granular fingerprinting – essentially a deep interrogation of cloud objects and their attributes – but also, manage ongoing reconciliation of infrastructure changes with respect to their representation of the provider’s cloud infrastructure as modeled within the service catalog. Note the “CloudSync Finite State Machine” chart, which is laced with Extension Points, where cloud administrators may insert custom logic on state transition for any given object within the model. Once collected, this inventory (e.g. a Chef Role) is presented to the cloud administrator for the ACM server for use within their cloud. Cloud administrators may choose to register the discovered objects for use by end users.
Cisco IAC Cloud Object Model – Chef
Cisco IAC Cloud Object Model – Puppet
For example, the cloud administrator may choose to register a Puppet Role as being available for end users to assign to a server. Registration of this role may include assignment of additional metadata, including price of the role as a one-time or recurring charge for use of the application and assignment of tenant permissions (whether to make the role available to all tenants or only select tenant(s)).
It’s through the relationships derived within the Cloud Object Model and assignment of tenant permissions that the specific applications are presented to a given end user. Service Resource Containers are used as a logical construct owned by the cloud administrator wherein tenant-specific resources may be hosted. Applications delivered to tenants may be created in a virtual data center that is serviced by either a Puppet Master or Chef Server. See the Cisco IAC documentation for further details on other constructs within these and other models.
Manage Applications – Node Classification
Approach to Node Classification
Once registered for use, applications become visible to end users, who may assign applications to their servers whether during initial server provisioning or to an existing server. Upon selection of application(s) by the end user, IAC classifies the node by writing a hiera file (Puppet) or by writing a run-list (Chef) on the respective ACM server and forces an immediate agent run to ensure application configuration is promptly enforced.
In this sense, IAC provides a common user experience for node classification irrespective of the underlying technology chosen by the cloud provider (the organization running and administering IAC). As the IAC product suite evolves, so has our approach in terms of classification via Puppet and the more programmatically effective use of a custom-written External Node Classifier, taking advantage of the ability for the node_terminus configuration to to interact with an ENC.
Application Configuration Management Highlights
- Integration with Puppet and Chef
- Connections to n number of servers
- System health checks for these servers
- Application infrastructure discovery (CloudSync)
- Bootstrapping of agents (green and brownfield)
- Financial Management
- Pricing of applications
- Showback for application orders
- Run rates including application consumption by user, org, tenant
Financial Management – Application Pricing & Showback
- Tenant-specific application catalogs
- Tenant/application consumption dashboards
- Application provisioning for virtual machines, physical servers
- “My Applications” interface for application management
- Service Offering Elections
- 3-tiers of control on enable/disable application configuration management services at provider, tenant and organization levels
- Multi-Cloud Platform Support
- Support same services ubiquitously across all platforms
Financial Management – Application Run Rates
- Application User Persona
- “My Application” interface for application management
- ACM Server and application usage dashboard
Cognizant of the plethora of application configuration management tools available to Cisco customers, including commercial, open source, and homegrown tools, we’re very interested to hear which ones you have found to be the best fit in your environment. Have you established revision control practices as you manage infrastructure as code?Having reviewed Cisco’s approach within its cloud management platform, IAC, whether you manage configuration of physical servers, virtual machines or use CM to build containers or hosts that run containers, how does your approach compare?
Tags: Chef, CIAC, cisco IAC, cloud-based applications, configuration management, enterprise applications, Puppet
The tenth OpenStack release codenamed Juno was released on October 16, 2014. This press release provides a good summary of what to expect in Juno. It also discusses important new capabilities included in the more than 340 new enhancements built in to Juno and highlights different usecases that showcase the diversity of workloads supported on OpenStack.
In the first part of the Cisco and Openstack Juno Release blog, I covered Cisco’s OpenStack team contributions to the Neutron project. Here I’ll provide details of our contributions to other OpenStack projects as well highlight our development efforts on StackForge. Cisco was the sixth top code reviewer for the Juno release across all projects in Juno release and is Foundation’s fifth largest company in terms of OpenStack membership.
This Nova blueprint was completed in Juno and provides support for configuration and provisioning of instances with SR-IOV port connectivity. The implementation generates SR-IOV specific libvirt domain and network configuration XML for the instances as well as includes the capability to schedule instances based on the compute nodes SR-IOV capabilities. One of the key use-cases for SR-IOV is Network Function Virtualization (NFV) that requires high performance traffic throughput in and out of a virtual machine providing network services (Virtual Network Function or VNF).
We proposed and implemented support for metering Network Services in Neutron using Ceilometer. This included new pollsters and notification handlers for Load Balancer as a Service (LBaaS), Firewall as a Service (FWaaS) and VPN as a Service (VPNaaS). The metrics are categorized into Provider or Service Level, providing different level of details. Provider level metrics help determine the type of implementation and its feature, whereas the Service level metrics provide more granular metric details on the service health and consumption. Separately, instance metrics were enhanced as part of this blueprint to support read and write metrics per instance disk device.
In the Cinder project, Fibre Channel Zone Manager allows FC SAN Zone/Access control management in conjunction with Fibre Channel block storage. It has a pluggable architecture and we contributed the Cisco FC Zoning plugin that automates creation, deletion and modification of zones in zonesets. Zones are configured automatically as part of the active zone set for the specified VSAN in the FC SAN to provide a more flexible and secure way of controlling access.
Enhancements to Horizon to enable configuration of IPv6 subnet modes is also part of the Juno release. This allows tenants to configure address and Route Advertisement (ra) mode for their subnets through the user dashboard. Neutron supports multiple IPv6 address configuration modes including SLAAC and DHCPv6 (both Stateful and Stateless modes).
The Cisco OpenStack team has been actively developing across different projects on StackForge as well. This provides an excellent platform for OpenStack related projects to make use of OpenStack project infrastructure and also continue to collaborate in the open.
OpenStack Services Puppet Modules – One of challenges that we hear about from our OpenStack customers is how to make OpenStack more manageable and deployable. There are several different deployment options for OpenStack and we have tremendous experience with automating the underlying system and service configuration via Puppet. We work with customers, partners and the community to enhance Puppet modules for OpenStack services and integrate with Cisco infrastructure as well. We also recently announced, in collaboration with RedHat, Cisco UCS Integrated Infrastructure that combines Cisco’s server, switching and management technologies with Red Hat’s enterprise-grade OpenStack platform.
Group Based Policy (GBP)– Currently staged on StackForge, this project aims to provide policy abstractions that extend the current Neutron API resources and introduces a declarative policy driven connectivity model that presents application-oriented interfaces to the user. The Group Based Policy framework implementation provides the flexibility for new API resources – End Points, End Point Groups, Contracts and Classifiers – that can be mapped to existing Neutron resources or passed directly to a third party controller. In addition to a mapping driver that supports all existing Neutron plugins, Cisco will also be releasing a driver to directly integrate GBP with its Application Policy Infrastructure Controller.
Nova Solver Scheduler – For resolving complex constraints based on policies and business rules, we have been collaborating with the community to develop a smart Nova Scheduler driver that models compute placement as a supply and demand problem. The intent is for the Solver Scheduler to integrate with the Gantt project that is aiming to separate out the Nova scheduler as a standalone project.
Cisco’s OpenStack team contributions are across numerous projects in OpenStack. Our aim is to work with the community, with our customers and partners to enable more successful OpenStack User Stories, resulting in a win-win situation. We are going to be presenting several general sessions that were selected as part of the community voting process at the upcoming Kilo Summit in Paris. You can find more details in this blog post and we look forward to seeing you there!
You can also download OpenStack Cisco Validated Designs, White papers, and more at www.cisco.com/go/openstack
Tags: ACI, Cinder, Cisco, horizon, IPv6, NOVA, OpenStack, Puppet, StackForge
[Note: This is the third a four-part series on the OpFlex protocol in Cisco ACI, how it enables an application-centric policy model, and why other SDN protocols do not. Part 1 | Part 2 | Part 4]
The Cisco ACI fabric is designed as an application-centric intelligent network. The Cisco APIC policy model is defined from the top down as a policy enforcement engine focused on the application itself and abstracting the networking functions underneath. The policy model unites with the advanced hardware capabilities of the Cisco ACI fabric underlying the business-application-focused control system.
The Cisco APIC policy object-oriented model is built on the distributed policy enforcement concepts for intelligent devices enabled by OpFlex and characterized by modern development and operations (DevOps) applications such as Puppet and Chef.
At the top level, the Cisco APIC policy model is built on a series of one or more tenants, which allows the network infrastructure administration and data flows to be segregated. Tenants can be customers, business units, or groups, depending on organization needs. Below tenants, the model provides a series of objects that define the application itself. These objects are endpoints and endpoint groups (EPGs) and the policies that define their relationships (see figure below). The relationship between two endpoints, which might be two virtual machines connected in a three-tier web application, can be implemented by routing traffic between the endpoints to firewalls and ADCs that enforce the appropriate security and quality of service (QoS) policies for the application and those endpoints.
Endpoints and Application Workloads Along with Tenants and Application Network Profiles Are the Foundation of the Cisco ACI Policy ModelEndpoints and Application Workloads Along with Tenants and Application Network Profiles Are the Foundation of the Cisco ACI Policy Model
For a more thorough description of the Cisco ACI application policy model, please refer to this whitepaper, or this one more specifically on Endpoint Groups.
For this discussion, the important feature to notice is the way that Cisco ACI policies are applied to application endpoints (physical and virtual workloads) and to EPGs. Configuration of individual network devices is ancillary to the requirements of the application and workloads. Individual devices do not require programmatic control as in prior SDN models, but are orchestrated according to the centrally defined and managed policies and according to application policies.
This model is catching hold in the industry and in the open source community. The OpenStack organization has begun work on including group-based policies to extend the OpenStack Neutron API for network orchestration with a declarative policy-based model based closely on EPG policies from Cisco ACI. (Note: “Declarative” refers to the orchestration model in which control is distributed to intelligent devices based on centralized policies, in contrast to retaining per-flow management control within the controller itself.)
Read More »
Tags: Chef, Cisco ACI, Cisco APIC, devops, Group Policy, Open Daylight, OpenStack, Puppet, SDN
When last we left our hero, he (that is, me, or I) was getting a crash course in Nexus programmability and trying to understand what all of this stuff meant. I had plied Jim* with beer in order to get him to explain to me – using the available napkins in the bar – what the technology was, what it meant, and why I should care. Read More »
Tags: Chef, nexus, onePK, programmability, Puppet, python, Ruby