Cisco Blogs

Automated Control Systems Risks

July 1, 2009 - 0 Comments

As discussed in the Cyber Risk Report (CRR) dated June 22–28, 2009, the recent crashes of Air France flight 447 and the Washington DC Metro Red Line commuter train have focused concerns over automated control systems, or computer-controlled systems. Preliminary findings in the ongoing investigations indicate that sensor systems malfunctioned or failed, and that the human interfaces of the systems either didn’t warn the air crew or train operator, or warned them too late. The preliminary investigations also indicate that the pilot disabled the autopilot and the train operator engaged the brake, but in both cases were unable to recover from their dire situation.Automated control systems are widely deployed and normally highly reliable. The systems are used to improve efficiency, reliability, productivity, and safety and security. They are used increasingly within homes, vehicles, manufacturing, financial trading systems, critical infrastructure and many other instances. Automated control systems range from systems used for fairly simple repetitive actions to highly complex systems that are capable of collecting and interpreting data from multiple sources and initiating actions with speed, accuracy and reliability levels not possible by humans. But these systems also include weaknesses that should be understood and considered by all users, operators, and managers of the systems. The critical fail-safe in nearly all of these systems is the human interface or management console, where information is presented to a human that can interact, correct, or take full control.Spurred by the recent tragedies, security experts and investigators are questioning if the human operators are adequately trained and qualified to understand and interact with the systems, or if we have become dependent on automated control systems that are so complex even a highly trained and experienced operator is not capable of assuming control and performing the actions necessary. However, we have not yet reached the point where the control systems are complex or sophisticated enough that they do not require the fail-safe human management.If this conundrum sounds familiar, it may be because physical and information security operators and managers increasingly find themselves in similar, if less fatal, positions; they are often overwhelmed with data, information and inaccurate or misunderstood alerts and warnings that could lead to incorrect decisions, and a lack of system understanding, training and experience needed to interact with these complex systems.But automated control systems will continue to increase their penetration in our lives and professions; they provide too many benefits to revert to older and simpler ways.Recommendations:

  • Understand the risks associated with these systems and include them in business continuity, disaster recovery, and incident response plans
  • Recognize and provide a high level of understanding, training, and experience that operators of these systems require
  • Perform regular tests, exercises, or simulations to rehearse incidents that rarely occur
  • Consider the critical ‘response time’ limitations, which reflect the amount of time an operator has to recognize a warning, alert, or situation and perform an action
  • Keep the system as simplified as possible by continuously refining the data and information input to those essential for decisions

In an effort to keep conversations fresh, Cisco Blogs closes comments after 60 days. Please visit the Cisco Blogs hub page for the latest content.