My last blog described how Cisco IT resolved an intermittent availability problem resulting from a hung Unity voicemail port. Here’s another example of an intermittent problem, this one related to call quality.
For more than a year our global technical response center (GTRC) received occasional calls from Cisco users with voicemail issues. These users reported that the system didn’t recognize their key entries, or just mysteriously disconnected them.
But the GTRC couldn’t verify the error because the problem was intermittent. When they tried, it worked perfectly, of course! They chalked it up to user error.
Cisco IT finally solved the mystery using Cisco Unified Service Monitor, which measures voice bearer quality and call disconnects. It collects information about quality directly from IP phones, and from sensors that we deploy in front of our Cisco Unity and Cisco Unified MeetingPlace servers.
When we looked at voice quality on the lines going into the affected Unity servers, we discovered that outbound quality was perfect, but inbound quality was so bad that it made speech unintelligible.
We eventually discovered that the devices on the subnet had not been configured for QoS! Under normal network loads, voice quality was fine. But during data center backups, when load was high, the network devices dropped packets. As a result, for around five percent of calls, the Unity server received enough data to keep the port open, but not enough to recognize DTMF tones and voice.
Without Cisco Unified Service Manager, we might never have diagnosed the problem, because 95 percent of calls to the Unity system worked fine. As it was, we got rid of an inconvenience that was costing time for a small group of our users every day.