UPDATE: This blog post is related to the redirection of domain name servers that occurred back in June 2013. This post is NOT related to the ongoing activity occuring July 16, 2013. Cisco TRAC is currently analyzing the ongoing issues with Network Solutions’ hosted domain names and has more information available here.
Multiple organizations with domain names registered under Network Solutions suffered problems with their domain names today, as their DNS nameservers were replaced with nameservers at ztomy.com. The nameservers at ztomy.com were configured to reply to DNS requests for the affected domains with IP addresses in the range 184.108.40.206/24. Cisco observed a large number of requests directed at these confluence-network IP addresses. Nearly 5000 domains may have been affected based on passive DNS data for those IPs.
Hijacking of a domain name’s DNS records is one of the worst attacks an organization can suffer. You literally have lost control over your domain. Network Solutions, having been the original registrar for .com, .net, and .org domain names, is quite an attractive target for attackers. Originally it was unclear whether the issue was the result of an attack or a misconfiguration. It turns out the problem was both attack-related and also the result of a misconfiguration. Network Solutions issued a statement claiming, “In the process of resolving a Distributed Denial of Service (DDoS) incident on Wednesday night, the websites of a small number of Network Solutions customers were inadvertently affected for up to several hours.”
Interestingly, several of these domains were setup under different nameservers at ztomy.com. For example, the domain usps.com was pointed to the DNS nameservers ns1621.ztomy.com and ns2621.ztomy.com. Yelp had their nameservers changed to ns1620.ztomy.com and ns2620.ztomy.com. Fidelity, meanwhile, was pointed at ns1622.ztomy.com and ns2622.ztomy.com. However, the fact that so many domains were displaced in such a highly visible way supports Network Solutions’ claim that this was indeed a configuration error.
TRAC recommends anyone who has a domain registered with Network Solutions verify that their DNS nameservers are pointed at the correct location. TRAC also recommends network administrators check the logs from their network devices for connections to the 220.127.116.11/24 subnet. Organizations need to carefully consider how they would swiftly identify unauthorized modifications to their DNS records and how they would react to such a situation.
Thank you to Henry Stern, Martin Lee, Mary Landesman and Gregg Conklin for their assistance writing this post.
An impressive share! I have just forwarded this onto a friend who was doing a little research on
this. And he actually bought me dinner due to
the fact that I stumbled upon it for him… lol. So allow me to reword this.
… Thank YOU for the meal!! But yeah, thanks for spending the
time to talk about this issue here on your internet site.
The configuration change for these domains such as fidelity.com and linked in.com does not look like a configuration error, but looks more like a deliberate configuration done by Network Solutions – perhaps while they were recovering from an attack. The various nameservers (and webservers) at ztomy.com IP space of 18.104.22.168/24 seem to be configured to answer mostly correct DNS responses to these domains (see below). Perhaps it is distributed anycast network such as cloud services providers. The network 22.214.171.124/23 seems to fit this criteria.
# nslookup http://www.linkedin.com ns1621.ztomy.com
http://www.linkedin.com canonical name = la.linkedin.com.
I live constantly on resolving the problem on my dns Hi pls
Looks like you’ll have some more pretty graphs for us…NS is having major issues again today. A great way to start the morning…I had a new article to post!
I can confirm… I don’t know what they’re doing to solve it, but I suddenly found a space between the host part and the domain name in our DNS console for all our A records… All domains we have that depend on networksolutions have problems. Domains registered somewhere else work fine.
Indeed. I have 2 sites down this morning on NS.
We are also experiencing ADNS issues from Network Solutions. I have called and they acknowledged the outage but didn’t say what it was from.
We have about 12 sites out.
Same here. July 17th 2013, and our website which has its domain name registered through Networksolutions, is unreachable due to DNS errors (access is fine direct to its IP address). I thought that using a large and VERY expensive service like NeSol that we would avoid these issues. Now if I could just work out what to do about it?
There’s not much that can really be done about it as moving from one DNS server to another will require you logging into Network Solutions to change it. 🙁
I also registered with the original and the biggest for the same reason–reliability. And to be honest, I’ve only had two outages (including this one) in about 9 years, so it’s not too bad. Especially when you look at the names of other companies who are with us in this outage.
I’m also suffering from the loss of all DNS entries hosted by Network Solutions. And the main NS site is not responding so no way to admin any changes.
We too are experiencing outages on all our site because nameservers are down. Unfortunately they are registered with Network Solutions..
What’s really bad is the fact that there’s no communication or a status update, and how come such a massive internet player does not have a business continuity plan in place.
We have a dozen domain names with Network Solutions and virtually all of them have DNS problems today.
My NS email sites just came back up. Lets hope they stay up!
Mine just came up too! 😀
I guess this should be our cue to switch DNS servers…
I wouldn’t change, and I personally won’t be changing. I’ve had almost zero problems for a decade with them. I can take a few hours of downtime every few decades, lol.
Nearly 2 decades here. But I’ve had more problems in the past 2 years than the first 15.
From Wikipedia: In August 2011, it was announced that Network Solutions will be bought by Web.com for $405 million and 18 million shares. On October 27, 2011, Web.com announced the completion of the Network Solutions acquisition. This was immediately followed by the departure of CEO Tim Kelly and other key leadership. Large scale employee layoffs began the following day. By December 31, 2011, over half of the office space in the Herndon, VA based headquarters had been vacated and returned to the property management. On March 31, 2012, the company’s Belleville, Illinois office was closed.
All the major players leaving a company is never a good sign. Jelsoft underwent the same problem when they were bought out by Internet Brands. Except, IB didn’t completely level the office. 😮
Same here, but seems only to be sites hosted by Network Solutions. Those hosted elsewhere are fine. Ah! Their own site is back up at 10:52 a.m. EDT, so maybe they’ve fixed it. Saw a notice on their site yesterday that advised they will be down for maintenance tomorrow night, so this could be related in some way.
Actually NetworkSolutions.com has been up and down repeatedly for the past couple hours. Down for me at the moment.
That’s what I’ve been seeing as well.
Same here, but only wordpress-based sites were affected. the hack page. Some hgroup calling themselves Islamic Ghosts Team took cred on the hack page. I was able to FTP in and changed filenames that were dated since yesterday morning to recover web access. Found a gif plant named animated-messages-small2.gif that redirected to126.96.36.199. I posted the hack replacement page’s header scripts if anyone is interested at http://www.galactickegger.com.
Ummm…I don’t think you’re experiencing the same problems as us.
To put it another way:
“these aren’t the droids you’re looking for…”
LOL Now that’s funny!
Then perhaps there were coordinated attacks on multiple fronts using multiple methods. Some Netsol WordPress sites got hacked last month; and of all the sites we have registered with them only pure WP sites were hacked yesterday. I’m not here to compare tool bags … I’m here to report what I found. My apologies if featyhers were ruffled.
I looked at your page… At least you’ve still got a sense of humor. “…islamic ghost limpdick society…” LMAO
No harm, no foul. I didn’t know if you were legit or some spammer trying to drive traffic to their site. 🙂 At least you gave me a good excuse to use a Star Wars line. 😀
You may definitely be on to something though as both of these outages seem to happen in the same timeframe. Where did those attacks originate from?
I don’t think it was limited to wordpress. I have a site on one of Network Solution’s shared JSP servers (*.netsoljsp.com), and on Tuesday night (7/16) a few of my users were redirected to Islamic Ghost pages instead.
No worries mate. I xfer’d from the department of redundancy department, so BUs are in place. Just can’t access email … manure occurs. Cheers!
Backups are always a good idea. 🙂 Glad to hear it’s not affecting you much. 😀
If it was truly multi-prong like we’re theorizing, it was probably distributed geo-locally and will have IPs from all over the world. 🙁
Farthest I was able to trace was 188.8.131.52 (Isp4p IT Services in Ludwigshafen Germany.)
And Cisco adds a statement at the top of this article as numerous tweets and tech360.com erronously report this article as happening today. :O
I don’t which is worse, reporting wrong bad news or the outage itself, lol.
And the article magically changes completely, lol. The Internet–Live!
We had several customers using network solutions with issues today.
I’ve seen intermittent emails being rejected with error “451 Temporary local problem – please try later”.
Issue seems to be fully resolved now.
Good to hear. 🙂 I didn’t know if I was getting all my email or not.
Nice act of plaigarism can be found here
Yay, our website is back!
Cisco has a new blog post about July 16/17 issues here http://blogs.cisco.com/security/network-solutions-customer-site-compromises-and-ddos/
Sweet. They should link to that post in this article’s header.
Fantastic issues altogether, you simply gained a new reader. What may you suggest about your submit that you simply made a few days in the past? Any sure?
An impressive share! thanks for spending the
time to talk about this issue here on your internet site.
So, um, if I am not being impertinent, please do tell me just who in the f**k is ZTOMY.COM?
I mean are they criminals, or what? What is their scam? Is it just another front for just another online business that will do anything, legal or illegal, to get clicks?
The WHOIS record is obfsucated, and nobody seems to be saying anything about just who the hell these people are… not Network Solutions, not Akami (who is giving that domain its base DNS), and apparently not Cisco either.
C’mon. Why is everybody so shy?
Comments are closed.