The New York Times reported a few days ago that “The F.B.I. seized Web servers in a raid on a data center early Tuesday, causing several Web sites, including those run by the New York publisher Curbed Network, to go offline.”
“The raid happened at 1:15 a.m. at a hosting facility in Reston, Va., used by DigitalOne, which is based in Switzerland.”
“In an e-mail to one of its clients on Tuesday afternoon, DigitalOne’s chief executive, Sergej Ostroumow, said: “This problem is caused by the F.B.I., not our company. In the night F.B.I. has taken 3 enclosures with equipment plugged into them, possibly including your server — we cannot check it.”
“Mr. Ostroumow said that the F.B.I. was only interested in one of the company’s clients but had taken servers used by “tens of clients.”
“He wrote: “After F.B.I.’s unprofessional ‘work’ we can not restart our own servers, that’s why our Web site is offline and support doesn’t work.” The company’s staff had been working to solve the problem for the previous 15 hours, he said.”
“Mr. Ostroumow declined to name the client targeted by the F.B.I. and said that he did not know why it had drawn their interest. It was also unclear why the agents took more servers with them than they sought, he said.”
“The sites of the Curbed Network, including popular blogs covering real estate, restaurants and other topics, were all unavailable Tuesday evening. Lockhart Steele, Curbed’s president, said his team realized that the company’s sites were down at around 3 a.m. and contacted DigitalOne. After initially declining to say what had happened, DigitalOne explained that the F.B.I. had raided the data center, Mr. Steele said.
“Our servers happened to be in with some naughty servers,” he said, adding that his sites were not the target of the raid. Curbed is working to get its sites back online”
“Another company effected by the raid was Instapaper, a popular service that saves articles for later reading.”
This case illustrates the problem with having your servers in the US even if your doing nothing wrong.
I think its fair to say that the US through server seizures, domain seizures and other means has gotten more aggressive than any western country in taking sites down, without notice and without a right to a prior hearing.
In the process of these domain name seizures and now server seizures a lot of innocent sites have gotten mixed up in the mess and take down along with.
We moved our servers even for this blog and our parked domains, up to Canada a few years ago for this exact reason.
There are other hosting facilities located offshore in the Bahamas and elsewhere to avoid this police state mentality that the US has seemed to adopted.
I think the 1st comment on the New York Time post said it best:
“I have to agree with the hosting company here. It’s like seizing the entire contents of an airplane because one passenger is believed to be carrying contraband in his suitcase. ”
“Except worse, because this has nothing to do with the hardware and they could have merely copied the hard drives of all the computers instead of taking anything at all”
“Except worse, because this has nothing to do with the hardware and they could have merely copied the hard drives of all the computers instead of taking anything at all”
You cannot just merely copy the data. They have to perform forensics on the actual hard drive platters to get at multiple deleted data. I think whats happening is, I’ve read, they’ve been trying aggressively to shut down the spam botnets. The botnet operators are using “control servers” and they’ve been quick to erase their tracks and even erasing while the raid is taking place at the same time! They are using only US based servers in the data centers to avoid standing out with a foreign IP address.
I cannot find the original article BUT here’s a link to get started:
http://www.wired.com/beyond_the_beyond/2011/03/microsoft-versus-rustock-botnet/
So far the Feds have been successful in taking down some “control servers”. It still ongoing though.
There’s alot of collateral damage to the other sites on those servers thats for sure.
could this be another risk of “cloud computing” (shared hosting)?
hosting companies can’t ensure the quality of the “neighborhood” (they can’t monitor every customer) and most customers probably don’t know who their “neighbors” are (who else they’re sharing the servers with).
imagine if this was amazon ec2.
feel bad for this hosting company as they’ll probably lose customers because of this.
Da
You raise a really good point.
What is there is a bad actor or suspected bad actor on Amazon’s huge server network?
How many sites would be effected if the FBI raided Amazon to grab servers based off of one site?
Very worthwhile post.
But, at the end of the day, the same severs could stop running for any number of reasons. A site backup and “secondary dns” would limit the exposure to something like this.
Can happen to Google,Microsoft and those big companies now.
Do you want to vote for Obama2012?
This is probably lulzsec related.
The feds are going after them with every ounce of law enforcement capacity they have.
nic is right. secondary dns, pointing to a server in some other data center. and/or if they can jsutify the costs, a cdn.
i’m no fan of akamai. they’re expensive and the number of extra dns lookups they cause is ridiculous- talk about “abusing” the dns. google’s “cdn”, otoh, is an amazing feat. it’s sounds silly, but a random google-hosted blog or google apps site probably has the same or better speed and resiliency as may of these data centers offering shared hosting. (can you remember the last time any google site was “down”?)
This was not “probably Lulzsec related”, this WAS totally Lulzsec related, Lulzsec “owned” the FBI several weeks back and of course they (FBI) were way behind the curve on this in trying to track them down, hence total overkill by muppets was the result.
Well, this article did it for me. Time to backup.
I can just see a pack of “good ol’ boyz” rushing in, guns drawn, threatening everybody like some kind of drug bust.
Just..WOW.
This is just insane, and the comment on NY Times is very accurate. Strange days we live in… and this police state is only getting worst.
Mike,
Thanks for posting this.
Yet another reason why any company should consider hosting outside the US to avoid being affected by fat-fingered raids like this one.
– Richard
Commercial clouds are not secure nor do they have a legal obligation to you. Your data could disappear. Not using them now or in the future.
FBI has tough job but could do it more elegantly.
DA,
Interesting points.
Normally the “bad guys” come into a data centre and order 10+ dedicated servers and move A LOT of bandwidth. They pay their bills fast and in cash.
Any experienced hosting company would spot this as unusual behaviour and investigate their client, ask questions, do the proper due diligence etc.
At our facility, we do thorough checks on potential clients and weed out the “bad guys” so they don’t get a chance to get onto our networks in the first place.
I expect most serious hosting companies do the same.
– Richard
The FBI is a bull in a china shop … rather than an orderly raid asking for the database of said potential criminal they kill legit businesses.
Should be a wake up call to backup your blogs etc and have another hosting firm ready to go.
Considering Google’s heavy-handed tactics with Adwords and Adsense users and now this kind of flagrant abuse from the FBI, one has to wonder who is looking out for online entrepreneurs.
This story will have me sleeping very uneasily tonight.
@John – take melatonin.
cheers richard.
just one clarification: i draw a rough distinction between “screening” (one-time) and “monitoring” (ongoing). in my opinion the former is easier, somewhat standardised and is more widely adopted, while the later is potentially more labour-intensive and the minimum standards are less clear. here i was referring to monitoring.
DA,
Another good point. We do the monitoring too, and yes it is more labour intensive, but if you want to protect all your customers & your business, it’s worth the work.
– Richard
The real question is “Can the FBI actually get the server rack enclousures they took up and running?”. Our datacenter, you could pull server racks, but you would need to take the massive database rack also otherwise the VM Server racks will not work. Pretty much you would need to sieze the entire datacenter. Then there is the backup datacenter…..
FWIW, I was formerly on the security incident response team for a major financial company and I can tell you that seizing servers has always been standard FBI procedure. Forensic data CAN be extracted in the field, but you have to be so careful with the handling of evidence if you’re going to prosecute. That’s why they normally take just everything and send it to their lab (documenting it in painstaking detail.)
Sounds very “police state”, but one minor screw-up can get all your evidence thrown out in court and potentially destroy a case that’s been in the works for months, if not years.
Site owners and hosting companies need to always have a good backup/disaster recovery plan in place – s*** happens. Own your own data.
Agree with MG.
What would happen if all customers suddenly asked to see a copy of their hosting company’s “disaster recovery plan”?
Levels of knowledge vary among people running datacenters, to say the least.
Caveat emptor.
What would happen if all customers suddenly asked to see a copy of their hosting company’s “disaster recovery plan”?
Every single customer should ask for it before entering a commercial datacenter !!!
That’s how you find out whether someone’s offering just an air-conditioned double floor equipped room or a truly professional DC.