Slashdot Log In
The Great Zero Challenge Remains Unaccepted
Posted by
timothy
on Sat Sep 06, 2008 02:21 PM
from the no-fair-guessing-porn-names- dept.
from the no-fair-guessing-porn-names- dept.
An anonymous reader writes "Not even data recovery companies will accept The Great Zero Challenge and only four months remain! We've all heard how easily data can be recovered from hard drives. We're told to make multiple overwrites with random data, to degauss drives and even physically destroy them just to be extra safe. Let's get the word out. The challenge is almost over! It's put up or shut up time. Can you recover the data?"
Related Stories
Firehose:The Great Zero Challenge Remains Unaccepted by Anonymous Coward
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Do many companies really do EFM recovery? (Score:5, Insightful)
Based on nothing more than personal suspicion, I think many professional recovery firms may be in the business of simply running expensive tools that scan through the partition and file table area and perhaps even the entire disk to locate data that has either been marked erased or had references removed (for a full disk scan) and then restoring it. Perhaps they'll also move the spindle from a dead drive into a new case to complete the operation, but I doubt there are many companies that will actually do electron force microscopy for you and even fewer that will do it at anything other than an astronomical fee. Powerful recovery tools can be purchased for a few hundred dollars now anyway. My opinion is that the recovery business is a focus around confidence that a professional will be doing the recovery and that you or your employees won't worsen the situation. In the event that a drive with critical data fails and you don't have a backup, who wants to be the person responsible for damaging the disk during recovery?
Anyway, IMHO this whole debate should be moot by now. If you want to secure your drive use full disk encryption (now freely available in TrueCrypt) and when it comes to destroying the data just overwrite the header area a thousand times with random garbage. It will take only a second or two, and the whole drive will be useless to anyone.
Of course it would also be nice if more manufacturers were producing encrypted disks as standard with verified schemes (there have been some lemons purporting to be secure that really aren't) so that we wouldn't have to do encryption in software.
Re:Do many companies really do EFM recovery? (Score:5, Interesting)
Parent
Re:Do many companies really do EFM recovery? (Score:5, Informative)
Although the drive has to be in a living system and not on the shelf, it's worth noting the cold boot attack
Not in this context because we're talking about how intentionally wipe the data from a drive, e.g. when you want to erase the data and dispose of the disk. The cold boot attack, although interesting, has nothing to do with recovering data from a drive after someone has attempted to destroy it, unless your implication is that someone would try to overwrite the header a split second before someone like the FBI breaks the door down. Even then, simply unmounting the volume will wipe the key from memory. If you have time to attempt an erasure you have time to unmount the disk. If you are in a situation where you have enough time to write zeros all over the drive, as in this challenge, you are certainly not at risk from the cold boot attack.
Parent
Re: (Score:3, Informative)
Re:Do many companies really do EFM recovery? (Score:5, Funny)
I can just picture it: The FBI kicks down your door at 3:40am, male voices scream "F-B-I", guns clicking, laser-sight dots hushing over the walls, someone jumps through your bedroom window, kicks you out of bed knocks you onto the floor, jams his knee into your neck... Then a nutty professor with fat glasses in a white coat runs onto the scene and screams "FREEZE!!!" as he sprays ice onto your RAM modules...
C'mon kids, won't happen. You've been to the movies too much. In the real world they just send you a letter. And you pay and/or get to clean some public spaces. And mommy will lock away the computer. That's it.
Parent
the drive must be in a living system??? (Score:3, Funny)
Critical line in the Challenge: (Score:4, Insightful)
So you're not allowed to (for example) exploit redundancy or error checking on the drive itself? If dd wrote zeros, that's what'll be read unles you can get "lower" than normal drive access.
This challenge has nothing to do with the security of your wipe. Rather, it has everything to do with dd successfully writing zeros given normal access.
Parent
I think you got it at the beginning. (Score:4, Insightful)
It's about money.
Since the "reward" offered seems to be less than the regular fee that a company would charge for such, why would any recovery company waste resources on it?
Parent
Re:I think you got it at the beginning. (Score:5, Insightful)
Parent
Re:it is PR (Score:5, Insightful)
And the drive being fake is a distinct possibility here. The guy has an agenda, that's pretty clear. And where's the accountability? Why should we believe him when he says what has been done to the drive? Any more than we believe British barristers representing the late Mr. Ongopongo of Nigeria in their claims that they have some millions of dollars they want to give you?
Because we want to believe him, because his claim is very plausible? Sorry, that doesn't increase the accountability or invalidity of this "challenge".
Unless acceptable witnesses can observe (a) the original status of the drive, (b) what was being done to it, and (c) the drive being kept secure from interference from (a) onwards, it must be treated as suspect. No matter how honorable the intent is. Intent is worth shit, and any company or researcher that would be foolish enough to enter this "challenge" would be tainted with same.
Parent
Re:it is PR (Score:4, Informative)
Encrypted by whom? Oh, that's right, by him.
Sorry, encryption doesn't lend any kind of credibility to the claim at all. That only makes it harder to change the list from now on, but doesn't validate that the list was correct in the first place. What would be stopping him from zeroing one drive and provide the list from another drive (or make one up), and then encrypt the wrong list? There's no verification process in place, which causes the addition of this encryption step to smell of snake oil, making it slightly less believable than if it had been all in the open.
I'm sorry, but you're taking his word on faith. Which is a very wrong thing to do, even if he is right. It's not the amount of money in question that's the big problem here, but the lack of accountability.
Parent
Re:it is PR (Score:4, Insightful)
Bingo. It's also worth pointing out that the $40 prize offered isn't even close to the normal fees that such companies charge to do data recovery. The cheapest fee I've *ever* seen quoted for a post-format recovery was $1700, and that was a special offer being made to our customer care because of a tech. support fuckup. (they didn't tell the customer that reinstalling the OS would delete all their pictures, and the customer raised a stink).
Such a "title" as the one offered by this so-called "challenge" is hardly worth the effort expended. Especially considering that this article is the first I've heard of it... How is this Slashdot-worthy?
Parent
Re:it is PR (Score:5, Insightful)
The terms of the challenge indicate that you cannot disassemble the drive.
Have you actually read the terms?
"If the challenger is an established data recovery business located in the United States of America (We would need to see Articles of Incorporation, a current business license and one other form of business identification in order to determine that they are indeed a professional, for-profit, established data recovery business) or a National government law enforcement or intelligence agency (NSA, CIA, FBI), then we will allow these type of organizations to disassemble the drive and to keep the drive for thirty (30) consecutive days. "
Parent
Re:Do many companies really do EFM recovery? (Score:5, Insightful)
Except, of course, that the point of the challenge is that instead of encrypting and whatnot (which can be a good idea for other reasons, but I digress), you could just overwrite the drive with 0's once and dispose of the drive safely. This is most likely substantially faster than what many people propose, like overwriting many times or physically destroying the disk.
However, I think their methodology is pretty flawed. The reward for completing the challenge is $40 and the drive itself (which is worth $40-60). You also have to pay shipping, which will run maybe $10-15. I know that it's really not worth it for me to spend any time trying to recover the data from the drive—probably a fairly lengthy process—just for $85.
Parent
Re:Do many companies really do EFM recovery? (Score:5, Insightful)
The conditions are also made to trick ignorant journalists. Anyone knowing a bit about file systems know that being able to restore some data from a drive is a heck of a lot easier than being able to restore file names, which they demand. Not only do you have to be able to restore the sectors that contain the file name metadata, but you need knowledge of the file system in question, and how exactly it stores its file names. If it's stored in byte swabbed format, you won't even recognize it as a file name.
Try to do a dd to a file of a working partition and then extract the file names from it. Unless it's a DOS partition or other ancient format, it's not easy, and that's with no zeroing.
Yes, the "contest" is a farce, and any company that enters into it will lose credibility just by entering.
Parent
Re:Not so. (Score:5, Insightful)
That is the cheapest publicity they would ever receive... and what publicity they would receive!
Yes, what publicity they would receive? :) I've never heard of 16systems.com before, their site is barebones with almost no articles [16systems.com]. I dare say they caught a lucky break with this Slashdot article. Maybe I'm wrong, but it seems that there is no obvious publicity to be had (before now). And should recovery firms respond to everyone with a small website who issues a challenge?
Parent
Re:Isn't that the POINT?? (Score:5, Funny)
For $40?
I don't do anything IT-related for $40. I'd charge $120 to lean down and press your power button.
Parent
Re: (Score:3, Informative)
I had an old drive which failed - one of those laptop Travelstar's that were known as 'deathstars' for the number of times they had died from overheating. Data recovery companies gave me a quote for anywhere between 300 pounds and 800 pounds, depending upon whether they would have to remove the spindle/platters from the drive and place them into a new one.
Fortunately, I managed to recover all the data from this drive for free, by putting it in external USB enclosure, place this in a freezer to cool it down,
Re:Do many companies really do EFM recovery? (Score:5, Funny)
It's always satisfying to fix a computer problem with a hammer, even though you are being very careful.
It's called percussive maintenance.
Parent
Re:Do many companies really do EFM recovery? (Score:4, Insightful)
You're absolutely right that we're talking big brother paranoid level security -- but if you write straight zeros, and writing a zero makes 1->0.05 and 0->0, it may be possible to tell the two states apart. As binary as the data may be, it's still getting written to a physical medium -- and the Real World lives in analogs. Even were this true, however, writing multiple passes of garbage would prevent an entity able to distinguish 0.00 and 0.01 from being able to determine the media's prior state -- and that's the whole point of this operation. Claiming that writing multiple passes of random garbage (or, better, patterns selected to-purpose -- see the Gutmann method) is somehow worse security than a single pass of zeros is complete bunk; the likely case is that it simply doesn't buy anything worthwhile at all, at a cost of time and electricity.
That said -- absolutely, this isn't a likely attack; if there were a cheap way to make equipment which could read data with that level of precision off of magnetic platters, we'd be using it to make higher-density magnetic platters... and tolerances for how the data is written to those platters is much, much lower today than it was twenty years ago. (Against a twenty-year-old hard drive, I'd expect the chances of someone with a STM and a lot of time to actually be quite good).
Parent
The problem is it isn't that simple (Score:4, Informative)
Long gone are the days when drives stored things in a simple modulation format. That's what MFM hardrives were (MFM means Modified Frequency Modulation). Now harddrives store an analogue wave, and analyze it to determine the maximumly likely result for a given waveform. It's called EPRML, Extended Partial Response, Maximum Likelihood. You can Google for the specifics of how it works, but the general idea is there isn't a certain threshold beyond which something is 1 or 0. Rather it is an analogue wave of varying intensity and by looking at how it changes, the drive's processor can pick out the binary stream it is most likely to represent. Sounds like voodoo, but works really well and is extremely reliable.
Well, that means that data recovery of overwritten data just became a hell of a lot harder. It isn't a matter of saying "Well the current data is a 0, however it is on the high end of 0 so it was probably a 1 before." No now you have to be able to tell what the wave looked like beforehand, and interpret that.
Now maybe there's a way that it is possible, but I'm rather doubtful. There is, of course, also the time factor. Supposing you can do this, how long does it take you to read one byte? A second? A minute? Ok, how long are you willing to spend scouring a drive that has five hundred billion of those bytes? So not only do you need to be able to do this, but you need to be able to do it quite quickly if you are to have any hope of scanning a modern drive in a timescale that is useful.
Parent
000 00 00000 000000000 (Score:5, Funny)
000 000, 0 000 0000 0000000 0 0 0 0000 00000! 000 0 000 000 0000000 000 000000 00000? 00 000 000000!
000 000 00 0000 000.
"....less than a zero percent chance" (Score:5, Funny)
That word "percent", I don't think it means what you think it means...
Re:"....less than a zero percent chance" (Score:5, Funny)
Parent
Re:The whole article is full of comedy gold (Score:4, Informative)
"It can't be done" is a little strong: On older (early-1980s) hard drives it probably could be done. Modern drives, less likely. No-disassembly rule, no chance whatsoever.
That said, "industry best practices" is what it is. When I'm wearing my data security hat for a company managing people's medical records, I'm going to advise that we follow whatever accepted standards are for wiping drives; if FIPS says to degauss the drives, we're damned well degaussing the drives. "Nobody ever got fired for choosing IBM" may be a lousy rule for procurement, but "nobody ever got fired for insisting on industry-accepted security practices" is right on the money.
Parent
Re:"....less than a zero percent chance" (Score:5, Funny)
Parent
Re:"....less than a zero percent chance" (Score:5, Funny)
Parent
Wow, what a prize! (Score:4, Insightful)
So the prize for winning is a $60 hard drive, plus $40? Damn, I don't know why people aren't just jumping all over that!
Also, disassembling the drive is against the rules of the challenge, unless you're a "established data recovery business ... or a National government law enforcement or intelligence agency".
This "challenge" is stupid.
Re:Wow, what a prize! (Score:5, Insightful)
Actually they also ask you to tell how you did it. Even though they claim it is not a scam it seems like a scam in the sense that they after this weird experiment have proven that recovery is impossible.
It is like me setting up a challenge - can ketchup stains be removed from my white t-shirt?
Send a self-addressed, postage-paid box you pay shipping both ways with packaging material to the address listed below along with a sixty $60 USD deposit United States Postal Service Money Order only and I will mail the t-shirt to you.
If you can remove the stain you get to keep the t-shirt and I will give you the amazing amount of money $50 and the right to become "official stain remover". Btw, if you can't prove you are a established ketchup removal business - you cannot use water or any other fluid.
If this challenge is not taken within a year I have the right to tell the world that the worlds dry cleaners can't remove ketchup stains. The whole clothes cleaning industry is a hoax.
Parent
Re:Wow, what a prize! (Score:4, Insightful)
The challenge isn't stupid, the rewards are.
If this were an X-prize type of deal, it'd be a lot better. Who's going to bother with EFMing a drive for $40? I guess some college students with access to those machines might, but those are very fickle and easy-to-fuck-up machines..aka..kept under lock, key, and password.
Parent
Jeez (Score:5, Insightful)
Interestingly, the most important thing is missing from the summary -- the prize. So, what the prize is you ask?
An incredible, unbelievable, astonishing and amazing amount of... wtf... fourty (40) US Dollars? Yes, you heard that right! No wonder nobody has shown any interest in participating.
Full quote from the site: Should someone win, they get to keep the drive. They also will receive $40.00 USD and the title "King (or Queen) of Data Recovery".
Re:Jeez (Score:5, Insightful)
> Interestingly, the most important thing is missing from the summary
Not only that, but also the fabulous restriction:
"You may not [...] disassemble the drive"
This is ridiculous. A drive overwritten with zero data will, by definition, returns 0s through ATA commands. The reason why some people overwrite sensible data several time is to guard against a possible scanning transmission electron microscopy, which, of course would need the disk to be disassembled to be performed.
How can this ends on slashdot ? Don't know...
Parent
Real price is $700 (Score:4, Informative)
$300? That's for running what's pretty much an "undelete" like any shareware program can do.
$3,000, and you might get what amounts to a sector dump.
Not at all true. I priced this out for a friend that had removed data beyond what the simple undelete commands you mentiioned can do. The real cost is more along the lines of $700, and you get real data files back.
$3000 is more along the lines of, the actual physical disk inside the case has been disturbed and you are talking about recovering whatever data you can. That starts to get real pricey, really quickly.
Parent
Where are the challengers? (Score:4, Insightful)
In addition, according to Wikipedia, [wikipedia.org] what he proposes is actually impossible, at the very least an electron microscope would be needed.
Can't say I'm entirely disappointed by this story, though. At least I learned something that I was ignorant of before.
Utter stupidity (Score:5, Insightful)
First of all, do data recovery firms ever *claim* they can recover from a zeroed drive? No, they don't. The claim is that government-level forensic analysis *might* be able to recover data with only a single overwrite, with very sensitive expensive equipment. Not terribly surprising the FBI wouldn't take them up on this challenge.
Second of all, someone is supposed to waste a lot of time and money for just a cheap drive and a piece of paper from some entity no one has ever heard of?
And they're doing this to "prove" that this type of data recovery can't be done?
This has to be the lamest challenge that's ever been issued.
why would anyone do this? (Score:5, Insightful)
Okay, here are my 3 reasons why a company would not accept this challenge:
(1) economical:
- I am asked to mail 60 USD to a random address, who claim they will return it to me if I send the harddisk back. This is a risk (how do I know it is not a scam?)
- In any case, I lose shipping charges both ways
- Maximum gain is 40$, plus an obscure web site calls me King of data recovery.
- Risk + Cost >> Gain
(2) International
I am asked to ship a US Postal money. A WHAT? Hello, creditcard? Paypal? Normal internaional cheque?
(3) Disassembly
All reasons I've heard for doing something more than dd is that there might be residual magnetic charge on the platter that is ignored by the filesystem. According to the rules of engagement, only some weird collection of institutions ("established data recovery business located in the United States of America" or "National government law enforcement or intelligence agency (NSA, CIA, FBI)") may disassemble the drive. How am I going to detect residual charge if I cannot disassemble it?
The last arguments compounds the first two, as only US Companies can disasseble, and disassembly voids the deposit, meaning I am certainly out 60$.
Next time that they want to be "noble and just to dispel myths, falsehoods and untruths", they should make a challenge that is actually interesting to any party to pick up.
From The Experts (Score:4, Insightful)
Given my general level of paranoia, I recommend overwriting zeros, and five times with a cryptographically secure pseudo-random sequence. Recent developments at the National Institute of Standards and Technology with electron-tunneling microscopes suggest even that might not be enough. Honestly, if your data is sufficiently valuable, assume that it is impossible to erase data complete off magnetic media. Bur or shred the media; it's cheaper to buy media new than to lose your secrets.
Because all data recovery companies have electron-tunneling microscopes on hand for recovery and aren't just running a Linux distro with a modified ext3fs to ignore "deleted" inodes. The longest AES key I've cracked is 28 bits (in Python, no less!). Yet we still use a minimum of 128, more likely 256. It's not the guys running recover [sourceforge.net] I'm worried about. It's the spooks with electron f'ing microscopes and a direct connection to AT&T.
An urban legend (Score:5, Interesting)
It's an urban legend. You can't recover erased bits. If you could it would imply that you can store at least two bits in the space of one. Disk companies have a pretty good idea what their heads and surfaces can do. Do you think they'd be passing up big $$$ by under-utilizing their disk's capacity?
There is that one Usenix conference "paper" foating around out there, but if you read it carefully it does not give a single example of one recovered bit.
If you've ever looked at the waveform coming off a disk head, you'd wonder with all the x/y noise and jitter how they can get even ONE bit out of that hairball. The answer is, they can, just barely, by applying all the sync, gating, PLL, and deglitching tricks, just barely reliably recover bits at the maximum recording density possible.
And all those pictures they show of bit patterns lingering under large erased areas are actually counter-examples. They prove that you can detect periodic bit patterns under large erased areas. Duh. In the real world the underlying data is not periodic, and the erasure isn't smooth or periodic either. If you overwrite real typical data with random data, you can't recover the original data. Shannon and company, you know.
It is NOT an "urban legend"... (Score:5, Informative)
This is of course no longer true what with much tighter tolerances, smaller and vertical magnetic domains, and so on. I think that is the point of this challenge.
Parent
damn straight! (Score:5, Funny)
Re:damn straight! (Score:5, Funny)
You have the same problem the Great 0 Challenge has, your prize is too small!
Parent
It is recoverable, but at a price. (Score:5, Informative)
It is likely that there is a hysteresis in the platter causing a "0" written on top of a "1" to be slightly "weaker" than a "0" written on top of a "0".
On old tape, this hysteresis was about 10%, and was actually visible with a magnetic loupe, so depending on s/n ratio, you could recover quite a bit, no pun intended.
The problem with a HDD is that the signal from the heads go through a lot of signal processing including Extended PRML or EPRML. There is also an algorithm like RZ to not have a long series of the same bit written physically. If you take the electrical output from the read head, you will have a big task reconstructing the data, even if there only good data.
The only places today that can analyze well what is read physically is at HDD manufacturers research lab, and probably using custom HW to read the platter that collects all the errors and offsets. For a recovery company to do this, they probably would have to invest millions of $$$, so they will not.
So bottom line is that you could send the drive in to Western Digital, and they could probably recover the raw data with about 90% accuracy. If that is enough for the error recovery to chew on, I am not sure, but here and there, long strings would be recovered. They can for sure give the exact probability for the recovery of a bit.
WD however does not have any incentives to demonstrate that wiping their drives with "0" is not sufficient. aux contrare, they may consider this an undesirable property. Therefore, the only ones that can recover this is unwilling.
So the challenge remains unaccepted.
Prize (Score:4, Funny)
This will sound totally paranoid, but (Score:5, Insightful)
The few people who MIGHT have the capability to look beyond what is written on the drive and see patterns remaining from previous data are most likely the ones who would prefer that the concept remain vague and unproven.
Dear sir, (Score:5, Funny)
Kindly sir, I am a Nigerian Prince trying to transfer some data from a zero-ed out hard drive to my cousin in the U.S.A. If you would kindly deposit $60 into my bank account, I will send you the hard drive. Upon your transmission of the data to my cousin, I will promptly return your $60, plus $40 for your effort. You may also keep the hard drive.
Your friend,
Prince Njeme Nawabi, P.O.S.
Well known (Score:4, Insightful)
The German computer magazine c't did try to get a disk that was overweritten once with zeros recoverd two years ago or so. All data recovery companies they contacted (all the major ones) said they could not do it and that it was likely impossible. So this is not newa at all. Even Gutman had an addendum that says tomething close for modern disks.
The source of all these stories is that it used to be possible, when disc coatings were more advanced than r/w head and electronics. That is not the case anymore. It is very likely that you cannot put much more data on the disk than a moder HDD does. That also means that a single overwrite is an unrecoverable deletion. Keep in mind, that due to the particulars of the modulation, an all zero overwrite does not take up less of the surfaces data storage cabaliluty as a fully random overwrite.
Basically the pople that claim recovery is possible are one or so decades behind the times. Nothing new.