Question about Hardrives/Partition Recovery

edit-reason: too many whacky theories of why my partition went bad.

gtoledo3 wrote:
As the OS was frozen, after around 10 minutes I decided to hard boot it off. On reboot, I saw that progress "strip" that happens after "bad stuff happens", like crashes...

That's ... not what that means. That strip is when a firmware update is trying to install, or when the machine is trying to restore from hibernation.

gtoledo3 wrote:
I guess, showing that the partition is trying to be mounted. After a bit, it loaded, and I didn't think anything else of it. Ran fine. No weirdness at all. I didn't get a message on reboot of any kind like "you should back this up", or "your system has recovered from blah blah", like I would expect.

Do you ever see those? I've never seen any explanation when a panic happens, or when the file system has been corrupted. (Now, whether or not it would be useful is a different matter ;)

gtoledo3 wrote:
Today, I booted into Leopard, and after awhile, I get a message that my SL partition is hosed, and that I should go ahead and back it up. I immediately question this and hope that Leopard is confused. Instead of backing it up, I go ahead and try to boot into SL a couple of times. Failure. I see the progress bar... it gets about a third of the way, goes back to the start, goes further, gives up, and turns the computer off.

Leopard cannot deal with all the FS features of 10.6 -- that said, it can usually handle it gracefully (though it will report some files as zero-length due to FS compression). When it warns that the partition is hosed, that's usually a sign that it's hosed (I got those more frequently in Tiger, but once in Leopard).

gtoledo3 wrote:
So, back to Leopard. Of course, the SL partition doesn't show at all anymore (not worried about that, didn't expect it to), so I use Disk Utility to go ahead and validate the entire disk, hoping that after it got through the first two partitions that it would jog the system into recognizing the messed up partition so I can copy it. Worked fine (as it has in the past), and I'm now copying the contents of my SL partition.

Here's the question: How can I now tell what actually caused the corruption? I've never been able to figure this out. I've looked at multiple online resources and haven't seen any constructive info. I have some theories, but is there a way to really figure it out? I don't think I was doing anything "wrong", and this is a pretty savage bug, but I can't really report it, because there is no info that I know of.

You can't. The corruption happens when a set of writes doesn't complete (due to hard poweroff / a panic / an unsafe eject), memory failure, or harddrive failures. After the fact, there's nothing you can do to see what caused it because that kind of information cannot be recorded. From my experience, these have happened due to bad memory, hard power-offs, and a loosely-connected harddrive that disconnected sometimes while the system was running (I wasn't careful when installing it). Because you panic your machine often, and hard power-off it, and probably don't verify it after every instance, it's impossible to tell if the problems accumulate over time.

gtoledo3 wrote:
A secondary question is - has anyone encountered any software that can fix the ACTUAL PARTITION in question into being bootable again? It's usually an irritating sort of song and dance to reinstall stuff, migrate files, and I'm not looking forward to it.

That's a non-question. the problem can be trivially recoverable (a few counts are wrong) to catastrophically wrong (you wrote zeros to the entire partition). Disk Utility seems to handle most of the simple cases well (I run it on my machine pariodically, and on my timemachine backup because it gets unexpectedly disconnected frequently), and a couple dodgy cases I've thrown at it. I've also had it totally fail to do anything useful for a few totally fubar partitions from bad memory that wrote trash all over the file system metadata.

Rather than trying to restore a broken partition, I generally find it faster/easier/more consistently pleasing to just restore from a time machine backup. I've had disk utility run for several hours trying to address something (and then fail), while a restore would have taken half as long, and not failed.

cwright wrote:

gtoledo3 wrote:
As the OS was frozen, after around 10 minutes I decided to hard boot it off. On reboot, I saw that progress "strip" that happens after "bad stuff happens", like crashes...

That's ... not what that means. That strip is when a firmware update is trying to install, or when the machine is trying to restore from hibernation.

Ok. I've had that happen after crashes before. That's odd... maybe it's linked to the hibernation thing for some reason? Thanks for the clarification, I don't doubt it, but I'm more puzzled now.

cwright wrote:

gtoledo3 wrote:
I guess, showing that the partition is trying to be mounted. After a bit, it loaded, and I didn't think anything else of it. Ran fine. No weirdness at all. I didn't get a message on reboot of any kind like "you should back this up", or "your system has recovered from blah blah", like I would expect.

Do you ever see those? I've never seen any explanation when a panic happens, or when the file system has been corrupted. (Now, whether or not it would be useful is a different matter ;)

Yes, I've had messages sometimes like "your system recovered from a serious crash because:" and then it lists the reasons. It looks like when an app crashes and you get prompted to send in a report. I have to admit, most of the time, I can directly link up what I did that caused "x" to happen, so I don't really look at that stuff, and just "click through" to get it off my screen quickly.

cwright wrote:

gtoledo3 wrote:
Today, I booted into Leopard, and after awhile, I get a message that my SL partition is hosed, and that I should go ahead and back it up. I immediately question this and hope that Leopard is confused. Instead of backing it up, I go ahead and try to boot into SL a couple of times. Failure. I see the progress bar... it gets about a third of the way, goes back to the start, goes further, gives up, and turns the computer off.

Leopard cannot deal with all the FS features of 10.6 -- that said, it can usually handle it gracefully (though it will report some files as zero-length due to FS compression). When it warns that the partition is hosed, that's usually a sign that it's hosed (I got those more frequently in Tiger, but once in Leopard).

I was thinking "hmm, maybe Leopard can't deal with some of the SL filesystem correctly", just like you describe, but I also had that lurking feeling that it was hosed, as you say. Disk Utility seems to give consistent results from 10.5/10.6.

cwright wrote:

gtoledo3 wrote:
So, back to Leopard. Of course, the SL partition doesn't show at all anymore (not worried about that, didn't expect it to), so I use Disk Utility to go ahead and validate the entire disk, hoping that after it got through the first two partitions that it would jog the system into recognizing the messed up partition so I can copy it. Worked fine (as it has in the past), and I'm now copying the contents of my SL partition.

Here's the question: How can I now tell what actually caused the corruption? I've never been able to figure this out. I've looked at multiple online resources and haven't seen any constructive info. I have some theories, but is there a way to really figure it out? I don't think I was doing anything "wrong", and this is a pretty savage bug, but I can't really report it, because there is no info that I know of.

You can't. The corruption happens when a set of writes doesn't complete (due to hard poweroff / a panic / an unsafe eject), memory failure, or harddrive failures. After the fact, there's nothing you can do to see what caused it because that kind of information cannot be recorded. From my experience, these have happened due to bad memory, hard power-offs, and a loosely-connected harddrive that disconnected sometimes while the system was running (I wasn't careful when installing it). Because you panic your machine often, and hard power-off it, and probably don't verify it after every instance, it's impossible to tell if the problems accumulate over time.

That's been my fear. I had a lot of panics on that partition when doing various OpenCL functions, mostly with patches that are broken, along the way to finding out what not to do to evoke pandemonium. I haven't had any panics in months, but had one a week or so ago, and then that freeze I described that was the first step of this recent partition loss.

How do I verify it after a panic? Do you mean by using disk validate, or something else?

I've had other "weird stuff" occasionally like some Finder crashes from viewing certain qtz's, and really large movie files that occasionally don't want to let me move to external hd's ("unexpected error").

cwright wrote:

gtoledo3 wrote:
A secondary question is - has anyone encountered any software that can fix the ACTUAL PARTITION in question into being bootable again? It's usually an irritating sort of song and dance to reinstall stuff, migrate files, and I'm not looking forward to it.

That's a non-question. the problem can be trivially recoverable (a few counts are wrong) to catastrophically wrong (you wrote zeros to the entire partition). Disk Utility seems to handle most of the simple cases well (I run it on my machine pariodically, and on my timemachine backup because it gets unexpectedly disconnected frequently), and a couple dodgy cases I've thrown at it. I've also had it totally fail to do anything useful for a few totally fubar partitions from bad memory that wrote trash all over the file system metadata.

Rather than trying to restore a broken partition, I generally find it faster/easier/more consistently pleasing to just restore from a time machine backup. I've had disk utility run for several hours trying to address something (and then fail), while a restore would have taken half as long, and not failed.

Point taken. I guess I should start using Time Machine again. My early experience with it was poor; it was a real hog, and it also didn't restore properly the one time I wanted it to. There seemed to be tons of useless replication of files, and it would go crazy when I would do offline renders that spanned over hours of time.

Thanks for your thoughts.

i would say to just restore the partition new as GUID table from your time machine as that may be quicker than running any disk aid utilities like drive genius or something. that is supposing you still have a good SMART status, if not you need to replace the drive. if the time machine is not an option try disk first aid or drive genius or any other drive diagnostic and or forensic type of program, maybe able to sort your problem.

i lost a whole year of work due to a SMART drive error and faulty backup. it sucks but i guess a backup of a backup is required now a days. they don't make drives like they used to.

i was bummed for a bit but you rebuild and do better the second time.

your data is relatively retrievable depending on how much it is worth to you. even after a format you can still get data back merging your header file with the datafork. norton is really good for this...

basically you pull out the datafork with a search by using the application the file was made with as your terms. once you get your datafork you need to merge that with a header which can be easily done by making a blank document in application your using as search terms and then merging this blank document with the datafork usually puts the file back together.

if you zeroed your drive you may be out of luck though.

gtoledo3 wrote:
Ok. I've had that happen after crashes before. That's odd... maybe it's linked to the hibernation thing for some reason? Thanks for the clarification, I don't doubt it, but I'm more puzzled now.

I have too, but I'm pretty sure it's unrelated to the crash -- Perhaps it's collecting a panic log?

gtoledo3 wrote:
Yes, I've had messages sometimes like "your system recovered from a serious crash because:" and then it lists the reasons. It looks like when an app crashes and you get prompted to send in a report. I have to admit, most of the time, I can directly link up what I did that caused "x" to happen, so I don't really look at that stuff, and just "click through" to get it off my screen quickly.

Does it say that (literally)? It doesn't sound like it recovered -- otherwise, you wouldn't have rebooted :).

The dialog you get is a panic reporter. It doesn't quite say "because:...", it gives a backtrace for the process that appears to have induced the panic, and some other data for the kernel engineers to poke at. When the GPU crashes (which is what it sounded like you experienced), the machine doesn't actually panic, it just becomes unusable (which is annoying). hard-power-offing the machine might interrupt disk writes, or it's possible the GPU crash already hung the kernel enough to prevent writes from completing.

gtoledo3 wrote:
That's been my fear. I had a lot of panics on that partition when doing various OpenCL functions, mostly with patches that are broken, along the way to finding out what not to do to evoke pandemonium. I haven't had any panics in months, but had one a week or so ago, and then that freeze I described that was the first step of this recent partition loss.

CL work really shouldn't be done on the GPU first. It's a shame QC barfs on CPU CL, because there at least you don't have to deal with unprotected drivers stomping on everything.

gtoledo3 wrote:
How do I verify it after a panic? Do you mean by using disk validate, or something else?

Disk Utility -> Verify Disk.

gtoledo3 wrote:
I've had other "weird stuff" occasionally like some Finder crashes from viewing certain qtz's, and really large movie files that occasionally don't want to let me move to external hd's ("unexpected error").

The "unexpected error" thing I think is something else (I'm nowhere near the finder/AFP/SMB teams to know) -- I get that from time to time, but it's usually due to a network hiccup that AFP can't recover from for whatever reason.

gtoledo3 wrote:
Point taken. I guess I should start using Time Machine again. My early experience with it was poor; it was a real hog, and it also didn't restore properly the one time I wanted it to. There seemed to be tons of useless replication of files, and it would go crazy when I would do offline renders that spanned over hours of time.

It's perhaps not much smarter now (I honestly haven't had any severe problems myself, but I hear lots of reports to the contrary), but the alternative (partition loss) is simply unacceptable to me. I've lost data to 1 harddrive failure in my life, and that's enough for me.

All my data is retrievable, it's just some kind of OS X wig out. I've never had partition failure that (in my opinion) is brought about by a software bug that has resulted in my files not being retrievable, only not being able to boot into the OS for that partition.

Oh, I was mixing up the terms verify/validate on the Disk Utility. There are many "missing thread records" and there's an invalid record count. "Error: Filesystem verify or repair failed." I wasn't expecting anything different to happen though. Oh well.

I've found time machine to be great. After the first backup, I don't really notice any negative effects from the regular backups at all, and it's recovered when I need it.

One important tip: do all your big renders and store big downloads and the like in a folder it's not backing up. I've got it ignoring my ~/Downloads folder, plus a '/No Backup' folder where I store random crap. I don't want it backing up a 3GB xcode download every couple of weeks :)

I managed to get an old g4 powerbook from work a couple of weeks back, I use that as a backup server for the whole house now. It was totally free but with a broken screen + gpu. I've removed the screen (as in the whole lid) and fixed the gpu with a heat gun (dodgy soldered joints.. thought it wouldn't be a problem, but would you believe the GPU was causing corruption even in remote desktop?!) It's awesome as a server - tiny, silent, fast, tons of storage on firewire for speed. It goes in the thin drawer meant for a keyboard + mouse on my desk :D

I'll try it out again. Now that you say that, I remember that at the end of my use with it, I had it ignoring my desktop, which was "ok", but I felt like it should be smart enough to not try to back up a file that was being written. Even then, with my white macbook, it just made stuff run too slow. I got into the habit of getting HD's and making bootable drives, and I'm just going to restore off of that, and go in and grab the files that I've made since my last backup off of the desktop.

Also, the time capsule itself kind of sucked initially, and required tons of updates. It seemed like I had a new software update everyday. It didn't inspire confidence. Then, when it failed to work the one time I wanted it to, I totally abandoned it. So, my time capsule is currently a big remote iTunes library.

I've been lucky in that I've never not been able to recover files when a partition goes bad because of software problems, only when there are actual hardware problems (only had that happen once with an HD that had been inactive for a long time).

It's a drag that there isn't any real way to tell what corrupted the partition. I'm pretty darn sure it is because of the software of a particular company, but I hardly want to try replicating the scenario.

Ibwas like you you with time machine for awhile. As the day my x wife and separated my computer also went down. As i was sitting at my dads house wondering what to do with my life os x just wouldn't boot up. I tried to reinstall and migrate from the cd my computer came with but my previously updated system was a different version than the os x saved on time machine. I remember thinking WTF is time machine good for if I can't restore from it when my computer craps itself. So like you George I gave up on it. A few months later like I mentioned before I had a couple drives go down. I had a couple options pay 20 bucks to the IT co outer repair guys on campus with no garentee they could retrieve anything. Send external back to the manufacturer with 3 grand in cash to retrieve the data from a clean room.

I really don,t know why I'm living in the sorry state I am but I would have liked to think all my data for a year would have ammounted to be worth over 3 grand and would warrant such a retrieval. I lost my first commercial development project that could have been worth lots of money on a backend for me. (radiology has not had any innovations in software for a decade, incase you where wondering, not at liberty to say anything else about the project ) so needless to say I didn't meet the convention deadline.

So besides all the awesome qc files I have made and collected from this site being toasted it was a big loss for me, but even then some digital files I hardly ever go back to is no comparison to loosing your child in a divorce. There is still a major prejudicial and gender inequality basis with the us cour systems but that's an entirely different topic.

What I decided to was go for the new drive so I could make new things because it's all about the new things for me and not the old ones or about the next one. So after loosing all my 2009 work I dint want to nuke my 2008 drive to back up my system that had smart status failure so I just replaced the faulty external. Maybe a bad decision I don't know but my laptops drive was ready to go so I had to act quick.

Wen I got my new drive I made a partition the same size as my internal drive for a time machine, then made a 10th partition to put a bootable os x on and left the rest of the drive for non time machine back ups. So I backed up my computer and returned it's drive back to apple subsequently had to get a new logic board as well but to my surprise when I migrated this time everything copied over like it was supposed to and faster than I expected thinking 300 gb would take 6 hours I think it was done in one and half. The only thing I did different was migrate with the migration application utility and not from the start up install disk. It seemed to make all the difference, now I love time machine. Unfortunately my machine had a new logic board which required me to re serialize preet much all my apps as they depend on Ethernet hardware a dress whichnwas now different so I had to reinstall a inch if apps but I think if my logic board had not been changed everything would have worked the way it supposed to.

Now when I remember to plug my time machine it's a geat tool particularly if I. Coding and a mess up and want to go back to the version I was working on ten min previously it's possible. I will tell you I have lots of drives and i dont like turning them on or even moving them so I pretty much hord away my data in a closet except for one time machine. So you see now I am paranoid about these things as i have never had a drive go down that wasn't repairable via software.

Have you tried mounting the HDD as a volume via Firewire ?

I have found that to be a pretty good way to gain examination of an otherwise recalcitrant MacBook HDD.

damn, i'm not even going to edit the above post as there are too many typos.... freaking ipad suggestive typing changed half my words. oh well there was no real useful info in the last post anyways.

I'll tell you what sucks though.

When I decided to "do whatever I was going to do" with that partition, I thought "hmm, I have the release right before GM, let me install that!". So I did that, did migration assistant from my Leopard partition (because I had never done that, and actually wanted a few Leopard apps and stuff, so I thought "what the heck").

Sooo.

I decide that I want to just update to 10.4, because I like the idea that everything is "clean" better than restoring from my last backup.

Nope.

I can't. I'm guessing because it's not the GM. It seems that I can't update to 10.6.1, 10.6.2 etc, all of which I actually kept the dmg's for. I didn't keep the one for the GM for some reason, and I never contacted Apple for a disc, even though when I bought my laptop it was in the time period where you could get SL free (it came with Leopard installed, but SL was imminent). It seems when I go on the Apple store, I can't download 10.6, or even get the DVD I thought I could get. Maybe this was a promo that has expired.

...and I let my ADC membership lapse because they stopped the mid-tier one.

Arghh. Wasted a serious amount of time. So, I guess I just need to restore the last time I backed up, even though I really don't want to. I'm not sure exactly what I want to do on this one yet. I don't want to get the ADC membership again just to download the 10A 432.

i think you can still get the SL disk for your computer. might not be free but maybe reduced ? i just smashed my snow leopard server disc, it was pissing me off. it was a completely legitimate official evaluation disc from apple, until i smashed it for a wanting password. maybe my understanding of evaluation is bit different ? as i didn't receive any serial with it.

its kind of like i'm being forced to download a pirate copy. some things are really messed up i buy dvd but i can not copy it to put it on my ipad, so i purchase the same show again this time in digital format from itunes, i can put it on my ipad but i can't play it externally out to my television. WTF did i buy an HDTV for ??? to watch hi-8 videos.

seriously its bad enough your forced to buy two copies of the same program but now i need to buy HDCP certified HDTV or something ?

my tv is only two years old, i mean i understand if it was my moms tv she had the same one for almost 2 decades but mine is less than 2 years old, which is subsequently now broken, not out of my own frustration but for crappy planned obsolescence reason i guess. since when do tv's last only a couple years ????

its like the only way to get something to work it seems is to steal it i guess. its one thing downloading a pirated movie or operating system but i don't entirely feel comfortable steeling a television.

deep breath 12345678910 ok much better don't want to vent to much here as the two HDTV's i have both do not turn on anymore and they are less than two years old. ( yes i have put new batteries in my remote )

one of them might be partially my fault as i built a multi-touch table out of it, although really i think it was the guy painting my house putting stuff on top of the exposed circuits. can not really blame anybody even if i know i didn't break it but now my other tv went down and i only turn it on a couple time a month.

so i'm leaning more towards electronics companies are money hungry greedy cost cutting rush to the market with crappy products that looks nice types of people that could care less about the consumer because they know the consumer doesnt have a choice but to by a new one.

with the exception of apple of course. (even though both my macbooks have shit the bed in less than two years as well) i'm telling myself i just got a bad apple and its not the company.

like i bought a new ipad and the screen was yellow, so it is easy enough to return it right ? no. you buy a brand new product that is broken and you return they send you a refurbished product back because you opened the box. so maybe the problem isn't with the company as i truly think apple wants to put out good products that change the world. it must be the manufacturing process now. doesn't matter if its a hard drive, a laptop, or a tv things are just junk nowadays.

the point is i have the same brand tv my mom does, she watches the tv everyday for two or three decades and the tv still works like new. i mean i tease her how old it is but i buy the same brand tv a couple years ago use it maybe a couple time a month and it won't work.

so as much as i don't want to say this george, i feel your frustration. just make your life easier and download 10432 from a torrent you will have it faster than you can get the disc, apple should be giving you for free.

Yep. I even have an install disc for sl that apparently will not install b/c it wasn't for this type of Mac! Funny. I don't really want to restore from my last backup b/c I feel like I just want to start from scratch, considering all the layers of seeds/reversions/etc that this particular clone would have. I would rather do a clean sl install and import my files that I recovered. Alas, for some reason I just have the disc right before GM around, and then a 10.2 that isn't working BC it's not for a MacBook pro I guess. I paid for, and had all of the seeds but got rid of a couple in a hasty moment. I thought I still had one of each but guess not. It sucks BC I paid for it three times, and can't just go on apple.com and dl it.

It is sort of outdated for the torrents as well... and I wouldn't feel confident about it- it would defeat the point of trying to make sure my install is rock solid.

I guess I'll just go to the store and get a disc, because I don't feel like calling apple support about the disc that I could get w/ my macbook pro and never got. I don't remember if I was supposed to order it by mail, or what? If I don't feel like that tomorrow morning, maybe I'll just restore from last week's backup.

Thought about it:

Don't feel like calling Apple about the SL disc that (I think) I am supposed to be eligible for by having bought my Macbook Pro at the time I did. I also wouldn't want to wait the time for it to come in the mail.

Don't want to restore to a system that I am dubious about either (eg., my 10.4 backup that does boot, but that is the result of upgrades/reversioning of a partition that has had everything from prerelease/test seeds/release SL to present day). Disc Utility says it's ok, but I still don't feel good about it. I want to "for sure" know that I'm starting at a point of something that should definitely be solid.

So, I guess I'll just go to the mall today and buy the disc for the "third time". Most expensive Mac OS X version I've ever bought... ;)

Well, I feel it was truly for the best. I ended up taking time on the install that I never took when I initially installed SL. When I first setup SL, I viewed it as a total "test" partition thing. Now, I have all of the stuff from my classic white macbook, plus the files from my SL install (I didn't lose a single file). So, it is truly the best of both worlds. I'm glad it happened, even though it was a hassle.