Macrium Support Forum

Macrium Support Forum
Macrium Support Forum    RSS 2.0  RSS Feed      KB
Home | Profile | Register | Active Topics | Members | Search | FAQ
Username:
Password:
Save Password
 All Forums
 Reflect v5
 Disk Imaging
 Defrag is bad for Differentials and Incrementals
 New Topic  Reply to Topic
 Printer Friendly
Author Previous Topic Topic Next Topic  

alan9182
Advanced Member

United Kingdom
527 Posts

Posted - January 25 2013 :  22:09:53  Show Profile  Reply with Quote
Reasons to refrain from defragging, and how I make a FULL image last a year.

C:\ has 11.4 GB of Used Space and a Full Image is 6.5 GB.
Daily Incremental backups are typically less than 100 MB,
and a monthly Differential will start at perhaps 500 MB and after a year it might become 2 GB,
at which time I start afresh with a new Full Image.
So far I have a total of only 78 GB for all images backups since installing Windows 7 on a new SSD on 10th March 2012.

All images are held on a secondary internal HDD,
and in case of disaster the FULL and Differential backups are copied via TeraCopy to an external eSATA HDD.

If I find a problem in an application I can roll-back to any previous date to determine if it was always like that;
or if it has been broken and how it was broken and how I may fix it.

I do NOT defrag unless I see a need - and I rarely see any need.

The penalty of a defrag with Acronis in the past, and no doubt with Macrium,
is that when a FULL image backup is around 6 GB, and a daily incremental might be 50 to 100 MB,
if the daily incremental is immediately followed by a defrag and then another differential,
the second differential is likely to be over 2 GB because more than 30% of all data will move by at least 1 sector.
I only consider performing a defrag before creating a FULL image.

This shows the analysis of my partition by the Piriform Defraggler
C:\	(C:\)	[c:\]	D:\	(D:\)	E:\	H:11500	11400	9620	755	653	411000	742	MB Used Space
435	426	175	116	45.4	246300	11.2	MB Size Fragmented Files
99	83	76	491	137	3779	42	No. Fragmented Files
638	566	233	1866	575	19346	119	Total Fragments
2	2	2	16	7	60	2	% Fragmentation

C:\ D:\ E:\ and H:\ are the results before running Piriform CCleaner to purge junk that accumulated over the last month.
(C:\) and (D:\) are the results after running CCleaner
[c:\] was the condition after 1 year without defrag of Windows 7 on my HDD before I switched to the SSD
I never felt the need for a defrag on the HDD and I sure see no need on my SSD.

CCleaner removed 100 MB of chaotic fragmented browser cache from Drive D:Much of the 45.4 MB remaining fragments is no doubt due to my 65 MB browser profile urlclassifier3.sqlite which gets touched every day I hit the Internet.

E:\ might benefit from a defrag, but I doubt that there would be more than a couple of seconds improvement in the 4 minutes it takes Macrium to read a total of 8 GB and restore 11.5 GB to my SSD.
E:\ also holds my downloads folder, and downloads are rarely used more than once,
so I see no point defragging those.

O/S: Microsoft Windows 7 Ultimate *64 SP1
Graphics: HIS ATI Radeon HD 4670 1GB GDDR3
Motherboard: ASUS M3A32-MVP DELUXE AMD 790FX
Processor: AMD Phemom 9500 Quad Core 2206 Mhz
SSD : OCZ-VERTEX2 ATA Device, 59 GB
Hard Drive: Samsung HD103SJ, 931GB, MBR, 32MB Cache
Hard Drive: WD Caviar Black, 640GB, GPT, 32MB Cache
RAM: Corsair XMS 6400 4GB (2X2GB) 800mhz

Gork
Advanced Member

USA
629 Posts

Posted - January 26 2013 :  01:28:09  Show Profile  Reply with Quote
With regard to defgragmentation there are two schools of thought that I'm aware of on the yes side. You spoke of one of them, speed increase, but the other is because a defragmented drive should last longer. Though there is a debate on these issues, and NTFS has certainly improved the situation, I have chosen to defrag. And I have learned I can do so AROUND my imaging/backup schedule so it doesn't interfere. I only defrag in-between a final incremental and the next full.

Were I to change my imaging schedule to be more like yours, instead of defgragging monthly I'd change it to once every six or twelve months, always after my last differential or incremental in the set and before the full image which would start a new set.

My current setup makes this very easy. I have three drives, C: D: and E:. C: a 300GB SSD where I have Win8 installed. D: is a 600GB SSD where I install my software. E: is a 2TB hardware RAID card where I store program data. (I have two 1TB hard drives hooked up to in in a RAID-0 configuration.) Since we do not want to defrag SSDs, I only have one drive to even consider - the RAID card. Since I'm currently using only 137GB of space on the array, it only takes about 5 mins to defrag once per month.

Most of my data is stored on network storage, a 2TB RAID-1 array device which uses a file system that doesn't need to be defragged. My image files are stored on a Linux box I set up as a file server containing a 1TB software RAID-1 array. (The 2TB device was too slow.) The file system is ext (ext4 I believe) which also supposedly doesn't need defragging.

As I partially noted in the other thread, I currently have one scheduled backup which runs as a VBScript on day 1, 8, 15, 22 and 27 of each month. It checks, and if it is day 1 it will perform a full backup. On the other days it will perform an incremental backup. On day 1, after the new full image is verified, the oldest set will be deleted thus always leaving me with one full set. Defrag runs on day 28 of every month so it doesn't cause problems with my imaging.

This method has worked well for me, as a "backup solution." And it has saved my hide on one occasion, long before Reflect handled my imaging, after I had a hard drive crash on a computer I no longer use. The reason your backup strategy interests me is you can go back to any time in the past year and restore files/partitions and without using a whole lot more disk space that my method uses. It sort of adds a "time capsule" into a backup strategy. In order to do this and still defrag, though, I'd have to adjust my defrag schedule to only once per year. But I doubt that'd be a problem - the single volume I would need to defrag is only 1-2% fragmented every month. Plus, as I said, the two drives are behind a hardware RAID card - and it keeps 'em pretty clean anyway.

The only things I want to do to finish off my strategy are: 1) Perform a file backup on irreplaceable files (photos, installation EXEs I've purchased etc) to removable media (flash drives or optical media, haven't decided for sure); and 2) POSSIBLY copy my latest image over to an external HDD of some sort I can store off site. I'd probably only take my full image backup off site once every few months. As far as #1, Reflect's file backup isn't robust enough for my needs so I have another piece of software in mind; so #1 is still on hold until my pocketbook catches up with my wannabe caviar budget.

When I mentioned before that a similar backup strategy saved my hide in the past, it was actually a file based backup to optical media. I had a set of 24 DVD-RWs, one set per month for a period of two years. After two years I'd rotate the media so I always had two years worth available, and once per year I'd move one piece of media off site. I learned two things: I never had need to go back as far as two years - that was overkill. For me, six months would probably be more than ample. And, the non-irreplaceable files? They can be a royal pain to lose! So my backup strategy after that was to image my C: (system) and D: (programs) volumes to a software E: (data) RAID-1 array in addition to keeping my file backups. When I bought this new computer a few years ago, though, and set up my two file servers I decided it was time for a system which is a little less invasive for me. In fact, when I start my file backups I may have them run from another "server computer" I use to serve a web site and run my free Google Voice "landline" telephone system. That way it'd run in the background on a computer which is always switched on instead of me having to deal with it on my main computer. I thought of this idea when I started thinking of the possibility of using flash drives for my file backups instead of optical media. (My plan since I bought the new computer was to go with a Blu-ray recordable drive.) I suppose I could rotate two USB drives around as well, we'll see what I end up going with.

Edited by - Gork on January 26 2013 01:39:14
Go to Top of Page

alan9182
Advanced Member

United Kingdom
527 Posts

Posted - January 26 2013 :  11:59:50  Show Profile  Reply with Quote
You have independently come to the same decision,
that Defrag is something to do before a FULL image.
We are brothers

I think the good thing about CD's and DVD's is that you cannot kill them by forgetting to "Safely Remove" them.
If only Flash drives were like that

My experience with CD's has mainly been as Bootable Recovery devices,
and I now prefer a Flash Drive because a Macrium Boot Recovery WinPE Flash "loads Windows Files" so much faster from Flash than it does from a Macrium Boot Recovery WinPE CD
I also find that it is faster creating a Macrium Boot Recovery WinPE Flash than creating an ISO and then burning it to a non-erasible CD,
and do not get me started on the frustration of burning the ISO to a re-writeable CD then taking a coffee break and coming back to find that it is still in the erase cycle that precedes burning

I strongly recommend Teracopy to perform file backup on irreplaceable files.
This computes MD5 checksums on each of the source files as it copies to the destination,
and then it reads the copies and computes their checksums and compares with the MD5 checksums of the source.
If any file is inaccessible it retries a few times before moving on with the rest of the files, unlike Windows Explorer which will stop dead at the first problem.
When TeraCopy has finished copying and comparing you can clear all successes from the screen,
and do whatever is needed to make accessible what had been inaccessible,
and then retry all the MD5 mismatch errors and inaccessible files again.
When Teracopy is done I know that the job is done right.

When I create a Macrium image backup it is automatically validated by Macrium.
There was a time when I used Windows Explorer to copy those backups from my Secondary internal HDD to an external USB2 connected HDD and then had Macrium revalidate those copies.
One day I copied about 8 off *.mrimg files in the 2 to 4 GB size range,
and half of them failed Macrium validation, even though the internal HDD files still validated.
I think one of the failed copies was also slightly smaller than the original.
I suspect a bad USB2 plug/socket caused the error and Windows ignored the error.
Since then I trust TeraCopy to do the job right and tell me of any problems,
and I no longer have Macrium revalidate the copies.
I still use Teracopy now that I have an eSATA external drive.

Teracopy comes from
http://codesector.com/downloads

If you are like me and prefer Portable Utilities then this tells you how :-
http://blog.codesector.com/2011/11/24/teracopy-portable/
Until today I was unaware of this new installer capability of portability.
My experience had only been with a very old but Portable only version 2.05.
I think the current version 2.27 is a lot more user friendly and it has a help file.

Regards
Alan

Edited by - alan9182 on January 26 2013 12:02:02
Go to Top of Page

Gork
Advanced Member

USA
629 Posts

Posted - January 26 2013 :  16:59:57  Show Profile  Reply with Quote
I also graduated from the WinPE bootable CD to flash drives. I found it was faster and, well, I keep the flash drives right next to my desk, so if I use them I don't have to walk across the room to grab the CD-RW. ;)

I will probably use Genie Backup Manager Pro. It is the latest in a line of backup software which initially started as a de facto standard in IT circles but was too expensive for home use. That company (can't remember the name) eventually released a cheaper version for home use but sold it off. I used one of the resulting pieces of software, BackupMyPC, for many years. (There's more to the story, rights being bought by another company which cornered the market for awhile, stuff like that, but I can't remember all the details.) If I use my old WinXP "server machine" with a flash drive for backups (instead of an optical drive) I might be able to use that old BackupMyPC software for now. But if not I'll probably go with Genie. I sure wish Reflect's file backup would work for me.

Genie lets you take incrementals, it compresses, it lets you quickly check boxes to choose directories or files which should be backed up, lets you pick a directory MINUS specific files and will back up any new files added later on... It verifies, it schedules, it slices, it dices... ;) BackupMyPC is the software which saved my bacon after the only hard drive crash I've ever had...

It's been a long time since I had the problems you describe with rewriteable optical media. I do use +RW DVD media, and it formats in about 30 seconds. I've moved beyond DVD media though and would have to move to (very expensive) 100GB tri-layer rewriteable Blu-ray media. I'm not sure what new problems might crop up with that; I haven't even purchased a drive yet! And I may not any time soon if I decide to use flash drives instead...

Yeah, I learned about the defrag problem long ago (way before I started using Reflect) after nearly pulling all my hair out trying to figure out why my incrementals were so huge! When I figured it out I felt really dumb 'cuz it made such perfect sense!

I'll check into Teracopy before making a file backup software purchase. Thank you for the referral. It still may be quite some time though. I'm still trying to pull myself out of the financial pit after upgrading my 160GB SSD to a 300GB and adding the 600GB SSD to my system some 2-3 months ago. It was a very expensive dumb move, but I'm most definitely in love!

As far as copying image files use Teracopy, doesn't Reflect have an option to make a duplicate copy at a secondary location now? As of 5.x I mean? I've never used it, so I'm not sure how well it works for verifying the duplicate copy or doing everything on a schedule without user input. But if you haven't looked into it, well, it may be worth your while to do so.
Go to Top of Page
  Previous Topic Topic Next Topic  
 New Topic  Reply to Topic
 Printer Friendly
Jump To:
Macrium Support Forum © Paramount Software UK Ltd Go To Top Of Page
This page was generated in 0.52 seconds. Snitz Forums 2000