Hard Drive Block Size Selection When Working with Adobe Bridge and XMP Files

[fike]

So I was considering tweaking the hard drive block size (range between 512 bytes to 64KB, 4KB is the default) on my archive hard drive (actually RAID 1 setup) to something higher, maximizing performance for the relatively big (20MB) raw files that populate the hard drive. 

Problem!!!

I realized that because I choose to use ACR with my Canon raw files, I have a 1KB XMP sidecar file for every single image.  That means I have a very small file and a very large file for every image.  This diminishes the performance benefit of using a larger block size because I would be increasing the relative size of each of those small files, wasting lots of space.  So where a pair of files might take 20MB for the raw file and 1KB for the XMP file, If I changed to a 64KB block, I would still have something very close to 20MB for that raw file yet the 1KB XMP file would take up 64KB. 

I can't decide if this is significant, or important.  In the whole year of 2010, I shot about 30,000 photos for a total of a little over 700GB of files (both raw and XMP).  I guess I can do some calculations to figure this out:

 1)  Current setup, 4KB blocks multiplied times 30,000 XMP files = 120MB of XMP files
 2)  Proposed setup, 64KB blocks multipled times 30,000 XMP files = 1,920MB of XMP files or almost 2GB of XMP files

Can someone check my math?

So, my conclusion is that it is probably no big deal to use the larger size, because I would only lose about 2GB of space in a year.

Does this make sense?

[John.Murray]

Marc:  Your math looks right (don't ya hate doing it in public???).

A couple of issues:

1) Controller - a majority of sata controllers process 512 byte blocks resulting in a 2^32 capacity of 2TB.  Granted more and more current chipsets are now supporting 4kb (although it's still unclear to me how they internally handle data).  Regardless - if your considering larger block sizes - your controller will be a choke point unless it natively handles your chosen block size, Adaptec and LSI make 'em.  Probably Promise as well - I've never messed with them.

2) Sector offset - you'll need to make sure your chosen block size physically aligns with the drives sector boundaries, otherwise every read/write operation will be broken out into multiples.

3) This is a really nice article that is more oriented toward database performance  - still, many relevant points.
http://www.anandtech.com/show/2739/2

4) It's all speculation until you compare & measure on your platform.
http://iometer.org/