Managing Megabytes...

[RedRebel]

Mark recently convinced me that using raw instead of jpeg would increase the image quality from my Canon 5D and L lens. He is right, using Photoshops CS raw converter it is possible to further increase the exposure, brightness, white balance, etc and get rid of purple fringing for example.

I already new that raw files from a 5D take about 13MB of disk space, while the jpeg is *only* 5-6mb. However when using Photoshop for raw conversion, a 75mb psd file is created and then I still have to export a suitable jpeg for printing (in a shop, I don't print my self).

This means roughly 90-100mb per image is needed for storage space... I am not shocked about the initial 13mb raw file size, but the total disk space needed is a bit high (although my 300GB disk -excluding 300GB backup disk- is only 25% used at the moment).

I wonder how others manage their raw processing and disk space resources. Do you simply accept the fact you need about 90~100mb per image, or do you have a different approach?

Second I wonder if you process every singly raw file manualy or use photoshops default raw converter settings (use auto adjustmens Ctrl+U) and only do manual processing when needed?

thanks..

[danag42]

I wonder how others manage their raw processing and disk space resources. Do you simply accept the fact you need about 90~100mb per image, or do you have a different approach?

thanks..
[a href=\"index.php?act=findpost&pid=77658\"][{POST_SNAPBACK}][/a]

I've found that keeping the original file (in DNG format) plus a flattened version of the final image allows me to conserve on disk space.  

Having said that, I'm using a RAID 1 array with 500GB of storage for my images, aside from endless DVD's as backup.

For serious shooters, there are arrays that have capacities of 4-6 terrabytes.  I'm going to have to get one of those eventually.  
                                                           
 

[61Dynamic]

Accept the fact images take up space. You won't accomplish squat worrying about it as there isn't much of anything you can do to change that in any significant way. It's not that big of an issue anyway. At about $0.45 USD per GB (for a 750GB drive) hard drives are cheap and getting cheaper. Heck, it was about 2 years ago I bought a 120GB drive that cost me about $250.

With new technologies Toshiba has been dropping strong hints that a single drive will reach 1.5TB by next summer. Given the cost of the largest drives always remain at $500 per, this means everything else will be dropping in price to match.

Second I wonder if you process every singly raw file manualy or use photoshops default raw converter settings (use auto adjustmens Ctrl+U) and only do manual processing when needed?

I do an initial quick edit for proofing if the defaults are too off whack or I intend for everything to be BW. Otherwise, I only hand-tune in the converter what I rate highly or will make prints of.

[nicolaasdb]

I feel your pain...but don't give in to quantity over quality...so keep on shooting RAW...and keep on buying storage.

I first tried to burn all backups to CD then DVD, but burning a days work...with my Canon ds1MkII.....about 2500 images which is about 35GB (+/-) to DVD..was such a pain...so I started buying LaCie drives...first 1TB and now 2TB.....within one year went from 1TB to 10TB and have about 1.5TB left free!! At least I don't have to buy cabinets to story contact sheets and neg's!

Now I recently bought a Aptus 5 MF back and shoot less per day (about 1200 images) but the space used is 70GB!!! So I estimate that by the same time next year I will have about 30TB in used space!!

 Ce La Vie!!

[ed j]

Mark recently convinced me that using raw instead of jpeg would increase the image quality from my Canon 5D and L lens. He is right, using Photoshops CS raw converter it is possible to further increase the exposure, brightness, white balance, etc and get rid of purple fringing for example.

I already new that raw files from a 5D take about 13MB of disk space, while the jpeg is *only* 5-6mb. However when using Photoshop for raw conversion, a 75mb psd file is created and then I still have to export a suitable jpeg for printing (in a shop, I don't print my self).

This means roughly 90-100mb per image is needed for storage space... I am not shocked about the initial 13mb raw file size, but the total disk space needed is a bit high (although my 300GB disk -excluding 300GB backup disk- is only 25% used at the moment).

I wonder how others manage their raw processing and disk space resources. Do you simply accept the fact you need about 90~100mb per image, or do you have a different approach?

Second I wonder if you process every singly raw file manualy or use photoshops default raw converter settings (use auto adjustmens Ctrl+U) and only do manual processing when needed?

thanks..
[a href=\"index.php?act=findpost&pid=77658\"][{POST_SNAPBACK}][/a]

i use to save as tif format. i hit 1.5 terra bytes. then i started thinking about who will want my pic when i die? nobody. what pic i do sell is far and few between. mainly i do it for myself and wife. thats it. what i do now is save the raw or nef
convert some to tif or jpg and print them. after there printed i delete the jpg, or tiff and just keep the raw file. they stay on my walls for a few months then i put up a new batch and give away the old ones at are 2 time a year garage sale.

was recently at  a garage sale and a ladie in her mid 70's had about 10,000 plus old photo prints she was selling of her family taken over a 110 year period by her great grand father, her monther and father and herself. very few where sold. she was selling boxes of them for $1

long term what kind of burden are u gonna put on your family when u die. sorting thru 20 hard drives of pic. after the frist few months of going thru them , i'm sure they will give up and pitch them.

now i keep all my family photos on 1 hard drive and everthing else on  others and clearly mark them. also have a packup of the original software on those hard drives.

[Tim Gray]

I shoot between 10 and 15k frames per year.  I process about 25% of those.  I keep the raw for the processed ones only and jpg, from which I print.  (the only exceptions are when I've spent real money to travel to a not likely to return to location - they all get kept)  For the 1% that are "the best" I end up reprocessing after having lived with a jpg print for a few months, and those I keep fully layered as PSDs.  HDD storage is reasonable in these circumstances.

[fike]

I haven't reached the levels of many of those who posted before me.  My redundant RAID array is up to about 350GB of data.  

I shoot RAW exclusively now.

What I don't do, is convert them all to PSD or TIF or anything else. I only save in one of these formats when I intend to print the image.  Otherwise, I keep the RAW file with the settings I liked in the XMP sidecar file.  To me that is like keeping the negative.  After shooting a thousand images on a trip, I may make 60 final images that are archived in PSD format.

[pixelpro]

Managing MB's

I have given up on DVD's and CD's and gone instead for a bank of harddrives attached viae firewire. I have different parts of my photography practice on different harddrives, they are like giant files. My image files are over 100MB each.

[JessicaLuchesi]

My workflow starts with selecting the images to be processed from the RAW files, converting only those into DNG. Then, from those, further selection, and processing, storing them in PSD files. And saving the final images either in JPG or TIFF format. It only depends on the client and their requirements. 9 times out of 10, it's JPG. Those are all stored in DVDs ( even if it takes 2 DVDs to store a single work ), with also a backup DVD ( meaning, 4 DVDs to store a 2DVD work ).

If disk space is an issue, the PSD version is tossed away. I also keep a record on CDs sent to clients, with exact copies of what was shipped, in case I need it.

[The View]

I weed out images that I do not need, especially multiple shots.

[Brad Proctor]

I use Lightroom and after uploading my images off the camera I delete the obvious really bad shots then I convert the rest to DNG.  The keepers I do my adjustments in Lightroom, then export a TIFF to finish it up in Photoshop.  The final flatten 16-bit TIFF, I save.  Since I don't do my own printing, I also create a JPG that is ready to print.  The DNGs get routinely backed up to DVD as well as the TIFFs and JPGs whenever there is enough of them to fill a DVD.

[sergio]

Whatever you do, have redundancy. I do want my photos to keep for the rest of my life. After that I don't really care much for what happens to them. I hope they serve a useful purpose for my son, instead of being a burden. If so he can ditch them.

The big problem in these digital days is that in the long run most of the worlds history in images will be lost because there are no practical and useful methods to archive images apart from paper.
So future archaeologists will use dvds as coasters.

[The View]

Don't get hung up on workflow issues alone.

Keep your creative process upfront.

If you run out of disk space, buy another disk.

Throw away what you don't like.

Stay a photographer, don't become a data manager. Because you'll end up managing more than photographing.

Some of this workflow software is so poor at its core, that it causes additional work by just delivering poor RAW conversions, for example.

So, don't go with the flow, but search around and go with what you like. Just because so many people are using LR or Aperture doesn't mean you have to.

[ErikKaffehr]

Hi!

Here is what I do, right or wrong...

1) I have something like 30000 picures right now...
2) I use Lightroom as main application
3) I essentially import all pictures as DNG directly from Lightroom
4) They are stored on a 1 TByte RAID-5 server used as NAS. They are stored as ../DNGImp/Year/Month/Day
5) I'm terribly bad at keywording, that's a bad thing!
6) The whole DNGImp folder is backed up to a local disk on my iMac

I generally only process or print a very small portion of my work. Here is what I do:

- I have folders Process_2006, Process_2007, Process_2008
- I have subfolders like Panorama, HDR
- So if I want to make a Panorama from Sella Pass I would export all pictures I would use to
  .../Digicams/Process/2008/Panorama/SellaPass
  using 16-bit TIFF in Adobe RGB.


Now, I should have an additional backup at a remote site, but I don't have that.

My DNGImp folder is 206GByte
---------------------
erik-kaffehrs-computer:/Network/Servers/192.168.0.7/srv/Digicams ekr$ du -hs DNGImp/
206G    DNGImp/
---------------------




Best regards
Erik
 

Mark recently convinced me that using raw instead of jpeg would increase the image quality from my Canon 5D and L lens. He is right, using Photoshops CS raw converter it is possible to further increase the exposure, brightness, white balance, etc and get rid of purple fringing for example.

I already new that raw files from a 5D take about 13MB of disk space, while the jpeg is *only* 5-6mb. However when using Photoshop for raw conversion, a 75mb psd file is created and then I still have to export a suitable jpeg for printing (in a shop, I don't print my self).

This means roughly 90-100mb per image is needed for storage space... I am not shocked about the initial 13mb raw file size, but the total disk space needed is a bit high (although my 300GB disk -excluding 300GB backup disk- is only 25% used at the moment).

I wonder how others manage their raw processing and disk space resources. Do you simply accept the fact you need about 90~100mb per image, or do you have a different approach?

Second I wonder if you process every singly raw file manualy or use photoshops default raw converter settings (use auto adjustmens Ctrl+U) and only do manual processing when needed?

thanks..
[a href=\"index.php?act=findpost&pid=77658\"][{POST_SNAPBACK}][/a]

[attachment=7425:attachment]

[stevekhart]

Stay a photographer, don't become a data manager. Because you'll end up managing more than photographing.

Oh no, too late!

I have a PC setup including a NAS 2Tb Acer Easystore, RAID5 which gives me 1.36Tb usuable storage, currently got just <400gb free, all my images.  I also have a 1Tb NAS device holding a compressed backup of the NAS. Since this is sat next to the acer, it's not really serving a purpose.  What I'd like to do is get the backup drive up to date with the acer, then take it off site and use a portable drive to take differential backups and apply them to the offsite backup. Is there any software that supports this?  In this way, for example, I could take the portable drive to the office on Friday, kick off the differential backup to run over the weekend, then bring it home Monday evening and have it apply the differential to the backup drive during the week.  Then if somebody broke in and nicked my drives at the weekend, at most I'd lose a week's worth of images (which I'd have on my laptop anyway).

[Farkled]

As the man said:  rightly or wrongly, here's what I do.

Download RAW images.
Pass 1 - Preview
Pass 2 - Grade and delete real losers (shots of lens cap, etc.)

If family shots, do a batch conversion to JPG and copy to DVD for family members.

I may or may not do any of the following according to time, interest and other priorities.

Convert low grade stuff to JPG and delete RAW
Work on better stuff in ACR and/or DxO
Stuff actually good enough to print get moved to PS and saved as PSD or TIF until printing is done.  I may then delete the PSD/TIF - especially if it's a 500mb+ pano file.
Really good stuff may get keyworded (I really need to be doing this, I just want to settle on imagebase software 1st.)

All images exist on 3 separate USB drives and MOZY.  Image backups happen to one drive every 2 hours and the other and to MOZY overnight.  Note that MOZY can only handle a couple GBs per day max - which is sufficient or me.  Some day I may start draining low grade JPGs to DVD and take off-line.  Probably not, especially with 500GB drives available for $150