Pages: [1]   Go Down

Author Topic: Batch-processing of RAW files  (Read 7463 times)

PeterLange

  • Guest
Batch-processing of RAW files
« on: November 12, 2006, 05:01:48 am »

Recently I came back from weekend trip with about 100+ Raw files calling for my attention.  So I started to process them based on my favorite workflow which is currently to do about 25% in Camera Raw and further 75% in Photoshop.

However, I realized that it cannot be that I have to spend another weekend to process these files.  Since then, I’m looking for an all-auto solution which delivers a somehow reasonable quality for the majority of images.  Thus, allowing me to focus and spend some time on the work with the few 'masterpieces'.

Actually this question about an ‘all-auto solution’ was inspired again by the current discussion on the Canon G7.  I’m quite sure that about 90% of said images could basically have been captured as JPEGs without great loss.  However, for the rest I need the undeniable RAW advantage and after-shot editing capabilities.  But now, I’m facing with a large number Raw files which I just what to process automatically – in a way which at least reaches an equal or favorably a bit superior quality compared to JPEGs…

Hope the core of my question is understandable.  And, I’m completely open for any equipment, software and procedure which may be suggested.  How do you proceed?

Thanks!

Peter

--
Logged

Julian Love

  • Guest
Batch-processing of RAW files
« Reply #1 on: November 12, 2006, 06:18:08 am »

I do a fair amount of travel stock photography, and will typcally shoot up to 200 RAW images a day while on location. So after a one week trip I will get back with approx 1000 images. I review them on my laptop each evening as I go along, just to make sure I am getting the shots I need, but I don't do any processing at this stage due to time and laptop screen constraints

My process when I get back home is as follows.

1) copy files over into the Import folder on my desktop
2) review and rank all images as 1,2,3 or trash using RawShooter Professional
3) re-review everything I have trashed, and then trash them :-)
4) rename all RAWs I am keeping to my filename system
5) make adjustments to all my Rank 1 images in RSP
6) export at 16 bit TIFF to my Working Files folder
7) Import all TIFFs into iView Media Pro
8) Opening from within iview, do dusting and spotting in PS, and any final editing although this is usually minimal, and convert to 8-bit
9) Caption and keyword in iView, and embed metadata in the TIFFs
10) Burn TIFFs onto a DVD and send to my stock agency
11) Transfer all RAWs and TIFFs onto my archive external disks using RSP for RAWs and iView for TIFFs (to maintain all edits and links, rather than using Windows Explorer)
12) Backup
13) Have a cup of coffee :-)

This works well for me. But I am seriously considering switching to Mac in order to use Aperture. The idea of being able to do all these things from within one application is very tempting....espacially not having to have lots of TIFFs lying around. 90% of my TIFFs are just spotted versions of my adjusted RAWs.

Hope this helps

Julian
Logged

Henrik Paul

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 87
    • http://www.henrik.paul.fi
Batch-processing of RAW files
« Reply #2 on: November 12, 2006, 07:31:13 am »

This friday I was shooting with another photographer a larger scale party which lasted for about 16 hours. I amassed over 500 photos from that day. When shooting for my own enjoyment, I don't shoot all that many, so I just copy the files to my hard drive and browse them with Adobe Bridge, editing the few I might like and store them separately.

This, naturally, wouldn't do for 500+ raws, so I decided to give Lightroom a shot. Previously when I have tried to get used to Lightroom, there was something I didn't really like about it. I can't put my finger on what exactly, but it might do with not being integrated enough with Photoshop. Anyhow, I gave it a fair shot.

0) All files are copied to the hard drive and imported to Lightroom's library
1) I began with looking through each and one of them, cropping a bit here, adjusting exposure and white balance there, also removing the duds.
2) Then I had a cup of coffee, did something else, and later returned to my photos. (this step I think is important, to get perspective)
3) I looked them through one more time, and whenever I saw something that might be above the average, i gave it one star. If I saw something I really liked already, I gave it two stars. I was down with about half of the photos.
4) I set Lightroom's filter to show all photos with one or more stars. Then I looked at "scenes" (for example all photos of one and the same speech) and picked out the two or three best photos of each and made sure that all the corrections were in place - crop, rotation, exposure, white balance.

I'm currently at step four-and-a-half, which included making a batch of max 800x800 images of each of the two-star photos (an export feature, very nice), had it churning on them for a while, and sent to the other photographer to 'sync' our photos (not to have too many photos of the same scene), and I'm currently waiting for his response. After I get his response, I will continue with the following:

5) Open Lightroom again, look at the individual two-star photos if there's a need for local adjustments - if so, I export them to Photoshop and do the stuff needed and save them somewhere; if not, I export them directly to where the rest of the photos are.

I'm thinking more and more highly of Lightroom every minute I work with it, and most probably will incorporate it in my 'private workflow' in the future. I'm just worried of what happens when the Beta license expires
Logged
You are welcome to look at my thoughts o

alangubbay

  • Full Member
  • ***
  • Offline Offline
  • Posts: 117
Batch-processing of RAW files
« Reply #3 on: November 12, 2006, 11:10:50 am »

Quote
Recently I came back from weekend trip with about 100+ Raw files calling for my attention.  So I started to process them based on my favorite workflow which is currently to do about 25% in Camera Raw and further 75% in Photoshop.

However, I realized that it cannot be that I have to spend another weekend to process these files.  Since then, I’m looking for an all-auto solution which delivers a somehow reasonable quality for the majority of images.  Thus, allowing me to focus and spend some time on the work with the few 'masterpieces'.

Actually this question about an ‘all-auto solution’ was inspired again by the current discussion on the Canon G7.  I’m quite sure that about 90% of said images could basically have been captured as JPEGs without great loss.  However, for the rest I need the undeniable RAW advantage and after-shot editing capabilities.  But now, I’m facing with a large number Raw files which I just what to process automatically – in a way which at least reaches an equal or favorably a bit superior quality compared to JPEGs…

Hope the core of my question is understandable.  And, I’m completely open for any equipment, software and procedure which may be suggested.  How do you proceed?

Thanks!

Peter

--
[a href=\"index.php?act=findpost&pid=84762\"][{POST_SNAPBACK}][/a]
« Last Edit: November 12, 2006, 11:15:25 am by alangubbay »
Logged

Roy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 196
    • http://
Batch-processing of RAW files
« Reply #4 on: November 12, 2006, 06:03:13 pm »

Quote
Actually this question about an ‘all-auto solution’ was inspired again by the current discussion on the Canon G7.  I’m quite sure that about 90% of said images could basically have been captured as JPEGs without great loss.  --
[a href=\"index.php?act=findpost&pid=84762\"][{POST_SNAPBACK}][/a]

The answer may be in your hands.  

A number of DSLRs (e.g. Nikon D200) can bee set to capture your shots in both raw and JPEG. Use the JPEG as your first source and process the raw file only if needed. The camera will probably make a more intelligent guess at how the scene should be processed than any software you might apply to automatically process the raw files after the fact.
Logged
Roy

PeterLange

  • Guest
Batch-processing of RAW files
« Reply #5 on: November 13, 2006, 03:08:00 pm »

Many thanks for your comments.

It reminded me that I should take more care about file management, pre-selection and maybe the capture itself (to avoid too much editing work).

However, I also understand that there doesn’t seem to be an “all-auto solution” which would do it more or less right for let’s 90% of the captures.  Something like: to press one button, drink one beer (or two) and let the computer process all Raw files overnight…  Maybe except shooting both JPG + RAW.  Maybe.

Cheers!

Peter

--
Logged

Andy M

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 333
    • http://
Batch-processing of RAW files
« Reply #6 on: November 13, 2006, 04:52:18 pm »

Quote
However, I also understand that there doesn’t seem to be an “all-auto solution” which would do it more or less right for let’s 90% of the captures.  Something like: to press one button, drink one beer (or two) and let the computer process all Raw files overnight…  Maybe except shooting both JPG + RAW.  Maybe.

I think we've all looked, but I'm certainly yet to find...
Logged

RedRebel

  • Guest
Batch-processing of RAW files
« Reply #7 on: November 14, 2006, 02:52:56 am »

When using Adobe Bridge you can open several raw images at once using the raw editor or whatever it is called. Normaly it uses automatic settings for exposure etc...and I think you can also configure some defaults, but i am not a real Adobe hero. When you have opened these images you can quickly review them and make some changes if needed. Having said that, it is very tempting to adjust image by image, but then you easy spend several minutes per image.

Like someone else stated, using the RAW+jpeg mode of your camera is easier...thats why I upgraded to a 4GB card
Logged

dlashier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 518
    • http://www.lashier.com/
Batch-processing of RAW files
« Reply #8 on: November 14, 2006, 03:59:06 am »

I'm not sure why no one has mentioned C1. Using quickproof you can batch output very high quality jpegs at the rate of 1 per second or faster, full process in the background as you tweak the next, or just toss them all in for batch processing while you go have dinner. Absolutely no reason to waste space and write time for raw+jpeg.

- DL
Logged

PeterLange

  • Guest
Batch-processing of RAW files
« Reply #9 on: November 14, 2006, 06:10:54 pm »

Quote
However, I also understand that there doesn’t seem to be an “all-auto solution” which would do it more or less right for let’s say 90% of the captures.  Something like: to press one button, drink one beer (or two) and let the computer process all Raw files overnight…  Maybe except shooting both JPG + RAW.  Maybe.
Quote
I think we've all looked, but I'm certainly yet to find...
Quote
I'm not sure why no one has mentioned C1. Using quickproof you can batch output very high quality jpegs at the rate of 1 per second or faster...


Quote
When using Adobe Bridge you can open several raw images at once using the raw editor or whatever it is called. Normally it uses automatic settings for exposure etc...and I think you can also configure some defaults, ...
[a href=\"index.php?act=findpost&pid=85086\"][{POST_SNAPBACK}][/a]
Well, that’s basically the approach which I’m trying to elaborate since some time – with a focus on broad applicability (at least for my purposes).  On this way I found it better to disable the auto functions in Camera Raw, even to set some controls completely to zero, in order to launch a more and more extensive Action in PS…  So I was wondering what other options I’m missing which might exist.

Thanks again.

Peter

--
Logged

b2martin

  • Full Member
  • ***
  • Offline Offline
  • Posts: 136
Batch-processing of RAW files
« Reply #10 on: November 14, 2006, 10:44:26 pm »

You can use Actions in Photoshop to process all your RAW files and write the result to a folder.  I place all my raw file from the latest shoot in a folder, select all and open in Bridge, make any adjustments (you can also just process without any adjustment), click Done when complete and then select the Photoshop Action that I saved to process all the files and output the results to another folder.
Logged

Gordon Buck

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 458
    • LightDescription
Batch-processing of RAW files
« Reply #11 on: November 15, 2006, 08:56:45 am »

I'm not a pro, just often take many photos.  I always shoot in RAW.

Like b2martin, I put all the RAW images into a single folder.  I use the Breeze Systems DownloaderPro and give each file a meaningful name with a serialized number.  With a minimum amount of processing in Adobe Bridge (often none), I then run an Action on each image through the CS2 batch processor.  I start the batch process, then go to bed.  

I call these actions "HQ JPG" because, in my mind, I'm making a High Quality JPG image file from the RAW.  I append "HQJPG" to the file name.

The Action (there are several variations) usually apply one or more of the PK Sharpener procedures, sometimes auto levels, Neat Image noise reduction if high ISO, sometimes increase saturation, etc., etc.  I take a guess at getting most of the images about right and let it run.  Sometimes I run a different variation later and then select the images I like best.

All the above is based on the idea that CS2 and plug-ins can do a better job at making a high quality jpg from RAW than could be done in-camera.  That may not be always true but it seems reasonable to me.

Thereafter, I may run another batch/action to generate 4x6 prints, images for a slide show, etc.

In the process of reviewing the images, I find the ones I like best and work those by hand from scratch.
Logged
Gordon
 [url=http://lightdescription.blog

PeterLange

  • Guest
Batch-processing of RAW files
« Reply #12 on: November 15, 2006, 03:09:47 pm »

Quote
...  With a minimum amount of processing in Adobe Bridge (often none), I then run an Action on each image through the CS2 batch processor.  I start the batch process, then go to bed. 

I call these actions "HQ JPG" because, in my mind, I'm making a High Quality JPG image file from the RAW.  I append "HQJPG" to the file name.

The Action (there are several variations) usually apply one or more of the PK Sharpener procedures, sometimes auto levels, Neat Image noise reduction if high ISO, sometimes increase saturation, etc., etc.  I take a guess at getting most of the images about right and let it run.  Sometimes I run a different variation later and then select the images I like best.
Very interesting.  Could you detail these ’fix’ settings in Bridge/ACR as well as the steps of your favorite ’HQJPG’ Action a little bit. Thanks!

Peter

--
Logged

Gordon Buck

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 458
    • LightDescription
Batch-processing of RAW files
« Reply #13 on: November 15, 2006, 10:11:22 pm »

Quote
Could you detail these ’fix’ settings in Bridge/ACR as well as the steps of your favorite ’HQJPG’ Action a little bit. Thanks!


Eventually, I was forced to realize that, the vast majority of the time, my manual processing of RAW files was essentially always the same.  When my wife returned from a vacation with almost 1000 RAW files and wanted prints of all the good ones (my interpretation:  every shot), I knew my processing methods had to change!

The first step is to accept that jpg files can produce good prints, screen images,  slide shows, etc.  It seemed reasonable to me that Photoshop could make a jpg of better quality than could the in-camera processor.  (I don't really know if that is true, but I prefer to think that it is.)  Even if the PS jpg was equal or slightly lower in quality as compared to the in-camera jpg, the PS jpg would be *my* interpretation.

So I learned how to write a simple Action in Photoshop CS2 that followed my typical processing steps.

My normal steps for processing RAW files include (usually in this order)

   - ACR settings to taste (use auto and defaults in the Action file)
   - levels  (set to auto in the Action)
   - noise reduction (set to auto profile at 50% in the Action) (I use Neat Image)
   - sharpen for the capture device (I use PK Sharpen)
   - local contrast sharpening (USM, 20, 50,0)
   - edge sharpening (PK Sharpen, medium setting)
   - Shadow and highlight reduction (CS2 settings of 5% and 15%)
   - Color saturation (5%)
   - Color saturation (20%)
   - strong color saturation using Velvia Vision (Fred Miranda Action)
   - convert to 8 bit mode
   - save as jpg at CS2 compression setting of 10

Don't do all the above every time!  Each step in a Photoshop Action can be turned off by unchecking the box on the left hand side by that particular step.  I've found it easier to have many "optional" steps in the Action and include or exclude each step according to the image (or how I feel/guess about it that day).

In the PS batch processor, I append "HQJPG" to each file name so that I know how it was processed.  Batched images are saved in a separate, new folder:  "HQJPG Images" under the main folder for those RAW images.

Actually, there is another step in the Action that saves the processed file as a 16bit Photoshop (.psd) file just in case I want to hack at it again.  Usually, this psd step is unchecked when processing many files but might be used for processing only one or two files that are to be manually tweaked.

In ACR, although I usually accept the camera auto white balance, sometimes I set the white balance to be the same for all images.  Most other settings are the auto default.  If I take the time to browse through the files and adjust exposure, contrast, etc then I uncheck the auto levels in the Action file.

According to the PK Sharpen scheme of sharpening (guess I should add "As I understand it"), images should be sharpened for the capture device, for creativity and for the output device.  I like this scheme - makes sense to me.  Notice that the above Action does not sharpen for the output device.  I have separate Actions for processing the HQJPG files according to output:  8x10, 4x6, slide show, screen size, emails, etc.

As examples, if the majority of the images are somewhat scenic landscape oriented, then the 5% and 20% color saturation might be unchecked and the Velvia Vision be checked.  If most of the images are closeups of "women of a certain age" (family reunion), then none of the sharpening steps and no color saturation steps are checked.

Sometimes I just run variations on the Action and use Breeze Browzer to pick the images I like best.

I often do slight retouching on an individual HQJPG file and then save it as (gasp, heresy!) another jpg file.

I don't claim the above method to be the best; in fact, surely there are better ways to process many files.  But this seems to work for me.

If it takes 1 minute to Action process each file, then 600 files take 10 hours!  Of course, I'm asleep.
Logged
Gordon
 [url=http://lightdescription.blog

PeterLange

  • Guest
Batch-processing of RAW files
« Reply #14 on: November 16, 2006, 05:09:23 pm »

Quote
Eventually, I was forced to realize that, the vast majority of the time, my manual processing of RAW files was essentially always the same.  When my wife returned from a vacation with almost 1000 RAW files and wanted prints of all the good ones (my interpretation:  every shot), I knew my processing methods had to change!

The first step is to accept that jpg files can produce good prints, screen images,  slide shows, etc.  It seemed reasonable to me that Photoshop could make a jpg of better quality than could the in-camera processor.  (I don't really know if that is true, but I prefer to think that it is.)  Even if the PS jpg was equal or slightly lower in quality as compared to the in-camera jpg, the PS jpg would be *my* interpretation.

So I learned how to write a simple Action in Photoshop CS2 that followed my typical processing steps.

My normal steps for processing RAW files include (usually in this order)

   - ACR settings to taste (use auto and defaults in the Action file)
   - levels  (set to auto in the Action)
   - noise reduction (set to auto profile at 50% in the Action) (I use Neat Image)
   - sharpen for the capture device (I use PK Sharpen)
   - local contrast sharpening (USM, 20, 50,0)
   - edge sharpening (PK Sharpen, medium setting)
   - Shadow and highlight reduction (CS2 settings of 5% and 15%)
   - Color saturation (5%)
   - Color saturation (20%)
   - strong color saturation using Velvia Vision (Fred Miranda Action)
   - convert to 8 bit mode
   - save as jpg at CS2 compression setting of 10

Don't do all the above every time!  Each step in a Photoshop Action can be turned off by unchecking the box on the left hand side by that particular step.  I've found it easier to have many "optional" steps in the Action and include or exclude each step according to the image (or how I feel/guess about it that day).

In the PS batch processor, I append "HQJPG" to each file name so that I know how it was processed.  Batched images are saved in a separate, new folder:  "HQJPG Images" under the main folder for those RAW images.

Actually, there is another step in the Action that saves the processed file as a 16bit Photoshop (.psd) file just in case I want to hack at it again.  Usually, this psd step is unchecked when processing many files but might be used for processing only one or two files that are to be manually tweaked.

In ACR, although I usually accept the camera auto white balance, sometimes I set the white balance to be the same for all images.  Most other settings are the auto default.  If I take the time to browse through the files and adjust exposure, contrast, etc then I uncheck the auto levels in the Action file.

According to the PK Sharpen scheme of sharpening (guess I should add "As I understand it"), images should be sharpened for the capture device, for creativity and for the output device.  I like this scheme - makes sense to me.  Notice that the above Action does not sharpen for the output device.  I have separate Actions for processing the HQJPG files according to output:  8x10, 4x6, slide show, screen size, emails, etc.

As examples, if the majority of the images are somewhat scenic landscape oriented, then the 5% and 20% color saturation might be unchecked and the Velvia Vision be checked.  If most of the images are closeups of "women of a certain age" (family reunion), then none of the sharpening steps and no color saturation steps are checked.

Sometimes I just run variations on the Action and use Breeze Browzer to pick the images I like best.

I often do slight retouching on an individual HQJPG file and then save it as (gasp, heresy!) another jpg file.

I don't claim the above method to be the best; in fact, surely there are better ways to process many files.  But this seems to work for me.

If it takes 1 minute to Action process each file, then 600 files take 10 hours!  Of course, I'm asleep.
It looks like we’re barking up the same tree, so to speak.

There are basically three functions which I’ve outsourced from ACR to an Action in order to facilitate such batch processing to something like HQ-JPGs:  First, sharpening - via a two pass approach and mainly based on procedures as described in literature e.g. by Ron Bigelow.  Second, the main tone curve (from brightness & contrast) - which now gets routinely equipped with a contrast mask to protect the highlights.  And third, some color specific ’perceptual’ tweaks: for example to prevent yellow-green hues from getting to bright, hot, dominant (as I see it with fresh plant green), or, to strengthen saturation with a weighting on blue hues (sky, water)…

Along this line I’ve given up to save multi-MB psd files.  If I wish re-edit a file, the options are either to start ab-initio in ACR and/or to launch said Action again on a single or few selected files – now without final saving steps (merge layers, goto 8 bit, …) so that the complete layer structure which was created as an intermediate becomes accessible for changes.

Sure there’s no one-size-fits-all solution, but for my purposes the results are encouraging enough to continue to elaborate this approach a little bit further … though being very open for alternatives such as suggested along this thread.

Peter

--
Logged
Pages: [1]   Go Up