Luminous Landscape Forum
Raw & Post Processing, Printing => Adobe Lightroom Q&A => Topic started by: mattmikulla on December 01, 2013, 07:13:11 pm
-
Hello all. I am working on my DAM workflow and wanted to poll you folks and maybe get some advice.
I use Lightroom for importing, editing, organization and printing. I shoot primarily RAW and convert to DNG. Occasionally I will open up a DNG as a TIFF or PSD in Photoshop for additional editing or tools like NIK silver effex or color effex.
Where should I store my PSD and TIFF files I create from the original DNG within Lightroom?
1. In another folder such as one labeled "Derivative" as recommended in The DAM Book?
2. Right next to the original DNG file for easy comparison and location?
Ideas, recommendations and advice appreciated.
-
Hi Matt.
My personal preference would be to have the the TIFF or PSD derivative right there with the original (DNG in your case).
In Lightroom if a requirement for a "separate folder" for these derivatives arises then simply create an appropriate Smart Collection that will house those derivatives.
In suggesting this arrangement I have made certain assumptions about your workflow that may not be valid.
Certain individuals only make derivatives for use external to Lightroom and do not then have Lightroom manage those derivatives.
Tony Jay
-
Key question is what you define as derivative. I too work in raw and now and then i need to take the image to photoshop to develop any further. In that case the tif or psd is an original or master, and stays in the folder where the raw file resides. If i derive an image to send to someone, to use for web publication etc, i put these in derivative folder and subfolders.
-
Key question is what you define as derivative. I too work in raw and now and then i need to take the image to photoshop to develop any further. In that case the tif or psd is an original or master, and stays in the folder where the raw file resides. If i derive an image to send to someone, to use for web publication etc, i put these in derivative folder and subfolders.
Exactly right!
Tony Jay
-
With Lightroom, it may make more sense, at least if you don't see the DNG as master reference anymore, to have the TIFF stacked on top of the original DNG.
That asks for the 2 files to be in the same folder.
-
Where should I store my PSD and TIFF files I create from the original DNG within Lightroom?
1. In another folder such as one labeled "Derivative" as recommended in The DAM Book?
2. Right next to the original DNG file for easy comparison and location?
A big part of the logic behind The DAM Book's recommendations is that the need to backup & restore is what should drive folder structure, because metadata and virtual containers like collections are what you use to find and categorise. So you adopt a "bucket" style of folder structure for your originals, building folder structures to match the capacity of write-once media such as DVDs or blu-ray. Subsequently adding new derivatives into the buckets would break the correspondence between your hard drive and these write once media, and that's why you then need a separate series of buckets for the derivatives.
A classic bucket system with separated derivatives doesn't mean you don't have easy comparison and location (do you mean finding them?). You no longer think of your folders as a way to find or categorise pictures, and the time that you'd waste doing so goes into adding more keywords and setting up smart collections.
John
-
PS don't forget you can now stack in (not smart) collections.
-
I have never seen the point of converting to DNG but, with that caveat, I allow Lightroom to keep any Tiffs that are created (e.g. when editing from within LR in CS6 or any of the Nik programs) adjacent to the original Raws in the Library module.
Reason? Sometimes, if revisiting an image for a new or different purpose, I may sometimes want to use the Raw as a starting point and may sometimes want to use the processed Tiff. By having them together, I can make an instant comparison and then work on a virtual copy of whichever I select.
-
By having them in a smart collection, "I can make an instant comparison and then work on a virtual copy of whichever I select". I can also see previous treatments of the same subject.
-
One of the great benefits of LR for me is that I no longer have to bother with separate folders for different file types from a shoot. They all go in the same folder and if I need to separate them out, I simply filter them and job done.
Collections, smart or otherwise are in my opinion the wrong way to go for that sort of separation.
-
In addition to the above comment i made and in line with other comments. If the image file stays under control of Lightroom, there is little reason to separate the storage into several different folders (The bucket approach described in The DAM book). However if the derived image(s) (f.i. an export to JPG etc) is(are) not under control of LR (not added to the catalog), then i use a folder for derived results. As i work per assignment or project if you will, my basic folder structure per assignment is: <Assignment>-ADMIN, <Assignment-EDIT>, <Assignment>-DERIVED
The ADMIN folder holds any assignment administration related files, like model-agreements, moodboard/soryboard, etc; the EDIT folder all the image files imported from cameras and any files created when going to Photoshop (LR stacks these by default), the DERIVED folder any derivatives not under control of LR, f.i. created with export command and not loaded into the catalog.
I use keywords extensively, and smart collections in combination with regular collections to manage my image development work the selection by customer and output per assignment.
-
write-once media such as DVDs or blu-ray.
does somebody really backup to those nowadays ?
-
does somebody really backup to those nowadays ?
Certainly.
-
I think that the secret of any effective back-up system is physical separation so that, if there is a house fire or similar calamity, the back-up survives.
I am afraid that I fail in terms of that criterion. My back-ups will protect against HDD failure but not against total destruction.
-
I don't think there's a single secret, though separation is one. The one I think is easiest to overlook is the ability to restore - we often know how to backup, but how familiar are we with restoring and reconstructing after a catastrophe? The beauty of the use of write-once media is there's no chance of overwriting the backup copies, and the "bucket" scheme is so low tech that you don't depend on knowing your backup software's restore features or even have it available (you could easily restore to a different operating system).
-
In addition to the above comment i made and in line with other comments. If the image file stays under control of Lightroom, there is little reason to separate the storage into several different folders (The bucket approach described in The DAM book). However if the derived image(s) (f.i. an export to JPG etc) is(are) not under control of LR (not added to the catalog), then i use a folder for derived results. As i work per assignment or project if you will, my basic folder structure per assignment is: <Assignment>-ADMIN, <Assignment-EDIT>, <Assignment>-DERIVED
The ADMIN folder holds any assignment administration related files, like model-agreements, moodboard/soryboard, etc; the EDIT folder all the image files imported from cameras and any files created when going to Photoshop (LR stacks these by default), the DERIVED folder any derivatives not under control of LR, f.i. created with export command and not loaded into the catalog.
Exported files from LR can be added to catalogue as they are exported, there's a checkbox in first part of Export dialogue to add/not add. Or is it that you choose not to add those exports? I would tend to add them to catalogue, to keep a check on what I've done so far from within LR.
With regard to your admin folder, this is where LR's limited file import drives me nuts. Files that are not photographs can also be a very useful part also a useful part of a photographic workflow. I don't care if LR cannot edit these files, but it would be handy to import information that for example may be text based such as everyone's names or titles which can then be used as keywords.
-
I think that the secret of any effective back-up system is physical separation so that, if there is a house fire or similar calamity, the back-up survives.
I am afraid that I fail in terms of that criterion. My back-ups will protect against HDD failure but not against total destruction.
A friend has a garage at end of his garden that he has converted into his office, so he has a physically separated area for his computers/back up, but all connected by a fast LAN network. I've been toying with a similar idea.
Having a friend/neighbour that you can leave stored data with is another option.
-
I don't think there's a single secret, though separation is one. The one I think is easiest to overlook is the ability to restore - we often know how to backup, but how familiar are we with restoring and reconstructing after a catastrophe? The beauty of the use of write-once media is there's no chance of overwriting the backup copies, and the "bucket" scheme is so low tech that you don't depend on knowing your backup software's restore features or even have it available (you could easily restore to a different operating system).
Sadly, low tech can also end up being out of date tech. And if you have many TBs of data…..
The big problem with write once media is when do you do the back up. As a lot of one's work takes place after the photo is taken and not necessarily immediately after. And work done on photographs can be as valuable as the shot itself.
-
Sure, I wonder why the high volume shooters I know are still using DVD rather than blu-ray, but they do so. After all, high tech methods have a nasty way of going out of date (or never catching on) before more primitive methods, and even Apple aren't so stupid that they no longer make DVD readers.
-
Apple don't make any computers with a DVD player these days and Blu-ray was never even an option on Macs. Hard drives tend to be seen as an easier and much cheaper option for back up than optical media. Which is different to archive, which is where Blu-Ray should be better, but how long will it even be around for? Tangible media is being got rid of as fast as possible it seems like with tech companies putting everything online, as nothing can go wrong there.
-
No, nothing can go wrong there, can it?
-
There is a segment in the "Where the *** are my pictures" video by Seth Resnick and Michael that addresses the backups issues. Worth listening to. I pretty much use that setup, except that I do not have a Drobo system as a final backup. These backup strategies work fine in the short term and medium term but they are all technology-dependent and doomed to obsolescence. An article by Joseph E. LaBarca, Pixel Preservation International, identifies a number of digital preservation issues. These include: media formats, file formats, data integrity, and technology advances. I have a box of obsolete Syquest cartridges that are useless since the drive died, and, even if i found a working one, it needs a SCSI connection, and who knows if my current OS would run the drivers. For true long-term preservation we need a technology-independent system. A human-readable system such as a family album or shoe-box of photos will outlast any machine-readable system. So, if it is important, print it.
As for the OP's question, I keep all my files together, same name except for the extension. With LR, there is little need to export files into TIF's or PSD's unless you need to do additional editing in PS.
Jean-Michel
-
Having a friend/neighbour that you can leave stored data with is another option.
http://crashplan.com has a software that does offline (full/incremental) backup (encrypted if you want to) to your family/friends/whatever free (unlike paid plan to backup to their own data storage)... and you do not need to worry about catalog structure on some obsolete DVDs or BluRays...
-
Sure, I wonder why the high volume shooters I know are still using DVD rather than blu-ray, but they do so. After all, high tech methods have a nasty way of going out of date (or never catching on) before more primitive methods, and even Apple aren't so stupid that they no longer make DVD readers.
blue rays are still low capacity hassle... HDDs and offline clouds (you can backup to 2 different providers to be even more safe).
-
No, nothing can go wrong there, can it?
sure, but nobody is forcing you to use just one cloud back service... use 2-3 different at once.
-
Why stick to just two or three? There's a whole industry that needs our support.
-
Cloud. Eventually, they evaporate, sometimes rather suddenly and violently.
Jean-Michel
-
Why stick to just two or three? There's a whole industry that needs our support.
no need to be sour... it is working and cheap, plus - you can backup to your own location(s) elsewhere, to your relatives, etc for free... way better than to juggle with plastic discs
-
Yawn.
-
For me, any file that might have further editing applied is kept in Lightroom. Lightroom makes this easy with the Edit In Photoshop function, since those images are automatically put into the catalog and in the same folder as the originals. Images that are exported in final form for specific purposes, such as web, print publication, etc., no matter if they are JPEG, TIFF or other, are kept in separate "Export" folder structure.
-
I would never allow Lightroom or any one type of software "manage" my file structure. I think JohnBeardy's bucket system if I understand it is what I do. Make a file with client name, then project based folders. I then have the RAW/Finals(usually flat TIF's/ JPEGS. Simple as that. If you want to use Bridge, CaptureOne, Acdsee, or Windows Explorer or Finder, things are at the OS level searchable. It is a marketing talk for the sheep to be guided into a proprietary means of structuring your files. :-P
Yes the catalog gives you an overlay of functions to search your images. Can be very powerful. But not seeing the folder level content without being part of the catalog is very inconvenient.
On the backup note, while I often do dump files ontot a DVD for a project, HArddrives are so cheap, you just fill it up alphabetically(a couple year or older files) and disconnect and secure it someplace. Off site preferred.
Forget clouds. If you LOVE the idea, make your own to a remote computer YOU manage. end of story.
This thread should be in the DAM catagory under Equipment and Techniques
-
I would never allow Lightroom or any one type of software "manage" my file structure ... It is a marketing talk for the sheep to be guided into a proprietary means of structuring your files.
I have to strongly disagree with these statements.
There is NOTHING proprietary in the way Lightroom does it file structure.
In fact that structure is entirely within the control of the operator.
One can create, delete, rename, and re-organize folders and files in the same way that the OS does.
The only difference is one does it in Lightroom so that the Lightroom database (read Catalog) can keep track of what is done.
I am not a Lightroom or Adobe apologist but the statements from Phil are just plain wrong.
Tony Jay
-
I have to strongly disagree with these statements.
There is NOTHING proprietary in the way Lightroom does it file structure.
In fact that structure is entirely within the control of the operator.
One can create, delete, rename, and re-organize folders and files in the same way that the OS does.
The only difference is one does it in Lightroom so that the Lightroom database (read Catalog) can keep track of what is done.
I am not a Lightroom or Adobe apologist but the statements from Phil are just plain wrong.
Tony Jay
Absolutely agree with Tony Jay.
My file naming and folder structure is unique (sort of, similar to many others) and "tuned" to what I like and the way I think or remember when/where things were done.
While I use one catalog, the file structure is not contained in one place. For example, I have one drive with current pictures and another with older, archived pictures. I include the 'iCloud Photo' folder (imported automatically with Jeffrey Friedl's 'Folder Watch'), so have easy access to images taken on any of my iDevices. I have separate access to a folder of greeting card templates and messages I have created. All of these are under the one catalog.
John
-
no need to be sour... it is working and cheap, plus - you can backup to your own location(s) elsewhere, to your relatives, etc for free... way better than to juggle with plastic discs
It's never been workable as far as my or many professional's needs go. The amount of data I accumulate always tends to outpace the ability to upload and that doesn't take into account my archive. They are fine for people with not much data, but once you get into multi TB amounts……
Oh and I have the fastest connection you can get in the UK [and have had it for many years now] and upload speeds are still pathetic. A colleague can upload faster via his phone it seems!
One company did offer to send you a drive to seed your back up to get it started, but only in the US and the drive was way too small anyway.
-
I would never allow Lightroom or any one type of software "manage" my file structure. I think JohnBeardy's bucket system if I understand it is what I do. Make a file with client name, then project based folders. I then have the RAW/Finals(usually flat TIF's/ JPEGS. Simple as that. If you want to use Bridge, CaptureOne, Acdsee, or Windows Explorer or Finder, things are at the OS level searchable. It is a marketing talk for the sheep to be guided into a proprietary means of structuring your files.
Good grief, people still believe this myth!
Using LR has made zero difference to my file structure setup or ability to access files via OS/Finder/Explorer/Bridge etc. It has however made it much, much easier to find photos.
-
First to the OP's question: I use the "bucket system" per Peter Krogh and any and all files associated with a specific project or shoot are dumped into that one bucket. Since I'm doing interiors/architecture, every shoot I do has a mailing address, so that's how I name the folder....and there are no sub-folders.
Can't help chiming in on the backup discussion. I think using DVD or other removable media is a bit like converting your music collection over to 8-track tapes. I'm down to a single device that even has a cd/dvd drive, and it's arguably past the end of it's useful service life. I'll be replacing it this winter and cd/dvd will no longer exist in my studio. I can understand not rushing to adopt the "bleeding-edge" technology, but using 20+ year-old technology seems equally silly.
As I import images into my archive, they're automatically backed up to 2 separate drives in addition to the "working" drive. As the backup drives fill up (I use 500GB externals), one goes into a desk drawer, and the other is shipped to a friend a couple of states away. She's got a Pelican case in her furnace closet full of my backup HDs. My Lightroom catalog still knows which files are where, so if something were to disappear off of my working drive, I'd know which external drive to look on. If my desktop computer melts or gets infected or whatever, I've got a desk drawer with everything, and if my house burns down, it's (almost) all in my friends closet, a FedEx box away.
-
It's not at all like 8 track tapes - you can write over them. The point of DVD etc is to use them as write-once media.
-
My Lightroom catalog still knows which files are where, so if something were to disappear off of my working drive, I'd know which external drive to look on
What do you do when your not using Lightroom any longer? Lets say 5 or so years from now ther is another cataloging standard? I know of a few applications that can replace LR, and the only reason I use it in conjunction with other DAMS is due to its Raw developer. But I also use other developers that have better quality is certain files and subject matters such as C1 on some files.
For me it would be short sighted to use LR as a organizer/Catalog as it doesn't openly allow other DAM's to replicate its database structure/conventions. So why would I not manage it myself? I also find the folder structure (older version of his book/bucket system) to be fast and most importantly not dependent on LR. If you already have a folder structure that is even just OK to manage. Stick with it , rather than try and manage with LR.
If you don't and you think you will die before LR is replaced, and your after life legacy manager will also be using LR, then go ahead and lock yourself in, and manage with LR, and its multiple collections and its sluggish workflow concept of Library mode vs Develop mode. Its a bit dinosaur like approach, but I don't see them breaking out of it just yet.
-
What do you do when your not using Lightroom any longer? Lets say 5 or so years from now ther is another cataloging standard? I know of a few applications that can replace LR, and the only reason I use it in conjunction with other DAMS is due to its Raw developer. But I also use other developers that have better quality is certain files and subject matters such as C1 on some files.
For me it would be short sighted to use LR as a organizer/Catalog as it doesn't openly allow other DAM's to replicate its database structure/conventions. So why would I not manage it myself? I also find the folder structure (older version of his book/bucket system) to be fast and most importantly not dependent on LR. If you already have a folder structure that is even just OK to manage. Stick with it , rather than try and manage with LR.
Do you even know how to use LR? LR does not, as mentioned several times above organise your work, you do. I am using the identical folder structure within LR that I used before LR even existed. Plus LR can import images into that structure quicker for me than when I had to do it manually. Any other DAM programme can parse my basic file structure, with no issues. If you use smart collections you can easily replicate them in other software that also does smart collections. And if the software does not do smart collections then frankly it's a rather useless.
If you don't and you think you will die before LR is replaced, and your after life legacy manager will also be using LR, then go ahead and lock yourself in, and manage with LR, and its multiple collections and its sluggish workflow concept of Library mode vs Develop mode. Its a bit dinosaur like approach, but I don't see them breaking out of it just yet.
If by dinosaur you meant old and out of date, then separating your cataloguing from your working on your images is the antiquated and sluggish method of working. With LR, there's no pointless exporting of one's organised work just so another programme can then utilise images and then having to re-save work, thus ending up with unnecessary copies and duplicates.
-
For me it would be short sighted to use LR as a organizer/Catalog as it doesn't openly allow other DAM's to replicate its database structure/conventions.
Hi Phil, while that statement is true in the strict sense, LR does use an open source SQL database widely used by application developers so like any SQL database it's not very difficult to write DDL script or in the case of someone without that experience to use a Database administrator application (many are opensource also) to export table data as text which can be imported into another applications' database.
Brian,
-
On reading the rest of Phil's post, he seems to be confusing the organisation of the physical folder structure and the database functionality of LR and not really getting how it works.
Which is more an issue with things like iMovie/Aperture where their database physically places images in folder structures that are of no use to humans or any other software.
-
On reading the rest of Phil's post, he seems to be confusing the organisation of the physical folder structure and the database functionality of LR and not really getting how it works.
Which is more an issue with things like iMovie/Aperture where their database physically places images in folder structures that are of no use to humans or any other software.
Absolutely no doubt.
Tony Jay