Pages: [1]   Go Down

Author Topic: Archive Strategy for MFDB Files (Sinar Users)  (Read 6059 times)

Murray Fredericks

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 295
    • http://www.murrayfredericks.com
Archive Strategy for MFDB Files (Sinar Users)
« on: January 30, 2008, 06:51:43 am »

I'm interested to hear how other Sinar users who are using the Brumbaer workflow are archiving their files.

Since upgrading to a MFDB 6 months ago, I have just purchased my 4th 1 terrabyte drive. That's a lot of space and expense...

Right now I am keeping my Raw files, the DNG conversions (which is equivalent to the Raw files as I convert the lot automatically in Brumbaer) and the final files which have been worked on and delivered to the client.

If time permitted I would be more selective about which files I keep from the shoot, but at this stage I guess I would like to dump either the raws or the DNGs. I re-name the DNGs so they match the files that are delivered to the client, so It would be better to dump the Raws.

I feel uneasy about keeping only the DNG's though, because if I need to go back............

Murray
Logged
Exhibition Website   http://www.murrayfr

Graham Mitchell

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2281
Archive Strategy for MFDB Files (Sinar Users)
« Reply #1 on: January 30, 2008, 07:08:02 am »

I wouldn't keep your work on hard drives.

As for the other issue, DNG is as safe a bet as any. Sinar will be switching over to DNG as the standard format with the Exposure software anyway.
Logged

Murray Fredericks

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 295
    • http://www.murrayfredericks.com
Archive Strategy for MFDB Files (Sinar Users)
« Reply #2 on: January 30, 2008, 07:11:08 am »

Quote from: foto-z,Jan 30 2008, 11:08 PM
I wouldn't keep your work on hard drives.

The hard drives are raid mirrored...
Logged
Exhibition Website   http://www.murrayfr

MichaelEzra

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1146
    • https://www.michaelezra.com
Archive Strategy for MFDB Files (Sinar Users)
« Reply #3 on: January 30, 2008, 07:26:21 am »

Here is what I do: my data drive is raid 10. It can survive a simultaneous crash of 2 drives. I have a full backup done once a week and a monthly backup to off-site storage. All raw files from either cameras or a film scanner are backed up to archival DVDs. The most valuable advise is to delete everything you don't really need.
Logged

samuel_js

  • Guest
Archive Strategy for MFDB Files (Sinar Users)
« Reply #4 on: January 30, 2008, 07:43:38 am »

Quote
I'm interested to hear how other Sinar users who are using the Brumbaer workflow are archiving their files.

Since upgrading to a MFDB 6 months ago, I have just purchased my 4th 1 terrabyte drive. That's a lot of space and expense...

Right now I am keeping my Raw files, the DNG conversions (which is equivalent to the Raw files as I convert the lot automatically in Brumbaer) and the final files which have been worked on and delivered to the client.

If time permitted I would be more selective about which files I keep from the shoot, but at this stage I guess I would like to dump either the raws or the DNGs. I re-name the DNGs so they match the files that are delivered to the client, so It would be better to dump the Raws.

I feel uneasy about keeping only the DNG's though, because if I need to go back............

Murray
[a href=\"index.php?act=findpost&pid=170919\"][{POST_SNAPBACK}][/a]

Hi,
I think almost 1 terrabyte per month is a huge amount of storage...
Of course your first step should be to keep just one kind of raw files. I'd keep the original raws if the new software has support of them in the future. If not, I'd keep the DNG's.
The other thing is to burn past projects to DVD, and keep in hardrives recent or actual projects only, this is a question of space too. Then use a CD/DVD library program to classify the DVDs. These programs uses metadata or file/folder names to find files, so it's very easy to find the pictures needed. Then you could use Lightroom to import previews from the DVDs, to be able to see all the pictures in the DVDs...
Logged

David WM

  • Full Member
  • ***
  • Offline Offline
  • Posts: 241
    • http://
Archive Strategy for MFDB Files (Sinar Users)
« Reply #5 on: January 30, 2008, 08:17:26 am »

I store on an internal drive and an external drive backup until I have the chance to cull and archive to DVD (x2).  It could be a good system if DVDs weren't so small.
David
Logged

Graham Mitchell

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2281
Archive Strategy for MFDB Files (Sinar Users)
« Reply #6 on: January 30, 2008, 10:42:19 am »

Quote
The hard drives are raid mirrored...
[a href=\"index.php?act=findpost&pid=170925\"][{POST_SNAPBACK}][/a]

Still, this doesn't seem to be the best way. The moment a drive fails, you have a lot of data resting only on one drive (which may be quite old by this stage). There may be worse consequences if the RAID controller itself fails. This solution also offers no protection against fire/theft as all the data is in one location.
Logged

Morgan_Moore

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2356
    • sammorganmoore.com
Archive Strategy for MFDB Files (Sinar Users)
« Reply #7 on: January 30, 2008, 11:15:13 am »

Quote
I'm interested to hear how other Sinar users who are using the Brumbaer workflow are archiving their files.

[a href=\"index.php?act=findpost&pid=170919\"][{POST_SNAPBACK}][/a]

THis is what I do, not recomend

DL onto the machine

Create DNGs

(If I like the shoot I keep the STIs but generally I delete them)

Delete the STIs

Keep those Dngs on a pair of HDs , one in the office one offsite

(during the period that they have not yet reached the offsite, monthly, I keep them on my computer too)

I just use cheapo 400gb drives and hope they both dont fail at the same time

Everyything is kept in a date folder and I record what I did when on a database so I can always find the DNGs by knowing the date shot and I have stickers on the HDs saying things like jan08 to march08

Simple but good I hope

Remember if you use raid and accidentally press delete you do it in stereo - offline storage is safer IMO as is offsite storage

SMM

further notes

-I dont bother with DVDs - way too slow
-It is important to clarify with clients the score either delete after a period of time or charge for not doing so
-My T+C promise to deliver the data once unless otherwise agreed ie keeping me out of a situation where I am oblidged to store 'thier' data
« Last Edit: January 30, 2008, 11:22:39 am by Morgan_Moore »
Logged
Sam Morgan Moore Bristol UK

MichaelEzra

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1146
    • https://www.michaelezra.com
Archive Strategy for MFDB Files (Sinar Users)
« Reply #8 on: January 30, 2008, 11:40:53 am »

I am writing a little program which will allow to catalogue archives and quickly find anything that was archived. Once complete I will post an update to this forum.
If there are any particular features anyone is interested in, feel free to post;)
Logged

Murray Fredericks

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 295
    • http://www.murrayfredericks.com
Archive Strategy for MFDB Files (Sinar Users)
« Reply #9 on: January 30, 2008, 04:54:36 pm »

Delete the STIs


This is what I was wondering...


Keep those Dngs on a pair of HDs , one in the office one offsite


What method of offsite storage do you use?


further notes

-I dont bother with DVDs - way too slow

I agree!!


-It is important to clarify with clients the score either delete after a period of time or charge for not doing so
-My T+C promise to deliver the data once unless otherwise agreed ie keeping me out of a situation where I am oblidged to store 'thier' data


I do this too - it's a good policy

Murray
Logged
Exhibition Website   http://www.murrayfr

bradleygibson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 828
    • http://GibsonPhotographic.com
Archive Strategy for MFDB Files (Sinar Users)
« Reply #10 on: January 30, 2008, 05:37:01 pm »

I get asked this question just about every week.

As MichaelEzra and foto-z point out, a single failure on a RAID-1 array means you are one failure away from losing everything you've stored there.  For those who think that double-drive failures aren't really something to worry about, I have some anecdotal experience to share:

Quote
This morning at about 6 AM GMT we had a failure in the RAID array of our forums database server. Our RAID-5 array would have been able to withstand losing one disk but apparently two died at the same time...

January 18th, 2008
Phil Askey
Editor, DPReview.com

(RAID-5, like RAID-1 can only withstand a single drive failure).

Any robust backup strategy requires a minimum of two backups--one local, and one remote.

Local Backup should protect against hard drive failure or even accidental/malicious deletion, depending on your situation.  (Do you have staff to worry about malice, and are you comfortable relying on the backups themselves to recover from accidents?)

My photography is very important to me, so I recommend doubly-redundant RAID.  There are only two common ways to get doubly redundant RAID: RAID-6 or RAID-50.  Either of these schemes will allow any two drives to fail and still maintain integrity.

I recommend an automatic backup system which can grow with your needs--typically that's RAID, but if everything you do will fit on a disk, external backup is also OK.  I am lazy.  I'll always be behind if I have to go back up stuff to CD or DVD.  Will the discs even work when it comes time to read them?  Just keep in mind that manual backup is very time consuming.

MichaelEzra, you should know that you are not fully doubly protected with your RAID-10 array.  If two drives from the same stripe set go down, it is true that you'll be safe, but if one drive from each stripe set goes down, you will lose all your data.  

So now that you have at least one backup copy of your data locally, what do you do in case of fire, flood, burglary, tornado, earthquake, sprinkler system, etc...?

Remote Backup keeps a copy of all your stuff on another PC somewhere far away from your main work.  Running across town to your brother's place with a CD or DVD or hard disk of new stuff is also a pain.  What's the solution?  Using a geeky tool like rsync works very well, but there's a friendlier way.  A service which keeps all your copies of all your files in sync all the time, automatically.

I'm sure there are others out there (and full disclosure, I also work for Microsoft), but consider the free service called FolderShare - http://www.FolderShare.com.  It will keep 2 (or more) computers in sync with one another over your network or over the internet.  Works on both Mac and PC.  It's a nice solution for lazy folks such as myself.

The only caveat is that there is a pretty serious limitation at the moment of 10 libraries (or top-level folders) with 10,000 files per library.  Perhaps there are folks aware of other services which do the same thing without these limitations?

Murray, I can't help you much regarding keeping your originals vs. your .DNG's, because I have not used the Brumbaer software.  I know that Adobe's DNG converter allows you to embed the original file in the DNG, to be safe, and that is a nice feature.  But I don't think Adobe supports Sinar's file format yet.

Hope that helps,
-Brad
« Last Edit: January 30, 2008, 05:44:16 pm by bradleygibson »
Logged
-Brad
 [url=http://GibsonPhotographic.com

Morgan_Moore

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2356
    • sammorganmoore.com
Archive Strategy for MFDB Files (Sinar Users)
« Reply #11 on: January 31, 2008, 01:04:18 am »

Quote
Delete the STIs

[a href=\"index.php?act=findpost&pid=171083\"][{POST_SNAPBACK}][/a]

NB I only do this after while once it has been seen that the DNGs are not corrupt

(Brums software seems stable and has never corrupted any of my files)

Bradley.

Web backup sounds great but what happens if you dump 10Gig off your camera into a watched folder

Its going at least to eat your bandwidth

I had some auto backup software running that used to reduce my whole network to snailspace for 2 hours every day - and that was backing up in the office

I dont see that creating HD backups is that difficult

Anyway I just take my HDs to a shop I have a contract with and they do it

S
Logged
Sam Morgan Moore Bristol UK

Dinarius

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1212
Archive Strategy for MFDB Files (Sinar Users)
« Reply #12 on: January 31, 2008, 02:40:25 am »

Not a Sinar user, but here's my tuppence worth anyway....

I copy from the flash card onto my desktop computer.

These unprocessed RAW images are kept on both the desktop HD and on an external HD.

I then process the images in ACR4.

After editing the images in ACR4, I create a set of DNGs. These are also stored on the desktop HD and on *two* external HDs.

The TIFFs are stored on the desktop HD and on an external HD. I delete the TIFFs from the desktop after a period of six months.

Most of the time, most of my editing is done in ACR. If a lot of post conversion work is done in CS3, or if I really like the tweaks I've made to an image in CS3, then I have a special folder for these TIFFs and they are not deleted.

In the end, I have three copies of DNGs (one on desktop, two external - all containing the ACR4 edits) and two copies of RAWs (one on desktop and one external - all untouched) and very few memory consuming TIFFs.

D.

ps. I am ruthless about deleting files that I will never convert. They take up a lot of space.
« Last Edit: January 31, 2008, 02:43:08 am by Dinarius »
Logged

Adina

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 70
Archive Strategy for MFDB Files (Sinar Users)
« Reply #13 on: January 31, 2008, 02:55:54 am »

Hi,

my workflow:

CF Card -> Brumbaer -> Bridge (for renaming with EXIF Timestamp) -> LR

in LR is a option in dng settings to use the dng compression (losless),
if you use the "refresh thumbnail and metadata" function

from ~49MB to ~11MB - 30MB ....

(Brumbaer & eXposure use per default the uncomresed dng's)

Greetings
Adina
Logged

rethmeier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 795
    • http://www.willemrethmeier.com
Archive Strategy for MFDB Files (Sinar Users)
« Reply #14 on: January 31, 2008, 04:22:44 am »

It looks like Adina got it sorted!
That will be my workflow,when i finally get my Hy6.
However I will certainly give eXposure a run for it's money.
Cheers,
WR
Logged
Willem Rethmeier
www.willemrethmeier.com

Jens_Langen

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 55
Archive Strategy for MFDB Files (Sinar Users)
« Reply #15 on: January 31, 2008, 05:04:27 pm »

I would like to use the Lightroom library instead of iView (our current solution) to locate images, since we're already using Lightroom and want to simplify workflow.  We burn all our images to sequentially numbered DVDs and then clear them off our drives.  With iView, we just scan the archive DVD and then all the thumbnails are referenced to that particular DVD (eg. DVD 456).  Can Lightroom direct you to your archived images on DVDs?  How to implement that?  It seems that you almost need to set up DVD folders on your hard drives ahead of time, but this prevents you from organizing folders and files optimally.  Has anyone implemented this workflow?
Logged

Adina

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 70
Archive Strategy for MFDB Files (Sinar Users)
« Reply #16 on: February 01, 2008, 03:01:17 am »

Quote
It looks like Adina got it sorted!
That will be my workflow,when i finally get my Hy6.
However I will certainly give eXposure a run for it's money.
Cheers,
WR
[{POST_SNAPBACK}][/a]

I have a very low image count ...
I don't have big archives ...

My hint is to use the dng compression ...

PS: in the current version of exposure i can not see the brumbaer dng's ...

in aperture you can not see the exposure dng's, you must use a patch like
the aperture2275 tool from brumbaer to see them

(in the Raw.plist file you must copy the "SINAR eMotion22" or "SINAR eMotion75" section from the aperture2275 tool and rename to "SINAR AG Body:Rollei 600x",
the part 'Rollei 600x' is the cameratyp you have selected on your back, after that run aperture2275 again because of
-> [a href=\"http://www.macosxhints.com/article.php?story=20071028133430814]http://www.macosxhints.com/article.php?sto...071028133430814[/url]
)

Greetings
Adina
Logged

MichaelEzra

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1146
    • https://www.michaelezra.com
Archive Strategy for MFDB Files (Sinar Users)
« Reply #17 on: February 06, 2008, 10:33:54 am »

I have written a small utility "Smart Disk Catalogue" for creating a comprehensively searchable catalog of on-line and off-line disks (backup DVD-s, etc).
It was written because I needed its functionality in my own workflow of cataloguing backups.

One of the features i found useful for myself is a search of duplicate filenames across all disks.

This program is released as freeware. If used for commercial purpose, voluntary donations are appreciated.

Utility can be downloaded from my website at www.michaelezra.com

The setup program itself a bit old and not too flashy, but this is all I have at my current disposal.




Enjoy!
Logged

MichaelEzra

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1146
    • https://www.michaelezra.com
Archive Strategy for MFDB Files (Sinar Users)
« Reply #18 on: February 09, 2008, 12:38:27 pm »

Any feedback would be appreciated
Logged

EricWHiss

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2639
    • Rolleiflex USA
Archive Strategy for MFDB Files (Sinar Users)
« Reply #19 on: February 10, 2008, 12:05:16 am »

Quote
Any feedback would be appreciated
[a href=\"index.php?act=findpost&pid=173527\"][{POST_SNAPBACK}][/a]

Well I'm not using a PC or I might have given this a go.  I'm suitably impressed that a good photographer can also make good looking software.

Regards,
Eric
Logged
Rolleiflex USA
Pages: [1]   Go Up