Best ROI to speed up workflow?

[BernardLanguillier]

Dear all,

I am starting to plan my next generation workstation/live storage unit.

I was initially looking at the 8 cores Mac Pro, but a casual analysis of my current workflow is showing that I waste most of my time waiting for disk accesses to my RAID5 NAS.

The logical conclusion appears to be that it is probably wiser to invest in a really fast SCSI320 RAID unit (I am looking at G-Technologies right now) together with a 4 core system rather than investing in an 8 core system and a new NAS.

I was wondering whether I was the only one thinking this way.

Regards,
Bernard

[frankric]

Hi Bernard

I'm also starting to think about the next box, though it won't be Mac.

From my understanding you would never fully utilise 8 cores. Even with Fully Buffered DIMMs and the fastest disks it seems to me that the processing power would be grossly underutilised. From what I've been reading, as far as Photoshop is concerned, the limitations these days are in memory bandwidth rather than processor speed.

My present box has a now-very-modest Athlon 4400 dual core and 4gb of RAM running XP x64. Using Photoshop, I don't think I've ever seen the processor go to 100%. I use a WD Raptor dedicated scratch disk - not as fast as a SCSI of course, but not bad. I have often seen the processor go to 100% with Lightroom though.

My own thinking for the next box is to use perhaps a Xeon 5150 and 6-8 gb of RAM (FB DIMMs). Most or all the current boards have 2 processor slots, so I reckon I'd start with one processor, then add a second if it seemed to be needed. I think I'd stick with the Raptor scratch disk in the expectation (or hope!) that the extra RAM above 4gb would take some of the load off the scratch disk.

All FWIW, but I agree that for still photography the money would be better spent on things other than 8 cores of processor.

Regards

Frank

[rdonson]

Dear all,

I am starting to plan my next generation workstation/live storage unit.

I was initially looking at the 8 cores Mac Pro, but a casual analysis of my current workflow is showing that I waste most of my time waiting for disk accesses to my RAID5 NAS.

The logical conclusion appears to be that it is probably wiser to invest in a really fast SCSI320 RAID unit (I am looking at G-Technologies right now) together with a 4 core system rather than investing in an 8 core system and a new NAS.

I was wondering whether I was the only one thinking this way.

Regards,
Bernard
[a href=\"index.php?act=findpost&pid=112492\"][{POST_SNAPBACK}][/a]

What is your network speed to the NAS?  Perhaps its time to invest in gigE.

[BernardLanguillier]

What is your network speed to the NAS?  Perhaps its time to invest in gigE.
[a href=\"index.php?act=findpost&pid=112723\"][{POST_SNAPBACK}][/a]

It has of course been giga-byte ethernet from day one.

Still, all actual test show a speed that hardly ever goes beyond 25 MB/s read/write on most NAS. My experience matches these results.

Cheers,
Bernard

[John.Murray]

I would agree that storage is probably the biggest bottleneck to overcome.  One thing you might consider with NAS is bridging network adapters - we do this for database solutions, the main reason, of course, is eliminating a single point of failure from the application servers to the databse server - but a nice side benefit is doubling or even tripling bandwidth at the cost of an extra network adapter or two.

I'm working on implementing this in my home BSD based FreeNAS machine, but I'm finding network adapter bridging is *not* trivial, at least in FreeBSD (ridiculously simple in Windows 2000 onward).  Also, latest build has iSCSI support which *should* eliminate the networked drive limitation of Lightroom:

http://www.freenas.org/index.php?option=co...id=24&Itemid=24

I'm personally committed to the idea of NAS for two reasons:

1) Application independance - my data is effectively partitioned from my workstation(s)
2) Independance of client platform (ie: Mac / Windows / *nix)

I'm currently running internal RAID10 (mirrored RAID5- separate controllers) SCSI320 on my development server - Lightroom is actually pretty "snappy"

Which G-Solutions product are you looking at?

Regards - John

[BernardLanguillier]

I'm currently running internal RAID10 (mirrored RAID5- separate controllers) SCSI320 on my development server - Lightroom is actually pretty "snappy"

Which G-Solutions product are you looking at?

Regards - John
[{POST_SNAPBACK}][/a]

Hello John,

Thanks for the reply. I have been considering their G Speed 3TB 6 disk Ultra 320 SCSI RAID:

[a href=\"http://www.bhphotovideo.com/bnh/controller/home?O=6546&A=details&Q=&sku=464182&is=REG&addedTroughType=categoryNavigation]http://www.bhphotovideo.com/bnh/controller...egoryNavigation[/url]

Any thoughts on this? One area of concern for me is noise, I couldn't find any review commenting on this aspect.

Regards,
Bernard

[theophilus]

Hi Bernard and John,

Can I ask why you are using a NAS solution for all your photo-editing?  I see it as a long term solution to storage, yet for "new" shoots I feel like all the initial processing could be done using an internal RAID solution with 10,000 rpm SATA drives.

I just can't imagine that you would get that much more benefit from a $3800 SCSI solution either, over using an eSATA solution for less than 1/4 the cost.

I'm guessing you have some reasons that I'm not aware of, so I'd interested in your philosophy.

thanks.

[BernardLanguillier]

Hi Bernard and John,

Can I ask why you are using a NAS solution for all your photo-editing?  I see it as a long term solution to storage, yet for "new" shoots I feel like all the initial processing could be done using an internal RAID solution with 10,000 rpm SATA drives.

I just can't imagine that you would get that much more benefit from a $3800 SCSI solution either, over using an eSATA solution for less than 1/4 the cost.
[a href=\"index.php?act=findpost&pid=113132\"][{POST_SNAPBACK}][/a]

As far as i am concerned, I keep working on a very large amount of old files, and need to have a live access to these. This is related to me doing lots of panoramas, and having often not the time to process them until months after a shoot.

Internal HD just don't cut it in terms of capacity, but I agree that having an internal RAID1 pair of large SATA disks when the latest files are stored first is the best option for many.

Regards,
Bernard

[theophilus]

That makes sense.  I think it will be something I will need to revisit in a couple of years.  I am also very interested in starting to do a lot of large pano's and the fill size ends up getting huge in a lot of cases, at least for the master Photoshop/TIFF file.

[naisan]

That makes sense.  I think it will be something I will need to revisit in a couple of years.  I am also very interested in starting to do a lot of large pano's and the fill size ends up getting huge in a lot of cases, at least for the master Photoshop/TIFF file.
[a href=\"index.php?act=findpost&pid=113288\"][{POST_SNAPBACK}][/a]

I am using a SATAII RAID0 arrangement with a dual-opteron box (effectively 4 processor cores) with 8GB RAM and a vista x64 box.

The RAID0 is a bit risky, but quick. I do lots of large panos (typically 3GB PS file size) and yes, the load time is a bit slow, as are things like sharpening. I would suggest using RAID0 arrangements with faster disks (10k). Internal SCSI and SATA will usually beat attached storage, but not always, esp in some cases of processor contention.

re: processor utilization I find all the cores are utilized by CS3 during a processor-intensive task.

I would humbly suggest that the best use of your funds would actually be to build two identical machines, and use a KVM switch or multiple monitors to get them both working in parallel.

That way, you start a process, switch to another machine, then go back when you're idle again.

I have this setup, and it works very well.

[jjj]

I would humbly suggest that the best use of your funds would actually be to build two identical machines, and use a KVM switch or multiple monitors to get them both working in parallel.

That way, you start a process, switch to another machine, then go back when you're idle again.

I have this setup, and it works very well.
[a href=\"index.php?act=findpost&pid=113601\"][{POST_SNAPBACK}][/a]

I was wondering about doing something similar, but doing office work, emails etc on the second lesser speced machine, whilst Bridge, PS or LR churned though images. But finding a good KVM seems to be the problem. One that can take USB input devices, I use a mouse for left hand, keyboard in the middle and a graphics tablet for right hand - all 3 are USB. And does sound too.
The other twist I was thinking of one machine being Mac and one PC. Just to make things trickier. I fancied a Mac Mini, but they are very overpriced for what you get and it won't drive two monitors either. $1060 for the less crippled one in the UK! A bargain, not. And $800 for the one with no DVD writer!

[Christopher]

Here is my current setup:

1st:
Intel Core 2 Duo 67er
4GB Ram
Win XP 64
2*NV 7300
2*150 Raptors in RADI0 (really fast)
2*500GB RAID0
1*750gb BACKUP
+
1*external 500GB which is backing up every night.

2nd:
Intel Core 2 Duo 63er @ 2,8
2GB Ram
Win XP
2*150GB RAID0
1*500GB

I have to TFTs on my desk, a Eizo CG220 and a Eizo S2100, the CG220 is only connected to the 1st computer the second one is connected with both.

So my normal work is the following.
If I have to do some office stuff than I'm doing it on the slower computer and I'm do the other stuff on the other computer.

If I only work on my images than I do the following I prepare the files on the first one, save them. Than use the Other computer through the network to load them and to create tha Panos, while it is doing it I can finish off some work or prepare more on the other computer.

I have two keyboards and mice on my desk, YES in the beginning that is kind of disturbing, but when you get used to is its so fast and wonderful.