Luminous Landscape Forum
Raw & Post Processing, Printing => Adobe Lightroom Q&A => Topic started by: Scott McGee on February 26, 2007, 03:10:01 am
-
Maybe I'm just way out in left field in thinking that Lightroom should be able to use a library that's located on a network drive. But it seems logical to me that it should be able to.
Here's my setup:
1) Dell desktop computer with four 18GB hard drives in RAID 5 (this is my C: drive)
2) ReadyNAS 2TB network attached storage in RAID 5 (this is my F: drive)
I have only Windows XP and my programs on the C: drive. I keep all my data files on the F: drive. Because of the limited space on my C: drive, I want to have my Lightroom library located on the F: drive. In Windows Explorer, I copied the Lightroom library from the default location in my C:\Documents and Settings folder to the F: drive. Then when opening Lightroom, I held down the Ctrl key to bring up the dialog box to select the library on the F: drive. When I selected it, I got a message in the dialog box that said Lightroom cannot use a library that's located on a network volume.
Does anyone know why this is, and if there's a workaround? Or do I just have to live with it?
Scott
-
Maybe I'm just way out in left field in thinking that Lightroom should be able to use a library that's located on a network drive. But it seems logical to me that it should be able to.
Here's my setup:
1) Dell desktop computer with four 18GB hard drives in RAID 5 (this is my C: drive)
2) ReadyNAS 2TB network attached storage in RAID 5 (this is my F: drive)
I have only Windows XP and my programs on the C: drive. I keep all my data files on the F: drive. Because of the limited space on my C: drive, I want to have my Lightroom library located on the F: drive. In Windows Explorer, I copied the Lightroom library from the default location in my C:\Documents and Settings folder to the F: drive. Then when opening Lightroom, I held down the Ctrl key to bring up the dialog box to select the library on the F: drive. When I selected it, I got a message in the dialog box that said Lightroom cannot use a library that's located on a network volume.
Does anyone know why this is, and if there's a workaround? Or do I just have to live with it?
Scott
[a href=\"index.php?act=findpost&pid=103184\"][{POST_SNAPBACK}][/a]
Not only that, I am finding it is choking on importing more than 1000 files from my Terastation NAS even though I am using the default library database location on my C: drive (I use Windows XP).
I currently backup to USB externals.
I am starting to change my strategy and look into using daisy-chained Firewire externals to hold my files, backup to the Terastation, and repurpose the USB drives.
I ordered a Western Digital 500GB My Book Pro just to see how it works on Firewire. Problem is, I hear they have a problem daisy-chaining.
-
Hello,
Just out of curiousity and not the change the main subject of this post, but why don't you use an online storage facility instead of buying multiple external hard drives? Photoshelter comes to mind and while the advertised pricing is high for large storage needs they do have unadvertised prices that are more reasonable and at least the data is offsite etc. There is also ProtectMyPhotos.com, but I just tried and sadly their service at least for the time being reads local drives only and will not work with mapped network drives like my Nas. Their prices are really good so I was disappointed to learn they couldn't see my Nas.
Just a thought....
I was going to buy LightRoom until I saw this post and as I use a ReadyNAS NV for my inoffice storage I guess I won't be able to use LightRoom. Find it hard to believe that Adobe would do something like this. A bit confusing to say the least.....
good luck
-
Hello,
Just out of curiousity and not the change the main subject of this post, but why don't you use an online storage facility instead of buying multiple external hard drives? Photoshelter comes to mind and while the advertised pricing is high for large storage needs they do have unadvertised prices that are more reasonable and at least the data is offsite etc. There is also ProtectMyPhotos.com, but I just tried and sadly their service at least for the time being reads local drives only and will not work with mapped network drives like my Nas. Their prices are really good so I was disappointed to learn they couldn't see my Nas.
Just a thought....
I was going to buy LightRoom until I saw this post and as I use a ReadyNAS NV for my inoffice storage I guess I won't be able to use LightRoom. Find it hard to believe that Adobe would do something like this. A bit confusing to say the least.....
good luck
[a href=\"index.php?act=findpost&pid=103329\"][{POST_SNAPBACK}][/a]
I'm not saying it is supposed to behave the way I describe. I have a hard to believing they would engineer it that way also.
I am only telling you what I am experiencing and I am also hearing similar banter on forums here and there. I can't wait for the situation to be solved if it is a bug, so I am moving my primary online image store from NAS to firewire volumes.
-
Bob,
just a quick fyi... I've had a Western Digital 500GB Pro Book for a few weeks now. I use as one of my photo file backups and connect via USB 2.. plenty fast. Thus far, a great drive.
p
-
Well, I've answered my own question. I found this at the Adobe Lightroom support website.
------------------------------------------------
Q) Can I store my library on a network drive?
A) No, this is not supported. There are too many variables in a network configuration to guarantee that the library database will not become corrupted. See the next question for how to use your library with multiple computers.
------------------------------------------------
So, Adobe does not now, nor ever will, support having the library on a network drive or NAS device connected via Ethernet. However, Lightroom will support a library that's located on an external hard drive that is connected to the host computer via USB or Firewire.
Unfortunately in my case, my ReadyNAS does not support connecting to the host computer via Firewire or USB. So I can't put the Lightroom library on the NAS.
-
Well, I've answered my own question. I found this at the Adobe Lightroom support website.
------------------------------------------------
Q) Can I store my library on a network drive?
A) No, this is not supported. There are too many variables in a network configuration to guarantee that the library database will not become corrupted. See the next question for how to use your library with multiple computers.
------------------------------------------------
So, Adobe does not now, nor ever will, support having the library on a network drive or NAS device connected via Ethernet. However, Lightroom will support a library that's located on an external hard drive that is connected to the host computer via USB or Firewire.
Unfortunately in my case, my ReadyNAS does not support connecting to the host computer via Firewire or USB. So I can't put the Lightroom library on the NAS.
[a href=\"index.php?act=findpost&pid=103413\"][{POST_SNAPBACK}][/a]
In theory you could locate the library on a local drive and have it point to files on a network drive if local storage space is your issue. You will get a performance hit especially when importing lots of files and especially with low-end network drives and NAS appliances and, of course, you won't be able to easily share the library from different computers.
Saving XMP files together with the image files might be a crude way to simulate library sharing but you must be extra careful to avoid the potential of double concurrent updates (i.e. only ever have Lightroom active on one computer).
A low-end iSCSI solution might be the answer for network library sharing. This will allow files (including, in theory, the library) to be located on a network drive but appear to the operating system (and thus Lightroom) to be located on a local drive. Unfortunately, low-end iSCSI solutions are not yet mainstream (and maybe they never will be).
iSCSI solutions involve an iSCSI host, which is typically an appliance or pc with internal or external disks, with appropriate software (or bootable firmware) to make it appear as a local drive to ethernet connected iSCSI clients which will be your computers with appropriate iSCSI drivers ( provided for free by Microsoft in the case of Windows).
For the computer savvy, iSCSI provides a method to encapsulate SCSI hard disk control coomands in a way that can be trasmitted over ethernet. The drivers on both ends have the job of encapsulating and de-encapsulating the SCSI commands. Low end iSCSI solutions support SATA protocols also.
Nikos
Athens, Greece
-
Just a general comment regarding Lightroom basic architecture. Adobe have decided (rightly or wrongly) to go with the database concept in the heart of Lightroom (same as Apple have done). Their library is essentially a sort of database.
This decision means, in principle, that they are bound to face now or in the future all sorts of issues (integrity, replication, merging, incremental backup, synchronisation, logging etc.) pertaining to all database systems. I am not aware of the internals of Lightroom database but if they have chosen not to use an industry standard embedded database system, but use propriatery code instead, they will have to re-invent the wheel in all these issues. Adobe is a large company with lots of resources but, to my knlowledge, this is the first time they are trying their hand on a database type application.
Nikos
Athens, Greece
-
...I am not aware of the internals of Lightroom database but if they have chosen not to use an industry standard embedded database system...
Lightroom uses SQLite. The library database can be accessed by any tools which work with SQLite. This is a good design decision by Adobe which opens up many possibilities for writing extensions to LR.
-
Lightroom uses SQLite. The library database can be accessed by any tools which work with SQLite.[a href=\"index.php?act=findpost&pid=103469\"][{POST_SNAPBACK}][/a]
Thank you. That's good to know.
So the answer to the OP's question about 'Why Not?' may be because SQLite, which is a flat file database, does not implement its own locking techniques and relies on the, often unreliable operating system locking and caching mechanisms, for ensuring integrity on network drives. Adobe wisely chose not to allow placement of the databse on a network drive since this could lead to corruption (and maybe in some extreme cases even when accesed by only a single client).
Also any hopes that Lightroom may grow up to be a multiuser application should probably be relegated to the distant future as SQLite, to my knowledge, is not really designed to be a robust multiuser database.
On the other hand, since tools to merge SQLite databases already exist, MR's hope of Lightroom supporting library merging in a future version is not overoptimistic.
Nikos
Athens, Greece
-
Well, I've answered my own question. I found this at the Adobe Lightroom support website.
------------------------------------------------
Q) Can I store my library on a network drive?
A) No, this is not supported. There are too many variables in a network configuration to guarantee that the library database will not become corrupted. See the next question for how to use your library with multiple computers.
Here's some additional input about this problem.
Not only are there many variables that can affect reliability in a networked disk setup, but there are also serious performance issues in CIFS (the network file system used by Windows and therefore most home/small office NAS devices).
Since you don't mention which ReadyNAS product you're using, it's a bit hard to say exactly what your performance will be like, compared to a FireWire 400/800 or USB 2 drive, but you should expect lower performance regardless of how good your network is.
According to the ReadyNAS Comparison Chart (http://www.infrant.com/products/compare.php), the best theoretical performance you can get is 32 MB/s read and 24 MB/s write (if jumbo frames are enabled throughout a Gigabit Ethernet network).
The best theoretical performance for internal SATA drives is about twice as good, and with significantly better latency figures. (Latency is, in brief, the time between the request for data to be read or written, and the point in time where the data has been fetched or written.)
FireWire comes reasonably close to internal drives. USB 2.0 may reach comparable performance to FireWire, but only if there are no other active USB devices on the same bus (sometimes the same controller).
The advice about looking at iSCSI is not unreasonable in itself, but keep in mind that iSCSI works better with dedicated network interfaces and cabling than when connected to a hub or a home office switch.
External SATA (eSATA) is becoming a competitive solution for home computers, but like FireWire and USB drives, the usable cable length is limited (I don't recall the exact numbers for either interface, sorry).
And if you're on a wireless network, you're in for more potential performance problems with network disks, depending on atmospheric conditions (yes, even inside your house) and radio activity (other wireless devices in your house or neighbouring houses).
All in all, for active image editing, NAS devices designed for home use are probably not the best for such purposes.
Getting decent performance from NAS or SAN (don't you love these acronyms?) is possible, however, it just costs money.
-
Here's some additional input about this problem.
Not only are there many variables that can affect reliability in a networked disk setup, but there are also serious performance issues in CIFS (the network file system used by Windows and therefore most home/small office NAS devices).
Since you don't mention which ReadyNAS product you're using, it's a bit hard to say exactly what your performance will be like, compared to a FireWire 400/800 or USB 2 drive, but you should expect lower performance regardless of how good your network is.
According to the ReadyNAS Comparison Chart (http://www.infrant.com/products/compare.php), the best theoretical performance you can get is 32 MB/s read and 24 MB/s write (if jumbo frames are enabled throughout a Gigabit Ethernet network).
The best theoretical performance for internal SATA drives is about twice as good, and with significantly better latency figures. (Latency is, in brief, the time between the request for data to be read or written, and the point in time where the data has been fetched or written.)
FireWire comes reasonably close to internal drives. USB 2.0 may reach comparable performance to FireWire, but only if there are no other active USB devices on the same bus (sometimes the same controller).
The advice about looking at iSCSI is not unreasonable in itself, but keep in mind that iSCSI works better with dedicated network interfaces and cabling than when connected to a hub or a home office switch.
External SATA (eSATA) is becoming a competitive solution for home computers, but like FireWire and USB drives, the usable cable length is limited (I don't recall the exact numbers for either interface, sorry).
And if you're on a wireless network, you're in for more potential performance problems with network disks, depending on atmospheric conditions (yes, even inside your house) and radio activity (other wireless devices in your house or neighbouring houses).
All in all, for active image editing, NAS devices designed for home use are probably not the best for such purposes.
Getting decent performance from NAS or SAN (don't you love these acronyms?) is possible, however, it just costs money.
[a href=\"index.php?act=findpost&pid=103608\"][{POST_SNAPBACK}][/a]
Thanks for that insight. It was clear, understandable, and informative. You have completely defined my situation and why I am going to daisy-chained firewire drives for primary file storage.
-
Thanks for that insight. It was clear, understandable, and informative. You have completely defined my situation and why I am going to daisy-chained firewire drives for primary file storage.
[a href=\"index.php?act=findpost&pid=103610\"][{POST_SNAPBACK}][/a]
I'd also like to add mine Jani. Very clear and well explained.
-
Also any hopes that Lightroom may grow up to be a multiuser application should probably be relegated to the distant future as SQLite, to my knowledge, is not really designed to be a robust multiuser database.
On the other hand, since tools to merge SQLite databases already exist, MR's hope of Lightroom supporting library merging in a future version is not overoptimistic.
Don't focus on SQLite - after all, you didn't know it was there until a couple of posts back. Applications like LR can sit relatively easily on other databases too (eg Extensis Portfolio) such as SQL Server or Oracle. Multi user might not be tomorrow, but nor is it the distant future.
The database back end is irrelevant to merging databases. In beta 4 there was a mechanism called binders - because of other priorities (folders) it didn't make the cut for the v1 release, but it's bound to be revived in some form.
John
-
Just stumbled on this thread and find it incredible that Adobe didn't just provide an ODBC interface that would allow the user to use their database of choice. Sounds like they're about 10 years behind the times. It's not an issue for me (or most amateurs) as long as I can store the images on a NAS but it's bound to be a killer for professional use. Everything these days is going to DB storage as it's the most flexible and best performing method. Bundling SQLite is fine as a default but not allowing the user to choose a REAL database is just plain stupid. There are plenty of robust DB engines that provide record locking, transactions etc., even free ones (Oracle, Firebird, MS SQl Express). I doubt that providing the ODBC flexibility would have been any more work than interfacing directly, and they could have avoided support issues with a disclaimer "use with other DB at your own risk".
- DL
-
Just stumbled on this thread and find it incredible that Adobe didn't just provide an ODBC interface that would allow the user to use their database of choice. Sounds like they're about 10 years behind the times. It's not an issue for me (or most amateurs) as long as I can store the images on a NAS but it's bound to be a killer for professional use. Everything these days is going to DB storage as it's the most flexible and best performing method. Bundling SQLite is fine as a default but not allowing the user to choose a REAL database is just plain stupid. There are plenty of robust DB engines that provide record locking, transactions etc., even free ones (Oracle, Firebird, MS SQl Express). I doubt that providing the ODBC flexibility would have been any more work than interfacing directly, and they could have avoided support issues with a disclaimer "use with other DB at your own risk".
- DL
[a href=\"index.php?act=findpost&pid=104523\"][{POST_SNAPBACK}][/a]
Agreed - kind of makes you wonder how they made such an obviously poor decision.
-
Hello,
When you create a new volume or partition in Windows you have an option to mount it into existing empty folder on any NTFS-formatted local drive.
Did anyone try this to trick LightRoom into thinking that it accesses local folder, while in fact the data is stored on NAS?
The explanation can be found here (about half way down, when they talk about Drive Paths):
http://www.microsoft.com/technet/prodtechn...e/10w2kadb.mspx (http://www.microsoft.com/technet/prodtechnol/windows2000serv/maintain/operate/10w2kadb.mspx)
I don't have a networked storage yet, so can't try it myself.
Allan
-
Agreed - kind of makes you wonder how they made such an obviously poor decision.
[a href=\"index.php?act=findpost&pid=104564\"][{POST_SNAPBACK}][/a]
Because they were developing a version 1 for two platforms and to a tight schedule. No doubt other back ends will come, but in their own good time.
John
-
If you want to run SQLite with ODBC, use an ODBC wrapper for SQLite (http://www.ch-werner.de/sqliteodbc/).
Edit: Typofix.
-
If you want to run SQLite with ODBC, use an ODCB wrapper for SQLite (http://www.ch-werner.de/sqliteodbc/).
[a href=\"index.php?act=findpost&pid=104645\"][{POST_SNAPBACK}][/a]
But does LR support the ODBC interface? If you're happy with sqlite there's no need for ODBC. I should add that this is not really an issue for me and most likely will be purchasing LR before the special is up, but I'm just scratching my head about this as a DB adds all sorts of extensibility/interface options and I'm already using and familiar with both mySQL and Firebird and life would be simpler if I could use one of these. The mail server I currently runs allows this flexibility as does my web engine - pick your DB weapon of choice.
- DL
-
No it doesn't. That suggestion is only useful if you want to examine the database with an external tool.
John
-
Also search Sourceforge for "SQLite Database Browser"
John
-
But does LR support the ODBC interface?
I thought the point was to use ODBC to access the database, not to access Lightroom.
As far as I know, Adobe doesn't provide an ODBC driver for Lightroom, but that seems orthogonal to the question asked.
If you're happy with sqlite there's no need for ODBC.
It is useful if you e.g. want to import the data into a different database, though.
-
Lightroom does indeed use SQLite, which also explains why network drives are not supported. Any database access across a network *must* support some form of concurrency or multi-user support. SQLite's method is applying a file level lock on the entire database file.
Although SQL methods such as Begin Transaction, Commit and Rollback are supported - the low-level result is the crude locking scheme described. Although you *might* get away with this for very small files - no way on even a modest lightroom db.
Looking at the internal tables and schema of the database is interesting.
http://www.sqlite.org/sqlite.html (http://www.sqlite.org/sqlite.html)
hth - john
-
I thought the point was to use ODBC to access the database, not to access Lightroom.
As far as I know, Adobe doesn't provide an ODBC driver for Lightroom, but that seems orthogonal to the question asked.
[a href=\"index.php?act=findpost&pid=104752\"][{POST_SNAPBACK}][/a]
The question was not if Adobe provides an ODBC interface but rather it they use one as this would permit the user to sub a more robust database that would work over a network.
- DL
-
Lightroom does indeed use SQLite, which also explains why network drives are not supported.
<snip>
hth - john
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=104851\")
SQLite databases may be shared on a network. There are various projects working on this topic. Check out:
[a href=\"http://www.sqlite.org/cvstrac/wiki?p=SqliteNetwork]http://www.sqlite.org/cvstrac/wiki?p=SqliteNetwork[/url]
Cheers,
Andrew
-
The question was not if Adobe provides an ODBC interface but rather it they use one as this would permit the user to sub a more robust database that would work over a network.
In that case, I misread the question completely.
I would be very surprised if a commercial company even considered using ODBC for what they consider an application-internal database. It's far more likely that they considered using Berkeley DB or other integrated solutions.
-
SQLite databases may be shared on a network.
Oh, the pain!
Why do people come up with those ideas? I know of people who try to use Access databases in a similar manner, too. It's awful.
Use a real database already.
-
I think there may be some confusion in this thread between 2 issues:
1) shared (multi-user) access to LR library database:
- clearly needs more than SQLite, more difficult to implement, quite reasonably not done in LR V1 but presumably not excluded as a future line of development by Adobe.
2) storing LR library database on a network volume:
- even in single-user mode, this can be useful for several reasons - some users (like me) put all their critical files on a server to take advantage of regular backup, others use NAS and so forth.
- as far as I know there is nothing in SQLite which prevents an SQLite database being on a network volume.
- therefore I don't quite understand why Adobe blocked this in LR v1. Maybe they were just being hyper-cautious, foreseeing that people would try to share an LR database on a network volume and screw it up. On the other hand, I can give access to my PC's hard disk to other users on my network, so even local hard disk storage is not immune to attempts at simultanous access by multiple users.
-
Oh, the pain!
Why do people come up with those ideas? I know of people who try to use Access databases in a similar manner, too. It's awful.
Use a real database already.
[a href=\"index.php?act=findpost&pid=104968\"][{POST_SNAPBACK}][/a]
Jani,
please note my wording: they *may* be shared over a network. I wasn't suggesting that this is a particulary good idea. In fact, I totally agree with everthing you pointed out.
I simply responded to the argument that SQLite databases cannot be shared over a network.
Cheers,
Andrew
-
You can put an Access DB on a network drive but there's hugh performance issues even with a single user (ignoring multiple access) because the actual DB engine is still on the local machine and just chunks hugh pieces of the DB across the network. What you really need for this scenario is a true database server with DB on server local drive where only the queries and query results will transit the network. SQLite is not a client-server and I suspect this is the real issue.
I would be very surprised if a commercial company even considered using ODBC for what they consider an application-internal database.
My mail server does this - there are options to store various stuff in a DB's and all is done via ODBC allowing you to choose Access, SQL Server, mySQL, SQL Express, SQLite, etc. My Radius server and web middleware also support choice of database via ODBC. I'm sure LR will allow this flexibility eventually also, I just think that in the long run it would have been simpler to start this way even if multi-user was not supported to begin with.
- DL
-
If you look at the structure of the LR Database you will quickly begin to see that Multi-User access is definately in the cards - I suspect, with a 1.0 release offering a radically different solution and workflow, Adobe chose to just not "go there" with multi-user or even network capabilities, in fact LR 1.0 opens it's database exclusively.
By *not* offering ODBC access, they effectivly prevent multi-user access - but that is not the same as saying they will not offer it or some other method in the future. I fully expect and hope to see eventual 3rd party offerings integrating LR with DAM and Content Management solutions - it would make great business sense for everyone.
-John
-
I think there may be some confusion in this thread between 2 issues:
1) shared (multi-user) access to LR library database:
- clearly needs more than SQLite, more difficult to implement, quite reasonably not done in LR V1 but presumably not excluded as a future line of development by Adobe.
2) storing LR library database on a network volume:
- even in single-user mode, this can be useful for several reasons - some users (like me) put all their critical files on a server to take advantage of regular backup, others use NAS and so forth.
- as far as I know there is nothing in SQLite which prevents an SQLite database being on a network volume.
- therefore I don't quite understand why Adobe blocked this in LR v1. Maybe they were just being hyper-cautious, foreseeing that people would try to share an LR database on a network volume and screw it up. On the other hand, I can give access to my PC's hard disk to other users on my network, so even local hard disk storage is not immune to attempts at simultanous access by multiple users.
[a href=\"index.php?act=findpost&pid=105006\"][{POST_SNAPBACK}][/a]
I could not agree more. Well said.
I fall into the second group, choosing to keep all critical data - and that includes images - on a RAID 5 array on a networked server. This is exported to my Windows XP workstation using NFS. (sorry for the techie bits) Just to make sure I don't loose data, the RAID array, is additionally backed-up to tape.
So, Mr. Adobe, insisting your application database is stored on a locally connected disk is fine for a beta, but for a mission-critical application I need something far more robust. Please, Please, Please get support for networked storage sorted.
After all, networked databases are hardly something new.
Andrew Gawthrope
HAPY Imaging