Sorting issue with a Collection

[jimrobertson]

(I started a new topic as the other sorting thread has to do with a different issue)

I am using LR5.7 on a WIN7 PC

I am having a sorting issue. All images have come from same file structure: Catalog > year/month > individual day.

I have been using this structure since day one with LR and have made many collections and done extensive sorting, this is the first time I had hit this.

Issue:
I have a collection of 2,900 images. I have been sorting them (click and drag) but I have found 50 or so (all adjacent to each other that were sorted into their position) that will no longer sort (trying to fine tune the sort). I can select some of them, drag them to where I want them to be, the black bar appears, but none of the images change position. I can't figure out what is going on. The images are from two folders. but other images from those folders are "sortable".

On further sorting I find that if I move other images close to the problematic ones, they will not sort properly. I can select and drag them to the appropriate spot, but they are dropped in the sorting action in locations other than what I picked. Last night I dragged some other files elsewhere in the collection to another area in the collection away from the problematic ones, and they too dropped into close-by spots, but not where I wanted.

But at the same time while dragging/sorting others, they behaved normally.

Anyone have any thoughts ??

Thanks

[RikkFlohr]

Are they all inside of a Stack?

[jimrobertson]

No Rikk they aren't.

I am trying to post a small update with a screen print of the folder structure but having internet connections issues with my PC where the catalogue is

Thanks for replying

[jimrobertson]

I have been continuing to work with the collection to fine tune the sort and remove images from within the collection.

I notice now that the "offending" images are from a number of subfolders within two folders (of the root of the catalog)

But there are other images from within the same subfolders that are able to be dragged/sorted with no problems.

I am posting a screen shot to show the folder structure

[fdisilvestro]

Hi,

Something may have got corrupted in the table that saves the sort order. Try duplicating the collection and test if now you can sort the images (it makes a difference in how the data stored in the catalog)

To duplicate a collection in LR on Windows, press CTRL while clicking on the name of the collection and drag until a thick gray lane appears between collections. It will create a new collection adding the word "copy".

If this work, then you can delete the original collection and rename the new one

Regards

[jimrobertson]

Thanks Fransico.

I was about to take some of the problem images into another collection and just see what happens. What I would have done if they were OK is another issue 

I'll try your solution and let you know what happens.

[jimrobertson]

Francisco: (got the spelling right this time I hope)

No joy 

Still the same problem.

Should I hold out for another option, or do you think I should go back to the original folder and add the problem images (I know of) to a new collection (or collections such as elephants only in one collection, giraffes in another etc) that might help?

If it does, I guess I would have to do that with all the problems images and when I am finished editing out the non-wanted ones and fine tuning the sequence, I'll have to recombine them all in another collection.

Thanks

[fdisilvestro]

Hi,

No worries about the spelling  

Looking at how the sorting information is stored in the ligthroom catalog (an SQLite database) I decided to try a test and was able to replicate your exact issue: You have hit a limitation of LR!!

Yes, the way that custom order is implemented (which is weird IMHO) is such that after many manual movements in the sort order, you can actually hit a limit, which is especially likely to happen in large collections.

The fact is that this limitation is "local" which means that you might have been sorting the affected area very often so to reach the limitation, while other areas in the catalog still accept sorting.

I'm attaching a screenshot of the table that store the image positions in the LR Catalog, and the area circled in red has reached the maximum amount of digits for the fields (15), so no further manual sorting is possible between those images. As you can see, further down in the column, the entries are using four digits (yy.xx) allowing further manual sorting.

Because of this being a technical limitation, the option that I see here is to remove the affected images from the collection and add again the images to the collection.


Edit:

WAIT!! There is a better solution: Move the affected images, even as a group, to an unaffected area in the catalog (where you can still sort) and problem solved! (you will need to sort back to the desired locations)

Regards

[jimrobertson]

Francisco:

I am not sure I fully understand it, but I gather the 2900 images in the collection is not the issue, but the problem is that I sorted some of the issues "too many times".

I will give your suggestion a try. I'll move the problem images (or at least some of them) to another area in the collection and see what happens.

I'll give it a try tomorrow morning as I am tied up on something else now.

Thanks

[fdisilvestro]

Francisco:

I am not sure I fully understand it, but I gather the 2900 images in the collection is not the issue, but the problem is that I sorted some of the issues images "too many times".

Thanks

Yes, exactly that!

[jimrobertson]

Francisco:

Got it. I'll see what happens tomorrow morning

Thanks again

[jimrobertson]

Francisco/Frank:

Your solution to move affected images to another part of the collection and sort them there worked !

Moving them back to their desired position though was problematic.

One group I moved back to where I wanted them worked great for some of the images, but others dispersed themselves to intermingle with other images adjacent to that location. If I moved them back to a slightly different location, they stayed sorted as I wanted.

If I restate my understanding of the issue a bit, I think I can explain the problem of the move back.

The issue is not the size of the collection per se, but the fact that there are areas within the collection that have been sorted beyond the "sort limitation" and I seem to have more of those locations in the collection than I thought.

(it isn't the individual image, as I had originally interpreted your explanation, that has been sorted too many times because if that was the case then why could I sort it in its temporary location?)

I have had collections as large as this one before, but due to the nature and purpose of this collection, I have been doing much more sorting than in the past and that's why I hit the "hidden" limitation this time.

In hindsight I should have made several smaller collections and done my sorting and editing ("keep vs delete" from collection) in those collections.

I think at this point that I either need to live with the over-sorted areas of the collection, or break-up the collection into several smaller ones.

I'm going to try to live with it, but be prepared to break it up if I have to.

Thanks for all your help, very much appreciated.

I had this issue posted on other forums (Adobe Communities as well as Lightroom Forum) and received no help, so I am very pleased someone suggested I try this forum.

[fdisilvestro]

Jim:

Glad I could help. Your understanding is correct. I guess that this issue has been rare if ever occurred to any user, and even Adobe did not anticipate the issue could manifest. It would actually be very easy to solve by Adobe as part of the "Optimise catalog" function (yes, I tested and it does not solve the issue)

Having said that, I have found a solution that can work for you, but please make sure you make a backup of the catalog before trying it.

- Go to your collection
- Select half of the collection (E.G. from the first image to the image 1450)
- Move the entire group to the end of the collection
- Now select again the images that are now at the beggining, from the "new" first to the "new" 1450
- Move again this group to the end of the collection

You will end up with the original order, since you moved twice the upper half (like if it were a stack of cards)

This solves the problem! I tested it! now you can start sorting like it was a new collection

Regards

[jimrobertson]

Thanks for the added solution and all your help

[jimrobertson]

Frank:

One last question....

How would I pass this issue on the Adobe (or do they watch these forum) ? Or maybe you already have.

Thanks again

[john beardsworth]

Jim, post it here. But I wouldn't want to raise your hopes. It's an obscure error - I've never seen it before - and one might suggest that collections of that scale are inherently unsuitable to manual sort orders. Adobe might be better employed extending the sort feature.

John

[jimrobertson]

Thanks John

I doubt if I'll opt for a collection that large again....

[jimrobertson]

See comment from Cletus Lee towards end at
http://www.lightroomforums.net/showthread.php?23053-Can-t-get-custom-order-option&p=160521#post160521

[john beardsworth]

To be frank, Cletus has seen "list" and associated your issue with a different bug with entirely different origins.

[fdisilvestro]

Hi,

No, I have not reported it. I agree with John that Adobe would assign a very low priority to this, on the other hand the solution is so easy that it would take a good developer less time than it takes to order pizza.

About sizes: 2900 is nothing (referred comment from Cleto Lee is not correct), SQLite can handle millions of records. The issue here is the way that manual sort is implemented (my guess is that it was designed for performance)