Adobe Lightroom!!!

Status
Not open for further replies.

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
Does anyone on here have extensive working knowledge of lightroom in conjunction with a NAS of any type? Obviously I will be using FreeNAS, but I am very curious about how I should try and set things up.

Currently, I have a horrible workflow. So lets not worry about how its set up now.

My plan is to put all of my photos (about 1 TB worth of pictures) on my freenas box. I am not sure how I would go about networking for this, maybe just set up my photo directory as a network storage drive, or iscsi target? I honestly have no idea, could use some advice here (I am also not very familiar with ISCSI...).

I was planning to have my lightroom catalog on my desktop in which case it will live on my RAID 0 SSD array which is plenty fast. And to help take the load and therefor bottleneck out of my gigabit ethernet, I would either build my entire library as a smart previews or 1:1 previews. Problem is, how much space do smart previews or 1:1 previews take? I can't seem to find concrete answers and I don't want to spend a week building previews just to find out the ~60 gigs or so I was planning on was a total misjudgment and it turns out its eating up my entire half TB worth of SSD space on my main rig.

Also, since I have never stored my images off my photo editing PC, weather that be on SSD or spinning rust, how does this workflow work? Is there any noticeable performance degradation, or does the smart/1:1 previews 100% eliminate that?

Any photography buffs in here who can let me know how they have set up their workflow and what they might advise?

Also to this point, is my planned FreeNAS box plenty viable for this type of data? I don't edit pictures all that often, maybe once a week, but if my 16 gigs of RAM for instance would be an issue, I guess there is always an easy way to add more ;)

Planned hardware:
8 x 4TB WD Red or Ironworlfs RAID Z2
i3-6100
MBD-X11SSM-F-O
M391A2K43BB1-CPB (16 GB ECC RAM)
 

gpsguy

Active Member
Joined
Jan 22, 2012
Messages
4,472

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
Well just to answer the question, what I had to do for my dad’s photo editing PC, since his 1TB SSD was full and I just could be bothered with getting a larger drive, or an additional one, I just got him a 4TB spinning rust.

Made 1:1 previews and smart previews of every picture which took up about 160 GB total, compared to the ~650 GB of photos. As far as moving around Lightroom goes, it’s a bit slower seemingly, but unless you have endless money to throw at SSDs, or it’s your business and not just a hobby, this seems to be a pretty solid solution. Obviously the catalog and previews are all stored on the SSD. I plan to do the same thing with my setup, except the actual photos will be on the FreeNAS box and not local drives. Hopefully will react the same.


Sent from my iPhone using Tapatalk
 
Last edited by a moderator:

styno

Patron
Joined
Apr 11, 2016
Messages
466
Yes, can confirm. Catalog on local disks (ssd) and pictures and catalog backups on the nas.
 
Joined
Feb 2, 2016
Messages
574
Not sure how I missed this many months ago. My apologies, @LIGISTX.

I would either build my entire library as a smart previews or 1:1 previews. Problem is, how much space do smart previews or 1:1 previews take?

We use standard sized, smart previews. With 15,000 photos in the current, active catalog, Lightroom's footprint on my internal, non-NAS drive is 17GB. Trivial. Smart previews let you edit images even if the originals are unavailable.

Is there any noticeable performance degradation, or does the smart/1:1 previews 100% eliminate that?

Whenever you move photos further away and involve more hardware, you're going to lose performance. Adding redundancy almost always degrades performance. That said, any performance penalty you do see will be mild and not at all an impediment to your workflow.

Any photography buffs in here who can let me know how they have set up their workflow and what they might advise?

My client with the highest volume of images is a wedding photographer. She will have 5,000-7,000 images at the end of the wedding day. Each image is a Nikon NEF weighing in at 40MB, give or take. Here is the workflow we found that works best for her...

1. Windows, Lightroom catalog, Photoshop scratch files on internal NVMe SSD (500GB).

2. Mirrored SSDs (250GB) internal to the workstation.

3. Ingest photos to mirrored SSDs.

4. Edit photos in Lightroom/Photoshop, saving edits to mirrored SSDs.

5. Complete assignment, deliver images to client.

6. Move photos from SSDs to FreeNAS (how-to).

7. Snapshot/Replicate from primary FreeNAS to secondary FreeNAS.

8. Wipe original SD/CF cards.

This workflow is for maximum efficiency and safety. At no point in the process is there ever just one device with one copy of the photos. Nothing is worse than a bride scorned.

Once the photos are on FreeNAS, additional edits and whatnot are done there. It is only during the bulk editing phase that the absolute maximum performance is required. Saving a second or two per photos when you're only editing ten isn't a big deal. When you're editing 500 images and going back and forth between images, or layering images, that's where you really feel the time savings.

For my own photography, I'm a bit more willy-nilly and am not concerned with the absolute best performance. My workflow is...

1. Windows, Lightroom catalog, Photoshop scratch files on internal NVMe SSD (500GB).

2. Ingest photos to FreeNAS.

3. Edit photos in Lightroom.

4. Upload to Facebook/Instagram, hope to get a few likes/hearts.

5. Wipe original SD/CF cards.

This workflow is not as fast as having the images on local SSDs but is still perfectly acceptable and faster than having the images on a local, single conventional hard drive.

Cheers,
Matt
 
Last edited:

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
Not sure how I missed this many months ago. My apologies, @LIGISTX.



We use standard sized, smart previews. With 15,000 photos in the current, active catalog, Lightroom's footprint on my internal, non-NAS drive is 17GB. Trivial. Smart previews let you edit images even if the originals are unavailable.



Whenever you move photos further away and involve more hardware, you're going to lose performance. Adding redundancy almost always degrades performance. That said, any performance penalty you do see will be mild and not at all an impediment to your workflow.



My client with the highest volume of images is a wedding photographer. She will have 5,000-7,000 images at the end of the day. Each image is a Nikon NEF weighing in at 40MB, give or take. Here is the workflow we found that works best for her...

1. Windows, Lightroom catalog, Photoshop scratch files on internal NVMe SSD (500GB).

2. Mirrored SSDs (250GB) internal to the workstation.

3. Ingest photos to mirrored SSDs.

4. Edit photos in Lightroom/Photoshop, saving edits to mirrored SSDs.

5. Complete assignment, deliver images to client.

6. Move photos from SSDs to FreeNAS (how-to).

7. Snapshot/Replicate from primary FreeNAS to secondary FreeNAS.

8. Wipe original SD/CF cards.

This workflow is for maximum efficiency and safety. At no point in the process is there ever just one device with one copy of the photos. Nothing is worse than a bride scorned.

Once the photos are on FreeNAS, additional edits and whatnot are done there. It is only during the bulk editing phase that the absolute maximum performance is required. Saving a second or two per photos when you're only editing ten isn't a big deal. When you're editing 500 images and going back and forth between images, or layering images, that's where you really feel the time savings.

For my own photography, I'm a bit more willy-nilly and am not concerned with the absolute best performance. My workflow is...

1. Operating system and Lightroom catalog on internal NVMe SSD (500GB).

2. Ingest photos to FreeNAS.

3. Edit photos in Lightroom.

4. Upload to Facebook/Instagram, hope to get a few likes/hearts.

5. Wipe original SD/CF cards.

This workflow is not as fast as having the images on local SSDs but is still perfectly acceptable and faster than having the images on a local, single conventional hard drive.

Cheers,
Matt


Awesome! Thanks so much! This is very similar to the workflow I sorta figured out.

I have an nvme 500 gb with 2 256 Sata SSD’s in RAID 0 (I know, it’s not “safe”, but I don’t think I will keep anything solely on it, either on CF still or freenas already, unsure), and keep catalog and previews on nvme drive.

I am not sure how I will ingest, possible to the RAID 0 volume. Remember, I’m no pro, and am not doing weddings or even have any clients. I am just a nerd who is also an amateur photographer haha.

Thanks so much for all the pointers, I may end up not needing the 1:1’s or your workflow works well without it. I currently build 1:1’s and smart’s on my dad’s system I am testing on. And I know 1:1 is only good for library, but I want to be able to scrub through the library quick, and peak in to full screen/zoom fast as well.


Sent from my iPhone using Tapatalk
 

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
Actually, is there a reason you don’t use 1:1’s? Seems like the one lady has a pretty serious workflow, and I can’t see myself needing anything she doesn’t lol.

What’s the reasoning for not using it? Simply just to much data and time devoted to storing and creating the 1:1’s to not be worth it? Or is it because she takes SO MANY photos it’s just not practical in her case?


Sent from my iPhone using Tapatalk
 
Joined
Feb 2, 2016
Messages
574
is there a reason you don’t use 1:1’s?

Her Nikon D800 has a resolution of 7360 × 4912 (her soon-to-arrive D850 is 8256 x 5504). At no point in the Lightroom processing is she going to see a 1:1 image on her 2560 x 1440 monitor. For those of us with HD monitors (1920 x 1080), we are especially unlikely to need to see a 1:1 image in Lightroom.

I know there are some people who do amazingly detailed editing in Lightroom. More typically though, Lightroom is for cataloging and doing adjustments of the entire image with a slider. Even selected area changes typically happen over large areas. For all these adjustments, a standard image size is more than enough.

For detailed adjustment, she's going to pop the image into Photoshop. At that point, she's working with the original image file in all its glory and not the Lightroom-created preview anyway.

If your entire workflow is in Lightroom without a secondary application for detailed editing, you could make a case for 1:1 previews.

Cheers,
Matt
 

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
Her Nikon D800 has a resolution of 7360 × 4912 (her soon-to-arrive D850 is 8256 x 5504). At no point in the Lightroom processing is she going to see a 1:1 image on her 2560 x 1440 monitor. For those of us with HD monitors (1920 x 1080), we are especially unlikely to need to see a 1:1 image in Lightroom.

I know there are some people who do amazingly detailed editing in Lightroom. More typically though, Lightroom is for cataloging and doing adjustments of the entire image with a slider. Even selected area changes typically happen over large areas. For all these adjustments, a standard image size is more than enough.

For detailed adjustment, she's going to pop the image into Photoshop. At that point, she's working with the original image file in all its glory and not the Lightroom-created preview anyway.

If your entire workflow is in Lightroom without a secondary application for detailed editing, you could make a case for 1:1 previews.

Cheers,
Matt


Ok that makes sense. I shoot d800/810 with an 850 on its way as well. So I fully understand the resolution size constraints. They are huge files, even when I drop it down to 12-Bit vs 14-bit.

But I guess that actually makes sense, unless your going to go and zoom in while in the library tab, there is no need for 1:1, IIRCA I read somewhere that double your screens resolution could be helpful so you can at least peak in 2x in library view before going to develop and really start working on the image. I may adjust to something like that.

That being said, my monitor is 3440x1440, but still, that is not even close to the res of my images. So you make a valid point there. And with my 1:1 previews my Lightroom folder is ~170 GB. With all my video games installed and a 10% over-partition on my SSD, room is getting scarce. If I do a smaller preview that number would drop drastically!


Sent from my iPhone using Tapatalk
 

JamesNisly

Dabbler
Joined
Nov 21, 2017
Messages
37
Is anybody here using a combo of lightroom with images stored on a NAS in a multi user environment?
 
Joined
Feb 2, 2016
Messages
574
lightroom with images stored on a NAS in a multi user environment?

{raises hand}

Images on NAS. Each workstation has its own Lightroom catalog stored local to the workstation. People import image directories as needed. No one catalog has all the images. Mostly people just load what they need to keep performance snappy.

Cheers,
Matt
 

JamesNisly

Dabbler
Joined
Nov 21, 2017
Messages
37
Ok, we do exactly the same. Had a glimmer of hope that there was a better way out there but I don't think there is. The fact that you can't store a catalog on a networked volume and that it doesn't support multiple concurrent users pretty much makes all of the cataloging and organizing functions in Lightroom useless for any studio with more than one person.
 
Joined
Feb 2, 2016
Messages
574
makes all of the cataloging and organizing functions in Lightroom useless for any studio with more than one person.

That's a feature, not a bug. Our users don't want to see the entire image archive. They just want to see the parts they need. Instead of loading more than four million images going back eight years, they load the images they need, tens of thousand at most. Even segmenting by year doesn't really help in our use case.

Lightroom is a single-user production tool not a shared-access digital asset management system. While many people treat it as a DAM, that only works when you have one person with a master catalog who copies that shared catalog to a handful of users who accept that their copy is often out of date or diverged from the master catalog.

For a true digital asset management system, check out MerlinOne. I installed and maintained one of their systems for a major market newspaper and was really impressed.

Cheers,
Matt
 

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
Just want to say my solution had worked out well. And I am currently syncing the catalog from the old main photo editing PC I used at my parents house with my server, which will then sync with my PC, so both computers (in the only one who ever edits, so still only one user who can’t ever be in the file at the same time) will be synced via syncthing. The entire photo dataset as well, which is only about 1 TB anyways, not like I take THAT MANY pictures compared to you guys.


Sent from my iPhone using Tapatalk
 

dreamerns

Dabbler
Joined
May 1, 2015
Messages
48
When it comes to 1:1 previews, you can set how long they will last in Lightroom. I usually set that for 1 week. After that they get deleted and standard previews remain. In case I don't get to edit them in a week, I just rebuild 1:1 previews later when I have time to edit (you can do that even if pictures are on NAS). I have very similar workflow as described above, but I don't move pictures from local to NAS using lightroom, I have Resilio sync do it for me in background. Also, as soon as pictures are on FreeNAS server I use another app to backup in cloud. But general rule is 1. have them locally until you finish editing, move to FreeNAS, keep catalog and smart previews locally.
 

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
Yea I have started using my setup, and I just keep 1:1’s forever (its sub 200GB of data), and import my images to my server directly. From what I have seen (it’s possible the new Lightroom is better and I’m missing out on the improvements) but it seems to me that Lightroom responds as quickly to scrubbing through timeline and loading pix in preview and develops modules just as fast weather it’s on my NAS or on local SSD. How that’s even possible, I have no idea. I chalk it up to Lightroom’s file handling to be very poor.

So because of that, I just import to NAS and create 1:1’s upon import.

But it has seemed to work well thus far. I also use syncthing with a Windows VM on my nas, syncthing on my parents PC and on my main rig in my apartment with two folder syncs. One handles syncing raw pix between a 4tb drive in parents PC with my NAS, and the other syncs Lightroom folders between parents PC and NAS and my main Rig. I use the NAS as an intermediary for the Lightroom sync since it’s always on, thus it will always end up with the latest Lightroom catalog data before I end up working on the other editing machine.


Sent from my iPhone using Tapatalk
 

Evan Richardson

Explorer
Joined
Dec 11, 2015
Messages
76
Late to the party here, but I can confirm this works with LR and Capture one pro

I have 12x WD 8TB reds in 2x6 Raid-z2 configuration
Supermicro X10SDV-2C-TP4F Motherboard
64GB DDR4 ECC RAM
2x Intel 3510 SSDs for boot

I don't save directly to the NAS though, as it sits in my garage. Here's my workflow:

1.) Using Lexar HR1 Workflow 4bay card reader with SDXC and XQD readers, I first download images into Capture One (I used to use Light room, but switched and haven't looked back). These images are saved locally on my image editing machine.
2.) Using Syncthing running on my image machine, plus as a docker container on another server in my rack, I have it sync my media folder directly to the NAS. This way any changes I make locally on my editing machine are directly synced to the NAS by way of Syncthing.
3.) using FreeNAS 11's Backblaze integration, I offload every 6 hours to BackBlaze.
 

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
Incase anyone happens across this thread in the future, my setup has worked GREAT for my needs. Just to summarize my workflow:

Editing PC 1 is in City A
Editing PC 2 is in City B
FreeNAS is in City B

Ingest images to either Editing PC depending where I am. If I am in City A, they go directly to a local 4TB drive where they stay. If in City B, they go directly to NAS. Regardless, 1:1 and smart previews are built. Both have some file structure so Lightroom catalog can find the images, City A editing PC is via local 4TB HD, City B editing PC is a mounted network location. Lightroom doesn’t care, at all, as long as structure and drive letter are the same.

On either machine, all Lightroom data is stored on local SSD.

City A editing PC has a syncthing sync for my /photography directory with all my raw images and a sync for /Lightroom data for all Lightroom data. This goes to my FreeNAS box which is on 24/7/365. City B editing PC has only the /Lightroom sync. This way my FreeNAS box always has the most current raw images and Lightroom data regardless of where I am working, and pushes updates to whichever editing PC is lacking the newest data.

Lightroom performance seems bang on in either situation. I am sure having 1:1 and smart previews locally on SSD makes this setup feel like it isn’t network attached or on a spinning HDD.


Sent from my iPhone using Tapatalk
 

guermantes

Patron
Joined
Sep 27, 2017
Messages
213
EDIT: Actually, I found the below is not entirely true. When re-linking the source files after having moved them to the NAS, something breaks in relation to the XMP sidecars. One has to select all files and then do right-click "Save XMP to file" in order to activate the link the XMP file and have edits be writen to them again. LR catalogue is of course, the primary location for where edits are stored, quire irrespective of whether I am writing out to XMP or not.

Does anyone have experience with Lightroom XMP sidecar files in the FreeNAS context?

When my raw files were on my desktop HDD, I ticked "save XMP data to sidecar file" as a second backup in case I wanted to move away from LR or something else unforseeable. When I moved my raw source files to my FreeNAS I discovered (or actually I discovered today, five months later) that the XMP files were not updated anymore when I made edits on old or newly imported files.

LR does not seem to mind, but I am wondering what purpose my XMP files serve at this moment in time. I am also curious as to why LR can't update the XMP files when it can write a backup of the catalog to the same share.
 
Last edited:
Status
Not open for further replies.
Top