FreeNAS with SuperMicro JBOD for Security Cameras SSD?

[07stuntar1]

I am currently using a
Dell R720 with 256GB Ram
(Write Drives) I combine 4 x ( raidz2 with 4 wide 14TB UltraStar 7200RPM drives.) If I did the math correctly it's about 255 MB/Sec per pool x 4.
2 10gb nic Lagg.
(Backup Drives) I combine 4 x (raidz2 with 11 wide 14TB UltraStar 7200RPM drives.)


I am looking to record about 200 IP cameras at 5-10 mbps per camera. I am worried about the iops. Will adding an SSD or NMVE Slog to the write or backup drives help with security cameras?
I am also worried about losing a pool. What if the front half of the jbod loses power or connection will this distory the pool if half the drives are located in the back?
How do I keep the DA numbers the same? Seems like the order gets mixed up.

 

[jgreco]

A SLOG is not a write cache or any sort of help.

Some insights into SLOG/ZIL with ZFS on FreeNAS

What is the ZIL? POSIX provides a facility for the system or an application to make sure that data requested to be written is actually committed to stable storage: a synchronous write request. Upon completion of a sync write request, the underlying filesystem is supposed to guarantee that a...

www.ixsystems.com

The best way to achieve IOPS is with mirror vdevs. For random IOPS this will give you very good throughput. Also be sure to keep sufficient free space on your pool.

 

[07stuntar1]

A SLOG is not a write cache or any sort of help.

Some insights into SLOG/ZIL with ZFS on FreeNAS

What is the ZIL? POSIX provides a facility for the system or an application to make sure that data requested to be written is actually committed to stable storage: a synchronous write request. Upon completion of a sync write request, the underlying filesystem is supposed to guarantee that a...

www.ixsystems.com

The best way to achieve IOPS is with mirror vdevs. For random IOPS this will give you very good throughput. Also be sure to keep sufficient free space on your pool.

I understand that. But with amount of storage I need mirroring cost to much.

 

[jgreco]

I understand that. But with amount of storage I need mirroring cost to much.

Not sure what you want here.

If you buy 200 IP cameras, presumably at least $50-$100 each, and get them installed, and consider the costs of running all the cabling and switchgear, you've probably invested $25K-$50K. If you wanted to be able to archive that footage, you should probably have planned on similar expenses for storage.

Most commercial NVR's support around 8 to 16 cameras on a single HDD. So what I did here to help establish a budget for you is to run over to NewEgg and looked at what was offered to end users. It seems that you can get an NVR with up to 128 channel support for somewhere between $12,000-$37,000. So you'd need two of those. The low end of that, the HikVision Blazer Pro, does not come with drives, and is limited to 8TB drives, so a more realistic low end is probably $4000 more to fill that with 8TB purple drives. So $16,000-$37,000 to support 128 cameras, or for your deployment, $32,000-$74,000 in costs for NVR and storage.

This stuff isn't magic. A RAIDZ vdev is really only capable of delivering a modest number of IOPS. It is designed for sequential archival storage. The kind of IOPS you're looking for is something ZFS does best with mirrors, or maybe with dozens of RAIDZ vdevs.

 

[07stuntar1]

Not sure what you want here.

If you buy 200 IP cameras, presumably at least $50-$100 each, and get them installed, and consider the costs of running all the cabling and switchgear, you've probably invested $25K-$50K. If you wanted to be able to archive that footage, you should probably have planned on similar expenses for storage.

Most commercial NVR's support around 8 to 16 cameras on a single HDD. So what I did here to help establish a budget for you is to run over to NewEgg and looked at what was offered to end users. It seems that you can get an NVR with up to 128 channel support for somewhere between $12,000-$37,000. So you'd need two of those. The low end of that, the HikVision Blazer Pro, does not come with drives, and is limited to 8TB drives, so a more realistic low end is probably $4000 more to fill that with 8TB purple drives. So $16,000-$37,000 to support 128 cameras, or for your deployment, $32,000-$74,000 in costs for NVR and storage.

This stuff isn't magic. A RAIDZ vdev is really only capable of delivering a modest number of IOPS. It is designed for sequential archival storage. The kind of IOPS you're looking for is something ZFS does best with mirrors, or maybe with dozens of RAIDZ vdevs.

Thank You for your explanation.
When I did the calculations I used 250 cameras at 10mbps. I calculated about 312.5MB/s. With 4 x 255 MB/s= 1020 MB/s i figured that was enough.
Am I doing the math wrong? So in this case a fast NMVE write cache will not help?

250 Streams
1 Camera 100GB Day = 25TB Day

10mbps =1.25 MB/S
2500mbps=312.5MB/s

 

[jgreco]

I'm sure a fast NVMe write cache would help. However, ZFS has no write cache capability other than the in-memory transaction group cache, so you'd be looking somewhere else for your storage solution, and a write cache would be limited by the underlying hard drives once filled in any case. In certain cases, smart write caches can aggregate discrete writes localized within an area, but ZFS being a CoW filesystem kinda does that to the maximal extent possible anyways, which is why you can get very good write performance if you keep your pool fill rates well below 50%. A NVR solution is definitely going to suffer fragmentation over time which will kill performance if you don't treat it similarly to how we treat databases or block storage.

The path to success for block storage

It seems like I haven't written a sticky for awhile, but just in the last week I've had to cover this topic several times. ZFS does two different things very well. One is storage of large sequentially-written files, such as archives, logs, or data files, where the file does not have the middle...

www.ixsystems.com

My first answer is still correct.