BUILD Designing a server for cold storage, low availability?

[anodos]

There's no good way to recover from the latter case.

Restore from backup? :p

 

[Ericloewe]

Restore from backup? :p

I said good.

Your day is already not a good one if you have to reach for the backups.

 

[Bidule0hm]

Your day is not a good one if you don't have to reach for the backups.

Fix'd :)

 

[Robert Trevellyan]

Your day is already not a good one if you have to reach for the backups.

Reminds me of this. "If it starts pointing toward space you are having a bad problem and you will not go to space today."

 

[star]

And, as it has been already said, nothing is stopping you from continuing using SyncToy, if that is what you like.

When you use NAS like this, you have to realize that if a file gets corrupted (or cryptolocked) on your workstation, the next synchronization may override a good file on NAS with a corrupted file from the local computer.

One way to soften the blow is to configure automatic snapshots on the FreeNAS, and save those snapshots for long time [forever maybe?]. If you notice a corrupted file, and you still have the snapshots dating back to when the file was good, you can restore the file from the snapshot.

So maybe 3 mirrored drives and 1 drive for snapshots?

Are snapshots 1:1 in size?

Just to add after reading through the thread...

1. WD RED NAS drives would be a better drive then the HGST drives for your application. The REDs require less power, run quieter and also cooler then the HGST.
2. Also, either add a second SSD drive OR use 2x thumb drives like the Cruzer FIT 32G and mirror them!

https://www.backblaze.com/blog/best-hard-drive/

Please don't hate me for anecdotal evidence, but I've experienced a high failure rate with WD _____ Drives, and so has Backblaze. That said, HGST is a WD company :P

 

[Robert Trevellyan]

Are snapshots 1:1 in size?

A snapshot only takes up as much space as the changes that have been made to the filesystem since the snapshot occurred. I'm going to politely suggest that you do some reading up on ZFS.

https://www.backblaze.com/blog/best-hard-drive/

I like how open BackBlaze is as a company, and the way they share so much data and their hardware designs and even some of their source code. I appreciate that they liberally sprinkle their blog posts about drive failure rates with "in our environment" and similar disclaimers. But I dislike their use of SEO-gimmick titles like "What is the Best Hard Drive" and slugs like "best-hard-drive".

If you look behind the BackBlaze data, and the oft-quoted Google study of hard drive failures, you'll find the same circular definition of hard drive failure: "A hard drive is considered 'failed' because it was replaced."
Why did you replace that hard drive?
Because it failed.
How do you know it failed?
Because we replaced it.
etc.

 

[star]

A snapshot only takes up as much space as the changes that have been made to the filesystem since the snapshot occurred. I'm going to politely suggest that you do some reading up on ZFS.

Are there any documents you can recommend for my purposes?

I like how open BackBlaze is as a company, and the way they share so much data and their hardware designs and even some of their source code. I appreciate that they liberally sprinkle their blog posts about drive failure rates with "in our environment" and similar disclaimers. But I dislike their use of SEO-gimmick titles like "What is the Best Hard Drive" and slugs like "best-hard-drive".

If you look behind the BackBlaze data, and the oft-quoted Google study of hard drive failures, you'll find the same circular definition of hard drive failure: "A hard drive is considered 'failed' because it was replaced."
Why did you replace that hard drive?
Because it failed.
How do you know it failed?
Because we replaced it.
etc.

I was disappointed that there was no readily available explanation on BackBlaze's website of what exactly constituted a drive failure. And yes, 'what is the best hard drive' is a dumb name for that page. Regardless, the data is more useful to me than forum recommendations and user reviews. Are there any other comparable data sets about current hard drive failure rates?

 

[Robert Trevellyan]

Are there any documents you can recommend for my purposes?

I would start with wikipedia and go from there.

I was disappointed that there was no readily available explanation on BackBlaze's website of what exactly constituted a drive failure.

Quoting directly from an earlier blog post: "A failure is when we have to replace a drive in a pod."

Quoting directly from the Google study: "... the most accurate definition we can present of a failure event for our study is: a drive is considered to have failed if it was replaced as part of a repairs procedure."

 

[star]

Alright, here's what I wound up going with:

• Motherboard/CPU: ASRock C2550D4I
• Disks: 3 x TOSHIBA PH3300U-1I72 3TB
  • OS: 80GB Kingston SSD
• Memory: Crucial 16GB (2 x 8GB) DDR3 ECC Unbuffered 1600 (PC3 12800) CT2KIT102472BD160B
• Power: SeaSonic S12II 430B 430W
• Chassis: Rosewill RSV-L4411

The question that is on my mind right now... is whether or not I should .zip/archive the files I know will never change. Is this worth doing?

 

[Bidule0hm]

Just leave the default compression enabled on the whole volume ;)

 

[Robert Smith]

And go with 4TB drives. I think they came down in price sufficiently. There is no such thing as too much storage, and with ZFS you do want to overprovision.

 

[star]

Just leave the default compression enabled on the whole volume ;)

What does the winky face add to this post, other than more confusion for me? I'm not sure what you mean by the default compression ;)

And go with 4TB drives. I think they came down in price sufficiently. There is no such thing as too much storage, and with ZFS you do want to overprovision.

Too late ;)


Am I a guru yet? ;)

 

[Robert Smith]

With a triple-mirror you will have 3TB before any overhead. You wanted 2.5TB of useful space, so it is cutting close.

Pictures and videos in already compressed formats do not additionally compress well.

Also, I am not sure on the power consumption of those 7200 RPM dives. There are other drives, such as WD-Red and Seagate-NAS, which spin a little slower and should consume less electricity.

 

[star]

With a triple-mirror you will have 3TB before any overhead. You wanted 2.5TB of useful space, so it is cutting close.

Pictures and videos in already compressed formats do not additionally compress well.

Also, I am not sure on the power consumption of those 7200 RPM dives. There are other drives, such as WD-Red and Seagate-NAS, which spin a little slower and should consume less electricity.

Can you even do triple mirrors in FreeNAS? Can three vdevs be mirrored to cross-reference 2 disks?

And I understand now what Bidule0hm meant by 'original compression.' I'm not worried about saving space or anything, I just wonder if sticking the files in an archive would potentially preserve them better?

 

[Robert Smith]

Can you even do triple mirrors in FreeNAS? Can three vdevs be mirrored to cross-reference 2 disks?

https://forums.freenas.org/index.ph...mirror-to-3-mirror-on-encrypted-volume.22562/
I guess you can buy one more disk and go RAIDZ2, if the triple-mirror thing does not pan out...

And I understand now what Bidule0hm meant by 'original compression.' I'm not worried about saving space or anything, I just wonder if sticking the files in an archive would potentially preserve them better?

Sounds like a workflow nightmare to me. Instead of compressing you may want to look into tools that let you checksum your files and keep that information for checking the checksums later...

 

[Bidule0hm]

FreeNAS has lz4 compression enabled by default, the overhead when a file is already compressed is very small because it'll not be compressed again that's why it's recommended to let it enabled and why it is enabled by default.

Now, the purpose is to save space but you just said you don't care about that so I'll answer to other question: no, the files won't be better preserved by using compression/archive algos like zip/tar/gzip/lz4/whatever, the purpose of those is to either save space or group a ton of files in a single file for convenience, or both.

 

[cyberjock]

Why is *anyone* talking about compressing the files into .zips or storing checksums. ZFS not only checksums the blocks the files are stored in, but the can compress the blocks too. So why would anyone want to do anything manually. Just set your compression to whatever you want and copy the files over as-is!

 

[Robert Smith]

Why is *anyone* talking about compressing the files into .zips or storing checksums. ZFS not only checksums the blocks the files are stored in, but the can compress the blocks too. So why would anyone want to do anything manually. Just set your compression to whatever you want and copy the files over as-is!

The OP does not have the benefit of ZFS on his local workstation where the master copies of the files are kept.

If he can break out of this, and designate his NAS stored files as the master copy, then yes, forget about extra checksumming.