BUILD New build - C2750D4I, to USB 3.0?

[TrevorX]

Hello,

First time with everything FreeBSD/FreeNAS etc - I'm a Microsoft-based architect and consultant, so I'm pretty good with everything Windows and Hyper-V, but I need a new reliable storage system at home and REFS is just too immature at this point, while FreeNAS seems to have come along in leaps and bounds the last five years (I've been reading sporadically from the sidelines).

My existing home workstation runs 8 disks in RAID6 on an Areca controller, but it wigged out one day last year and I got about 5% corruption (not sure of cause, chkdsk just started prompting out of the blue, I noticed files went missing, several passes of chkdsk restored some of them but a whole lot just evaporated). My work stuff is all well backed up, but there are too many movies on there for me to have a comprehensive backup so some of that got damaged and lost. Considering the expense and effort I went to to design a system that should be impervious to that (the Areca card even does redundancy scrubbing checks against parity data, similar to ZFS), I felt a bit cheated.

So now I want to separate my computer from the data, like I would with any work environment. I want the file server to do just that - while streaming media is useful, I'm not doing anything stupid with it like domain management or VM hosting - other systems can handle those tasks without complicating (and possibly compromising) the task of data storage.

Sorry for the background - I'll cut to the chase. Here's my build. I've skimmed through the FreeNAS manual (and read several sections properly) and I'm feeling fairly confident I'm across the basics, but in my experience it never hurts to ask stupid questions of people who are experts. So if you wouldn't mind, I'd love to have my specs critiqued, possible issues pointed out and one specific question answered.

That question is USB 3.0 external drive(s) for backup. I realise ideally I'd build two identical FreeNAS boxes and have them replicate to each other, but that's not within my budget. To be honest, a full backup isn't within the budget either; I have to accept that total loss of the box will lose movies - blu-ray discs are just too large to keep two uncompressed copies of everything. So that's ok - what I'd like to do is have the box do regular backups of everything else to something like external USB 3.0 drives, so I know that total loss of the box will be survivable. So are external USB 3.0 drives a good solution and is my idea feasible, or should I be looking at other backup options?

Here's the hardware list:
Asrock C2750D4I
Lian-Li PC-Q35B
Kingston KVR16LE11-8I DDR3 ECC RAM x4 (32GB)
Corsair CS450M PSU
Hitachi Deskstar NAS 6GB drives x6 (RAIDZ2)
5-to-3 3.5" to 5.25" SATA hot-swap bay
3-to-2 3.5" to 5.25" SATA hot-swap bay
Kingston DT101G2 and Sandisk Cruzer Edge 32GB flash drives for FreeNAS OS

Please feel free to point out anything I've missed/haven't thought of/got wrong

 

[Ericloewe]

I'd avoid Kingston RAM in favor of something else from ASRock's QVL. We've had bad experiences with Kingston's shady business practices. The more popular RAM for the ASRock C2x50s is probably Crucial, due to its good availability.

I'd also go with a better PSU. The Seasonic G-450 is significantly better and shouldn't cost too much more.

 

[Robert Trevellyan]

That question is USB 3.0 external drive(s) for backup.

For your partial backups, have you considered combining something like this with something like this? It's not quite as elegant as an external drive, but it's more flexible. For example, you can use a mirror for the backup. I've used exactly this setup when I wanted to reconfigure my pool - replicate the datasets I cared about to a mirror, destroy/recreate and replicate back.

Edit: plus something like this to keep the drives cool.

I'm pretty sure USB3 on FreeNAS isn't considered reliable yet, so that route may not be a good idea.

 

[Fraoch]

Looks like you've read and thought this through thoroughly.

Kingston DT101G2 and Sandisk Cruzer Edge 32GB flash drives for FreeNAS OS

Even this - have you chosen two different flash drives so that they won't both succumb to the same issue?

Unfortunately this may prove to be problematic. In order to mirror the boot drive, the amount of bytes on each device must be exactly the same. And you'll find that 32 GB from one manufacturer is not (exactly) equal to 32 GB from another manufacturer. My boot devices are two 64 GB SSDs, yet:

Code:

64023257088    # mediasize in bytes (59G)

and

Code:

63023063040    # mediasize in bytes (58G)

So I have to install FreeNAS to the smaller device first, then mirror the larger device, never the other way around or I get an "out of space" error.

It's best to get two identical devices.

 

[TrevorX]

Thanks very much for taking the time to reply :)

We've had bad experiences with Kingston's shady business practices.

Can you please elaborate on that?

I've used Corsair & Crucial memory as a preference for years, but my suppliers no longer stock either in ECC anymore. I probably should have pointed out I'm in Australia, so availability of components is rather different (and the exchange rate has gone to hell over the past 18 months).

I'll take a look around and see if we still have local channel supply of Crucial anywhere.

I'd also go with a better PSU. The Seasonic G-450 is significantly better and shouldn't cost too much more.

Do you have any references for why the Corsair unit is significantly worse? Seasonic is not a brand I have experience with, but Corsair are hands down the most reliable consumer PSU brand I've used. I realise they use different OEMs for different lines though, so if there's a problem with that category I'd be very interested to hear about it :)

 

[TrevorX]

For your partial backups, have you considered combining something like this with something like this? It's not quite as elegant as an external drive, but it's more flexible. For example, you can use a mirror for the backup. I've used exactly this setup when I wanted to reconfigure my pool - replicate the datasets I cared about to a mirror, destroy/recreate and replicate back.

Edit: plus something like this to keep the drives cool.

I'm pretty sure USB3 on FreeNAS isn't considered reliable yet, so that route may not be a good idea.

Hmm eSATA is doable. I didn't realise USB 3.0 wasn't considered reliable. I would use a proper eSATA enclosure rather than bare drives - that's not something I'd be comfortable with as a permanent solution. I'll have to do some more reading.

Quick question - are automated partial backups not something readily supported/available?

 

[TrevorX]

Looks like you've read and thought this through thoroughly.

Even this - have you chosen two different flash drives so that they won't both succumb to the same issue?

...

It's best to get two identical devices.

Thanks for that. Yes, that's precisely why I'd chosen them - normally I would match devices for any sort of mirror, but then I started thinking about how unreliable flash drives can be unless you pay top dollar and I thought I don't want to get caught out having two drives fail because I've unknowingly selected a bad model (no one reviews low end USB 2.0 drives so it's very difficult to know without direct experience). SSDs are massive overkill. Maybe I'll buy one, put it through its paces on a bench test for a few weeks, then purchase a second if it holds up OK.

 

[Ericloewe]

For starters, the Seasonics' build quality is leaps and bounds ahead of the Corsair CX/CS - all Japanese caps instead of second-rate stuff like Teapo, a much better fan... Output quality is also significantly better.
HardOCP and JonnyGuru have informative reviews of Seasonic G-Series and Corsair CS models.

 

[Fraoch]

Thanks for that. Yes, that's precisely why I'd chosen them - normally I would match devices for any sort of mirror, but then I started thinking about how unreliable flash drives can be unless you pay top dollar and I thought I don't want to get caught out having two drives fail because I've unknowingly selected a bad model (no one reviews low end USB 2.0 drives so it's very difficult to know without direct experience). SSDs are massive overkill. Maybe I'll buy one, put it through its paces on a bench test for a few weeks, then purchase a second if it holds up OK.

iXsystems officially recommend SATA DOMs - they're $80 around here. The forum goes with USB drives - a good 32 GB one (Corsair) is on sale for $25 this week - regular price $45. Given that low-end 64 GB SSDs can be found for $50, that seemed like a no-brainer for me - they're faster than SATA DOMs, cheaper than SATA DOMs and way more reliable than USB drives. Getting two at once would have been somewhat costly though, but I had an old Crucial M4 sitting around doing nothing, so I started with that and added a base model SanDisk later when 9.3 was introduced with its boot mirroring feature.

BTW SMART reports on my M4:

Code:

Power_On_Hours 19150

:D

 

[Ericloewe]

BTW SMART reports on my M4:

Code:

Power_On_Hours 19150

:D

You call that a knife? This is a knife:

 

[Fraoch]

:)

Now a USB stick that has that many continuous hours on it, that would be something. I have USB drives older than my M4 but they only get a few hours use a month.

 

[Tywin]

You call that a knife? This is a knife:

Bah! I thought I had you, but my M4 is only at 21084 :P

 

[JoeVulture]

Hmm eSATA is doable. I didn't realise USB 3.0 wasn't considered reliable. I would use a proper eSATA enclosure rather than bare drives - that's not something I'd be comfortable with as a permanent solution. I'll have to do some more reading.

Quick question - are automated partial backups not something readily supported/available?

FreeNAS doesn't have the capability built-in, but you can use rsync from a script to do the backups. I have a script that does nightly incremental backs to a pair of eSATA drives (hard drives in an eSATA dock) and I rotate the drives weekly.

I should upload the script here and document it (and thus improve it in the process), but you can read up on it and grab it from here for now.

-- Joe

 

[TrevorX]

For starters, the Seasonics' build quality is leaps and bounds ahead of the Corsair CX/CS - all Japanese caps instead of second-rate stuff like Teapo, a much better fan... Output quality is also significantly better.
HardOCP and JonnyGuru have informative reviews of Seasonic G-Series and Corsair CS models.

Thanks Eric, appreciate it I'll do some further reading.

 

[TrevorX]

iXsystems officially recommend SATA DOMs - they're $80 around here. The forum goes with USB drives - a good 32 GB one (Corsair) is on sale for $25 this week - regular price $45. Given that low-end 64 GB SSDs can be found for $50, that seemed like a no-brainer for me - they're faster than SATA DOMs, cheaper than SATA DOMs and way more reliable than USB drives. Getting two at once would have been somewhat costly though, but I had an old Crucial M4 sitting around doing nothing, so I started with that and added a base model SanDisk later when 9.3 was introduced with its boot mirroring feature.

BTW SMART reports on my M4:

Code:

Power_On_Hours 19150

:D

Thanks Fraoch Yeah I originally only considered SSDs, but read so many recommendations to use USB Flash because the OS gets loaded into memory and then the flash drive is ignored that I figure it will probably only get accessed three or four times a year (unless patching requires more frequent reboots?). It seems pointless going to SSDs when they won't be used for caching or anything. I'm all for spending what it takes to do a job properly, but my initial look at SSD pricing suggested a single drive would be three times the cost of two new USB flash drives, for highly questionable benefits.

Do you have a different take on this?

 

[TrevorX]

T

FreeNAS doesn't have the capability built-in, but you can use rsync from a script to do the backups. I have a script that does nightly incremental backs to a pair of eSATA drives (hard drives in an eSATA dock) and I rotate the drives weekly.

I should upload the script here and document it (and thus improve it in the process), but you can read up on it and grab it from here for now.

-- Joe

Thanks Joe, appreciate the suggestion. I've used rsync in the past so I'm sure I'll be able to put a workable script together using your pointers. I'll hit you up if I run into issues ;-)

 

[Fraoch]

Thanks Fraoch Yeah I originally only considered SSDs, but read so many recommendations to use USB Flash because the OS gets loaded into memory and then the flash drive is ignored that I figure it will probably only get accessed three or four times a year (unless patching requires more frequent reboots?). It seems pointless going to SSDs when they won't be used for caching or anything. I'm all for spending what it takes to do a job properly, but my initial look at SSD pricing suggested a single drive would be three times the cost of two new USB flash drives, for highly questionable benefits.

Do you have a different take on this?

If you stayed with the same 9.3 build then yes, an SSD would be wasted. You're correct, FreeNAS runs from RAM so the OS drive is only accessed for updates and startup.

However 9.3 is very actively developed, there are at least 2 updates a week. This week was unusual, there have been 4 updates in the past 4 days so there has been a lot of updating and restarting.

My decision was easy because I already had a surplus SSD. The only noticeable benefit is the increase in reliability, I see quite a few reports of flash drives failing in the forum. If I was starting from scratch though, getting two of them simultaneously would have been a significant premium over 16 GB flash drives and a small premium over 32 GB flash drives. And apart from the reliability and higher speed updating and restarting, there are no benefits to a well-running FreeNAS.

 

[Robert Trevellyan]

Quick question - are automated partial backups not something readily supported/available?

If you have scheduled snapshots configured you can use a Replication task with localhost as the destination.