how to build a backup server with FreeNAS?

[Jailer]

I don't understand why RAID is recommend for Data backup

You're missing the point. RAID is not a backup and no one here has suggested it is. It's is for redundancy and data protection for WHEN a hard drive fails. If you care about your data then redundancy plus a good backup solution should be part of your overall build plan.

 

[depasseg]

I don't understand why RAID is recommend for Data backup
why not using that harddisk for snaphosts storage instead of RAID

You are really confusing me.

For me, I use RAID on my backup location for 2 reasons:
1) I have more data than will fit on a single drive, so I have to use RAID of some sort
2) I use RAID-Z2 on my backup location (and ZFS) so that I can use ZFS snapshots and ZFS replication and so that my backup data doesn't get silently corrupted.

RAID-Z and backups are not mutually exclusive. They work together very well.

 

[danb35]

Reverting to snapshots is very reliable, though they're time-based rather than true versioning. But, again, you're drawing a false dichotomy between RAID and versioning/snapshots. The best solution, IMO, is to do both.

ZFS is intended to pool storage together, so that, for example, your 3 x 4 TB disks could be very easily combined into a single 12 TB volume. But you then have no redundancy, and when any one of those disks fails, you lose all your data. Even if the data on your server is only backups, you normally don't want that to happen. That's one reason to add redundancy.

The other reason is that ZFS will checksum all your data, and can therefore detect if there's been any data corruption on disk. If there's redundancy present (e.g., a RAIDZ array), ZFS can also repair that corruption on the fly. Without redundancy, you're stuck with corrupt data.

 

[yashiharu]

You're missing the point. RAID is not a backup and no one here has suggested it is. It's is for redundancy and data protection for WHEN a hard drive fails. If you care about your data then redundancy plus a good backup solution should be part of your overall build plan.

let's talk with example: 3TB x3 harddisks (or 30TB LVM / zpool)
A) 3TB for Data storage + 3TB as RAID 1 + 3TB remote or backup snapshots
B) 3TB for Data storage + 3TB local snapshots + 3TB remote backup snapshots

Why would it be better to choose A) over B)?

 

[depasseg]

You need to define what your understanding of "data storage", "RAID 1", "remote or backup snapshots" and "local snapshots" all mean. Except for whatever you mean by RAID 1 (which doesn't exist in FreeNAS), you can have RAIDZ supporting all of the other items. And LVM is out the window too.

Just so we are clear. When you read about using RAID in FreeNAS, it does not mean hardware RAID. It means FreeNAS controlled Software RAID-Z1, Z2, Z3 or mirrors (or a single disk).

 

[depasseg]

Have you read the intro to FreeNAS and ZFS presentation? I think it would help clear up some things.

 

[yashiharu]

Have you read the intro to FreeNAS and ZFS presentation? I think it would help clear up some things.

yes i did.

RAIDZ is benefit on "so-called checksum/ auto-healing" AND saving down time

for home user
as restoration from snapshots is not that difficult
down time is not that important like those web servers in DC

If i do have budget, i should go for RAIDZ + snapshot backup over 2 snapshot backup
else, goes for snapshot backup over RAIDZ, right?

 

[depasseg]

If you read it, you didn't understand it. Snapshots don't exist on a separate drive. They are on the same drive(s) as your data.

simple example: running ZFS on a single Drive taking snapshots every hour. If that drive dies, all data is gone. If you add a second drive and make it a different dataset, then you can replicate the snapshots from the first dataset to the second. So if your first drive fails, you still have data, but it's in a different dataset name, so it's not going to work the same. Alternatively, if you had those 2 drives in a Mirrored pool, the Data and the snapshots would exist on both drives, so if one died, the other would keep serving data as if nothing happened. You could then replace the failed drive and not have any impact.

If i do have budget, i should go for RAIDZ + snapshot backup over 2 snapshot backup
else, goes for snapshot backup over RAIDZ, right?

Can you provide your proposed system options in terms of drives in each pool/dataset? I'm confused.

For example:
Option A) - Main dataset - 6x3TB drives in RAID-Z2+ Backup Dataset of 6x3TB drives in RAID Z1
Option B) - Main dataset - 12x3TB drives in RAID-Z3

 

[yashiharu]

If you read it, you didn't understand it. Snapshots don't exist on a separate drive. They are on the same drive(s) as your data.

oh, i am wrong about snapshot
I thought snapshot can be storage on the other hd at the first place (like a compressed DD)
sorry, i have to understand what solutions are available and the reasons behind it. I know i am annoying.


6TB data storage as source

3TB hdd x4 on hands (next upgrade can be 2 more 3TB)
it's going to be a backup server (not source)

what do you recommend?

 

[depasseg]

Recommendation for what? It looks like you are building a backup system to be a replication target for your 6TB of data? Is your 6TB data currently running on FreeNAS? What are you data protection concerns?

 

[yashiharu]

Recommendation for what? It looks like you are building a backup system to be a replication target for your 6TB of data? Is your 6TB data currently running on FreeNAS? What are you data protection concerns?

yes, a backup system, copies from original.
i like FreeNAS monitoring GUI and ZFS

RAIDZ1 or just cron-DD ...?

 

[depasseg]

RAIDZ1 or just cron-DD ...?

This is like asking: "should I have lunch or walk to work". They are completely different topics.

Use RAID Z2 for your backup pool, setup 4 hour snapshots on your source data, create a replication job from your source to destination and you are all set.

 

[pirateghost]

yes, a backup system, copies from original.
i like FreeNAS monitoring GUI and ZFS

RAIDZ1 or just cron-DD ...?

neither. raidz1 should not be used. DD should not be used.

You appear to be only giving us bits and pieces. What is the end goal? What are you trying to accomplish?

 

[yashiharu]

This is like asking: "should I have lunch or walk to work". They are completely different topics.

Use RAID Z2 for your backup pool, setup 4 hour snapshots on your source data, create a replication job from your source to destination and you are all set.

ok thanks. sound good

RAIDZ2 required 5 disks
and i had to add 1 more disks for snapshots replication

 

[pirateghost]

ok thanks. sound good

RAIDZ2 required 5 disks
and i had to add 1 more disks for snapshots replication

technically you can do RAIDZ2 with 4 disks, but at that point it would be more beneficial to just use 2 mirrors instead(in my opinion)

1 more disk for snapshot replication? what does that mean?

 

[yashiharu]

neither. raidz1 should not be used. DD should not be used.

You appear to be only giving us bits and pieces. What is the end goal? What are you trying to accomplish?

1) a backup server stores full and/or incremental backups
2) a gui to monitoring everything on backup server
3) 6TB data sources from devices
4) a new server with 4 3TB harddisk (max. 6)

i think FreeNAS is good for both.
and depasseg explain a lot and his suggestion seems good enough for me.

 

[yashiharu]

technically you can do RAIDZ2 with 4 disks, but at that point it would be more beneficial to just use 2 mirrors instead(in my opinion)

1 more disk for snapshot replication? what does that mean?

he suggested: " create a replication job from your source to destination"
does it mean, copy the snapshots from RAIDZ2 to another harddisk as backup?

 

[depasseg]

Until you re-read the newbie guide and post your system specs, I can not help you. I'm sorry.

 

[Robert Trevellyan]

RAIDZ is benefit on "so-called checksum/ auto-healing" AND saving down time

It seems to me that you're closer to understanding the point with this statement than anything else I read that you wrote.

Having redundancy in the system, regardless of the way the system is used and whether or not a backup exists, is a way of improving data availability. In the case of ZFS, it also allows for detection and automatic correction of silent corruption. Without ZFS (ignoring BTRFS and proprietary solutions), you can have any number of backups and still end up losing data to silent corruption.

 

[yashiharu]

Until you re-read the newbie guide and post your system specs, I can not help you. I'm sorry.

you just give me a solution.
OK

FreeNAS:
version 9.3.1 stable 64 bits http://download.freenas.org/9.3.1/STABLE/201509022158/x64/FreeNAS-9.3-STABLE-201509022158.iso

Hardware:
HP ML10v2 G3240 814483-421
CPU: Intel G3240
RAM: Kingston 8GB CL11 1600Mhz ECC ram x2, total 16GB
Hard Drives: WD Red 3TB x4, Sandisk USB 3.0 16GB stick x2
no Add-On cards
no VM, no esxi
and the above hardware is on the way shipping

6TB data need to backup to the above machine.
Hope to have a solution to secure the data.

I think your solution is good: RAIDZ2 + snapshot replication.
Whats wrong?