Offsite backups in a low bandwidth environment

[Crustacean]

Sorry if this is the wrong board but..

I'd like to have a nightly or a weekly offsite backups of my server bare metal recoveries. My NAS is currently my only backup. If the building blows up, im out of luck.

I have a pair of crucial SSDs and enclosures. I plan to use rsync or a scheduled task on the server to copy the latest recovery to the drives during the day, then have an employee & myself take the drives home nightly. I chose to go with SSDs because of the constant movement the drives will endure.

Cloud backups are not an option. The data changes often, is 110gb, and we only have 5mbps up.

I've heard much discussion (and hate) on USB drives on the board.. I cant tell if its because zfs doesn't play nice with them, or if people just don't trust rotating media in a portable environment.

Does this sound like a solid plan to you guys? Should I periodically run tests on my SSDs?

Should i stick with a robocopy script, and leave the drives attached to a windows machine, or use rsync and connect the drives directly to the freenas.

lots of questions.. i know.

 

[cyberjock]

You brought a tear to my eye when you mentioned doing SSDs because of constant movement. That was beautiful!

There's a few reasons why USB is bad. The big ones for your particular situation are:
- On a disk error the USB bridge typically disconnects. If you are doing SSD you'll probably be okay because SSDs typically work or don't work. There is no intermediate and no error recovery if the flash memory goes bad so that problem is gone.
- SMART virtually doesn't exist for SSDs at all. So not too much lost if you can't do SMART on your SSDs. I'm referring to both SMART testing and SMART monitoring.
- Do realize that USB will be slow... 30MB/sec tops. :(

It sounds like you have a solid plan. If robocopy gives the kind of performance you need then stick with what works! No reason to change if what you have meets your needs. Both robocopy scripts and rsync will work. Surely you are experienced enough that you can do robocopy in the other direction in the event you need to restore, but you may not be very experienced with restoring from rsync. The last thing you need is to have a need to use the backup and not be 100% sure you can do the restore without any problems. ;)

Good work! Nice to see people willing to spend a little extra money to make sure they've done it "right".

 

[Crustacean]

I did get USB 3.0 enclosures, but we'll see if they actually perform to spec.
Ill get an esata pci card if its a problem.

I should only be copying 100gb or so, and I have all day to do so. USB speeds should be fine. I might have the drive connected to one of the employees computers (to make it easy for her to grab), so it might be good not to saturate her nic!

I really don't want to have to perform smart tests on the SSDs, so it'll be great if ill just be able to check them once a week to see if backups are being written. I'm lazy and i want things to be simple.

I like the simplicity and reliability things in linux. Everything just seems more elegant than scheduled tasks and batch files in windows. Restoration won't be too much of an issue, if the place burned down, I'd be able to just plug my SSDs into my new system after we rebuild.

Honestly the SSDs weren't very expensive. Time are a changin'. I just can't backup everything i have. Luckily only 100-150gb or so is critical. I'm glad to hear my plan is solid. I've been working on it for a while!

I'll probably use rsync. Will i need to make mounting the drives part of my job/script if i have the drives connected to the nas? and can i use rsync over the network (with the drive connected to a windows machine)?

Thanks for the quick reply.

 

[cyberjock]

USB3 is disabled on FreeNAS. So if you plan to directly connect them to your server you will be limited to USB2 speeds.

You're going to have to go into the WebGUI, mount the drives, setup the rsync task, then unmount the drives to remove the disk if you want to do rsync locally.

If you are doing rsync through windows you'll need an rsync application like Deltacopy. I know Deltacopy will work on Windows from FreeNAS, but you will be on your own to set that up since we don't directly support Deltacopy here.

Keep in mind if you are doing an rsync while a copy is open the file you have on your SSD may be corrupted because of the inconsistent state of the file. You really should be doing backups when nobody is actually using the server. I'm not sure if this ruins your "let it run all day" comment. You ideally should be starting the backup at the end of the workday and then taking the drive home when its done.

 

[eraser]

Instead of USB what about using an external eSATA dock? Something like this:

http://www.newegg.com/Product/Product.aspx?Item=N82E16817153071

 

[Crustacean]

The freenas is idle during the day. It's for backups only (employees don't even know it exists).
There are some cron jobs that run during the day to delete old backups, but i plan to work around them.

My windows server backups start at 6pm, and will be copied to the drives the next day. Our offsite backups will be a day old, but i can totally live with that.

I'll probably use robocopy, since i'll get some more speed and wont have to worry about mounting drives. Plus it will take me all of 2 second to create that batch file.

I might go for the dock, but my motherboards don't have esata. I could get PCI cards, but i feel like I'm already spending quite a bit of cash this month. SSDs and the ECC ram for the nas :|. Plus the enclosures i ordered are supposedly pretty fast, and they'll help to protect the drives since employees will be handling these. I dont know what id do if an employee scratched off some of he contacts or broke the connector on the end of the SSD and i needed them for a recovery. I dont want to solder connectors onto any ssds.

Docks look great though.. and it would make things easier for those handling the drives. Maybe in the future, good suggestion!

 

[Ericloewe]

The freenas is idle during the day. It's for backups only (employees don't even know it exists).
There are some cron jobs that run during the day to delete old backups, but i plan to work around them.

My windows server backups start at 6pm, and will be copied to the drives the next day. Our offsite backups will be a day old, but i can totally live with that.

I'll probably use robocopy, since i'll get some more speed and wont have to worry about mounting drives. Plus it will take me all of 2 second to create that batch file.

I might go for the dock, but my motherboards don't have esata. I could get PCI cards, but i feel like I'm already spending quite a bit of cash this month. SSDs and the ECC ram for the nas :|. Plus the enclosures i ordered are supposedly pretty fast, and they'll help to protect the drives since employees will be handling these. I dont know what id do if an employee scratched off some of he contacts or broke the connector on the end of the SSD and i needed them for a recovery.

Docks look great though.. and it would make things easier for those handling the drives. Maybe in the future, good suggestion!

If you have available SATA ports, you can just use a SATA-eSATA adapter, it should work fine and you get to avoid USB altogether.