cloud + intermediary monitor vs. mirrored freeNAS boxes

[jickerson]

I know this is going to sound like the often asked question "how do i backup my freeNAS to the cloud", but it is in fact different and to the best of my knowledge, not been asked before.

I currently have an external array at work with ~300 gigs of data (which may grow over time to 1-2 TB, but this hasn't been resolved yet) and a freeNAS box at my house. The ultimate goal I would like to achieve is to have a copy of my work data reside on my home NAS so I can easily access it over my LAN to grab any needed data when working from my house. The data will stay mostly untouched from day to day, although there may be a few instances a week when some of the data is modified or files are added (lets say around 1 gigabyte/day max will be changed). Also, I don't need it instantaneously accessible from my home, although it would be nice if I had it synced by that evening.

My biggest issue is that the external HD with the data is being accessed from a work laptop which can't stay on all the time. In addition, I'm worried about my home internet connection being reliable/fast enough to sync all the data changed in a day before having to leave work. Otherwise, I'd just try and sync straight from my work laptop to my home freeNAS box at the end of the day and then jet.

Also, I know I can't just put it on a google drive and/or dropbox and have it synced directly from the freeNAS box, as has been asked multiple times by others.

The two options I can think of are the following:
1. Have the files synced by either dropbox or google drive continuously while at work. Meanwhile, use a raspberry pi I always have on at my house to monitor the cloud drive of choice and if a file changes, push it from the cloud drive to my NAS. This seems like the more affordable choice and able to update files after the initial sync (which I can do with a portable hdd)

2. Set up a second freeNAS box at work. Run a backup program on my work laptop to sync everything to the work NAS before leaving for the day. Then have the work NAS and the home NAS in a mirrored setup (rsync?) so there will be a copy of the files on the home box later on. This seems like the more expensive option, and possibly problematic to get approval from my work to run a full time NAS with outside access (assuming I'd have to run some sort of VPN/tunnel between the two boxes?)

Are either one/both of these options feasible. If they would both work, are there particular merits to either that make it the obvious choice? I wasn't sure how much overhead the pi would add if it acted as an intermediary.

 

[anodos]

I think option 2 is better. ZFS replication is tunnelled over ssh and goes in one direction. This means that you probably won't have to make any firewall changes at work. You can keep it relatively low cost and low power.

I don't trust cloud services with sensitive work data.

Whatever you do make sure to communicate clearly with management and IT. You don't want to get in trouble / violate company policy.

 

[jickerson]

Anodos, thank you for the advice. I thought it would probably be the better option, but unfortunately it is the more expensive.

Is the first option also viable, or are there other problems with running the raspberry pi as an intermediary?

 

[AlainD]

Hi

I wouldn't use a cloud service without encryption, but get it properly encrypted.

Is the work related stuff only available on 1 external HDD?? --> What if it's got stolen?

A second external HDD could be a simple solution.

 

[jickerson]

The work related files are only on 1 external hdd, but it stays in the secured office. I use it over my laptop hdd for future expandability.

I have thought of using a second hdd, but I'm looking for an automatic solution that doesn't require me plugging in another hdd, syncing files, lugging it home with me every time, plugging it into the nas, and copying the files over.

 

[jickerson]

Sorry to bump a topic that has stalled somewhat, but I was hoping to get one final clarification.

Does anyone have experience with option 1 above, using a raspberry pi to pull from the cloud and push to a NAS? Will the throughput be cut substantially going through the pi or will it still outpace the typical broadband connection, which would be the bottleneck?

 

[cyberjock]

Depends on what you consider "typical broadband". If I'm not mistaken the pi has only a 10/100 connection, so you aren't under any circumstances going to do more than about 12MB/sec. However in my case I have internet that exceeds 100Mb, so I could easily bottleneck any load by using a Pi.

 

[anodos]

Sorry to bump a topic that has stalled somewhat, but I was hoping to get one final clarification.

Does anyone have experience with option 1 above, using a raspberry pi to pull from the cloud and push to a NAS? Will the throughput be cut substantially going through the pi or will it still outpace the typical broadband connection, which would be the bottleneck?

I believe that ethernet on the Pi is provided by a built-in USB-ethernet adapter. In my mind "USB-ethernet" is not associated with "stable" or "high-throughput".

 

[Whattteva]

USB protocol (at least up to 2.0, not sure about 3.0) suffers from incurring a rather significant overhead.