Problem setting up SSH between two FreeNAS servers (for snapshot replication)

[BlackObs]

Both servers are running 8.01 BETA4 amd64.

So I'm trying to set up snapshot replication, following http://doc.freenas.org/index.php/Replication_Tasks, and the snapshots are getting created just fine, but they're not replicating to the remote FreeNAS box. This appears to be an SSH issue, since when I try "ssh -i /data/ssh/replication NAS002" from an SSH session with NAS001, I get "The authenticity of host NAS002 can't be established. RSA key fingerprint is (stuff)." This happens whether I use the name or IP of NAS002.

The linked Replication_Tasks page tells me that this means that SSH key-based authentication isn't working. It tells me to make sure I've entered the right keys, which I'm sure I've done, and there are no spaces or line breaks in the keys.

One issue I've come across is that either NAS box's public key seems to vary depending on how I request it.
If I enter "cat /data/ssh/replication.pub" in a PuTTY session on NAS001, I get one key
If I enter "ssh-keyscan {IP of NAS001}" from NAS002, I get a different key
In desperation, I've entered all 4 keys (2 from NAS001, 2 from NAS002) into authorized_keys in /root/.ssh/ on both NAS boxes, with no change. Whether I include the leading "ssh-rsa" at the start of the key doesn't seem to matter (since there's a space after it, it ends up on its own line anyway). I've cycled the SSH service off and back on several times so far. About the only thing I haven't done is reboot either box.

What could I be doing wrong?

 

[bmcdonald74]

I had the same problem as you do. What I did to fix it was to copy the /data/ssh/replication.pub from the local server to the remote server using scp. Below is the command I used, you'll have to replace the remote_ip with the IP address of your remote server.

scp /data/ssh/replication.pub root@[I]remote_ip[/I]:.ssh/authorized_keys2

 

[bobrebelbob]

I have just setup two FreeNAS 8.01 RC1 boxes. Each has 4 x 2TB drives and each set up RAID 5 so each NAS box is about 6TB. I want to know if there is a step by step RSYNC or Remote Replication for a noob?? I see users here writing about commands that I assume are included in the version of Debian that FreeNAS is built on. I just want to setup the fastest form of backup from server A to server B. Home network is Windows only. I think BlackObs is doing (or about to do) what I need to do and was wondering if there is a document that I can follow to achieve that setup? The note bmcdonald74 adds is very useful and may be the final "key" to making it all work.

Please advise and thanx for sharing,
bob

 

[BlackObs]

The command supplied by bmcdonald74 did indeed work. However, it turns out that replicating snapshots is not what I want, because the system isn't smart enough to also replicate the initial dataset, without which the snapshots are entirely useless.

The good news is that I think there's a much easier way to get the result we want: rsync.

First, make sure that the backup box has no datasets on it, aside from the default mount point. Then follow these instructions to create an rsync module on said backup box. With that done, follow these other instructions to create the rsync task on your main box.

If your primary FreeNAS array already holds a lot of data (as mine does), the initial rsync is going to take a rather long time. But after that, at each time interval you specify, it'll only sync the difference to keep the backup array up to date, which will take much less time.

I'm about to set this up today myself, and I'll let you know if I run into any problems or if there are any steps in the documentation that are missing/could be made more apparent.

 

[bobrebelbob]

I must correct my error: I hope no one slings arrows for my mistake in my previous post. I said, "version of Debian that FreeNAS is built on" when I should have said FreeBSD that FreeNAS is built on.

1000 pardons,
bob

 

[ProtoSD]

I must correct my error: I hope no one slings arrows for my mistake in my previous post. I said, "version of Debian that FreeNAS is built on" when I should have said FreeBSD that FreeNAS is built on.

1000 pardons,
bob

I was about to say something but I'm glad you realized it on your own! ;)

 

[James]

This page of the documentation should be much clearer now (and doesn't require enabling root SSH access).

 

[bobrebelbob]

It works BUT

I did finally spend a little time and figure out how to get the RSYNC to work. However, it is not the solution for me since it does not take care of anything but copying of additions to the Main server. If you delete anything from the Main server it won't delete from the backup server and will just remain forever or until you manually delete. Looks like I will need to look at the SNAPSHOT so both boxes stay in sync. That is unless you could give me a hint how to make RSYNC work?

Thanks for pointing me to the directions because they did work and I did ask. If it were not for a community such as this, I know I would be forever searching for answers and never getting to do anything with the hardware.

Bob

 

[mikeyr]

RSYNC and/or Snapshots

bob,
(RSYNC hijack of the thread)
If you set the "delete" option for RSYNC, then it will delete things on the destination that have been deleted from the source. Be wary though, there was a post in the "old" days (version .69 or maybe.7) where if the source is not carefully set up, there can be a disk failure case where the RSYNC sees an empty filesystem on the source and proceeds to delete everything on the remote copy-- obviously not what you want!

http://sourceforge.net/apps/phpbb/freenas/viewtopic.php?f=51&t=4131&p=26129&hilit=rsync+delete+many+files#p26129

(End of RSYNC hijack)

However, I must also say that I was having problems with snapshots and *about* to follow your path of using RSYNC (had some experience with it on a previous r.69 FreeNAS box)-- but suddenly ("magically"??) snapshots and replication started working for me.

My source machine that I want to replicate has a~500 GB of data on it. I haven't verified everything yet, but the machines "talked" at full network bandwidth for about 12 hours and my remote replication machine shows that 500 GB is used up, so the initial replication apparently copied my entire data set over to the remote machine (not just the snapshot "delta changes" to the filesystem as blackobs presumed above). I assume that snapshots after this will only send the delta-changes though, but if i understand correctly the entire filesystem should be reconstructable from the original replication plus the "delta-changes" incremental snapshots on the remote machine.

Now I need to figure out how to mount and verify the entire data set and snapshots on the remote machine... will post more as I learn more. Follow my progress on another related thread here:

http://forums.freenas.org/showthread.php?2548-Snapshots-not-working