Replication -How to setup 2x PUSH to 1x PULL, advice needed.

[Daniel Claesson]

Hi all,

I'm having some questions on setting up a replication "workflow" on the FreeNAS systems that i have deployed, i really need some advice on how to proceed.
Hope the forums collective knowledge can pitch in here.

My goal is to replicate the snapshots from PUSH 1 and PUSH 2 to one single PULL system.

Quickspecs:
PUSH 1
Hardware: IX System 2U FreeNAS server 2x Intel Xeon E5-2609v2 / 32GB RAM/ 6x4TB SAS / 480GB L2ARC / 200GB ZIL / 4x1GbE / 2x10GbE
System Build: FreeNAS 9.10.1
Services: AFP, CIFS, SMB, SSH, iSCSI
Main usage: iSCSI provider to VMWare vSphere/vCenter and SMB and AFP filesharing.

PUSH 2
Hardware: HP Proliant MicroServer Gen8 1x Intel Celeron G1610T / 16GB RAM / 4x2TB SATA / 2x1GbE / 1x10GbE
System Build: FreeNAS 9.10.1-U1
Services: AFP, iSCSI, SMB, SSH
Main usage: iSCSI provider to VMWare vSphere/vCenter (Used for HA), SMB filesharing for Veaam Backup and Replication and AFP filesharing for TimeMachine backups of OS X server VM's

PULL
Hardware: HP Proliant MicroServer Gen8 1x Intel Celeron G1610T / 16GB RAM / 4x4TB SATA / 2x1GbE
System Build: FreeNAS 9.10.1-U1
Services: SSH
Main usage: Snapshot replication receiver from PUSH 1 and 2.

Configuration thoughts
On PULL system
2 different users with SSH key from PUSH 1 and 2 respectively, ex: User "push1" with SSH key from PUSH1 system and user "push2" with SSH key from PUSH2 system.
Both these users have Sudo access and home directory in /mnt/Tank

On PUSH1 and 2 systems
Do i have to create the same "exact" user on each corresponding system? (User push1 on PUSH1 system and user push2 on PUSH2 system)
In PUSH1 system i config the "Add replication task" with dedicated user: push1 and for PUSH2 system i use dedicated user: push2.
Is the SSH key extracted from those users on PULL with the same username?

I think i need to be able to fetch the correct SSH key from each of these systems. In the manual there is only explained how to do this with the "root" account, and to my knowledge this setup is only applicable in a 1 to 1 configuration, correct?

Question: Is this the way to set it up?

PS.
I have it working in a 1 to 1 config.

Best Regards
Daniel Claesson

 

[Stux]

I have this setup.

I have a "tank/replicas" dataset, and then nested in there, I have two datasets, one for each server.

Each server replicates the datasets I care about to its dataset on the backup server.

I just replicated via root.

 

[Ericloewe]

Each replication task is 1-to-1, but that has no bearing on the possibility of replicating from two different systems to two different datasets on the same system.

There really shouldn't be any catches, just follow the manual for each task and everything should just work.

 

[depasseg]

Do i have to create the same "exact" user on each corresponding system? (User push1 on PUSH1 system and user push2 on PUSH2 system)

No. Just follow the directions on each push1 and 2 system.

And make sure you use different destination datasets on Pull.

 

[Daniel Claesson]

Ok i am partially withy you guys on this. What i don't understand is how i differentiate the SSH key on PULL for the two PUSH systems.

Say i use root fort both PUSH systems, how do i enter 2 different SSH keys so SSH check clears on PULL?

Regards
Daniel

 

[Ericloewe]

You append them to the list.

 

[Daniel Claesson]

Ok, i think i get it.

I added the two SSH key's to the root user on PULL. I have setup a test replication task from PUSH2 to PULL now. Same way as i have done on the 1 to 1 config except that i specified a specific dataset and added two SSH keys to the root user.
Hope this works.

Regards
Daniel Claesson

 

[Daniel Claesson]

Ok here is an update.

It works but is painfully slow. In Meraki Dashboard i can observer a traffic between the two of around 2-2,5 Mbit/s.
Normal SMB file transfer to either of the servers i can achieve ruffly 100MB/s from my client computer.

What can be causing this ? I will do rSync test between to clients to see if that also yield that kind of slow performance.

Regards
Daniel Claesson

 

[Daniel Claesson]

I did some testing and this is the results.

PULL (kk-s01-ext-backup)
Replication from kk-s01-backup = 2.5Mbps
Rsync from client = 170Mbps
AFP/SMB = ca 350Mbps


PUSH2 (kk-s01-backup)
Replication to kk-s01-ext-backup = 2.5Mbps
AFP/SMB = ca 850Mbps
Rsync from client = 780Mbps

Rsync test with client was done from a Mac mini with Carbon Copy Cloner, SMB/AFP file transfer was from the same Mac mini through Finder.
The Mac mini is connected to the same Meraki switch as the PULL. UP-Link from that switch is the routed through the Meraki Firewall to a another Meraki switch where the PUSH2 system is connected. (PUSH2 and PULL are in different parts of the building, but same LAN and vlan)
The hardware specs for both PUSH2 and PULL are the same. Only difference is the drives.

PULL = Seagate ST4000VN000
PUSH2 = Seagate ST3200641AS

Network speeds for PUSH2 is all within reason for 1GbE on all test except the replication task with PULL. My "conclusion" is that the network is fine between Mac mini and PUSH2 (cables, switches and firewall). The problem should be with PULL, or with the network config on PULL (defective cable to it?)

All network traffic to PULL is very low, even the "manual" file transfer. But the biggest issue here is the exceptionally low speed between PUSH2 and PULL.

I will get a new cable between PULL and the switch and retest.

Regards
Daniel Claesson

 

[Daniel Claesson]

Yet another update...

The suspect cable between switch and PULL is now replaced with a new CAT6. Speed bumped up almost instant to around 600-700Mbps.
The faulty cable was a "good looking" CAT5e, so no visible clues to it being bad.

Any thoughts on my setup, config etc?

Regards
Daniel Claesson