Need help with iSCSI Snapshot/Replication

Status
Not open for further replies.

Lukazy

Dabbler
Joined
Jun 25, 2017
Messages
11
Hello,

I have a freenas system, that offers a iSCSI target for 2 windows servers for backup purposes (server iSCSI targets only differ in size and its a test setup, no production environment so there is no auth on the iSCSI).
I attached screenshots of the iSCSI Share settings for all details of its implementation.

In the next step I want to backup that data to another freenas system offsite (connected via VPN) because the windows server is sitting in the same room as the freenas system.

So I read a lot about snapshots and replication and wanted to implement that.

But I must have missed something in the documentation because it wont let me create a snapshot (for both servers) giving me the error e-mail with:

Can someone explain why snapshots are not working? I'm trying to create the first snapshot, there are no snapshots yet.
I thought that the first snapshot does not take up much space as there is no changed data yet?

EDIT: I know windows backup is not the best and I will use other software later on, it is just a test setup as of right now
EDIT 2: Here is the output of zfs list and zfs list -o space (note that sizes vary between this and the screenshots, as there are some days passed in between):
Code:
[root@FreeNAS ~]# zfs list																										
NAME															USED  AVAIL  REFER  MOUNTPOINT									
Backup														 6.94T  75.3G   139K  /mnt/Backup									
Backup/Backup-SQL											  5.46T  3.00T  2.52T  -											
Backup/Backup-WTS											  1.49T  1.01T   561G  -											
Backup/jails													117K  75.3G   117K  /mnt/Backup/jails							
Kamera-01													   439G  3.08T   439G  /mnt/Kamera-01								
freenas-boot												   2.18G  26.6G	64K  none											
freenas-boot/.system											536M  26.6G   510M  legacy										
freenas-boot/.system/configs-839d4bf50898424ab2b76c72b7c93def  21.8M  26.6G  21.8M  legacy										
freenas-boot/.system/cores									 4.86M  26.6G  4.86M  legacy										
freenas-boot/.system/rrd-839d4bf50898424ab2b76c72b7c93def		29K  26.6G	29K  legacy										
freenas-boot/.system/samba4									 116K  26.6G   116K  legacy										
freenas-boot/.system/syslog-839d4bf50898424ab2b76c72b7c93def	 29K  26.6G	29K  legacy										
freenas-boot/ROOT											  1.64G  26.6G	29K  none											
freenas-boot/ROOT/11.1-U5									  1.64G  26.6G   845M  /											
freenas-boot/ROOT/Initial-Install								 1K  26.6G   825M  legacy										
freenas-boot/ROOT/default									  3.74M  26.6G   829M  legacy										
freenas-boot/grub											  6.85M  26.6G  6.85M  legacy										
[root@FreeNAS ~]# ^C

Code:
[root@FreeNAS ~]# zfs list -o space																								
NAME														   AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD			
Backup														 75.3G  6.94T		 0	139K			  0	  6.94T			
Backup/Backup-SQL											  3.00T  5.46T		 0   2.52T		  2.93T		  0			
Backup/Backup-WTS											  1.01T  1.49T		 0	561G		   962G		  0			
Backup/jails												   75.3G   117K		 0	117K			  0		  0			
Kamera-01													  3.08T   439G		 0	439G			  0	  15.6M			
freenas-boot												   26.6G  2.18G		 0	 64K			  0	  2.17G			
freenas-boot/.system										   26.6G   536M		 0	510M			  0	  26.8M			
freenas-boot/.system/configs-839d4bf50898424ab2b76c72b7c93def  26.6G  21.8M		 0   21.8M			  0		  0			
freenas-boot/.system/cores									 26.6G  4.86M		 0   4.86M			  0		  0			
freenas-boot/.system/rrd-839d4bf50898424ab2b76c72b7c93def	  26.6G	29K		 0	 29K			  0		  0			
freenas-boot/.system/samba4									26.6G   116K		 0	116K			  0		  0			
freenas-boot/.system/syslog-839d4bf50898424ab2b76c72b7c93def   26.6G	29K		 0	 29K			  0		  0			
freenas-boot/ROOT											  26.6G  1.64G		 0	 29K			  0	  1.64G			
freenas-boot/ROOT/11.1-U5									  26.6G  1.64G	  830M	845M			  0		  0			
freenas-boot/ROOT/Initial-Install							  26.6G	 1K		 0	  1K			  0		  0			
freenas-boot/ROOT/default									  26.6G  3.74M		 0   3.74M			  0		  0			
freenas-boot/grub											  26.6G  6.85M		 0   6.85M			  0		  0			
[root@FreeNAS ~]#


I foud the following command to be helpful in a way that it shows more info, but I have yet to pinpoint the problem or settings thats keeps me from creating snapshots
Code:
root@FreeNAS:~ # zfs get all | grep Backup-SQL

Backup/Backup-SQL											  type					 volume										 -
Backup/Backup-SQL											  creation				 Thu Feb 22 19:20 2018						  -
Backup/Backup-SQL											  used					 5.46T										  -
Backup/Backup-SQL											  available				3.00T										  -
Backup/Backup-SQL											  referenced			   2.53T										  -
Backup/Backup-SQL											  compressratio			1.56x										  -
Backup/Backup-SQL											  reservation			  none										   default
Backup/Backup-SQL											  volsize				  5.37T										  local
Backup/Backup-SQL											  volblocksize			 16K											-
Backup/Backup-SQL											  checksum				 on											 default
Backup/Backup-SQL											  compression			  lz4											inherited from Backup
Backup/Backup-SQL											  readonly				 off											default
Backup/Backup-SQL											  copies				   1											  default
Backup/Backup-SQL											  refreservation		   5.46T										  local
Backup/Backup-SQL											  primarycache			 all											default
Backup/Backup-SQL											  secondarycache		   all											default
Backup/Backup-SQL											  usedbysnapshots		  0											  -
Backup/Backup-SQL											  usedbydataset			2.53T										  -
Backup/Backup-SQL											  usedbychildren		   0											  -
Backup/Backup-SQL											  usedbyrefreservation	 2.93T										  -
Backup/Backup-SQL											  logbias				  latency										default
Backup/Backup-SQL											  dedup					off											default
Backup/Backup-SQL											  mlslabel																-
Backup/Backup-SQL											  sync					 standard									   default
Backup/Backup-SQL											  refcompressratio		 1.56x										  -
Backup/Backup-SQL											  written				  2.53T										  -
Backup/Backup-SQL											  logicalused			  3.50T										  -
Backup/Backup-SQL											  logicalreferenced		3.50T										  -
Backup/Backup-SQL											  volmode				  default										default
Backup/Backup-SQL											  snapshot_limit		   none										   default
Backup/Backup-SQL											  snapshot_count		   none										   default
Backup/Backup-SQL											  redundant_metadata	   all											default
Backup/Backup-SQL											  org.freenas:description  Backup-SQL									 local
root@FreeNAS:~ # 


regards
lukazy
 

Attachments

  • 1.jpg
    1.jpg
    44.2 KB · Views: 591
  • 2.jpg
    2.jpg
    30.8 KB · Views: 535
  • 3.jpg
    3.jpg
    28.7 KB · Views: 509
  • 4.jpg
    4.jpg
    34.6 KB · Views: 538
  • 5.jpg
    5.jpg
    41.9 KB · Views: 549
  • 6.jpg
    6.jpg
    23.8 KB · Views: 551
  • sql.jpg
    sql.jpg
    91.9 KB · Views: 545
  • wts.jpg
    wts.jpg
    83.7 KB · Views: 523
Last edited:

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457
Snapshots hold all of the changed data. Windows backup is absolutely the worst backup software ever made (outside of symantec backup exec). It keeps backups until the drives is filled! this is not advisable for ZFS as its is a Copy-On-Write file system (thats art of what makes it so safe) If the filesystem or in your case the zvol is full, there is no room to store this old data as it is "overwritten". There is no magic way of storing data without using space.
If you use a backup software that can set some saine retention policy, this will work. Again, you will still need space for and delta data. It should also be noted that snapshots for versioned backup data are arguably less useful then other use cases. I would still use snapshots on my backup target (and I do) but the retention is very short, about 1 month and overlap when I start new backup chains.
 

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457
Take a look at your snapshots and see how much space is used by them. That's where you will find the "missing" space.
 

Lukazy

Dabbler
Joined
Jun 25, 2017
Messages
11
Take a look at your snapshots and see how much space is used by them. That's where you will find the "missing" space.



I currently have no snapshots, I was trying to create the first snapshot for this zvol. That is why I'm so confused that it said "out of space" because the first snapshot should not use that much space as there is no changed data yet.


Backup software will be coming in the future, as of right now Im testing with windows backup and I have a script that will delete old backups if there is no more space left just for the sake of testing. Snapshot creation should work no matter what backup software is used.

Lets take the SQL-Server Backup Volume as an example:


In FreeNAS
  • zvol size: 5.5 TB (oddly enough when I edit the zvol is shows 5.37T as 'Size for this zvol)
  • available space: 2 TB

In Windows
  • Volume size: 5.5 TB
  • Used space: ~ 5 TB

Where do I need free space for snapshots? I thought that with 5.5 TB used and 2 TB free it should be enough to create a snapshot (as its the first it should not take up that much space).

What am I missing here?


regards
Alkahna
 
Last edited:

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457
can you provide the output of zfs list
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,365
Fwiw, I like veeam backup agent for windows. Works very well backing up to a FreeNAS SMB share.

And it’s free.
 

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457
Yep now try zfs list -o space and look at the USEDSNAP column.
Code:
root@Pico:~ # zfs list pico/vmblock
NAME		   USED  AVAIL  REFER  MOUNTPOINT
pico/vmblock   425G  2.08T   423G  -
root@Pico:~ #
root@Pico:~ #
root@Pico:~ # zfs list -o space pico/vmblock
NAME		  AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
pico/vmblock  2.08T   425G	 1.64G	423G			  0		  0
root@Pico:~ #

Il bet if I checked, I'm "missing" 1.64G from pico/vmblock
 

Lukazy

Dabbler
Joined
Jun 25, 2017
Messages
11
Yep now try zfs list -o space and look at the USEDSNAP column.
see original post for the output, there is nothing in that column which seems to line up with the facts that no snapshots have been created thus far I guess.
 

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457
From Oracle:
refreservation

Value: Number (or none)

Default: none

Sets the minimum amount of disk space that is guaranteed to a dataset, not including descendents, such as snapshots and clones. When the amount of disk space used is below this value, the dataset is treated as if it were taking up the amount of space specified by refreservation. The refreservation reservation is accounted for in the parent dataset's disk space used, and counts against the parent dataset's quotas and reservations.

If refreservation is set, a snapshot is only allowed if enough free pool space is available outside of this reservation to accommodate the current number of referenced bytes in the dataset.

The property abbreviation is refreserv.
Are we talking about Backup/Backup-SQL or Backup/Backup-WTS? Your screenshots are showing Backup/Backup-WTS (1.5T) on Windows backup and Backup/Backup-SQL on the FreeNAS iSCSI settings.
Either way, this "USEDREFRESERV" would seem to be the issue.
 

Lukazy

Dabbler
Joined
Jun 25, 2017
Messages
11
From Oracle:
Are we talking about Backup/Backup-SQL or Backup/Backup-WTS? Your screenshots are showing Backup/Backup-WTS (1.5T) on Windows backup and Backup/Backup-SQL on the FreeNAS iSCSI settings.
Either way, this "USEDREFRESERV" would seem to be the issue.
Im trying to get both zvol's to be set up up with snapshots (So Backup/Backup-SQL & Backup/Backup-WTS) and there is a windows screenshot of both and iSCSI-Settings just for Backup/Backup-SQL as its settings are the same for Backup/Backup-WTS (they just differ in size)

What is the column "USEDREFRESERV" for or what does it indicate?
 

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457

Lukazy

Dabbler
Joined
Jun 25, 2017
Messages
11

Attachments

  • zvol_edit.jpg
    zvol_edit.jpg
    90.3 KB · Views: 519

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457

Lukazy

Dabbler
Joined
Jun 25, 2017
Messages
11
That's just the size of the zvol. Perhaps this may contain more information. I don't know anything more on this subject than you. I am only searching using google.
https://docs.oracle.com/cd/E19253-01/819-5461/gazvb/index.html
Thanks for the heads up on that topic, I will add the output of
Code:
zfs get all | grep Backup-SQL
in my original post as there is a little more info on what space is used for which purpose although I have not figured out why snapshots wont work yet.

I found this at the end of your linked article:

"If refreservation is set, a snapshot is only allowed if sufficient unreserved pool space exists outside of this reservation to accommodate the current number of referenced bytes in the dataset."

Lets take backup-sql as an example and by looking at the output (
zfs list -o space)
from my original post there is 2.52T USEDDS and 2.93T USEDREFRESERV which adds up to 5.46T USED in total. In the last output we can see that there is 2.53T referenced which leaves not much space left as we have only 3T unused free space
but windows sees it as ~5.5T of which ~5T are used so I dont get the logic here...
 
Last edited:
Status
Not open for further replies.
Top