What are your ARC statistics?

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
No clue about the "(Adaptive)" part. I didn't notice that until you mentioned it.

Strangely my FreeNAS Mini displays properly, but my main server does not. Both have two pools too... hmm.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
  • VM backups
  • 11.8TB / 14.5TB (vmbackups)
  • 7.91
    (Adaptive)GB (MRU: 7.70GB, MFU: 218.84GB) / 16.00GB
  • Hit ratio -> 92.66% (higher is better)
  • Prefetch -> (higher is better)
  • Hit MFU:MRU -> 73.39%:13.25% (higher ratio is better)
  • Hit MRU Ghost -> 1.31% (lower is better)
  • Hit MFU Ghost -> 2.69% (lower is better)
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
  • archive
  • 1.83GB / 17.9GB (freenas-boot)
  • 17.8TB / 40TB (archive)
  • 21.40GB (MRU: 1.97GB, MFU: 19.44GB) / 32.00GB
  • Hit ratio -> 76.25% (higher is better)
  • Prefetch -> 1.64% (higher is better)
  • Hit MFU:MRU -> 56.40%:42.59% (higher ratio is better)
  • Hit MRU Ghost -> 0.09% (lower is better)
  • Hit MFU Ghost -> 0.22% (lower is better)
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
I know where the "(Adaptive)" is coming from. Forgot I had added an L2ARC to this box as an experiment and never removed it...

Check out the output...

# python /usr/local/www/freenasUI/tools/arc_summary.py | grep "ARC Size:"
ARC Size: 77.23% 23.16 GiB
L2 ARC Size: (Adaptive) 27.59 GiB

It's grabbing the last line and using it.

For those that want to know how much my L2ARC is used in a home environment...

Code:

L2 ARC Summary: (HEALTHY)
  Passed Headroom:  27.25m
  Tried Lock Failures:  153.21k
  IO In Progress:  348
  Low Memory Aborts:  174
  Free on Write:  454.24k
  Writes While Full:  45.85k
  R/W Clashes:  166
  Bad Checksums:  0
  IO Errors:  0
  SPA Mismatch:  107.57m

L2 ARC Size: (Adaptive)  27.59  GiB
  Header Size:  0.19%  52.54  MiB

L2 ARC Evicts:
  Lock Retries:  361
  Upon Reading:  0

L2 ARC Breakdown:  86.61m
  Hit Ratio:  11.83%  10.25m
  Miss Ratio:  88.17%  76.36m
  Feeds:  466.73k

L2 ARC Buffer:
  Bytes Scanned:  33.12  TiB
  Buffer Iterations:  466.73k
  List Iterations:  29.41m
  NULL List Iterations:  916.60k

L2 ARC Writes:
  Writes Sent:  100.00% 101.65k


See? Utterly useless. Miss ratio of 88%. ;)

ZFS doesn't even really try to use it. A whole 4GB out of a 32GB drive is used (the most I've ever seen it use before).
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
  • general fileservice/mirrors
  • 9.80TB / 21.8TB (fileserver)
  • 3.56GB (MRU: 228.33GB, MFU: 3.34GB) / 8.00GB
  • Hit ratio -> 99.48% (higher is better)
  • Prefetch -> 33.41% (higher is better)
  • Hit MFU:MRU -> 96.67%:0.36% (higher ratio is better)
  • Hit MRU Ghost -> 0.24% (lower is better)
  • Hit MFU Ghost -> 0.26% (lower is better)
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Mine too, it's lightly (but maybe during more time, about 2/3 of the day) loaded and my CPU usage never exceeded 20%, even during a scrub (probably a little bit overkill CPU; I had chosen a 4330 but a flash discount on the 4360 changed my mind (it was only 15€ more expensive than the 4330...)). I just can't explain your MRU:MFU size ratio (about 1:1 where everyone else is at about 10:1).

@cyberjock Ah ok, good catch.

More WTF with jgreco stats: "(MRU: 7.70GB, MFU: 218.84GB) / 16.00GB" seems that my script is far from being bulletproof. I think it's because his MFU is so low it's in MB and I append GB in the script... I'll see if I can fix that ;) <-- script fixed.
 
Last edited:

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
More WTF with jgreco stats: "(MRU: 7.70GB, MFU: 218.84GB) / 16.00GB" seems that my script is far from being bulletproof. I think it's because his MFU is so low it's in MB and I append GB in the script... I'll see if I can fix that ;)

Frequently Used Cache Size: 2.96% 237.68 MiB

.... oops, hahahahaha.
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Yep, fixed ;) If you can edit your post and change GB to MB it would be great ;)

And I took the opportunity to change TB to TiB, GB to GiB, ... for the pools sizes as well :)
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
It's the right unit :p I too prefer to use GB instead of GiB even if I talk about GiB but it's inexact...
 
Last edited:

yis

Dabbler
Joined
Jan 24, 2015
Messages
30
@Bidule0hm
Thanks for your help last night.. and here are my stats. It look those stats are better when the server was just booted.

  • Put your data type(s) here...
  • 88.6GiB / 8.12TiB (Main)
  • 2.12GiB / 14.9GiB (freenas-boot)
  • 8.25GiB (MRU: 8.22GiB, MFU: 22.73MiB) / 12.00GiB
  • Hit ratio -> 73.32% (higher is better)
  • Prefetch -> 0.00% (higher is better)
  • Hit MFU:MRU -> 35.94%:63.78% (higher ratio is better)
  • Hit MRU Ghost -> 0.00% (lower is better)
  • Hit MFU Ghost -> 0.05% (lower is better)
 

yis

Dabbler
Joined
Jan 24, 2015
Messages
30
@Bidule0hm
Thanks for your help last night.. and here are my stats. It look those stats are better when the server was just booted.

  • Put your data type(s) here...
  • 88.6GiB / 8.12TiB (Main)
  • 2.12GiB / 14.9GiB (freenas-boot)
  • 8.25GiB (MRU: 8.22GiB, MFU: 22.73MiB) / 12.00GiB
  • Hit ratio -> 73.32% (higher is better)
  • Prefetch -> 0.00% (higher is better)
  • Hit MFU:MRU -> 35.94%:63.78% (higher ratio is better)
  • Hit MRU Ghost -> 0.00% (lower is better)
  • Hit MFU Ghost -> 0.05% (lower is better)
rebooted my freenas, got more interesting results :)


  • Put your data type(s) here...
  • 94.0GiB / 8.12TiB (Main)
  • 2.12GiB / 14.9GiB (freenas-boot)
  • 160.36MiB (MRU: 5.20GiB, MFU: 5.20GiB) / 12.00GiB
  • Hit ratio -> 98.33% (higher is better)
  • Prefetch -> 1.22% (higher is better)
  • Hit MFU:MRU -> 90.15%:7.76% (higher ratio is better)
  • Hit MRU Ghost -> 0.00% (lower is better)
  • Hit MFU Ghost -> 0.00% (lower is better)
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Thanks for the stats (and to not having deleted the old stats because the new are more or less irrelevant, I'll explain...) ;)

This is called cache warm-up: after a reboot the cache is (more or less) slowly repopulated as you use the server. For example actually I have an uptime of 18 days and my cache is still warming-up, but very slowly now (about +0.4% for the last 24h).

So the stats right after a reboot are irrelevant. Except for you, in your case as you can see that the stats are worse and worse with the time, it's pretty important and as we saw on the other thread it's because your cache is too small.
 
  • Like
Reactions: yis

dgoodrich

Dabbler
Joined
Dec 18, 2013
Messages
10
Funny I saw this thread, I was just looking for ARC information trying to determine the cause of our poor performance. Both boxes are in 45 drives cases, FreeNAS-01 has the POD 2 upgrade. What do my ARC stats say about the boxes?

[root@FreeNAS-01]/var/tmp# ./arcstats.sh
  • ISCSI Extents to VEEAM, RSYNC Targets, and IP Camera video storage
  • 15.0TiB / 18.1TiB (Backup-Vol1)
  • 15.0TiB / 18.1TiB (Backup-Vol2)
  • 15.0TiB / 18.1TiB (Backup-Vol3)
  • 15.0TiB / 18.1TiB (Backup-Vol4)
  • 19.2TiB / 32.5TiB (Backup-Vol5)
  • 9.70TiB / 18.1TiB (Data-Vol1)
  • 15.1TiB / 18.1TiB (Video-Vol1)
  • 15.0TiB / 18.1TiB (Video-Vol2)
  • 11.41GiB (MRU: 10.70GiB, MFU: 722.80MiB) / 16.00GiB
  • Hit ratio -> 77.54% (higher is better)
  • Prefetch -> 40.65% (higher is better)
  • Hit MFU:MRU -> 39.70%:48.76% (higher ratio is better)
  • Hit MRU Ghost -> 0.24% (lower is better)
  • Hit MFU Ghost -> 0.26% (lower is better)

[root@FreeNAS-02] /var/tmp# ./arcstats.sh
  • ISCSI Extents to VEEAM and RSYNC Targets
  • 23.4TiB / 32.5TiB (Backup-Vol6)
  • 8.69TiB / 18.1TiB (Data-Vol1)
  • 9.56GiB (MRU: 10.56GiB, MFU: 891.81MiB) / 16.00GiB
  • Hit ratio -> 81.63% (higher is better)
  • Prefetch -> 30.26% (higher is better)
  • Hit MFU:MRU -> 46.47%:44.31% (higher ratio is better)
  • Hit MRU Ghost -> 0.09% (lower is better)
  • Hit MFU Ghost -> 1.49% (lower is better)


DAve
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Well, you clearly don't have enough RAM. You can see this by the hit ratio and the hit MFU:MRU ratio that are very low (especially for the first server) ;)
 

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
  • backup destination for Arq, BitTorrent Sync, 3 small Ubuntu server VMs in VirtualBox, DVD images for occasional streaming (not Plex)
  • 1.93GiB / 3.59GiB (freenas-boot)
  • 768GiB / 2.70TiB (pool0)
  • 11.36GiB (MRU: 9.48GiB, MFU: 1.91GiB) / 16.00GiB
  • Hit ratio -> 96.77% (higher is better)
  • Prefetch -> 44.40% (higher is better)
  • Hit MFU:MRU -> 98.42%:1.22% (higher ratio is better)
  • Hit MRU Ghost -> 0.01% (lower is better)
  • Hit MFU Ghost -> 0.02% (lower is better)
  • up 5 days
 

Serverbaboon

Dabbler
Joined
Aug 12, 2013
Messages
45
Creaking N54L with 16GB Ram

  • Data Raid Z2 (4 x 2TB) - Photos, Music, VMWARE NFS and General Files
  • VMWARE Mirror (2 x 1TB) - VMWARE ISCSI Data Store
  • 1.20TiB / 7.25TiB (Data)
  • 116GiB / 928GiB (VMSTORE)
  • 3.79GiB / 7.44GiB (freenas-boot)
  • 9.86GiB (MRU: 7.98GiB, MFU: 1.89GiB) / 16.00GiB
  • Hit ratio -> 92.62% (higher is better)
  • Prefetch -> 17.80% (higher is better)
  • Hit MFU:MRU -> 69.28%:29.79% (higher ratio is better)
  • Hit MRU Ghost -> 0.29% (lower is better)
  • Hit MFU Ghost -> 0.52% (lower is better)
 
Last edited:

Apollo

Wizard
Joined
Jun 13, 2013
Messages
1,449
  • Miscellaneous...
  • 12.4TiB / 21.8TiB (tank)
  • 3.30GiB / 7.19GiB (freenas-boot)
  • 23.69GiB (MRU: 14.82GiB, MFU: 8.90GiB) / 32.00GiB
  • Hit ratio -> 80.73% (higher is better)
  • Prefetch -> 11.42% (higher is better)
  • Hit MFU:MRU -> 69.02%:0.24% (higher ratio is better)
  • Hit MRU Ghost -> 0.00% (lower is better)
  • Hit MFU Ghost -> 0.07% (lower is better)
 

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
  • media/fileshare
  • 3.47GiB / 29.8GiB (freenas-boot)
  • 17.9TiB / 21.8TiB (tank)
  • 2.60GiB
    (Adaptive)2.25 (MRU: 61.53GiB, MFU: 61.53GiB) / 128.00GiB
  • Hit ratio -> 98.61% (higher is better)
  • Prefetch -> 6.72% (higher is better)
  • Hit MFU:MRU -> 96.45%:3.42% (higher ratio is better)
  • Hit MRU Ghost -> 0.00% (lower is better)
  • Hit MFU Ghost -> 0.00% (lower is better)
 

dgoodrich

Dabbler
Joined
Dec 18, 2013
Messages
10
Well, you clearly don't have enough RAM. You can see this by the hit ratio and the hit MFU:MRU ratio that are very low (especially for the first server) ;)

Yea, it's a drag. We have the max ram possible in the 45drives chassis, 16GB. To increase ram requires we pretty much throw out everything but the power supplies. I am still testing and collecting data in hopes we can find another solution for poor transfer rates. If it's just the lack of ram, well, it just is.

Thanks,

DAve
 
Top