Just upgraded from 4GB to 16GB memory. Now what?

[_h9]

Hello.

Hopefully this is a quick question!

I have a 10x2TB FreeNAS 8.3.1 server, with a 3.2GHZ AMD Quad-Core processor and 4GB of ram.
I also have a pair of 300GB 10000RPM disks mirrored as a zlog. Probably for no real reason.

The 10 disks are set up in a RAIDZ2, and the pair of disks is its ZLOG.

It worked pretty well, though I'd occasionally get a bit of stuttering while multiple video playback was happening.

I happened to need RAM for my newer backup server, so I thought I'd upgrade the ram in system and pass it down.

I went from 4x1GB --> 4x4GB unbuffered ECC memory.
From what I can tell I'm not getting any stuttering anymore even while running 3 or more movies from the server.

I have a screen-cap of the memory graph from before and after the RAM upgrade [here]

My question is, is having 16GB of ram over-kill? I stream media from this server, and do backups from workstations but not much else.

last pid: 6904; load averages: 4.11, 4.15, 4.17 up 0+01:19:51 13:47:02
50 processes: 2 running, 48 sleeping
CPU: 7.7% user, 0.0% nice, 72.4% system, 7.4% interrupt, 12.5% idle
Mem: 126M Active, 1313M Inact, 1084M Wired, 2388K Cache, 188M Buf, 13G Free
ARC: 690M Total, 166M MFU, 506M MRU, 34K Anon, 14M Header, 3284K Other
Swap: 24G Total, 24G Free

[c@fileserver3] /mnt/ztank/storage# zpool status -v ztank
pool: ztank
state: ONLINE
scan: scrub in progress since Sat Mar 30 12:50:24 2013
1.35T scanned out of 8.86T at 536M/s, 4h4m to go
0 repaired, 15.27% done
config:

NAME STATE READ WRITE CKSUM
ztank ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
gptid/31c8998a-01c6-11e2-a282-10bf48bc0a58 ONLINE 0 0 0
gptid/3264d8e4-01c6-11e2-a282-10bf48bc0a58 ONLINE 0 0 0
gptid/32f906d3-01c6-11e2-a282-10bf48bc0a58 ONLINE 0 0 0
gptid/33927e63-01c6-11e2-a282-10bf48bc0a58 ONLINE 0 0 0
gptid/34323a0a-01c6-11e2-a282-10bf48bc0a58 ONLINE 0 0 0
gptid/34d6ef03-01c6-11e2-a282-10bf48bc0a58 ONLINE 0 0 0
gptid/357d9e9d-01c6-11e2-a282-10bf48bc0a58 ONLINE 0 0 0
gptid/36215b70-01c6-11e2-a282-10bf48bc0a58 ONLINE 0 0 0
gptid/36cafc55-01c6-11e2-a282-10bf48bc0a58 ONLINE 0 0 0
gptid/37484e46-01c6-11e2-a282-10bf48bc0a58 ONLINE 0 0 0
logs
mirror-1 ONLINE 0 0 0
gptid/37bad92e-01c6-11e2-a282-10bf48bc0a58 ONLINE 0 0 0
gptid/38018501-01c6-11e2-a282-10bf48bc0a58 ONLINE 0 0 0

errors: No known data errors
[c@fileserver3] /mnt/ztank/storage#

I haven't done any tuning, other than the "Enable Autotune" Feature, which added the following.

net.inet.tcp.sendbuf_max 2097152 Generated by autotune True
kern.ipc.maxsockbuf 2097152 Generated by autotune True
net.inet.tcp.recvbuf_max 2097152 Generated by autotune True

Prefetch is also enabled, which I believe is what I want for my setup. There aren't any ZFS variables in my /boot/loader.conf file, either.

 

[gpsguy]

Overkill - absolutely not. And, as you said "Prefetch is also enabled", now that you have more than 4Gb RAM.

Read section 1.4.2 of the manual, regarding RAM requirements.

If you asked for confirmation before you built the system, an answer might have been to buy a board that will accommodate 32Gb of RAM and start with a minimum of 16Gb.

My question is, is having 16GB of ram over-kill?

 

[RAJOD]

Overkill? depends on who ya ask. To some 128 gig would not be overkill, they would also tell you to overclock.


I experiment for a 6 TB NAS. from 2, 8 and 16 gb.

2 to 8 big jump in intel NAS bench tests. File copies went from 30s to 70-90MBS across my home net.
I don't have compression on, and don't use that dup feature (which eats ram)

From 8 to 16 almost zero difference, but I did not attempt to do 3 HD streams from diff TVs in house as that will never happen in my situation.

I did not test 2 and 4 gig DDR 3 which would help me determine how much of the increase was due to cpu change.
I don't have any small DDR3 sticks or I would.

having 16 might pull a lil more juice from the wall socket but other than that why take it out?

For most people 8 Gb is probably the point of diminishing returns. And if you not using ZFS that point is even less.

 

[_h9]

One idea was I could put 8GB in this server, and 8GB in my backup server, and get the same performance.
But knowing myself I will probably just buy 8GB more.

This motherboard also supports 32GB out of the box, its an ASUS M5A97 PRO.
It has an ECC option in the BIOS that seems to work with my unbuffered ECC memory.
The cost of the sticks was the "got-cha" for me.

The motherboard I have for the "backup" server is a ASUS M4A7BLT-M LE (with Athlon II Dual Core) which according to the manual also supports ECC memory option in the BIOS.


I guess I feel like I still don't know if there is any additional tweaks I should do after upgrading the memory.

I know ZFS tuning is sort of frowned upon, so I am reluctant too.

I did notice that there are some ZFS tweaks put into the auto-tune fields now.

I'm just sitting on so much "free" memory according to the charts, does that translate directly into ZFS performance as the box is put under load? Or is the ZFS performance (max arc size etc) indicated completely by the variables in /boot/loader.conf?

Overkill? depends on who ya ask. To some 128 gig would not be overkill, they would also tell you to overclock.


I experiment for a 6 TB NAS. from 2, 8 and 16 gb.

2 to 8 big jump in intel NAS bench tests. File copies went from 30s to 70-90MBS across my home net.
I don't have compression on, and don't use that dup feature (which eats ram)

From 8 to 16 almost zero difference, but I did not attempt to do 3 HD streams from diff TVs in house as that will never happen in my situation.

I did not test 2 and 4 gig DDR 3 which would help me determine how much of the increase was due to cpu change.
I don't have any small DDR3 sticks or I would.

having 16 might pull a lil more juice from the wall socket but other than that why take it out?

For most people 8 Gb is probably the point of diminishing returns. And if you not using ZFS that point is even less.

 

[Stephens]

I guess I feel like I still don't know if there is any additional tweaks I should do after upgrading the memory.

Try autotune.

 

[joeschmuck]

My two cents...

16GB is not overkill at all and since you have already turned on the autotune then those values should be set to use your RAM better. If you are using CIFS you might also consider adding a few optional parameters which have proven to help out some people... Refer to this posting: http://forums.freenas.org/showthread.php?11925-FreeNAS-8-3-1-RELEASE&p=56252&viewfull=1#post56252

Now as far as your system goes, the log drives, well I just don't see any use for those in a system that will only be streaming a few movies and doing backups in. Log drives are great for material that is repeatedly accessed and that is it. You should remove the log from your system and see if you can watch 3 movies at the same time, should be no difference. I'm sure those drives could be used in some other system you have. Or maybe make a cache out of them for your FreeNAS system, that could increase your write throughput on your backups, assuming you haven't already max'ed out your network adapter.