Freenas not using all ram. Disabling autotune – thoughts?

[Bobuser]

All, I've googled this topic and really did not find an answer.


I have a 64 bit system with 5gb ram, using zfs for basic file storage (sync from windows).

I know… supposed to have A LOT of ram to use zfs but I'm not using compression or deduplication and it seems to behave well – so far. I know more is better but for now 5gb is all I have. At this point I'm just testing the system.


At first, looking at reports, it showed all 5gb ram being used as expected. After a while – it went to a flat line 1gb no matter what the load. I finally suspect the “autotune” kicked in….

I modified the “tuneables” by doubling the numbers – figuring x gives me 1gb, 2x should at double it . It did not – I used 3x the tuneables number (assuming bytes) – still no difference in the memory used.


What I did was disable autotune, and delete all the tuneables and sysctl values – rebooted and now my memory “seems” to be used as expected –increases up to 5 gb with usage and decreases when not used. It very nice to see it behave as expected!!

My question is… do you see a issues with not using autotune – when using it actually RESTRICED resources being used. No matter how much of a load I tried – the report and top would never get above 1gb

Interesting observation is, I start a large copy and watch free memory in top… it slowly climbs until it almost uses 5gb (about 3-5 min) question here is that process uses that much ram – if another process starts – where does its ram come from?

This link has screen shots of my system. ANY feedback is welcome. Thanks!

https://sites.google.com/site/racdebugging/freenas

Bob

 

[cyberjock]

Autotune is only recommended for systems with 16GB+ of RAM. Not sure why autotune would cause the effect you are seeing where all is fine and then suddenly decreases to 1GB of RAM used. The Autotune adjusts the RAM settings on bootup. They wouldn't be changing while you are using the machine. My first guess is that the ZFS cache is being forced to release itself completely because some other process needs more RAM. But I don't have any scientific evidence for your exact system since every system performs differently.

If another process needs RAM, one of two things happens:

1. Kernel panic. You can render your zpool unmountable(loss of all data). This is the main reason why you shouldn't have less than 6GB of RAM.
2. Other things using RAM are forced to resize themselves down. Usually the ZFS caches suffer, which hurts performance.


In your case you appear to be lucky in that #2 seems to be happening frequently. If you have #1, you may be in for a rude awakening in terms of where your ZFS data went. #2 is only hurting your ZFS system performance.. right now.

 

[Bobuser]

ok, I hear you. But I think it’s very strange that disabling auto tune "freed up" my memory. I looked at a report of today and seems the ram is pegged out, with no real workload.

I’m just confused on the memory being locked and now wide open.
New image uploaded
https://sites.google.com/site/racdebugging/freenas

I think if data integrity was compromised ("lost") due to less than required memory - that wouldn’t be much of a system. I have more faith in freenas than that. + (my backups have backups) But that’s just me. I just want to USE all the resource I have. Seeing 1gb used was not good.

Id bet If I re enabe autotune I would see the same resource constraint

I

 

[cyberjock]

You just might.. but as I said.. autotune should be off unless you have 16GB+ of RAM.

Your RAM should be near 100% utilized all the time. The cache will consume all RAM(minus a very small percentage) and will never flush unless you force the cache to flush, reboot, export the zpool, etc. Unlike Windows, the intent is to use all available resources to maximize performance.

The key is that "percentage" might be 2%(not sure what it is.. made up the number), which at 5GB is only 102.4MB, but if you have 16GB of RAM, its 327.68MB. Guess which system will be more reliable?

As for your comment that it wouldn't be much of a system with less required memory.. keyword.. "required". Thats how much you should have.. at the minimum. If you chose to ignore what the designers recommend, well, that's your own fault if you lose your data. And to be technically correct, the thumbrule is 6GB of RAM + 1GB per TB of storage. You're below the thumbrule with zero storage.

 

[drich1982]

I have been reading and this seems like the best place for my question as.

I have a simple setup with:
4 x 7,200 RPM drives
6 core 3.7 64 bit processor
32GB of DDR3 (1866 non ECC)
LSI 9122-8i with IT firmware
Intel 1GB nic (PCI express)

I am using ZFS with an ISCSI share to my Win2008 R2 server.

I am seeing max w/r speeds of 70mb/s but it creeps down to an average of 27~34mb/s. I notice my RAM is not being used as well. I have enabled auto-tune rebooted and do not see a difference at all. I am looking at the GUI and using top command. I am also using Roadkil to test speeds.

I am not sure if the auto-tune is really making a difference as was stated above the RAM should almost be used up at all times. I am only seeing about 1GB used max of 3 GB under load. Any suggestions or additional tests that would be good to do?


Thanks!!!

 

[cyberjock]

Someone else just asked a question about iscsi on ZFS. You should find that thread as it includes a link to a support ticket with stuff you can do. In short, iscsi on ZFS isn't always the best way to go. If you have anything else to say you really should start your own thread.

 

[drich1982]

Sorry. I guess I was just trying to give as much info as possible. I was really aiming at the system not using all the RAM. I will look into the other post as well. I definitely was not trying to hijack the post just seemed like the same issue I was experiencing.

 

[cyberjock]

If you are planning to use ZFS, not using all of the system RAM really isn't an option. ZFS performs well because of RAM. ZFS, by default, uses pretty much all the system RAM you have.

 

[papageorgi]

as cyberjock was saying zfs does love ram. i am using an older consumer desktop motherboard so i'm limited to 16GB of RAM so i maxed it out. my pool performs well with this configuration and autotune but when it had less it was a dog and slow with zfs but not ufs. since you need a backup of the data on the server you may want to use that file system instead. since i'm at the ram limit of the board i may use ssd's since today they're inexpensive to improve write performance since read is already sufficient for my needs.