Unable to use all physical RAM

[ErebusMaligan]

I am relatively new to FreeNAS and *nix systems in general.

I have been running a FreeNAS setup for maybe 3 months. I currently have 22 disks totaling 47 TB running in 11, 2 disk mirror pools.

This is strictly a single user, home use scenario, mostly video file storage. Generally reads and writes are done only by one other computer, not multiple simultaneously. While I am not currently seeing any performance issues, I did not notice until recently that only around half of the physically installed ram is being reported by the OS.

I am running (Yes I am aware these are not server components, and yes I am aware that I'm not using ECC ram... moving on):

OS: FreeNAS-9.3-STABLE-201501301837

CPU: AMD Phenom II 1055T Processor (6 core, non black)
MOBO: ASRock 990FX Extreme 9 (Max addressable ram 64 gb)
RAM: G.Skill Ripjaws 7-8-7-24 1600 (4 * 4GB)

The motherboard was new for this build, the other components were from one of my previous desktop builds.

The BIOS sees all the ram, the ram all passes memory test. The Web UI reports 8135 MB, running grep -i memory /var/run/dmesg.boot says:

real memory = 9126805504 (8704 MB)
avail memory = 8202178560 (7822 MB)

I took it apart this weekend and tried reseating all the RAM... although I doubted this would work since the BIOS sees it all fine, and the result was the same.


Aside from trying different known working hardware configurations, is there anything else I can try to get access to the other missing 8GB of ram?

 

[DrKK]

It's very hard to diagnose this. Your system is so insanely and ridiculously sub-par for this pool, that none of us are going to want to touch this with a ten foot pole, sir.

All I can tell you is that anything can, and does, happen when you run AMD desktop grade stuff, and are generally improperly specced.

 

[ErebusMaligan]

Thank you for taking the time to respond.

While i'm not really debating that my components aren't server grade... your post seems more hyperbolic than helpful. Insanely AND rediculously... that's quite a combo, and seems redundant.

Other than ECC ram and accompanying hardware that can support those features I'm not really sure what the difference (other than an unnecessary price markup) between desktop and server grade components would really be for the usage scenario I have.

The cpu is largely negligible as it gets almost no usage except when doing scrubs.

Most of the single socket server boards (including some I've seen recommended here) really don't seem to be anything special compared to most desktop boards (other than supporting ECC ram), other than a pointless price markup and less flexibility in add in card slots.

Obviously more ram is better, but considering I can't even get the 16 i have to register, that seems like a pointless investment.

Honestly I'm not really sure why the level of my components is relevant to what you what steps could even be remotely suggested in the scenario. Unless your blanket statement should be interpreted as FreeNAS (and BSD in general) has horrendous driver support... buy intel! kthxbye! Which... is basically just above useless.

 

[DrKK]

Have a good day sir.

 

[marbus90]

Go to a Tesla dealership (FreeNAS forums) and ask that they fix up your shitty old Lada (something something Desktop AMD).

nope.avi and have a good day, sir.

 

[ErebusMaligan]

Go to a Tesla dealership (FreeNAS forums) and ask that they fix up your shitty old Lada (something something Desktop AMD).

nope.avi and have a good day, sir.

Your analogy is fun, if a bit specious.

Let me provide a much more apt one:

I am a man dying of thirst... who shows up to the country club (FreeNAS forums) in my normal clothes (something something Desktop AMD). I ask you for a suggesting on how i would prevent dying of thirst. However because I am not wearing required dress, you cannot be bothered to answer.

Your analogy would be more accurately portrayed as: it 's more like i am asking "hey my car makes this loud humming/grinding noise when the window is down coasting at high speeds, any ballpark idea what i should look at?" And you knowing full well that it might be a bad wheel bearing refuse to answer because it's not a tesla.

Both analogies accurately reflect the amount of help being given... but I feel like only mine truly captures the amount of posturing and snobbery accurately.


Edit: It should be noted that I wasn't asking anyone to fix anything, i was asking for any ballpark suggestions as to what i could look at. I'm not asking someone to be responsible for my hardware choices, i'm responsible for my own problems... was just asking if anyone had an inkling of a suggestion aside from switching hardware... which I may end up doing when i get the money anyway.

It's also worth noting that no, I do not think i'm entitled to help, considering it is voluntary, especially if I choose to operate outside recommended specs. However responding to a help request with "i'm not going to help you" as opposed to just not replying, can only serve to be condescending... there is absolutely no other valid motive behind it.

 

[mjws00]

LOL. I wouldn't let you in the club in normal clothes. But we would lend you a jacket. Plus the sprinklers will come on at 11pm... feel free to wait. Also happy to change Lada bearings wouldn't go near Tesla. In addition Intel or GTFO. Plus kthanksbye.

Now onto more useful less tongue in cheek comments:

Isolate the hardware vs. Software. Will BSD 9.3 see all the ram? How about windows, linux, or even a fresh install of FN. What about 9.2.1.9... any changes?

What about 1,2 3, or 4 of the sticks of RAM. What do different combos do? Does a different speed or slot matter. Break it down into components and test one thing at a time.

Unfortunately I don't know that hardware and have no interest in it... but there is no difference troubleshooting it vs any other Intel board. The difference is you may fire up BSD vs Linux or Windows and find out the board just doesn't work as anticipated.

Good luck.

 

[ErebusMaligan]

Thanks for you response and sense of humor.

It's quite possible i'd just happen upon the sprinklers by accident then and solve that problem indeed. However in the case of my ram, the analogy loses translation, and am unlikely to just stumble on an answer there.

Many of your suggestions are exactly what I was going to try next.

My next step was going to be trying a linux livecd and see if it reports the ram properly.

I have already considered testing the ram sticks individually or in varying combos to identify if it's a stick, or a slot, or what, that FN is not seeing. Because of the fact that the 4u case is in a rack without rails, and is extremely heavy, it's time consuming to do... all the disks have to be removed in order to move it... etc.

So i figured before doing that I would just check if there was something obvious I was missing or could check software/config-wise.

It is absolutely possible the board though it reports fine in bios, may not be allowing the os to detect the ram... i never used the board before in any previous build, so it is indeed possible it has been bad from the start and I just didn't notice.

Just before reading this I thought well maybe just another FN re-install would help... but I guess i'll save that for last, as if the board really is the problem that won't likely help one bit.

 

[mjws00]

A FN reinstall MAY help. Dunno if something got limited or borked along the way. That is a quick test. I'd plug in a 9.2.1.9 usb all day, it takes seconds. But my gut says dmesg should be written by the kernel, if we can't see the RAM there then we are dead long before ZFS or any significant FN components are in play. i.e ZFS can't use the mem but FN sees it.

Truthfully, I hate the choice of mobo, proc, 8GB (working), in a 22 disk 47TB rig. It smells like pain is coming. But you owned your choices... so be it.

 

[ErebusMaligan]

A FN reinstall MAY help. Dunno if something got limited or borked along the way. That is a quick test. I'd plug in a 9.2.1.9 usb all day, it takes seconds. But my gut says dmesg should be written by the kernel, if we can't see the RAM there then we are dead long before ZFS or any significant FN components are in play. i.e ZFS can't use the mem but FN sees it.

Truthfully, I hate the choice of mobo, proc, 8GB (working), in a 22 disk 47TB rig. It smells like pain is coming. But you owned your choices... so be it.

Well, it's more out of necessity than choice that I have the components I have. Now granted, trying FN was a choice... and overall I'm pretty happy with that, but I simply didn't and don't have the money to support better components at the moment. My previous storage solution was just a standard windows 7 x64 box that i used robocopy scripts to backup from copy A to copy B of each drive daily. Honestly that worked just fine for what I do (infrequent changes of data, mostly one time write storage forever)... but FN just seemed like a fun project to try out.

I tried the mint cd i had laying around, and it only sees 8gb as well... so it's looking like a board issue at this point anyway. But i'm probably going to take it apart and figure out what slots or what sticks are specifically having problems anyway. Might be the push I need to switch components anyway... since even if I wanted to RMA the board with the manufacturer, I can't be without a working system while I wait.

Thanks for your help.

 

[depasseg]

Looks like it's an issue with that hardware. Not a FN issue.

http://lmgtfy.com/?q=ASRock+990FX+Extreme+only+sees+half+ram

 

[ErebusMaligan]

Looks like it's an issue with that hardware. Not a FN issue.

http://lmgtfy.com/?q=ASRock+990FX+Extreme+only+sees+half+ram

Thanks, but none of those links are actually specific to my motherboard model, and in most of those the causes and symptoms seem to all be wildly different from each other... (in some cases certain banks aren't working, in some cases all banks seems to be working but still only reporting half, in some cases the bios isn't properly detecting, etc.)

All the same I'm inclined to agree it's a hardware issue of some kind at this juncture.

 

[Ericloewe]

Go to a Tesla dealership (FreeNAS forums) and ask that they fix up your shitty old Lada (something something Desktop AMD).

nope.avi and have a good day, sir.

Did someone call?

*Heavy steps*
*Hat levitates*
Nope.
*Stretches neck*

In all seriousness, let's break this down a bit:

• You'll never get all the RAM available. PCI address space, Chipset configuration space,... all take up room. Windows helpfully tells me it's 107MB out of 16GB on my desktop.
  
• 900MB sounds like way too much. Got any video cards trying to grab some system RAM?
• Any weird BIOS settings?

Other than that, you'll have to follow general troubleshooting steps. No way around it, unfortunately.

 

[HoneyBadger]

Check your memory slots. The 990FX chipset is notorious for not behaving with large sticks of RAM in dual channel mode - move your DIMMs to A1 and A2. The small loss from going to single-channel will be more than made up for by having 16gb instead of 8gb.

And yeah, usual disclaimer about consumer-grade boards, non-ECC RAM, etc. If you care about this data, get onto a proper solution ASAP.

 

[ErebusMaligan]

Check your memory slots. The 990FX chipset is notorious for not behaving with large sticks of RAM in dual channel mode - move your DIMMs to A1 and A2. The small loss from going to single-channel will be more than made up for by having 16gb instead of 8gb.

And yeah, usual disclaimer about consumer-grade boards, non-ECC RAM, etc. If you care about this data, get onto a proper solution ASAP.

It's definitely an issue with the board. Not sure if it's a problem endemic to the model, or if I just have a specific bad board. I have 4 4gb sticks so it's not really an option to switch them, either way it's reducing the ram to a max of 8. Tried other ram, same issues. Getting the board replaced, arriving tomorrow.

As for a proper solution, I have a third entire set of disks that is not on a freenas system that is an offsite backup... the data is backed up from the original source not the freenas system, so I'm not super concerned with data loss. If necessary, rebuilding the data from the offsite backup is an annoyance, but not an impossibility. The offsite disks are also not in a system once filled, so no additional wear. Absolute worst case, and all 3 disks fail/corrupt etc... none of the data is irreplaceable, just would take time to reacquire.

 

[HoneyBadger]

It's definitely an issue with the board. Not sure if it's a problem endemic to the model, or if I just have a specific bad board. I have 4 4gb sticks so it's not really an option to switch them, either way it's reducing the ram to a max of 8. Tried other ram, same issues. Getting the board replaced, arriving tomorrow.

Pretty specific to the 990FX chipset itself as far as I can tell. Ask my why my current gaming rig is an Intel. ;)

As for a proper solution, I have a third entire set of disks that is not on a freenas system that is an offsite backup... the data is backed up from the original source not the freenas system, so I'm not super concerned with data loss. If necessary, rebuilding the data from the offsite backup is an annoyance, but not an impossibility. The offsite disks are also not in a system once filled, so no additional wear. Absolute worst case, and all 3 disks fail/corrupt etc... none of the data is irreplaceable, just would take time to reacquire.

Emphasis mine; as long as you realize that the data's at risk of corruption and validate as such, that gets a (conditional) pass.

 

[ErebusMaligan]

Pretty specific to the 990FX chipset itself as far as I can tell. Ask my why my current gaming rig is an Intel. ;)

So is mine now. Previously it was the chip and ram that is in that FN box. The intel upgrade and the 980gtx I just put in my main rig are part of the reason the FN box gets a pass on component upgrades for the time being. Only so much in the way of funds available, I chose my priorities.

Pretty much done with AMD components all around for the future. I haven't had much in the way of problems with them other than a dual monitor glitch in their video cards that has been around for years that never got fixed... But overall they just aren't keeping up, and they use too much power and run too hot because of it. Despite using AMD exclusively for over a decade, I'd really only recommend them now for budget desktops with their apu line.

Emphasis mine; as long as you realize that the data's at risk of corruption and validate as such, that gets a (conditional) pass.

Absolutely. As I've said, I would like to update to more recommended components when I can... but in the meantime, for my specific use case and setup, the risk of irretrievable data is low. Even if it comes to pass, it would be just disappointing, not a life changing event.