Hi all,
background: I am building a new fileserver box for our organization: As we work with digital video in multiple locations, whatever our replacement is has to be reliable for days (months, years) on end. Our current solution is quite stable.. but limited hard drive space, redundancy, and security, so I am working to deploy freeNAS as a replacement.
"Hardware": FreeNAS is running on hyper-v server, with the following "specs" for the VM: 3 cores of an AMD FX 6100 bulldozer (of 6), 5 GB of DDR3 ram (of currently 8) The network interface is a legacy network interface, FreeNAS is installed to an "IDE hard drive" (4 GB, vhd file, on the hyper v system drive right now). It also has attached (direct physical hard drisk, attached as IDE) a 1 TB drive (NTFS, I know, I didn't have a choice due to a shortage of drives to shuffle off the data and reformat: See second disk for future fix), which is setup as a mount point. Also attached, also as a physical drive passthrough to IDE, is a 2 TB, unformatted, unmounted drive (which will eventually be paired with another 2 TB drive, when it arrives, so I can setup raidz, move all the data to it, and get the dirty ntfs out of my system). I have been proceeding on the assumption that an odd number of processor cores and an odd amount of ram will not cause problems, but I wanted to list everything to be sure. The only other active VM on the physical server is a very small (one core, 512 MB of ram) PfSense (also bsd based) router/vpn/etc. FTP is enabled, no security, and there is an rsync module active (but this problem existed before I enabled rsync, and I think before ftp)
Problem: The webgui will "hang": If I have left it open, I can click on a couple things which I assume are already loaded on the local machine, but any new tasks, expanding anything, configuring a service, always results in a long wait state. Occasionally, the page will eventually load, but I have clocked it as at least ten minutes (10 minutes, 12 minutes, etc). Firefox seems to have the best rate of eventually loading it, but I assume this is just longer timeouts. I have tried it with firefox, chrome, and IE, and from multiple machines (virtual and physical) running windows 7 and also a linux machine (just to make sure). After the webgui hangs, new tabs opened, or new browsers pointed at the address, or refreshing the page will immediately come up with the title (which seems to indicate lighttpd is at least starting to respond) and then just stop responding.
Replication: I am not sure of the exact timeframe, but it happens every time: I have rebooted the system several times (Enough that I had to detach the disk and run a chkdsk in windows because the ntfs driver complained the volume had a problem, which chkdsk repaired)(this means that the problem witht he webgui existed before I had any ntfs problems that I am aware of), and the webgui works great after a reboot, but eventually always fails in the same way.
Troubleshooting/information: The hyper v manager tells me that when the webgui is taking forever to respond, it does not seem to be a resource problem, as the CPU runs from 0-2%. I can still access the "physical machine" to get to the console, and it works, responds correctly, etc. Running top, nothing seems to be eating CPU power, and there is still ram free. (the webgui is locked up right now, and the mem in top is listed as 117 active, 4001 inactive, 588 wired, 576(K) cache, 520 buf, 245 free. Load averages is 0.00, 0.01, 0.00, uptime 1 day, 19 hours. From another recommendation on this forum for a webgui problem, I saw someone had recommended "service nginx restart", which I tried, but I get an error that nginx does not exist in /etc/rc.d or the local startup (/usr/local/etc/rc.d). I can still access the ftp, ping the machine, so it seems to be only a webgui problem. Obviously, I can't really just reboot the machine once a day, and also, I want to find the root cause and fix it, instead of taking the "windows approach" of shrug and reboot. Since we are often transferring 3+ gb up or down to the file server over residential internet connections, it needs strong uptime because the speed on the other end is... sometimes not awesome.
Does anyone have any ideas? I am newish to BSD, but I have some experience in linux, mostly just enough to think things transfer, and then be confused when they do not. As this is not yet production, and I can snapshot the VM, I can easily take drastic steps to try to fix it, it just has to end up stable. ZFS features is what *really* sold me on freeNAS, although I am glad I eventually read that I cannot upconvert, so I have to wait for my additional drive to ship before I can build a raidz. The 1 TB hard drive is a borrowed drive from another machine, and is about 70% full and slowly growing, so I do have a time limit to get this all working.
background: I am building a new fileserver box for our organization: As we work with digital video in multiple locations, whatever our replacement is has to be reliable for days (months, years) on end. Our current solution is quite stable.. but limited hard drive space, redundancy, and security, so I am working to deploy freeNAS as a replacement.
"Hardware": FreeNAS is running on hyper-v server, with the following "specs" for the VM: 3 cores of an AMD FX 6100 bulldozer (of 6), 5 GB of DDR3 ram (of currently 8) The network interface is a legacy network interface, FreeNAS is installed to an "IDE hard drive" (4 GB, vhd file, on the hyper v system drive right now). It also has attached (direct physical hard drisk, attached as IDE) a 1 TB drive (NTFS, I know, I didn't have a choice due to a shortage of drives to shuffle off the data and reformat: See second disk for future fix), which is setup as a mount point. Also attached, also as a physical drive passthrough to IDE, is a 2 TB, unformatted, unmounted drive (which will eventually be paired with another 2 TB drive, when it arrives, so I can setup raidz, move all the data to it, and get the dirty ntfs out of my system). I have been proceeding on the assumption that an odd number of processor cores and an odd amount of ram will not cause problems, but I wanted to list everything to be sure. The only other active VM on the physical server is a very small (one core, 512 MB of ram) PfSense (also bsd based) router/vpn/etc. FTP is enabled, no security, and there is an rsync module active (but this problem existed before I enabled rsync, and I think before ftp)
Problem: The webgui will "hang": If I have left it open, I can click on a couple things which I assume are already loaded on the local machine, but any new tasks, expanding anything, configuring a service, always results in a long wait state. Occasionally, the page will eventually load, but I have clocked it as at least ten minutes (10 minutes, 12 minutes, etc). Firefox seems to have the best rate of eventually loading it, but I assume this is just longer timeouts. I have tried it with firefox, chrome, and IE, and from multiple machines (virtual and physical) running windows 7 and also a linux machine (just to make sure). After the webgui hangs, new tabs opened, or new browsers pointed at the address, or refreshing the page will immediately come up with the title (which seems to indicate lighttpd is at least starting to respond) and then just stop responding.
Replication: I am not sure of the exact timeframe, but it happens every time: I have rebooted the system several times (Enough that I had to detach the disk and run a chkdsk in windows because the ntfs driver complained the volume had a problem, which chkdsk repaired)(this means that the problem witht he webgui existed before I had any ntfs problems that I am aware of), and the webgui works great after a reboot, but eventually always fails in the same way.
Troubleshooting/information: The hyper v manager tells me that when the webgui is taking forever to respond, it does not seem to be a resource problem, as the CPU runs from 0-2%. I can still access the "physical machine" to get to the console, and it works, responds correctly, etc. Running top, nothing seems to be eating CPU power, and there is still ram free. (the webgui is locked up right now, and the mem in top is listed as 117 active, 4001 inactive, 588 wired, 576(K) cache, 520 buf, 245 free. Load averages is 0.00, 0.01, 0.00, uptime 1 day, 19 hours. From another recommendation on this forum for a webgui problem, I saw someone had recommended "service nginx restart", which I tried, but I get an error that nginx does not exist in /etc/rc.d or the local startup (/usr/local/etc/rc.d). I can still access the ftp, ping the machine, so it seems to be only a webgui problem. Obviously, I can't really just reboot the machine once a day, and also, I want to find the root cause and fix it, instead of taking the "windows approach" of shrug and reboot. Since we are often transferring 3+ gb up or down to the file server over residential internet connections, it needs strong uptime because the speed on the other end is... sometimes not awesome.
Does anyone have any ideas? I am newish to BSD, but I have some experience in linux, mostly just enough to think things transfer, and then be confused when they do not. As this is not yet production, and I can snapshot the VM, I can easily take drastic steps to try to fix it, it just has to end up stable. ZFS features is what *really* sold me on freeNAS, although I am glad I eventually read that I cannot upconvert, so I have to wait for my additional drive to ship before I can build a raidz. The 1 TB hard drive is a borrowed drive from another machine, and is about 70% full and slowly growing, so I do have a time limit to get this all working.
