SOLVED Copying 1GB Files From NAS to Windows Causes Network Crash

[ttabbal]

I've been using 192.168.1.105 as my Static IP for the Intel NIC, is that much different then 10.10.2? I know 192.1.68.X.X is a local IP based on network, verses global IP seen from the outside, what kind of IP is 10.X.X.X, is it a IPv4 setting? The current netmask is 255.255.0.0.

Which address range you use isn't important. I was looking to ensure you could connect without DHCP and such. Sounds like you can. The 10.0.0.0/8 range is ipv4, it's reserved for private networks so it's safe to use without issues.

From the more recent post, it sure sounds like Windows 10, at least the install you're using, is causing a problem.

As for server grade hardware, it is nice. And it doesn't have to cost a ton. Shop around for previous generation gear, used server gear is all over. I prefer to test it thoroughly, but if it checks out, I'd take that over a new consumer grade setup any day. As a reference point, I got a dual cpu board, 2x quad core Xeons, and 98gb ECC ram for about 160 shipped. Nothing really special, but more than enough for a nice filer.

That said, my previous setup was a gaming board, AMD CPU, 8gb ECC RAM. Worked great for about 7 years. Good quality consumer gear is out there.

 

[Baptiste]

Which address range you use isn't important. I was looking to ensure you could connect without DHCP and such. Sounds like you can. The 10.0.0.0/8 range is ipv4, it's reserved for private networks so it's safe to use without issues.

From the more recent post, it sure sounds like Windows 10, at least the install you're using, is causing a problem.

Yeah, when I get the chance I'll run a fresh install of Windows 10 and 7 off a USB and see if either work. If neither work though... I'll be just totally lost.

As for server grade hardware, it is nice. And it doesn't have to cost a ton. Shop around for previous generation gear, used server gear is all over. I prefer to test it thoroughly, but if it checks out, I'd take that over a new consumer grade setup any day. As a reference point, I got a dual cpu board, 2x quad core Xeons, and 98gb ECC ram for about 160 shipped. Nothing really special, but more than enough for a nice filer.

That said, my previous setup was a gaming board, AMD CPU, 8gb ECC RAM. Worked great for about 7 years. Good quality consumer gear is out there.

Hey, your previous set up is basically exactly what I am using now, minus ECC. I did do a little shopping online for some server gear, found a Intel NIC 4gb/s Optical Fiber for dirt cheap, picked that off ebay and shopping around for a second. That being said, not a fan of buying used. I personally wouldn't buy a CPU or Mobo (basically core components) off of ebay. Like the only reason I bought the Asus board from my friend is cause it was still under warranty.

 

[maglin]

Server components as a package is usually either end of life and probably still good for some time or something that was upgraded to save on power usage. It's amazing how spending money to replace working gear can save you a ton in the long haul.

I would say it's an issue with your windows. Did you uninstall your windows NIC drivers and reinstall them yet? Maybe the cable going to that machine has a bad crimp on a connector. Check there.


Sent from my iPhone using Tapatalk

 

[Baptiste]

Server components as a package is usually either end of life and probably still good for some time or something that was upgraded to save on power usage. It's amazing how spending money to replace working gear can save you a ton in the long haul.

So I've noticed! Like I said I picked up a 4gb's Intel NIC for dirt cheap online before and found a second I'm working on now.

I would say it's an issue with your windows. Did you uninstall your windows NIC drivers and reinstall them yet? Maybe the cable going to that machine has a bad crimp on a connector. Check there.

I did update the NIC driver, will uninstall and reinstall when I get home.
NOTE: Seeing has how I said before I had the AMD board in my current computer, I also went and completely removed any AMD driver that might be conflicting.
I really hope it's not a Windows 10 permeant issue cause going back to Windows 7 will be a major hassle. But if it must be done it must be done, I'll do my fresh reinstall on the USB check when I get home.

 

[jgreco]

...idly wonders what a "4gb's Intel NIC" is. If you've actually picked up a fiber channel 4GB card, that's useless. Plus probably not worth what you paid...

 

[Sakuru]

...idly wonders what a "4gb's Intel NIC" is.

I'm assuming a 4 port card

 

[Baptiste]

...idly wonders what a "4gb's Intel NIC" is. If you've actually picked up a fiber channel 4GB card, that's useless. Plus probably not worth what you paid...

I just meant a 4Gb/sec NIC. Paid 12 bucks for it and its Fiber Optics, why is that wrong?

Again not good with the whole network thing.

 

[ttabbal]

I just meant a 4Gb/sec NIC. Paid 12 bucks for it and its Fiber Optics, why is that wrong?

Again not good with the whole network thing.

I suspect what you got is a FC card, fiber channel. It's basically a disk controller. If you had a matching one, you might be able to link them, but it's likely undocumented and more difficult than it's worth.

If you want faster, you probably need 10gb SFP+ cards and matching transceivers and fiber. Probably better to get 1gb running solid before messing with it.

 

[Baptiste]

I suspect what you got is a FC card, fiber channel. It's basically a disk controller. If you had a matching one, you might be able to link them, but it's likely undocumented and more difficult than it's worth.

If you want faster, you probably need 10gb SFP+ cards and matching transceivers and fiber. Probably better to get 1gb running solid before messing with it.

I figured 4Gb/s was enough as its equal to 500MB/s which is the speed of the SSD I plan on putting in the NAS as the cache SSD.
I was looking for a second 4Gb/s to link them together. How come FC is useless?
Here's the card in question: http://www.ebay.com/itm/331310378018

And I wasn't planning on installing this at all till I get the 1Gb working.

 

[Baptiste]

Btw, working on installing Win 10 to a USB now, and them there I'll create a RAM disk again and see whats up.

EDIT: As a note, since I had both FreeNAS and Xpenology create raids and do who knows what to my 5 HDDs, so running DBAN on them to give them a nice fresh start and formatted USB with the OSs on them so I'll do a clean install of those too.

 

[ttabbal]

I figured 4Gb/s was enough as its equal to 500MB/s which is the speed of the SSD I plan on putting in the NAS as the cache SSD.
I was looking for a second 4Gb/s to link them together. How come FC is useless?
Here's the card in question: http://www.ebay.com/itm/331310378018

And I wasn't planning on installing this at all till I get the 1Gb working.

FC is technically a network, but it's not a general purpose network. It's designed to connect to storage systems. It might be possible to do a mode where ethernet frames are passed over the link, but I don't think I've ever heard of it being done. It was/is more common with infiniband, but I don't think freenas supports IB. You can get a pair of real 10gb cards for $30 on ebay. One example of a working card..

http://m.ebay.com/itm/282042392246

The DAC is only useful for short links. You need to add transceivers and fiber for longer runs. I believe that's a Mellanox card, which is kind of supported in freenas. I had to compile the driver. I think an update is supposed to add the driver soonish. That card does ethernet frames, and is directly compatible with 10gb ethernet switches and other cards.

I did find some info on IP over FC... So I guess it's possible. But even people who have done it don't recommend it....

http://serverfault.com/questions/300699/how-do-i-set-up-ip-over-fc

 

[Baptiste]

FC is technically a network, but it's not a general purpose network. It's designed to connect to storage systems. It might be possible to do a mode where ethernet frames are passed over the link, but I don't think I've ever heard of it being done. It was/is more common with infiniband, but I don't think freenas supports IB. You can get a pair of real 10gb cards for $30 on ebay. One example of a working card..

http://m.ebay.com/itm/282042392246

The DAC is only useful for short links. You need to add transceivers and fiber for longer runs. I believe that's a Mellanox card, which is kind of supported in freenas. I had to compile the driver. I think an update is supposed to add the driver soonish. That card does ethernet frames, and is directly compatible with 10gb ethernet switches and other cards.

I did find some info on IP over FC... So I guess it's possible. But even people who have done it don't recommend it....

http://serverfault.com/questions/300699/how-do-i-set-up-ip-over-fc

Thanks for the info! I'll look into it.
Seems like a totally great price for a 10Gbs card, too bad it's not Intel :p

As for me testing a fresh Win 10, it'll have to wait till tomorrow. DBAN said it would take 2 hours, and 2 hours later it's only 55 percent done. Sucks, but oh well.

EDIT: Is there a page perhaps you can link me to so I can read about RJ45, FC, FO, and stuff like that?

 

[maglin]

The 40G IB stuff is cheap and will work at 10G speeds but you have to get an IB switch like the topspin 120.

All the big boys are upgrading their clusters to the newer 100G IB and soon the 200GB IB that is coming. That means a lot of cheap 40G gear. The IB does require you to install it's driver on FreeNAS so updates are a bit tougher.

Your 4G stuff is very old and probably not worth messing with. Don't waste any more money on it. Look at what is currently supported and get it. The 10G stuff is getting cheaper. It's the switch that is still kind of pricey.


Sent from my iPhone using Tapatalk

 

[ttabbal]

Thanks for the info! I'll look into it.
Seems like a totally great price for a 10Gbs card, too bad it's not Intel [emoji14]

As for me testing a fresh Win 10, it'll have to wait till tomorrow. DBAN said it would take 2 hours, and 2 hours later it's only 55 percent done. Sucks, but oh well.

EDIT: Is there a page perhaps you can link me to so I can read about RJ45, FC, FO, and stuff like that?

Heh.... Intel is the go to for 1G, but there are plenty of options for 10G. Great place to start.

https://forums.freenas.org/index.php?threads/25749/

 

[Baptiste]

Heh.... Intel is the go to for 1G, but there are plenty of options for 10G. Great place to start.

https://forums.freenas.org/index.php?threads/25749/

Oh so if it's 10Gbs Intel is not really needed as much? How come? Just more driver support for them in FreeNAS? And thanks, I'll keep that link open in a tab and read it when I get a chance!
On that note, DBAN just finished, so maybe I'll have time to test fresh Win 10 tonight.

 

[Baptiste]

The 40G IB stuff is cheap and will work at 10G speeds but you have to get an IB switch like the topspin 120.

All the big boys are upgrading their clusters to the newer 100G IB and soon the 200GB IB that is coming. That means a lot of cheap 40G gear. The IB does require you to install it's driver on FreeNAS so updates are a bit tougher.

Your 4G stuff is very old and probably not worth messing with. Don't waste any more money on it. Look at what is currently supported and get it. The 10G stuff is getting cheaper. It's the switch that is still kind of pricey.


Sent from my iPhone using Tapatalk

Thank you, I'll look into it!

 

[jgreco]

Oh so if it's 10Gbs Intel is not really needed as much? How come? Just more driver support for them in FreeNAS? And thanks, I'll keep that link open in a tab and read it when I get a chance!
On that note, DBAN just finished, so maybe I'll have time to test fresh Win 10 tonight.

The right Intel stuff is fine for FreeNAS, but there are more challenges with 10G. For example, I believe we just had a member that was having a lot of problems with the X520 on Windows 10, whereas if I had to pick an Intel 10G card to use with FreeNAS, it'd be the X520. In general, FreeNAS does better with slightly older, heavily deployed hardware. For 10G, the Intel choice is probably the X520. But the Chelsio T420-CR may be a better choice.

 

[Baptiste]

The right Intel stuff is fine for FreeNAS, but there are more challenges with 10G. For example, I believe we just had a member that was having a lot of problems with the X520 on Windows 10, whereas if I had to pick an Intel 10G card to use with FreeNAS, it'd be the X520. In general, FreeNAS does better with slightly older, heavily deployed hardware. For 10G, the Intel choice is probably the X520. But the Chelsio T420-CR may be a better choice.

Oh, I think I understand.

Basically newer stuff or cheaper stuff, Intel is preferred as it is mostly supported. But for higher end things, old is better as long as the company isn't some weird shady company. This being due to older NICs having better driver deployment and the bugs all ironed out?

 

[Baptiste]

On that note, I have some really really weird and interesting (and possibly good) news.
I think I got it working at 10MB/s speeds, but I have no idea how. Also I need to do more testing to confirm that it works, maybe I just got lucky last night.

The following are more or less my steps exactly.

So like I said before, after installing FreeNAS and then Xpenology I wanted to DBAN my drives just to get a fresh start. I did the quick format (just zeroes out all the sectors), 1 pass. While that was happening I did a few things.
I wanted to also make sure my USBs were clean before doing a fresh reinstall of FreeNAS. Now some of my USBs had been formatted on Mac OS which leaves behind a EFI that simply formatting in Windows won't get rid of. So I went ahead and did a DISKPART, selected my disks, and did a CLEAN command on each of them. Then I reinstalled the FreeNAS installer and installed FreeNAS to a USB. Meanwhile, I had tried to install Windows 10 to a USB to boot from, I must have done something wrong as I couldn't get it to boot. So I went to my BIOS to see if maybe Fast Boot or some stupid crap was on.
THIS IS THE IMPORTANT PART.
Once in my BIOS (remember I have the Asus Z87 motherboard) I noticed something I have never seen before. At the top, where it says AI Tweakier, Advanced, Boot, Tools, ETC... I saw 2 circles. One blue and one black. This was at the very very top right, they have never been there and a quick google search before going to bed and I could not find it. Once clicking that black circle, it brought me to a whole new BIOS page with a NIC (might have just been called network or something) settings. Besides showing info such as Intel NIC, it had two options. A Wake On Lan option that I left alone. And a second setting that I forgot the name, might have been network speed or something. It had 5 Options:
- Auto (What was selected)
- 10mb Half
- 10mb Full
- 100mb Half
- 100mb Full

For shits and giggles just to see what would happen I clicked 100mb Full. Saved BIOS settings and restarted. From there I booted into my normal Windows environment, connect to the WebGUI for FreeNAS and did the basic set up (the initial Wizard, made a pool, a share for Windows, and a new user). I connected Windows by mapping the NAS as a Network Drive (what I normally do). I then began my normal tests. Moving the 1GB movie file back and forth. It worked fine, so I tried moving 10 X 1GB files back and forth at the same time, still fine (thought it did start off strong at 100MB/s for about 1/3 the transfer and then dip down to around 20MB/s for the rest). Now normally It would have crashed by now. So I went and did the Blackmagic Speed test for 15 minutes, no issues were it would normally crash after about 2-5. I couldn't believe it, so I did the final test which was make a Adobe Premiere project, and have it play back the clip. The clip in question btw is a 1GB 4K Res RAW Sony FS7 clip. It played back with 0 issues.

I have no idea what I did to "fix it" (again, maybe I got lucky last night, need to do more tests and for all I know it might crash the second I try again later today) but it seems to work. So to try to figure out what it was I did in the BIOS, I booted back to my BIOS, and as mysteriously as that second page appeared, it was gone. I could not get to the network settings anymore. I really want to try to figure out how to get there so I can set it back to "Auto" and see if that was causing the crash. But I have 0 clue how to get there.

 

[jgreco]

Oh, I think I understand.

Basically newer stuff or cheaper stuff, Intel is preferred as it is mostly supported. But for higher end things, old is better as long as the company isn't some weird shady company. This being due to older NICs having better driver deployment and the bugs all ironed out?

Something like that. With FreeBSD and Linux, there's certain hardware you're likely to find in the data center, and the newest stuff usually is a bad choice because usefully meaningful support in the form of stable drivers in a released version of the OS aren't available for ~6-12 months. There are people who will plow their way through the quagmire and try to use it anyways, but it's for the masochistic.

With Windows, no one wants to go and write new Windows 10 drivers for their old gear. Not even Intel.