Netork Domain Configuration advice please

[Zaaphod]

I am new to FreeNAS and would like some networking advice to get the most out of my system. I am running FreeNAS on a SuperMicro A1SA7-2750F Motherboard with 32GB of ECC RAM installed.

I have always had trouble with my network, and I'm thinking FreeNAS could help me solve my problems. it has always been a workgroup configuration where one Windows 7 Professional computer stored the majority of the files, but with other shares as well. My goal with FreeNAS is to get away from this completely and make it a true Server / Client arrangement, with FreeNAS storing all datafiles for all computers. I do not even want it to be possible to save files in my documents or my pictures on client computers.

For Client Computers I have the following:

4 Windows 10 Tablet PCs

2 Windows 10 Desktop PCs

1 windows 8.1 Laptop

1 Windows 7 Professional - I need to keep this one windows 7 because I need to run a W98 VM with Microsoft Virtual PC 2007 and it won't run on 8 or higher, and I can't get it to work on virtualbox correctly

1 Windows 7 Home Premium Laptop

4 Windows XP - Small single application work stations with no need to upgrade

3 Windows XP VMs - used to run older software no longer available

1 Windows 98 Virtual PC VM (this is the one above mentioned w98 VM)

5 Windows 3.11 - don't laugh, there is a reason... these 5 computers run CNC cutting equipment that must communicate with special motion controller cards that are ONLY available as ISA cards... it would cost THOUSTANDS to upgrade these, these computer can network with the W98 VM just fine, and the win XP through win 10 computers can also network with W98

1 Ubuntu Desktop PC

3 Ubuntu VMs

7 Android devices - phones tablets.. etc, one of which is a WiFi Tablet with root access and networked the others I would like to access network shares via apps like AndSMB

1 Konica Minolta BIZHUB C203 network color laser printer/MFP - I mention this because it's network seems to work the same way as a Windows XP network.. when XP computers can't access shares, neither can this MFP. This device has my only high speed scanner and it needs to log into a network share to transfer the scans.. It is essential that this connection is reliable.. right now, it is not

With a workgroup configuration, things are never reliable, for example, I can have it mostly working pretty well, then boot up the W8.1 laptop, and as soon as it is up, the rest of the network no longer functions properly, shares can't be found, other computers need to be re-started... etc, it's a disaster pretty much.

I have tried all kinds of things like disabling master browser on all but one machine, and lots of work around, like assigning fixed IP addresses to every single computer, but it's still not reliable. It seems the network names are the worst problem, looking up computers by name often fails, some computers show up in the network browser, others don't.. but if I type in the IP of the failed connection, it's just fine.. once in a while even pining IP addresses that I know are online fails

I want to change over to a true Domain type system, but after searching the internet, I can't seem to find a good clear set of instructions on how to set this up from the beginning, so I am looking for some guidance on how to set up FreeNAS to be the Domain Controller and then how to set up my workstations to use it. As I said, they all have fixed IP addresses.. but those are fixed because I programmed the Router to assign a fixed address to each MAC address, the computers are all still receiving their IP address from the router with DHCP. My Subnet mask is currently 255.255.0.0, as I had a few devices like PLCs and such that are already set at 192.168.1.xxx but the rest of my network is at 192.168.0.xxx just to put things in groups, and since I had the subnet as such, I also used some addresses like 192.168.10.xxx and others.

The one thing thats odd about my network is the Win 3.11 requirement, at the moment, I am using a w98 VM to communicate with both the Win 3.11 computers and the more modern computers, however this has one catch that I'm not so happy about... that is W98 cannot re-share one of it's shares (that I know of) so I have it set that the w98 VM is the server for the files that must go to the machines connected to Win 3.11 computers. I would love to do away with this, but if I can't make FreeNas compatible with Win 3.11, I'll be stuck with it. for some reason the 3.11 computers don't see shares correctly with TCP/IP, but they work fine with NETBEUI, I am pretty sure NETBEUI traffic is something that clutters up my network, so I installed two network cards in the host computer that runs the W98 VM, the W98VM can access both cards, with NETBEUI turned on only one card, and TCP/IP only turned on the other card.. the host itself has the NETBEUI card disabled.

Ok, so, where do I start with converting over to a Domain and creating a real network? Pretty much the only fields I understand how to fill in when I look at the settings for domain controller are Administrator Password and Confirm Administrator password. Is there a guide, manual, book on how to network, course I need to take.. etc that I need? I'm not computer illiterate by any means, I've been networking computers since arcnet and lantastic were popular, but I just have no domain environment experience at all.. it's always been struggling with workgroups...

I greatly appreciate any advice anyone has to offer!

 

[Ericloewe]

set up FreeNAS to be the Domain Controlle

That's a genuine Very Bad Idea (tm).

Samba specifically advises against doing this. Use either a separate server running Samba in the DC role or a real Windows Server.

 

[Zaaphod]

That's a genuine Very Bad Idea (tm).

Samba specifically advises against doing this. Use either a separate server running Samba in the DC role or a real Windows Server.

I did not know this, thank you for the information. I did read somewhere that if you have linux clients, you are better off with a Samba domain controller than an Windows server, I'm not sure if that's true or not, but I want to be as compatible as possible. I also want to use the least power possible especially when nothing is using the network or file server (hence the atom based NAS) what is the smallest computer I could run a reliable Domain Controller on? could something like a raspberry pi work? or do I need something with a gigabit Ethernet adapter? perhaps a smaller super-micro atom based server board? Would it be better for the domain conroller to use ECC RAM or is that irellevant? and last question, do I need any special domain controler package or software, or is it something Samba does on it's own?

 

[anodos]

I am new to FreeNAS and would like some networking advice to get the most out of my system. I am running FreeNAS on a SuperMicro A1SA7-2750F Motherboard with 32GB of ECC RAM installed.

I have always had trouble with my network, and I'm thinking FreeNAS could help me solve my problems. it has always been a workgroup configuration where one Windows 7 Professional computer stored the majority of the files, but with other shares as well. My goal with FreeNAS is to get away from this completely and make it a true Server / Client arrangement, with FreeNAS storing all datafiles for all computers. I do not even want it to be possible to save files in my documents or my pictures on client computers.

For Client Computers I have the following:

4 Windows 10 Tablet PCs

2 Windows 10 Desktop PCs

1 windows 8.1 Laptop

1 Windows 7 Professional - I need to keep this one windows 7 because I need to run a W98 VM with Microsoft Virtual PC 2007 and it won't run on 8 or higher, and I can't get it to work on virtualbox correctly

1 Windows 7 Home Premium Laptop

4 Windows XP - Small single application work stations with no need to upgrade

3 Windows XP VMs - used to run older software no longer available

1 Windows 98 Virtual PC VM (this is the one above mentioned w98 VM)

5 Windows 3.11 - don't laugh, there is a reason... these 5 computers run CNC cutting equipment that must communicate with special motion controller cards that are ONLY available as ISA cards... it would cost THOUSTANDS to upgrade these, these computer can network with the W98 VM just fine, and the win XP through win 10 computers can also network with W98

1 Ubuntu Desktop PC

3 Ubuntu VMs

7 Android devices - phones tablets.. etc, one of which is a WiFi Tablet with root access and networked the others I would like to access network shares via apps like AndSMB

1 Konica Minolta BIZHUB C203 network color laser printer/MFP - I mention this because it's network seems to work the same way as a Windows XP network.. when XP computers can't access shares, neither can this MFP. This device has my only high speed scanner and it needs to log into a network share to transfer the scans.. It is essential that this connection is reliable.. right now, it is not

With a workgroup configuration, things are never reliable, for example, I can have it mostly working pretty well, then boot up the W8.1 laptop, and as soon as it is up, the rest of the network no longer functions properly, shares can't be found, other computers need to be re-started... etc, it's a disaster pretty much.

I have tried all kinds of things like disabling master browser on all but one machine, and lots of work around, like assigning fixed IP addresses to every single computer, but it's still not reliable. It seems the network names are the worst problem, looking up computers by name often fails, some computers show up in the network browser, others don't.. but if I type in the IP of the failed connection, it's just fine.. once in a while even pining IP addresses that I know are online fails

I want to change over to a true Domain type system, but after searching the internet, I can't seem to find a good clear set of instructions on how to set this up from the beginning, so I am looking for some guidance on how to set up FreeNAS to be the Domain Controller and then how to set up my workstations to use it. As I said, they all have fixed IP addresses.. but those are fixed because I programmed the Router to assign a fixed address to each MAC address, the computers are all still receiving their IP address from the router with DHCP. My Subnet mask is currently 255.255.0.0, as I had a few devices like PLCs and such that are already set at 192.168.1.xxx but the rest of my network is at 192.168.0.xxx just to put things in groups, and since I had the subnet as such, I also used some addresses like 192.168.10.xxx and others.

The one thing thats odd about my network is the Win 3.11 requirement, at the moment, I am using a w98 VM to communicate with both the Win 3.11 computers and the more modern computers, however this has one catch that I'm not so happy about... that is W98 cannot re-share one of it's shares (that I know of) so I have it set that the w98 VM is the server for the files that must go to the machines connected to Win 3.11 computers. I would love to do away with this, but if I can't make FreeNas compatible with Win 3.11, I'll be stuck with it. for some reason the 3.11 computers don't see shares correctly with TCP/IP, but they work fine with NETBEUI, I am pretty sure NETBEUI traffic is something that clutters up my network, so I installed two network cards in the host computer that runs the W98 VM, the W98VM can access both cards, with NETBEUI turned on only one card, and TCP/IP only turned on the other card.. the host itself has the NETBEUI card disabled.

Ok, so, where do I start with converting over to a Domain and creating a real network? Pretty much the only fields I understand how to fill in when I look at the settings for domain controller are Administrator Password and Confirm Administrator password. Is there a guide, manual, book on how to network, course I need to take.. etc that I need? I'm not computer illiterate by any means, I've been networking computers since arcnet and lantastic were popular, but I just have no domain environment experience at all.. it's always been struggling with workgroups...

I greatly appreciate any advice anyone has to offer!

A few quick points in addition to what ericloewe wrote:

• This is a very significant change to how your network functions. Typically, you would do extensive testing on a separate test network before deploying into production. If you skip the testing phase, you stand a good chance of breaking things. Downtime is expensive and something to be avoided.
• You will not be able to join anything older than Windows XP to an AD domain.
• Win 3.11, W98, Windows XP systems should be air-gapped, or at least placed on a separate network from your other computers.
• Active directory domain controllers should be virtualized when possible. I'd have a dedicated VM server (esxi or xenserver) and create a samba4 DC following the steps in the samba wiki here: https://wiki.samba.org/index.php/Setup_a_Samba_Active_Directory_Domain_Controller. That said, I run Windows Server 2012R2 VMs for my DCs.

 

[anodos]

I did not know this, thank you for the information. I did read somewhere that if you have linux clients, you are better off with a Samba domain controller than an Windows server, I'm not sure if that's true or not, but I want to be as compatible as possible. I also want to use the least power possible especially when nothing is using the network or file server (hence the atom based NAS) what is the smallest computer I could run a reliable Domain Controller on? could something like a raspberry pi work? or do I need something with a gigabit Ethernet adapter? perhaps a smaller super-micro atom based server board? Would it be better for the domain conroller to use ECC RAM or is that irellevant? and last question, do I need any special domain controler package or software, or is it something Samba does on it's own?

I'd look into buying a proper server to operate as an esxi host and migrate your existing VMs to it (as well as using it to host your AD DC). A C2750 would do the job well, but it'd probably be cheaper to get a Dell T20 or a Lenovo TS140 to use as the basis for your VM server (although I'd personally seriously consider getting a supermicro bare-bones kit and a decent hardware raid card). If your VMs aren't particularly large, then using SSDs for your VM storage can be a big performance win. Of course this is out of scope of what I typically discuss in these forums.

 

[Robert Trevellyan]

5 Windows 3.11 ... special motion controller cards that are ONLY available as ISA cards... it would cost THOUSTANDS to upgrade these

Are you 100% sure you wouldn't be better off upgrading? How do you determine the real cost of keeping obsolete systems running? I mean, I love it when someone pays me good money for something obsolete on eBay, but it always leaves me with questions. And here you are with a major networking project on your hands...

 

[Zaaphod]

A few quick points in addition to what ericloewe wrote:

• This is a very significant change to how your network functions. Typically, you would do extensive testing on a separate test network before deploying into production. If you skip the testing phase, you stand a good chance of breaking things. Downtime is expensive and something to be avoided.
• You will not be able to join anything older than Windows XP to an AD domain.
• Win 3.11, W98, Windows XP systems should be air-gapped, or at least placed on a separate network from your other computers.
• Active directory domain controllers should be virtualized when possible. I'd have a dedicated VM server (esxi or xenserver) and create a samba4 DC following the steps in the samba wiki here: https://wiki.samba.org/index.php/Setup_a_Samba_Active_Directory_Domain_Controller. That said, I run Windows Server 2012R2 VMs for my DCs.

At the moment, the network is broken more than it is working, so it's already causing delays, however, it's good advice to test things, and this could easily be done by setting up a few rarely used computers to test with before trying to move the entire network over.
If the win3.11, 98 and xp systems are on a separate network, is it possible to bridge it to the main network? It would be nice to have access to that network from the modern computers, the files needed are very small, but they need to be updated frequently
I'm curious why the domain controller should be virtuized and not just running on it's own hardware?

 

[Zaaphod]

I'd look into buying a proper server to operate as an esxi host and migrate your existing VMs to it (as well as using it to host your AD DC). A C2750 would do the job well, but it'd probably be cheaper to get a Dell T20 or a Lenovo TS140 to use as the basis for your VM server (although I'd personally seriously consider getting a supermicro bare-bones kit and a decent hardware raid card). If your VMs aren't particularly large, then using SSDs for your VM storage can be a big performance win. Of course this is out of scope of what I typically discuss in these forums.

This sounds interesting, could you tell me more about how this would work? I'm not familiar with esxi, could I run my w98 VM on this system? How are these VM's running on an exsi server normally accessed? Is it something like remote desktop or something else? I'm just wondering if video performance would decrease accessing the VMs over the network as opposed to running on the local machine. I like the idea of unloading the VM processing from the desktop computers, and it would also give a huge benefit that they would not need to be shut down when the desktop does, which has always been a pain. I like supermicro hardware, and none of my VM's are very large so I'm sure they would all fit on SSDs so I may take your advice on this.

 

[Zaaphod]

Would it be a good idea to use something like a supermicro A1SRM-LN7F-2758 For an esxi server? It has 7 Lan ports, so I could assign different ports to different VMs and perhaps get better performance, and also have the ports wired to different networks as well. Any thoughts on this board? Am I correct in understanding this that the best way to set it up is to have the NAS system dedicated to one system, and the Virtual machines on another, and one of the VMs could also be the domain controller?

 

[anodos]

Adding a DC isn't going to fix your networking problems. AD is about single-sign-on and simplifying administrative tasks.

Most network problems are cable / switch related. Try to narrow down your problem, eliminate wirless, perhaps have someone come out and run / certify your network cables. You may need shielded cables & jacks for lines running near CNC equipment.

The only acceptable bridge for an airgap is a pair of sneakers and a USB drive. :) although I don't think you need to go this far. I'd probably have two switches that are not connected to each other. One for modern computers and one for your ancient things. The 'modern computers' switch connects to your router, gateway thing. The 'ancient computers' switch isn't connected. Everything on ancient computers has a static IP address. Connect your NAS and VM server to both networks (you will need at least two NICs in both NAS and VM server. In W98 vms you should only have the NIC connected to the ancient lan available, in the DC VM only the modern computer LAN will be available. The samba server in freenas will automatically listen for connections from both LANs and be available on both of them.

Lots of hand-waving going on here because I'm using a phone to type.

 

[Zaaphod]

Most of what seems to be happening is the windows names keep getting lost. If I type in the IP address, then I can connect and it and the share works fine. I've been reading about setting up wins servers and things like that to help with name resolution, but the articles I read usually indicate that workgroup configurations and all this masterbrowser business, and the many different ways microsoft has implemented workgroup being not really compatible with itself has made it so the only real reliable way for computers to know where each other are of all kinds of different operating systems is to switch to a domain environment which doesn't use master browsers and an election process.. etc... the domain controller takes care of it. With a work group environment, I can have all my network shares working all day long, then someone turns on a windows 8 laptop and poof, all the shares that were working fine, now can't be seen, yet I can ping IP addresses and they are fine, and if I just connect using the ip address, it's also fine..

I think the solution with 2 NICs on the NAS and two seperate networks sounds like the best solution to the ancient network. Luckily the board I picked out for a server board already has 2 Lan ports (not including the management port) so that sounds like the best solution to that.

Actually now that I think of it... with the NAS up and running, It is really going to be irrelevant if any computers can see each other or not, as long as they can ALL always see the NAS is all that will matter. I'm just wanting the network to be as reliable as possible before I start migrating all the data to the NAS, however there will be a period of time where the NAS is just a mirror of the existing structure anyway, so there is little risk.

 

[anodos]

Most of what seems to be happening is the windows names keep getting lost. If I type in the IP address, then I can connect and it and the share works fine. I've been reading about setting up wins servers and things like that to help with name resolution, but the articles I read usually indicate that workgroup configurations and all this masterbrowser business, and the many different ways microsoft has implemented workgroup being not really compatible with itself has made it so the only real reliable way for computers to know where each other are of all kinds of different operating systems is to switch to a domain environment which doesn't use master browsers and an election process.. etc... the domain controller takes care of it. With a work group environment, I can have all my network shares working all day long, then someone turns on a windows 8 laptop and poof, all the shares that were working fine, now can't be seen, yet I can ping IP addresses and they are fine, and if I just connect using the ip address, it's also fine..

I think the solution with 2 NICs on the NAS and two seperate networks sounds like the best solution to the ancient network. Luckily the board I picked out for a server board already has 2 Lan ports (not including the management port) so that sounds like the best solution to that.

Actually now that I think of it... with the NAS up and running, It is really going to be irrelevant if any computers can see each other or not, as long as they can ALL always see the NAS is all that will matter. I'm just wanting the network to be as reliable as possible before I start migrating all the data to the NAS, however there will be a period of time where the NAS is just a mirror of the existing structure anyway, so there is little risk.

Just a quick FYI - home editions of windows can't join an AD domain.

 

[Zaaphod]

Just a quick FYI - home editions of windows can't join an AD domain.

thanks for the info, I'll take that into consideration

 

[tvsjr]

I'll be the negative one and throw this out there... let me start by saying - please don't take offense to what I'm going to say.

I'm assuming this is some sort of small business... I'd suggest it's time to take a step back and consider things at a slightly higher level. You have infrastructure problems... dropping a FreeNAS or any box in the middle of that isn't going to solve your issues, if you don't understand and resolve the underlying problems. If I'm correct and this is a business, every failure is costing the business money and productivity (not to mention the frustration).

If I were working on this environment, I'd start with documentation of what exists today. Networks, VLANs, physical devices, all client systems (and their requirements), etc. Then go from there. You definitely need multiple zones and a firewall in the middle, if nothing else than to airgap your WinXP/Win98/Win3.1 machines (USB and sneakernet is the best option, but there are options to remain safe and connected). Dropping multiple NICs into a box and trying to use that to bridge multiple networks isn't a great solution.

Once the physical issues are fixed (if any exist), I'd consider building a clean, parallel environment then doing a cutover.

You might consider bringing in a consultant for a day or two to help you at least develop a strategy. I've done similar engagements for several small businesses, and they've always said they found it worth the money.

 

[Zaaphod]

I'll be the negative one and throw this out there... let me start by saying - please don't take offense to what I'm going to say.

No offense taken at all.. I was looking for advice, and any advice is appreciated

I'm assuming this is some sort of small business... I'd suggest it's time to take a step back and consider things at a slightly higher level. You have infrastructure problems... dropping a FreeNAS or any box in the middle of that isn't going to solve your issues, if you don't understand and resolve the underlying problems. If I'm correct and this is a business, every failure is costing the business money and productivity (not to mention the frustration).

Well, it's kind of an un-common situation I have. I owned a small business from 1994 until 2008 and then everything went down hill with the economy crashing and on top of that oil spill in the gulf... I had no choice but to shut down and ever since, I've been single highhandedly doing only technical support, repairs, and maintenance for my existing customers. In the last few years however, things are really going well for my customers, and therefore they have been wanting me to do larger projects like upgrades to their equipment etc. I've actually managed to get to a position where I can now afford to build my own building and own it rent and mortgage free, which is what I'm in the process of doing, once that is done, I will re-build my manufacturing process to include all the ways I always wished it worked but could never stop working long enough to fix. So, as the computer and network situation was always something I needed to address, it seems like the perfect time to figure it all out... There is no production being done at the moment, so the entire thing is a full scale test environment and learning experience for me. If all my computers were down for a week it wouldn't hurt anything right now, however in the future, that will change, that is why I'm trying to switch from peer to peer to real servers and a restructure of the network.

If I were working on this environment, I'd start with documentation of what exists today. Networks, VLANs, physical devices, all client systems (and their requirements), etc. Then go from there. You definitely need multiple zones and a firewall in the middle, if nothing else than to airgap your WinXP/Win98/Win3.1 machines (USB and sneakernet is the best option, but there are options to remain safe and connected). Dropping multiple NICs into a box and trying to use that to bridge multiple networks isn't a great solution.

The win98/win3.1 machines would be fine on an isolated network, and even the XP machines could possibly be upgraded or replaced, however I have my BizHub color laser printer / mfp which seems to network the same way as XP does... when my XP computers have difficulty, that printer does not see my computers to send scans to. It's an extremely large, extremely heavy, and for me.. extremely expensive device, it's the most expensive price of computer related hardware I ever purchased, so I'm really wanting to find a solution to make it a reliable part of the network. It does have settings to allow it to log into a domain environment, so I'm hoping perhaps getting away from workgroups and using that would help the reliability issues.

 

[Zaaphod]

I have a question.... Is there a common, popular method of networking where none of the computers on the network are aware of each other at all, and they all only communicate to the router and the servers? It seems to me that would solve a lot of my problems.. I wouldn't have this whole master-browser situation going on FreeNAS screen is constantly putting up messages that it is the master browser and then that it is not the master browsers.. etc...

I've been thinking about it and, unless I'm missing something, I can't think of a single reason why I would need computer A to directly communicate with computer B when they both communicate to the server and the server is where all the files are stored. I suppose it would be nice if I could use remote desktop occasionally, but even then, I could do the with Chrome's remote desktop and that seems to always work, even if I'm on my cell phone on the cellular network, so my computers still never need to communicate with each other.

It's kind of nice to look at the networking just to see what computers are on-line, but that could be done by looking at the router DHCP table, or by writing a script that just pings all the IP addresses and prints me a report.

So, is there a way to set it up so computers all only see the Router, MFP network printer, FreeNAS, and EXSi box, and don't even attempt to connect to each other? Would I still use a domain to manage this? What would the limitations of this be? at the moment, I'm only really interested in the XP through Win 10 computers. I'm convinced that keeping Win98/WFW3.11 on a separate network is the way to go. XP would be nice, but could also be optional if it helps.

 

[tvsjr]

Ok, so you've got a big test environment, at least for now. That certainly gives you the luxury of time.

For your environment, a small Active Directory really makes sense. You can do this with native Windows, or Samba 4.x can serve as a complete AD environment by itself. You'll have a LOT more challenges with Samba than Windows.
Anything XP or older will have challenges connecting to newer domain controllers... you'll need to relax some security settings to make it work.
Typically, your computers are "aware" of each other, but they aren't doing a great deal of communicating with one another. Best practice is to have some sort of central repository (file server, etc.) where files and other resources live... computer-to-computer contact would typically be limited to IT support sort of work.

I would strongly suggest creating a new environment. This will let you build, learn, play, and make mistakes... in the worst case, you nuke it from orbit and begin again.

First, you need to consider the network. My suggestion would be a firewall/router - either something free like pfSense or the free version of Sophos UTM (as long as you have 50 or less devices), or a dedicated device (Cisco ASA-X, etc.) Create an outside, inside, and untrusted (for your Win98/Win3.X systems) zone. These should exit the box either as 3 separate NICs (to either three separate layer-2 switches, or one managed layer-3 switch with VLAN support), or as a single VLAN-trunked interface (again, requiring a managed switch).

A VM host (or two, if you want redundancy) would be the next step. Set up two domain controllers, create your forest/domain, etc. Then, you could add your FreeNAS (if that's the direction you want to go for file storage), a client system, etc. Join the client to the domain. Use your firewall appliance for DHCP services, and use the DCs for DNS (AD-integrated DNS works quite well).

Keep this in mind - what you're trying to do is done by millions of companies every day. If something seems hard, or requires a klugey solution (like writing scripts), at least at this level, you're probably going at it the wrong way.

 

[anodos]

Ok, so you've got a big test environment, at least for now. That certainly gives you the luxury of time.

For your environment, a small Active Directory really makes sense. You can do this with native Windows, or Samba 4.x can serve as a complete AD environment by itself. You'll have a LOT more challenges with Samba than Windows.
Anything XP or older will have challenges connecting to newer domain controllers... you'll need to relax some security settings to make it work.
Typically, your computers are "aware" of each other, but they aren't doing a great deal of communicating with one another. Best practice is to have some sort of central repository (file server, etc.) where files and other resources live... computer-to-computer contact would typically be limited to IT support sort of work.

I would strongly suggest creating a new environment. This will let you build, learn, play, and make mistakes... in the worst case, you nuke it from orbit and begin again.

First, you need to consider the network. My suggestion would be a firewall/router - either something free like pfSense or the free version of Sophos UTM (as long as you have 50 or less devices), or a dedicated device (Cisco ASA-X, etc.) Create an outside, inside, and untrusted (for your Win98/Win3.X systems) zone. These should exit the box either as 3 separate NICs (to either three separate layer-2 switches, or one managed layer-3 switch with VLAN support), or as a single VLAN-trunked interface (again, requiring a managed switch).

A VM host (or two, if you want redundancy) would be the next step. Set up two domain controllers, create your forest/domain, etc. Then, you could add your FreeNAS (if that's the direction you want to go for file storage), a client system, etc. Join the client to the domain. Use your firewall appliance for DHCP services, and use the DCs for DNS (AD-integrated DNS works quite well).

Keep this in mind - what you're trying to do is done by millions of companies every day. If something seems hard, or requires a klugey solution (like writing scripts), at least at this level, you're probably going at it the wrong way.

This is good information. A few more random thoughts:

• Since it sounds like you're dealing with new construction, you should make sure the the company you hire to run your network cables 'certifies' the cables after they're installed. This involves using some fancy expensive equipment (google "fluke cable certifier"). This has the added advantage of filtering out contractors who don't know what they're doing. I've seen low-voltage installers untwist CAT5 about 4+ inches from a patch panel (not good) and ziptie cables in all sorts of kinky ways. It's worth paying to have it done right.
• Each license for Windows Server 2012R2 Standard permits two Windows VMs on the same physical hardware. This means you can set up a VM as a DC and a second one as a Windows update server (lets you check status of updates) / Windows deployment server (lets you quickly deploy standard OS images). MS recommends having at least two DCs (for failover / redundancy). Whenever possible I try to put these on different physical servers. This means having at least two licenses for Windows Server.
• If you go the Windows DC route, you will need to purchase client access licenses (CALs). MS licensing can be complicated. It's best to work with a vendor, licensing specialist, or at least a consultant. An example is that reimaging rights (ability to use WDS to deploy windows images) are only conferred if you have a volume licensing agreement in place with a volume license for the OS version you are deploying.
• Configuring DNS for your network will fix many name resolution problems. It won't fix problems with having printers and servers appear when you click on the "network" button in Windows Explorer. If you set up a pfsense firewall, you can configure its DNS resolver to handle name resolution for your LAN (by registering machine hostnames when devices request DHCP leases, etc).
• I'd try to standardize on a single OS version. "Simplify" and "standardize" are two good ways to have a happy network.

 

[gpsguy]

Given that the OP has "5 computers run CNC cutting equipment", that would probably violate the home user license for the "free version of Sophos". Their low end SG xxx series UTM's with Full Guard are affordable.

 

[Zaaphod]

Thank you very much for all the advice. I decided to put together an EXSi machine, and I'm ordering parts for it today. I think I can learn a lot by setting up a bunch of VMs on it and try to get the networking between the VMs to work the way I want it to, this would make a nice test environment and I would not have to wonder if a cable issue is causing a problem.