economical home storage music production parts list proposal

[ramar]

reading another hardware list in PDF format, https://forums.freenas.org/index.php?threads/hardware-recommendations-guide-discussion-thread.46494/ i see 10GBe mentioned in the motherboard section. they also say that generally 2 motherboards are not very helpful. since i am using NAS in time sensitive music production, would a second motherboard decrease latency?
quote: Xeon-D A very wide variety of Xeon-D boards is available from both Supermicro and ASRock Rack. Most boards are available with a choice of CPU, allowing for different core counts. These tend to be very expensive, but extremely competent for the size and power envelope. Most make use of Xeon-D’s integrated 10GbE controller, exposing two such ports – most boards employ 10GBaseT, but SFP+ boards are also available. ~end quote

i see a few Xeon-Ds listed here: http://www.supermicro.com/products/nfo/Xeon-D.cfm
2 here for the FreeNAS site: https://forums.freenas.org/index.php?threads/hardware-recommendations-read-this-first.23069/ the first one https://www.superbiiz.com/detail.php?name=MB-X10SLLF&c=CJ has PCIe slots, so does it support a 10GBe card?
how about this $300 one https://www.ebay.com/itm/NEW-LATEST...711400?hash=item4d655771e8:g:YB0AAOSw7ylZ4jct
i'd like 3 or 4 @ 10GBe ports. why do these motherboards only have 2 ethernet ports? i thought these were for networking. perhaps they daisy chain ethernet?
this other $300 motherboard might be the one: X10SRi-F https://www.amazon.com/Supermicro-D...9120&sr=1-1&keywords=X10SRi-F#customerReviews
and another freeNAS hardware guide, PDF:

 

[Ericloewe]

they also say that generally 2 motherboards are not very helpful. since i am using NAS in time sensitive music production, would a second motherboard decrease latency?

You're reading that wrong, because I never wrote that. A second motherboard is a second computer, so your question doesn't make sense. If you mean second CPU, the answer is most certainly "no". I mean, it'll still be fine, it's audio, for crying out loud, that's what buffers are for. No audio is time-sensitive enough that it can't wait in a 16 ms buffer. If it is, for some voodoo reason, it should not be subject to any sort of digital signal processing, because "digital" means "delays".

Rule of thumb: more stuff in parallel == bigger buffers

i see a few Xeon-Ds listed here: http://www.supermicro.com/products/nfo/Xeon-D.cfm
2 here for the FreeNAS site: https://forums.freenas.org/index.php?threads/hardware-recommendations-read-this-first.23069/ the first one https://www.superbiiz.com/detail.php?name=MB-X10SLLF&c=CJ has PCIe slots, so does it support a 10GBe card?
how about this $300 one https://www.ebay.com/itm/NEW-LATEST...711400?hash=item4d655771e8:g:YB0AAOSw7ylZ4jct
i'd like 3 or 4 @ 10GBe ports. why do these motherboards only have 2 ethernet ports? i thought these were for networking. perhaps they daisy chain ethernet?
this other $300 motherboard might be the one: X10SRi-F https://www.amazon.com/Supermicro-D...9120&sr=1-1&keywords=X10SRi-F#customerReviews
and another freeNAS hardware guide, PDF:

No offense, but are you sure you're qualified for this project? You can't hope to set up a server of any kind without knowing the basics of networking. I'm not familiar with any good resources for the very beginner, but someone else might be and help you along.

 

[ramar]

I strongly recommend that you do some research on your own about NASes, Hypervisors, and networking,

really good points. i'll spend some time reading about this and the functionality of DAWs. after looking at motherboards and seeing often they have 4 ports, to ethernet and 2, something ...SF+. what i think are optical, and seeing the posts about plex, i assume there are other components, patch bays, that connect computers and NAS on the one side and hard drives and NAS (or are part of NAS) on the other. thank you and everyone who posted. i need read for a while. i didn't realize this was a whole big thing, i thought i could buy a few components and stick them together and i have a NAS box which i could connect my computers to and go. now i think, "yes and no" but to do it correctly, i need to do a lot of other things. i just saw commercial on tv last night for car insurance that showed to guy assembling furniture from a kit. they had called tech support and were on hold and then a computer voice came on asking what type of furniture they wanted help with. a helpful
'lizard" answered but no one could agree on what it was that they were assembling, and by the looks of what they had assembled, i understood the confusion. after seeing your post this morning, sadly i could relate. but it's totally fine. i have wasted a lot peoples time, in a way and i hope that is ok because i think a lot of artists (music, video, graphic and ?) who come here maybe can read my posts and come to the same conclusion i have, there is a reason there is a body of knowledge, hardware and software and whole career field for this. on the other hand, i may be one of a very few who feels they have the ability to tackle this with $1500 and a few posts, never having heard of NAS until 2 weeks ago. well i hope this helps someone and i'll be back, probably sooner than you want me.

 

[ramar]

You're reading that wrong, because I never wrote that

i am quite sure you are correct. i am sure you said 2 CPUs, but i not only got what you said wrong, i got what i read wrong, and that (what i read) is was what i was referring to. i saw that you posted this post while i was still writing my explanation and apology to the group for my basic ignorance on this subject, so no offense taken, and i hope you will accept my apology. i will read up and start over or enlist someone to help

 

[ramar]

I'm not familiar with any good resources for the very beginner, but someone else might be and help you along.

anyone, feel free to post. however i may get farther just reading resources here, and watching tutorials on youtube for year or so. i do not know how to access technical info about Music editing software, the marketers to seem to have thought supplying these facts will drive sales down, that's my impression anyway. i guess i have to start from the very beginning and i am not sure that is my best use of time. i have spent years learning how to operate Music software and i have only scratched the surface of Logic Pro X and Avid ProTools First (it's free) and must be to ProTools 12 HD what Garage Band is to Logic Pro X. now i have the task of learning WaveForm Tracktion and Harrison Mixbus (DAWs) because i want to migrate to Linux. Tracktion and Mixbus work in Windows and Mac OS X too, but i'd like to start producing music at some point. so, i need to spend some time evaluating what i need to do. i may not learn about NAS.

 

[Nick2253]

Hopefully the questions below will help you begin your research. Until you get a good grasp on the basics, the resources here (FreeNAS forums) aren't going to be super helpful. Really, there are three steps of difficulty/specificity of knowledge, and you just jumped right in the deep end: general server/networking concepts, general NAS/storage concepts, and specifically FreeNAS/ZFS concepts.

Before pulling the trigger on any hardware (DIY or otherwise), I would make sure you feel comfortable answering the following questions:

• What is a NAS?
• What protocol(s) does a client use to connect to a NAS?
• How do I connect to my NAS using all of my client operating systems?
  
• What is a switch? What is a router?
• What is an IP address? What is DHCP?
• What is a FQDN? What is a DNS server/name server?
• Where are the DHCP and DNS servers on my network?
• How does my DAW access data? What are the requirements for the datasource for my DAW?
• How does RAID work?
• What is the difference between the different RAID levels?
• What is the difference between hardware/software RAID?

Before pulling the trigger on DIY hardware, I would make sure you feel comfortable answering the following questions (in addition to the above):

• What are the names for all the components found in a computer? (E.g., a motherboard, CPU, PCI card, RAM, video card, PSU, case, backplane)
• What do those components do?
  
• What are all the ports on a motherboard, and what devices plug into what?
  
• Can I remove the side panel of my computer and identify all the above ports and components?
• What capabilities are limited by my motherboard? What capabilities can I add to my computer through expansion cards?
  
• What is static electricity, and how can it ruin my day?
• How do I assemble a computer?
• What is a CPU socket, and why does it matter for picking a CPU?
• What is the difference between DDR/2/3/4 memory? What is ECC memory? What is registered memory?
• What is an HCL? How do components end up on an HCL? Why does it matter that I buy hardware on an HCL?

Before setting up the NAS, I would make sure you have answers for the following questions (these are specific to your use, as opposed to general questions):

• What permissions and security do I need for my data?
• How much data am I generating?
• What tolerance do I have for data loss?
• What secondary uses do I have for my NAS?
• How do I plan to backup my data?

I would be happy to answer any of the above, but I firmly believe that you will gain much more by doing the research to answer these yourself. If you do not think you have the time to set up and configure this NAS, and you need it for your work, I would recommend that you seek out a local IT professional that can help you. There are many consultants out there that focus on supporting small businesses, and can help you put together something that would meet your needs, as well as help support it if/when things go wrong.

 

[rogerh]

I would just add to all the good advice that you certainly want a DAW with its own working fast storage big enough for any single project. If you want a NAS at all (and if you do you also want a place to back up everything on the NAS) it is to store files you are not currently editing, and therefore it does not have to be enormously fast. Or at least it does not need really low latency connections to the DAW even if you want to transfer files very quickly. Provided you look at it as two separate projects I think you will find it easier to plan.

 

[ramar]

If you want a NAS at all (and if you do you also want a place to back up everything on the NAS) it is to store files you are not currently editing, and therefore it does not have to be enormously fast

i was wondering about this, currently i record to a 2TB USB3.0 5400 RPM portable external hard drive. why not keep that and backup to NAS? then i started to wonder if it would be better to record directly to NAS. what i don't understand and i need to read up on is if there are errors from my computer to my portable external hard drive and then i store that am i introducing the possibility of more errors that way

 

[Constantin]

I've found the Mobius 5 series of DAS' to be a pretty good starting point. Reliable hardware controller RAID (up to RAID6), many interfaces (eSATA, USB 3, FW800), 5 drive tool-less hot swap, and attractive price. They make a good backup platform for my FreeNAS server.

Are they as reliable, bulletproof, etc. as a properly-set-up FreeNAS server? Of course not. But, unlike a FreeNAS server, you'll have one up and running in minutes. For the money you'd spend on a proper FreeNAS box, you can buy two of these Mobius 5 enclosures and keep one off-site, then cycle them.

Bitlocker or FileVault them depending on your OS preferences and then use a good backup program to do the rest. On the mac, Carbon-Copy-Cloner works very well, I've found SyncBack to work very well in Windows.

Speed-wise, you won't be setting records but 100+MB/s is not unusual.

 

[ramar]

Hopefully the questions below will help you begin your research. Until you get a good grasp on the basics, the resources here (FreeNAS forums) aren't going to be super helpful.

thank you for taking the time to put this together. i will look these things up and go from there.

 

[ramar]

I've found the Mobius 5 series of DAS' to be a pretty good starting point.

good to know, my main collaborator is writing 3 songs a day and wants me to get going. so i'll have to continue with a portable hard drive and work my way in and this is a good option to know about, thanks

 

[Constantin]

good to know, my main collaborator is writing 3 songs a day and wants me to get going. so i'll have to continue with a portable hard drive and work my way in and this is a good option to know about, thanks

Sure thing. Just keep in mind that the Mobius 5 is a Desktop-Attached-Storage (DAS) device. You have to attach it to a computer to make it work, it is not standalone like a NAS.

 

[ramar]

You have to attach it to a computer to make it work

sure thing. i have my iMac 27 in what i believe is a Ming Dynasty cabinet. printer on top and we'll see what happens.

 

[ramar]

Before pulling the trigger on any hardware (DIY or otherwise), I would make sure you feel comfortable answering the following questions:

very wise tip. i was reading in the hardware list that there is a slight increase in performance with motherboards that have a component ...
quote below from link: https://forums.freenas.org/index.php?threads/hardware-recommendations-read-this-first.23069/
quote:
Supermicro X10s (32GB of RAM max)

• The X10SLL+-F ($170) is a very good board if you are looking for a box with 6 disks or less. It supports the E3-1200v3 series as well as 4th gen i3s, Pentiums and Celerons.
• The X10SL7-F ($240) is amazing if you are planning to go with more than 6-8 disks but don't want the cost jump with going beyond 32GB of RAM. It has a built-in LSI SAS 6Gbps controller which you can reflash to IT mode fairly easily which saves you from the mess of buying an M1015 and using it as an add-on card (not to mention the cost savings). On-board gives some small performance gains and this is a clear choice if you are looking for a good long-term system that is highly expandable. It has IPMI, dual Intel Gb LAN, lots of SATA and SAS and is a proven winner for many users in the forum. This is the current winner for those that are looking for that ultimately expandable server for home use.

~end quote
me:
jumping ahead in my studies (i have started doing my homework and i am fascinated by these few points), when choosing a motherboard (speaking theoretically, in actuality i will not buy any parts that cost over $10 that i do not value over $500 for at least 6 months) for those home users saving computational data in short time frame, under 7 milliseconds for sure, it seems possible that "built-in LSI SAS 6 GBps controller which i can reflash to IT mode fairly easily" would be a plus.

same goes for ZILs and L2ARCs: quote from the same "hardware recommendations (read this first)" link above:

6. SATA2 and SATA3 don't play a big part in your server's performance. Most rotating platter media can't even saturate SATA1 speeds, so don't spend money on that board that has all SATA3 because you think it will matter. Your bottleneck is almost certainly going to be your Gigabit LAN. SATA3 is useful for ZILs and L2ARCs which means large (and expensive) systems will benefit but home users won't.

the last post in this thread is looking at saving 1 millisecond.
https://forums.freenas.org/index.php?threads/l2arc-question-what-is-populated-in-the-l2arc.10245/

are other considerations besides which computational resources the digital audio workstation computer program is using? i know i am the one who is to answer this and i wanted to share what i assume are computer related audio hardware modules, things i will eventually purchase which until i do, the computer itself fulfills the functionality of and part of what may be going on is a high priority placement of 0 and 1s in the data stream we (my friends and i) generate similar to the difference between watching TV video generated in the 1980s and now.

audio synchronization. i do not believe this is synchronization between audio and video, as google thinks, but more of a stabilization (i am not sure it is stabilization, the links states is as support, maybe someone here understands this) of the audio sample rate by a master clock. i am posting this because i am presenting information to assist in designing a suitable NAS system.
https://www.sweetwater.com/store/detail/10MX--antelope-audio-10mx
3 more
https://www.sweetwater.com/store/detail/OCXHD
https://www.sweetwater.com/store/detail/SyncGen
with redundant power supply:
https://www.sweetwater.com/store/detail/Trinity

another computer sound module is the analog to digital converter and there are also digital to analog converters sometimes these are separate units and sometimes not and most of us just have the computer do this.
https://www.sweetwater.com/store/detail/HEDDQuantum

what we all (music producers) have are audio interfaces that may assist the computer do sampling (the work of the master clock) and AD/DA conversion and audio pre amplification to prepare the signal for AD conversion.

here are some audio interfaces (over lap in functionality with the above)
these 2 are "class compliant" by having functionality in any operating system, Windows, Mac OS X, Linux, iSO, Android and use USB to connect and power themselves
https://www.sweetwater.com/store/detail/OnyxProducer22
https://www.sweetwater.com/store/detail/USBPre2

i am going out of my way to create a clear (with computer created emulated distortion of audio which is very different from incidental digital distortion) and unique audio signal in the digital realm and so perhaps preserving it as best i can is in order

 

[Nick2253]

when choosing a motherboard ... for those home users saving computational data in short time frame, under 7 milliseconds for sure, it seems possible that "built-in LSI SAS 6 GBps controller which i can reflash to IT mode fairly easily" would be a plus.

Practically speaking, this won't make any difference. Most likely, the built-in controller is wired through the PCIe bus, so it wouldn't even have that benefit over an addon card.

the last post in this thread is looking at saving 1 millisecond.

I think you are completely misunderstanding what is being discussed here. What is being discussed here is the benefits of the L2ARC: when your disks are relatively idle, an L2ARC may only save you milliseconds on access. However, when your disks are very busy, you may see dramatic improvements in speed.

However, tuning a ZFS system with a SLOG and L2ARC is an advanced topic. The fact that you are already jumping so far ahead without a firm grasp on the (relatively) basic topics is continuing to take you down paths that are unproductive.

I also think your latency concerns are completely misplaced here. I don't remember who said it, and I'm not going to be bothered going back through three pages, but it's highly impractical to work raw files directly from a NAS. Most likely you'll do your work on your specific project locally, and then keep the files on the NAS when you're not working on them. Furthermore, why does storage latency matter whatsoever for post-processing? There is no reason your software can't buffer the audio from storage into memory. I can see how latency would be critically important in audio capture applications, but it doesn't seem like that's anything like what you are doing. And, again, if you were dealing with audio capture, your audio capture should have no problem capturing audio directly into super-fast RAM, and then writing it out to your slow HDD or NAS without a care in the world.

 

[Ericloewe]

Most likely

Not likely, certainly. There is literally no other bus that comes close to being suitable, in a modern affordable architecture.

OP, let's take a step back here.

Why do you care about latency? It seems absolutely irrelevant to your use case, at least for any remotely viable system.

 

[ramar]

it's highly impractical to work raw files directly from a NAS. Most likely you'll do your work on your specific project locally, and then keep the files on the NAS when you're not working on them.

thank you. i totally misunderstood that. i was curious about this. i really appreciate your posts, now i will go back to reading up on the basics. thank you

 

[ramar]

Why do you care about latency? It seems absolutely irrelevant to your use case, at least for any remotely viable system.

thanks for explaining and confirming. i need to go back to studying the basics of NAS. i will use an external hard drive as my "projects drive" and save my projects to NAS after the low latency is required, so now i understand that NAS latency is irrelevant, which means i can use an affordable system

 

[rogerh]

Furthermore, why does storage latency matter whatsoever for post-processing? There is no reason your software can't buffer the audio from storage into memory. I can see how latency would be critically important in audio capture applications, but it doesn't seem like that's anything like what you are doing. And, again, if you were dealing with audio capture, your audio capture should have no problem capturing audio directly into super-fast RAM, and then writing it out to your slow HDD or NAS without a care in the world.

When using a audio workstation you may be using 12 to 20 channels at up to 192 kHz with 24bit samples. Several of these channels may be sent via complex real time DSP filters and back to storage. Making a few more parallel channels. And having been played and re-recorded, some with and some without processing, they have to remain in sync within a few milliseconds. So latency of reading and writing all this data is really quite important. This is why recording (in the sense of processing prerecorded tracks and perhaps recording additional tracks all in sync) to a NAS is probably a non-starter.
Edit: It may be possible to do this all in RAM, but it would probably need hundreds of Gigabytes, and if you want to save it all to disk pretty promptly you need a fast storage system - thought I agree this might solve the latency problem.

 

[ramar]

This is why recording (in the sense of processing prerecorded tracks and perhaps recording additional tracks all in sync) to a NAS is probably a non-starter.

i have had a track synchronization fail while recording an artist. it was interesting because it seemed like everything was ok during the recording, but when we played it back his part was about 1/2 second behind and he was embarrassed claiming that he didn't play it like that. it was easy to to advance his part relative to the other tracks but he lost confidence in me, i am sure.
while video editing uses way more computational resources than audio, i am not sure how much of the time spent video editing is comparable in the following analogy, playing the CGI (compared with DSP audio) project back while filming (visually recording to a computer) an actor in front of a green screen that is also imaging additional CGI emulations for the actor to respond to. the equivalent in audio is common, so it is a big computational demand, this in regards to having NAS record in real time and how lagging might effect other computational processes related to latency, which is not a concern at this point and would be expensive (in hundreds of gigabytes of RAM) to deal with if it were a concern. i am speaking from an intuitive perspective because i do not have the computer knowledge, nor do i understand other factors from NAS that would need consideration

edit: speculation, looking at these components
Supermicro ATX DDR4 LGA 2011 Motherboard X10SRL-F-O $275
CPU Intel Xeon E5-2697 v2 Twelve-core Processor 2.7GHz 8.0GT/s 30MB LGA 2011 CPU BX80635E52697V2 $2,225.02
RAM samsung M386A8K40BM1-CPB 64GB DDR4-2133 4Rx4 LP ECC LRDIMM Server Memory x8= $6,480 = $8980 without drives
this seems a bit beyond my budget, there is no need for experts here to check compatibility at the moment, i am curious if these components along with M.2 drives might be recordable to from a DAW. i know, i am the one to answer that after i do more homework. i will answer in about 6 months if i learn enough by then, in the mean time i'll record to my portable external hard drive and worry about the integrity of my data. should i consider tape backups like google? i know, i should answer that or record to a drive and then save to NAS, as that is the way it's done generally.

edit: article on audio master clocks. i saw one that cost much more than $6,000 and i was looking for it and found this technical explanation of clocks. just to demonstrate the extremes audiophiles go to re sampling, there are other aspects of digital audio quality such as brands of plugins, or to the source of the sound such as the shape of the room or the microphone. https://www.soundstageultra.com/ind...idium-audio-clock#most-read-equipment-reviews

edit: from amazon user Chris Lopez: "the motherboard x10srl-f-o works v3/v4 family and the processor Intel Xeon E5-1620 is v2 family, so it doesn't work." i'm off to a great theoretical start

here we go, i want to protect my data, here is a CPU,
338-BJFK Dell Intel Xeon E5-2699 v4 2.20GHz compatible Product by NETCNA for $13,571.63
https://www.amazon.com/338-BJFK-E5-...30&sr=8-8&keywords=xeon+e52699+v4&tag=ozlp-20

with the following features:

• Enhanced SpeedStep Technology; Turbo Boost Technology; Intel Extended Memory 64 Technology; Execute Disable Bit Capability; Intel Virtualization Technology (VT-X); Intel Virtualization Technology for Directed I/O (VT-d); Intel VT-X with Extended Page Tables (EPT); Trusted Execution Technology; Idle States; Hyper-Threading Technology; Demand Based Switching; Thermal Monitoring Technologies; Intel vPro Technology; AES New Instructions.