First time FreeNAS

[iliak]

hi,
im new to the storage part of DevOps (a new responsibility at my work place)

we should receive a server that will be mostly used as read intensive jobs (approx 30-40 Gbs ) will be connected with 40gb network

we need to make 2 shared storage's :
1. 80% of the capacity to store the data files (the clients will read them)
2. 20% of the capacity as temporary scratch location( the output of the clients will be stored here for total time of one week)

what is the best configuration to do so
the reads are very have approx 30-40 Gbs (at peaks) up to 1k iops
the write are up to 1Gbs

i thought of making raid 5 (using all the 8 ssds), and make two volumes one for each.

this is not for production (that why 1 hdd fail should be enough)

what do you think?

 

[Chris Moore]

i thought of making raid 5 (using all the 8 ssds), and make two volumes one for each.

this is not for production (that why 1 hdd fail should be enough)

No opinion, that is not just a wild guess, is possible without knowing the hardware involved. Please follow this guidance and give details:

Updated Forum Rules 12/5/18
https://forums.freenas.org/index.php?threads/updated-forum-rules-12-5-18.45124/

 

[iliak]

HW for the server:
SuperMicro Server 1U 1027R-72BRFTP W/ 2 X E5- 2630v2 CPU's 64GB & 8x800gb ssd sas
Mellanox ConnectX-3 40Gigabit

i think the ssds are sandisk Optimus Extreme™ SAS SSDs (or something similar)

 

[Chris Moore]

we should receive a server that will be mostly used as read intensive jobs (approx 30-40 Gbs ) will be connected with 40gb network

From the way you describe this, I am guessing that it is already ordered and on the way. That is really a bit of a bad deal because it might not even be hardware that is compatible with FreeNAS and the ZFS file system.

 

[Chris Moore]

HW for the server:
SuperMicro Server 1U 1027R-72BRFTP W/ 2 X E5- 2630v2 CPU's 64GB & 8x800gb ssd sas
Mellanox ConnectX-3 40Gigabit

i think the ssds are sandisk Optimus Extreme™ SAS SSDs (or something similar)

That is all fine, but we also need to know what drive controller is being used to access those SAS SSDs and it might be nice to know what you plan to use for the boot pool.

 

[iliak]

We had tight budget and this is the best that fit our budget, i hope it will work with freenass.(if not we will have to pick another solution)

For boot there will be a SATADIMM - VIKING TECHNOLOGY SATADIMM 100GB 240P SSD Storage Controller,
about the drive controller i dont know,, ill ask

 

[Chris Moore]

SuperMicro Server 1U 1027R-72BRFTP

The default system board for that model server includes an integrated Broadcom 2208 hardware RAID controller which is completely unsuitable for use with FreeNAS.
There are PCI-E slots for expansion, so you could disable the onboard controller and install a SAS HBA, but routing the cables might be a problem in a 1U chassis.

 

[Chris Moore]

For boot there will be a SATADIMM - VIKING TECHNOLOGY SATADIMM 100GB 240P SSD Storage Controller

https://www.ebay.com/itm/Viking-Tec...-SSD-Storage-VRFSD3100GNCTMTKS1-/222736128417
That is pretty insane looking. It is more storage than you need for the FreeNAS boot pool. The FreeNAS boot pool is ZFS and it is best to have a mirror for the boot pool even if it is on SSD to guard against data corruption.
The information to correct checksum errors comes from the redundancy information, so having no redundancy means there would be no ability for ZFS to correct an error, just report it.

 

[iliak]

we did not receive the hardware yet,, ill try to change it for a model with different raid controller

 

[Chris Moore]

we did not receive the hardware yet,, ill try to change it for a model with different raid controller

No RAID controller. It needs to be a SAS HBA in IT mode.

 

[Chris Moore]

ZFS, the file system of FreeNAS, needs direct access to the disks and ZFS handles mapping them into storage pools.

 

[Chris Moore]

Now, more background on ZFS. If you are going to be managing this, you should probably spend some time learning about it...

i thought of making raid 5 (using all the 8 ssds), and make two volumes one for each.

Your statement above included "raid 5" and that is not the terminology of ZFS, so I thought you might want to spend some time reviewing these documents to help you understand the technology and terminology better.

Slideshow explaining VDev, zpool, ZIL and L2ARC
https://forums.freenas.org/index.ph...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/

Terminology and Abbreviations Primer
https://forums.freenas.org/index.php?threads/terminology-and-abbreviations-primer.28174/

 

[Chris Moore]

what is the best configuration to do so
the reads are very have approx 30-40 Gbs (at peaks) up to 1k iops
the write are up to 1Gbs

About this. Your mileage may vary since you are using SSDs, but I don't think you will get that many IOPS from a RAIDz pool.
@jgreco might be in a better position to comment on that. I don't use SSD based storage for my work.

 

[iliak]

our iops requiremnt is low, but each iop is for large amount of data

 

[iliak]

does controler like "AOC-S2308L-L8e" will work ?

 

[Chris Moore]

does controler like "AOC-S2308L-L8e" will work ?

Sure, that is Supermicro's model number, but it should work. I have a LSI/Broadcom SAS9207-8i in my home server and the 2308 is just a newer chipset of the same kind of controller.

 

[iliak]

Sure, that is Supermicro's model number, but it should work. I have a LSI/Broadcom SAS9207-8i in my home server and the 2308 is just a newer chipset of the same kind of controller.

because there might be some stoking issues, does the following controllers work as well?

AOC-S3008L-L8e
AOC-S2308L-L8e ( you confirmed it should in previous post)

 

[Chris Moore]

because there might be some stoking issues, does the following controllers work as well?

AOC-S3008L-L8e
AOC-S2308L-L8e ( you confirmed it should in previous post)

Yes. That should be fine.

 

[iliak]

Yes. That should be fine.

The server will arrive in few days with AOC-S2308L-L8e

there is the final hw that is going to arrive

• SuperMicro Server 1U 1027R-72BRFTP
• CPU 2 X E5- 2630v2 CPU's
• RAM 64GB ecc
• BOOT - SATADIMM - VIKING TECHNOLOGY SATADIMM 100GB
• STORAGE 8x800gb SSD sas 6
• Controller AOC-S2308L-L8e
• Mellanox ConnectX-3 40Gigabit

any tips on how to configure

steps i thought of making before os install ( we do it with all our severs):

1. configure\upgrade ipmi
2. integrade ipmi to domain
3. upgrade bios
4. validate all hw is recognized
5. hw test \ built in test

Here we go freenass setup (first time)? what now ?

what is the optimal configuration based on my needs: explained in first post

 

[Chris Moore]

i thought of making raid 5 (using all the 8 ssds), and make two volumes one for each.

ZFS can take all the drives and map them into a single pool:
https://www.ixsystems.com/documentation/freenas/11.2/storage.html#pools
You said 'raid 5' which in ZFS terms would be RAIDz1. From your storage pool you can create datasets that are dedicated to various purposes.
https://www.ixsystems.com/documentation/freenas/11.2/storage.html#adding-datasets

we need to make 2 shared storage's :
1. 80% of the capacity to store the data files (the clients will read them)
2. 20% of the capacity as temporary scratch location( the output of the clients will be stored here for total time of one week)

The datasets can be can be separate shares with totally different properties.
https://www.ixsystems.com/documentation/freenas/11.2/sharing.html#sharing
The following is a fairly good guide for configuring the NAS, even though it was written for a previous version of the GUI, most of the information in the guide is still very useful and appropriate for configuring a new system.

Uncle Fester's Basic FreeNAS Configuration Guide
https://www.familybrown.org/dokuwiki/doku.php?id=fester:intro

The guide suggests doing a 'burn-in' test on the drives, but that is not needed with solid state drives. If you review the other links in the "Useful Links" button in my signature and the information in the guide, I think it will get you up and running. If you have any questions, please ask. It is better to ask than to have problems later.