optimal configuration for 8 ssd for read only tasks

[iliak]

Hi, i should get the server in a day or two

the configuration for the server:

• SuperMicro Server 1U 1027R-72BRFTP
• CPU 2 X E5- 2630v2 CPU's
• RAM 64GB ecc
• BOOT DRIVE - SATADIMM - VIKING TECHNOLOGY SATADIMM 100GB
• Controller AOC-S2308L-L8e
• Mellanox ConnectX-3 40Gigabit ( will be connected to compatible switch)
• STORAGE 8x800gb SSD sas 6 - will be used for read only pool ( will be updated once a day)
• fusion io 1.2tb (mlc ssd )- will be used as network cache and temporary shared folder for some tasks(will be deleted daily)

what is the optimal configuration for the 8*ssd ,z1? or 4*2 ? the pool will be for non production use, and will be updated once a day for approx (3GB of new data)
the container will hold approx 1000 files each with average of 2GB)
the storage should support relatively low iops (up to 100 requests in second). but very high throughput (each request can be for a 500GB data) if it possible to configure it as first request firsd servers would be best.

 

[Chris Moore]

Depends on your tolerance for potential failure of any of the SSDs. If you want maximum IOPS, you make a straight stripe giving you eight vdevs, with each vdev being a single disk. This configuration gives you no fault tolerance, but maximum speed. If you want fault tolerance, you need mirror vdevs, which takes two SSDs per vdev, giving you four vdevs instead of eight. This cuts the IOPS in half but it gives you the ability to survive a disk fault and correct data errors.

Mellanox ConnectX-3 40Gigabit ( will be connected to compatible switch)

Throughput will be limited by the interface of the drives, not by the network

 

[iliak]

I think the minimal fault tolerance (of 1 hdd) should be enough, this is not production and we can mange few hours\day until hdd is replaced.

each ssd should give approx 700-750MB read/write (enterprise grade ssds, let's hope the bottleneck will be the ssds :))

the server support only 8 hdds, so no future upgrade will be possible(only more ram can be added), any future upgrade will probably be another server.

what configuration would you suggest ?

 

[Chris Moore]

With RAIDz1 (one drive redundancy) you will only have about 4.7 TB of usable capacity, allowing for the fact that you should not fill a ZFS pool beyond 80% capacity. The throughput should be adequate for your requirements.

RAM 64GB ecc

I think you would be wise to include more memory because ZFS uses the system RAM for ARC (adaptive replacement cache) so it can respond to requests for data from memory instead of needing to access the disks if the data is still in RAM. It accelerates performance and reduces the wear on the SSDs. I would suggest getting as much memory as possible.

 

[Chris Moore]

fusion io 1.2tb (mlc ssd )- will be used as network cache and temporary shared folder for some tasks(will be deleted daily)

I am not sure how you plan to use this. It is unclear to me.

 

[iliak]

I am not sure how you plan to use this. It is unclear to me.

a shard folder across multiple machines. the main shared storage for spark/yarn grid. it is require