ilmarmors
Dabbler
- Joined
- Dec 27, 2014
- Messages
- 25
Hello,
I'm looking for a most efficient way how to spend 10000 EUR (including VAT) for 2 cold storage servers. All the components must be purchased in EU. Max budged limit is cut in stone. I want to use 2 identical or almost identical servers. I imagine, there is possibility to play with memory and disks - slightly different configs for primary and secondary server.
Purpose:
SuperMicro SuperStorage Server 5018A-AR12L
http://www.supermicro.com/products/system/1U/5018/SSG-5018A-AR12L.cfm
~1500-1600 EUR
http://www.asrockrack.com/general/productdetail.asp?Model=1U12LW-C2750
~1150 EUR in DE, but I won't be able to buy that cheap in my country, so it would be just ~200 EUR savings, but there are many cons:
~380 EUR
Unfortunatelly tested memory list for Supermicro motherboard is small and 16GB module on that list (or in general?) cost arm and leg > 400 EUR for one module.
HDD options:
12 x WD Red WD40EFRX ~2200 EUR
12 x H3IKNAS40003272SE (7200rpm) ~2200-2300 EUR
12 x WD Red WD50EFRX ~2900 EUR (dangerously close to limit)
12 x HGST IDK Deskstar NAS H3IKNAS500012872SE (probably over the limit)
option for secondary server only:
12 x Seagate Archive 8TB (will require to use 4TB in primary server for sure because of total cost limit) (probably will be available by time servers will actually be bought) (can be interesting option from future perspective - if there are second primary server, then this secondary server can be secondary server for 2 primary servers, which have 2 time smaller disks each)
SATA DOM: Adata SATA III DOM Memory Card 16GB, MLC, 0 to 70C, 7pin, Horizontal (I can get hands on such model, not big selection here for me). If second SATA DOM for OS mirror, then it will need to be sticked into LSI ports + power. SuperMicro has only 1 SATA port with power for SATA DOM on motherboard.
I'm thinking about doing copying from primary to secondary via rsync. And taking snapshots from time to time on secondary too. In theory that should exclude possibility that secondary server ZFS pool can be broken if replication is done by zfs send and ZFS goes nuts on primary server, right?
There is question about most appropriate pool config. Options are:
Thanks,
Ilmārs
I'm looking for a most efficient way how to spend 10000 EUR (including VAT) for 2 cold storage servers. All the components must be purchased in EU. Max budged limit is cut in stone. I want to use 2 identical or almost identical servers. I imagine, there is possibility to play with memory and disks - slightly different configs for primary and secondary server.
Purpose:
- Storing 2nd and 3rd copy of digitized objects master files - images, audio, video. Those files are not accessible to end users.
- Files are copied to primary storage server over 1Gbps Internet connection from few places.
- Files from primary storage server are copied to secondary storage server automatically.
- Bit rot safe storage. That's why I'm looking at ZFS.
- Should be able to saturate 1Gbps link.
- Mostly sequential workload. Files once written will stay there forever.
- Must handle at least 10M files in total.
- Rack-mountable case. The smaller the better. 1U ideal. Definitely no more than 2U.
- Max 10000 EUR (with VAT) for 2 running servers
- No video trans-coding, no jails or other CPU intensive tasks on servers
- Compression of files are on. Image files are stored mainly as loss-less uncompressed TIFFs, that can be compressed.
SuperMicro SuperStorage Server 5018A-AR12L
http://www.supermicro.com/products/system/1U/5018/SSG-5018A-AR12L.cfm
~1500-1600 EUR
- Intel Atom processor C2750
- 12x 3.5" Hot-swap HDD bays
- Up to 64GB Unbuffered ECC DDR3 1600MHz in 4x DIMM sockets
- Dual GbE LAN ports
- Integrated IPMI 2.0 and KVM with Dedicated LAN
- 400W High-efficiency redundant short depth Power Supplies
- Proprietary form factor motherboard form factor - http://www.supermicro.com/products/motherboard/ATOM/X10/A1SA7-2750F.cfm
- LSI 2116 SW controller for 16x SATA3 (6Gbps) ports; SATA3 or SAS2 with IT mode
http://www.asrockrack.com/general/productdetail.asp?Model=1U12LW-C2750
~1150 EUR in DE, but I won't be able to buy that cheap in my country, so it would be just ~200 EUR savings, but there are many cons:
- Simple non hot-swappable 250W PSU
- All 12 SATA ports on motherboard are used up, so FreeNAS will need to be run from USB sticks
- Using Marvell ports makes be nervous, if HBA card is added, then savings are even less, if at all.
~380 EUR
Unfortunatelly tested memory list for Supermicro motherboard is small and 16GB module on that list (or in general?) cost arm and leg > 400 EUR for one module.
HDD options:
12 x WD Red WD40EFRX ~2200 EUR
12 x H3IKNAS40003272SE (7200rpm) ~2200-2300 EUR
12 x WD Red WD50EFRX ~2900 EUR (dangerously close to limit)
12 x HGST IDK Deskstar NAS H3IKNAS500012872SE (probably over the limit)
option for secondary server only:
12 x Seagate Archive 8TB (will require to use 4TB in primary server for sure because of total cost limit) (probably will be available by time servers will actually be bought) (can be interesting option from future perspective - if there are second primary server, then this secondary server can be secondary server for 2 primary servers, which have 2 time smaller disks each)
SATA DOM: Adata SATA III DOM Memory Card 16GB, MLC, 0 to 70C, 7pin, Horizontal (I can get hands on such model, not big selection here for me). If second SATA DOM for OS mirror, then it will need to be sticked into LSI ports + power. SuperMicro has only 1 SATA port with power for SATA DOM on motherboard.
I'm thinking about doing copying from primary to secondary via rsync. And taking snapshots from time to time on secondary too. In theory that should exclude possibility that secondary server ZFS pool can be broken if replication is done by zfs send and ZFS goes nuts on primary server, right?
There is question about most appropriate pool config. Options are:
- one vdev with 12 disk RAID-Z3
- one vdev with 11 disk RAID-Z3 (although "Since FreeNAS utilizes compression by default (and there are 0 cases where it makes sense to change the default!), any attempts to optimize ZFS with the vdev configuration are foiled by the compressor.")
- one vdev with 12 disk RAID-Z2 (feels too risky, although there should be backup copy on other server)
- two vdevs with 6 disk RAID-Z2
Thanks,
Ilmārs