Ace22
Cadet
- Joined
- Sep 13, 2017
- Messages
- 2
Hi folks,
Problem:
When I drag a folder containing multiple sub folders/files from (Z: on Freenas file server) to (D: or C: on PC Workstation) (some small, but mostly large video files broken down into 4GB increments) I get transfer speeds of 20-50MB/s ??? I am getting >1GB/s in all other transfer situations.
- Is this what i should be seeing?
- Is there a flaw in my raid/pools design based on my performance need?
- Have I missed some settings, performance tweaking?
- Is this a hardware issue?
- Where is my bottleneck??
Any help or insight would be greatly appreciated.
If extra info required - I won't have access to the server until mid next week, and will retrieve any info requests/screenshots etc. then!
Thanks so much everyone!
Background info and system specs:
This is my first Freenas setup - I have spent many hours reading the manual and forums. Despite this have I butchered the setup?
This is a storage server for a production company using 8k video files. The requirement is to have high speed transfer between 1 primary editing machine via the mellanox connection.
I used this performance tuning guide for the mellanox 10GB connection. Down the line 10GB ethernet to other machines on the network, but we'll worry about that when the time comes.
Transferring between (C: ) and (D: ) on the editing rig I can achieve speeds of 1.7GB/s - regardless of single file or entire folder.
Transferring from the Editing rig regardless of single file or entire folder (D: ) to the Freenas Storage server (Z: ) I get variable speeds peaking around 1Gb/s (I get frequent error msgs that when accepted/ignored transfer the file anyways, but I can live with for now/will deal with that separately - not the reason I am posting here)
When transferring a large single file from storage server (Z: ) to editing (D: or C: ), I have sustained transfer speed of 1.2GB/s.
To summarize this:
- C: can read/write over 1GB/s
- D: can read/write over 1GB/s
- Z: can write at 1GB/s (performance fluctuates, I do get errors, but overall speeds are good enough)
- Mellanox network can sustain 1.2GB/s transfers both directions
*Note: I am maxing out the 64GB of RAM after about 30 seconds into a transfer, I have another 128GB arriving in a few days.
Storage Server Specs:
Freenas 9.10 or 9.11 (can't remember off the top of my head but I used latest stable avail 3 months ago)
x9 supermicro board
e5 2650 v2
24 bay chasis w/backplane
4x16gb =64GB Rdimm 1066mhz ecc
lsi 9211 8i HBA
(Z: ) 16 HGST Ultrasta 3TB HDD *All 16 drives are in a single raidz2, deduplication disabled*
Mellanox Connect ex3 running at 10GB
Editing Rig Specs:
x99 Workstation mobo
e5 2690v4
64GB ecc RAM
(C: ) 1 Samsung 960 Pro NVME
(D: ) 6x10TB Seagate Enterprise He (windows raid 0 - yes i know, no redundancy here 3 copies of data are kept so failure here is ok - I need max speed for editing 8k files) *max block size of 64k
Problem:
When I drag a folder containing multiple sub folders/files from (Z: on Freenas file server) to (D: or C: on PC Workstation) (some small, but mostly large video files broken down into 4GB increments) I get transfer speeds of 20-50MB/s ??? I am getting >1GB/s in all other transfer situations.
- Is this what i should be seeing?
- Is there a flaw in my raid/pools design based on my performance need?
- Have I missed some settings, performance tweaking?
- Is this a hardware issue?
- Where is my bottleneck??
Any help or insight would be greatly appreciated.
If extra info required - I won't have access to the server until mid next week, and will retrieve any info requests/screenshots etc. then!
Thanks so much everyone!
Background info and system specs:
This is my first Freenas setup - I have spent many hours reading the manual and forums. Despite this have I butchered the setup?
This is a storage server for a production company using 8k video files. The requirement is to have high speed transfer between 1 primary editing machine via the mellanox connection.
I used this performance tuning guide for the mellanox 10GB connection. Down the line 10GB ethernet to other machines on the network, but we'll worry about that when the time comes.
Transferring between (C: ) and (D: ) on the editing rig I can achieve speeds of 1.7GB/s - regardless of single file or entire folder.
Transferring from the Editing rig regardless of single file or entire folder (D: ) to the Freenas Storage server (Z: ) I get variable speeds peaking around 1Gb/s (I get frequent error msgs that when accepted/ignored transfer the file anyways, but I can live with for now/will deal with that separately - not the reason I am posting here)
When transferring a large single file from storage server (Z: ) to editing (D: or C: ), I have sustained transfer speed of 1.2GB/s.
To summarize this:
- C: can read/write over 1GB/s
- D: can read/write over 1GB/s
- Z: can write at 1GB/s (performance fluctuates, I do get errors, but overall speeds are good enough)
- Mellanox network can sustain 1.2GB/s transfers both directions
*Note: I am maxing out the 64GB of RAM after about 30 seconds into a transfer, I have another 128GB arriving in a few days.
Storage Server Specs:
Freenas 9.10 or 9.11 (can't remember off the top of my head but I used latest stable avail 3 months ago)
x9 supermicro board
e5 2650 v2
24 bay chasis w/backplane
4x16gb =64GB Rdimm 1066mhz ecc
lsi 9211 8i HBA
(Z: ) 16 HGST Ultrasta 3TB HDD *All 16 drives are in a single raidz2, deduplication disabled*
Mellanox Connect ex3 running at 10GB
Editing Rig Specs:
x99 Workstation mobo
e5 2690v4
64GB ecc RAM
(C: ) 1 Samsung 960 Pro NVME
(D: ) 6x10TB Seagate Enterprise He (windows raid 0 - yes i know, no redundancy here 3 copies of data are kept so failure here is ok - I need max speed for editing 8k files) *max block size of 64k
Last edited: