TrueNAS SCALE 22.12.1 write speeds start fast then drop to nothing.

[oumpa31]

I've been running Scale for years and rebuilt my server on TrueNAS Scale 22.12.1 with 12 Seagate Exos X16 14TB 7.2K RPM drives set up in a raid Z3. My network is a Unifi 10GB network and i am able to get 1GB transfers from 1 windows tower to another.
24 core Threadripper 64Gb ram 10gb intel x550 nic

It took some time but i moved all my data to these new drives. When i set this pool up i turned on dedupe cause I'm sure i have multiple copy's of some stuff in different folders. I just don't have the time to find them all. But now that I'm using these drives when i go to transfer a movie file (11GB) to these drives my write speeds will start off around 500MB jump to 800MB then drop to nothing and sit that way for a minute or longer. My old drives (8) 12TB WD red 5400 rpm that didn't have dedupe on i would get consistent 300+MB transfers at a minimum.

My video files are in there own folder set up in TrueNAS Scale. Do i need to turn off dedupe to get the write speeds back to normal?

I turned some of the old drives into vdev drives for ISCSI for myself and my kids and i get write and read speeds of almost 2GB and a consistent 300MB when moving a 100GB 4k video on the same machine. So its not the hardware of the machine. I just wouldn't think dedupe would kill a transfer like that.

 

[ZPrime]

You should go RTFM about Dedupe. It is incredibly RAM-intensive, and also very disk-intensive (because the table of hashed chunks has to sit on the disk when it isn't in RAM). TFM suggests that you should have an SSD-based "special" VDEV in your pool dedicated solely to the dedupe table data ("DDT").

ZFS Deduplication - Would it worth ...

Hi, I've got a use case where I'm very thinking about enabling zfs deduplication. I've got two separate datasets : - Medias : For Plex movies, tvshows and other - Qbittorrent : Where qbt is watching, downloading and seeding files Today, when I'm downloading video file that I want to put in...

www.truenas.com

TrueNAS Backup Target - Deduplication Impact

A TrueNAS backup taget is an ideal solution for home or small business usage. Thanks to deduplication in TrueNAS...

www.storagereview.com

Both of those threads are worth a read.

"Multiple copies of some stuff in some folders" probably isn't worth turning on dedupe, unless "some stuff" is actually "multiple TB worth of data".

 

[oumpa31]

You should go RTFM about Dedupe. It is incredibly RAM-intensive, and also very disk-intensive (because the table of hashed chunks has to sit on the disk when it isn't in RAM). TFM suggests that you should have an SSD-based "special" VDEV in your pool dedicated solely to the dedupe table data ("DDT").

ZFS Deduplication - Would it worth ...

Hi, I've got a use case where I'm very thinking about enabling zfs deduplication. I've got two separate datasets : - Medias : For Plex movies, tvshows and other - Qbittorrent : Where qbt is watching, downloading and seeding files Today, when I'm downloading video file that I want to put in...

www.truenas.com

TrueNAS Backup Target - Deduplication Impact

A TrueNAS backup taget is an ideal solution for home or small business usage. Thanks to deduplication in TrueNAS...

www.storagereview.com

Both of those threads are worth a read.

"Multiple copies of some stuff in some folders" probably isn't worth turning on dedupe, unless "some stuff" is actually "multiple TB worth of data".

Thanks for the reply. I'll give those a read