Very low speed with small files

[bohrz]

Hi, i just finished building a small home server with TrueNAS. I'm new to the whole subject, so I apologize in advance.

The parts I used are:

• CPU: Core i3 10100
• MB: Gigabyte H510M H
• OS Drive: Kingston A400 240GB SSD
• RAM: Crucial 16GB DDR4 2666MHz
• Storage: 2x8TB Seagate Ironwolf mirrored

I work with Salesforce development, so it's basically a NodeJS project with lots of small files that a I need to transfer from my main PC to the server, also, I would like to edit from the server.

Now, I know my parts are low spec, also small files are a nightmare for speed, but I'm getting speeds below 1KB/s, is that normal?

What can I do to diagnose the problem or find bottlenecks?

 

[homer27081990]

No, it is not.

• Can you try moving one large file?
• Check your network configuration, cables, router, switches
• Make sure you do not have any throttling on the array or the network
• Try moving files between another (third) computer and TrueNAS, and then your PC
• Check your RAM usage: [go to the shell and run top - If you have high RAM usage check swap partition
• Make sure your boot drive is healthy (SMART, at the very least)
• Check the health of the array drives
• Run a scrub on the array

 

[c77dk]

You have a Realtek NIC on that board - those are cr**. Try putting in a Intel NIC.

But other than the nic, how are the datasets setup regarding sync?

 

[bohrz]

Hi,
I made some file transfers from my desktop W10 PC. I copied a blu-ray file (~100GB), speeds were between 85 to 100 MB/s. Since the NAS and the PC have gigabit, it seems OK. I tried moving one of my projects, it wasn't fast, but it took less than 6 minutes for moving ~30K files (about 1GB), so I'm content with that.

When I tried moving a project from my company laptop W10, the performance was much much slower, so much so I canceled the transfer. I'll check with the IT department if there's any company security measure which might be impacting so much. The laptop I tested using wireless AND wired, both to no avail.

The last test I made using my M1 MacBook Air, I moved a small SvelteKit project from the laptop to the server, 170MB, took some 5 minutes.

About the the tests you asked, I scheduled two SMART tests (long and short) to run on the SSD, but I can't find the test logs/results. The RAM usage command you listed appears as []FILE]top{/FILE], which does not run. But the my RAM usage is listed by the dashboard as:

• 15.8 GiB total available;
• Free: 0.4GiB;
• ZFS Cache: 12.2GiB;
• Services: 3.2GiB.

I can provide some images if needed.

One thing I noticed, my disks are constantly spinning, even though I'm barely using them, doesn't seem right.

Would setting up an Apple Share (AFP) improve my transfer speeds when using the Mac Air?
I found a cheap Intel NIC, the EXPI9301CT, would it improve small file transfer speeds?

For the other tests/information you need, could you point me how to do? Like, how do I check the array sync setup? I tried google, but did not find this info.

I started a scrub as well.

Thank you very much for the help!

 

[Samuel Tai]

There's a ZFS copy bug that results in slow performance with small blocks. See
https://www.truenas.com/community/threads/truenas-13-way-slower-than-12.103082/post-711029. This will get fixed in 13.0-U2.

As for your corporate laptop, the usual culprit is virus scanning on every file transfer.

For your Mac, AFP is deprecated, and Apple recommends SMB nowadays. Slow copies aren't helped by Spotlight indexing of the network drive. Try exempting the SMB share from indexing.

 

[bohrz]

Understood, so I should wait for the fix, right?

As for the Intel NIC, the one I listed, will do any good?

 

[HoneyBadger]

If the worst of the slowness is felt with small files from macOS, it could be SMB strict sync in effect here.

Try disabling SMB strict sync; in the SMB auxiliary settings (Services -> SMB -> Little Wrench Icon) set strict sync = no

 

[bohrz]

The worst offender is the company laptop, but I suspect it is doing some scan or something like that for every file.
Another issue I'm having is when I try to run npm install (to install the node packages for the project) from my PC inside a server folder. It's taking more than 5 minutes to run the command and install the packages, which usually takes less than a minute locally.
I created a jail with OpenSpeedTest and its reporting 400mbps/200mbps, which is my full internet speed. I disabled the strict sync, let me see how it goes.

About the NIC I listed, Intel EXPI9301CT, would it do me any good?

 

[bohrz]

So, I installed the latest updates from TrueNAS and it seems the problem persists. Nevertheless, I think I can survive. If anyone has any other suggestion, I'll be happy to try.

Would I see any improvements with the NIC I mentioned above?