First build

[fjmp24]

Hi, I come back to you. What do you think of this configuration to run TrueNAS? Knowing that:

• I forget ECC
• I have to use mi-itx format
• I have to run around 20 containers (Handbrake, flexget, jellyfin, nodered, zigbee2mqtt, gitea..)
• I may need 2-3 virtual machines
• I need video transcoding
• I would like to have 4 SSD drives for data
• I would like TrueNAS installed on an M.2 PCIe
• I would like an M.2 PCIe for the cache

Motherboard: ASRock Z790M-ITX Wi-Fi LGA1700 ITX Motherboard (DDR5)
RAM: Corsair Vengeance DDR5 64GB (2x32GB) 5200MHz
Memory Processor: i5-13600 or i7-13700

 

[Heracles]

Well.... What you wrote in your post points more to the need for an hypervisor like ProxMox than to TrueNAS. TrueNAS is a storage appliance. It can run some jails / plugins or VMs but storage is the first mission.

Second thing is, considering the VMs, numerous jails / plugins, etc., your storage does not look good...

You need a maximum IOPS for VMs and parallel accesses. For that, you need a maximum number of vDev. With 4 drives, that means only 2 mirror vDev. That also means you have the usable space of only 2 drives. Not much for a personal media server, VMs and more.

From here, you seam to be on the wrong track...

 

[fjmp24]

Sorry I forgot to say I need to store 1TB of files

 

[Heracles]

So your need day 1 is 1TB. Considering a pool must not go ver 80%, that means you need 1.25T. Because it is hard to add space, it is not wise to design a system that will be loaded up to 50% on first use. So that means 2.5 T of usable space day 1 if you wish to go with only the very minimum recommended. Because you would have the usable space of only 2 drives, that means you need SSDs of at least 1.25T each.

So again, this design does not seam to be on the right track...

 

[fjmp24]

Should I use proxmox and truenas?

 

[Heracles]

TrueNAS is an appliance built for storage using ZFS. Nothing in what you described is related to ZFS storage. Because of that, TrueNAS does not look like the solution you need.

You can go with Proxmox and create your VMs in it. Proxmox will manage the local storage and present it to VMs for each to take and use what they need.

To run containers, you would be better to deploy Docker in one of these VMs and run your containers from that host.

Should you need to share files between your VMs, a simple file sharing VM with any linux distro like Ubuntu can do it over NFS or SMB.

Can you name me 3 features of ZFS that would be of interest to you ? If you can, please do so and then we will be able to tell you how to do it in an optimal way. If you can not identify ZFS features that would be of use to you, why would you go with ZFS then ?

For storage, a simple linux system with LVM / ext4 is probably what you need.
About running apps, a Docker host looks like what you need.
Both of these can run from a Proxmox hypervisor next to your other VMs.

At the end, TrueNAS is nowhere in the plan because you would not use it for its main role (ZFS) and would even put you at risk by trying to use it for what it is not meant.

 

[fjmp24]

Ok, I'll start from the beginning.

Currently, I have a Synology DS423+ NAS which allows me:
- to store all my files (document, image, video, music)
- save photos from our phones
- to host my docker containers and a virtual machine
- to secure the storage of my files (RAID5)
- to have a modification history on particular directories
- ...

The problem is that the synology is not powerful enough to do what I want to do. Hence the idea of creating a powerful machine that would allow me to do what I want.
TrueNAS interests me because the ZFS file system allows:
- to have data integrity
- take snapshots
- to encrypt data
- to create redundancy

After I saw people using TrueNAS with Proxmox to have more freedom.

 

[Heracles]

Currently, I have a Synology DS423+ NAS which allows me:
- to store all my files (document, image, video, music)
- save photos from our phones
- to host my docker containers and a virtual machine
- to secure the storage of my files (RAID5)
- to have a modification history on particular directories
- ...

The problem is that the synology is not powerful enough to do what I want to do.

This part of your reply points to the need for an application server.

TrueNAS interests me because the ZFS file system allows:
- to have data integrity
- take snapshots
- to encrypt data
- to create redundancy

Here we come in the ZFS world and TrueNAS.

So about encryption, you should not touch that before you took a few years of experience with TrueNAS and ZFS. A single mistake and ZFS encryption will turn to a self-inflicted ransomware. Considering the almost non-existent cases where that encryption is of any use, do not touch this.

ZFS is surely very good at ensuring data integrity when all its requirements are satisfied. Unfortunately, you already discarded one of them (ECC).

Snapshots is also a fundamental feature that ZFS is very good at. Good that you separated snapshots and redundancy because indeed, snapshots are not redundant nor are they backups.

About redundancy, this point can be at different level (disk level, data level, server level). At disk level, we are talking about RaidZ and mirrors. Know that RaidZ1 is not considered safe anymore and it will betray you down the road. You should go for RaidZ2 or mirrors at least.

At data level, ZFS can create redundant copies within the same pool. That is of very little use and your resources will be better used by leaving that copy=1 (default) and do different redundancy.

Server level redundancy will require more than 1 server. Do you have a second server ? If you do, we will need the specs for the other one.

So for disk level redundancy, with only 4 drives, that leaves you with the usable space of only 2 drives.
It is dangerous to load a pool beyond 80%.
Because it is often difficult / costly to add space in a pool, it is not recommended to design a pool that will be loaded up to 50% day 1.

So that means you need each of your SSD drive be bigger than 100% of your actual data volume. I doubt this is the case but please confirm.

From what you described, you need an application server more than anything else. Of course, every application is meant to handle data, so you do need data storage.

TrueNAS is storage first, apps second and way behind.
Proxmox is application first, storage way behind.
Unraid and Synology try to score somewhere in between for both functions.

TrueNAS is specialized and enterprise-class solution but that means it also has some enterprise-class requirements.
Others are more consumer and end-user oriented, so their requirements like ECC are not the same.

 

[fjmp24]

On the TrueNAS discord I was told that ECC was not a necessity for a personal NAS and that ZFS did not require it.

ECC protects you against bit flips of data on hold in memory or during transfer. But it is not a "common"
thing or a big problem as still most PCs don't use it.
It is commonly used in servers though because there data integrity usually matters a great deal, you don't
want a bit flip causing a transaction to become billions instead of millions or a multiple day long job to
fail due to it.
If you store your data on your local machine without ECC, you could easily store it on a NAS without ECC.
People just like to use ECC because it gives an extra layer of stability and the given platform supports it

It does not, even ZFS developers stated that ZFS is not different from other filesystems in that regard
https://www.klennet.com/notes/2022-11-25-zfs-and-ecc-rant.aspx
TLDR: Many other things will cause you data loss before ECC becomes relevant
That doesn't mean ECC has no value, you just need to weight if it is worth going for given all the constraints
it entails on consumer hardware

If I well understand, I need two machines:
- One for backups (TrueNAS or Synology)
- One for my applications (Proxmox)

 

[Heracles]

That would be better indeed. The Proxmox one will run the VM and applications while TrueNAS will do the storage. The old Synology can then turn to the backup system and you will have a complete solution that way.

 

[fjmp24]

How Proxmox and TrueNAS can work together ?

 

[Heracles]

I am not using Proxmox myself (using ESXi here...). Still, the concepts are probably the same.

You create shares in your TrueNAS server (here I use NFS but SMB is also available) and then mount it from other systems. Either Proxmox mounts the share and present it to its VM or you install the VMs using only Proxmox local storage and then you mount your data inside each VM. As for me, I would rather go with the second option because there is one less layer of abstraction to manage.

--Create a small Linux VM in your Proxmox using only local storage
--Install Docker in that Linux VM and that will be your application server
--Deploy your applications as Docker containers (ex: Jellyfin) and mount your data from TrueNAS using an NFS Volume inside these containers

If you are not that familiar with Docker, it could be a good thing to install Portainer as your first Docker container and then use Portainer and its Web user interface to create everything else, including the NFS Volume to re-map inside your containers.