FreeNAS® is © 2011-2019 iXsystems

FreeNAS® and the FreeNAS® logo are registered trademarks of iXsystems

FreeBSD® is a registered trademark of the FreeBSD Foundation

Written by users of the FreeNAS® network-attached storage operating system.

Version 11.1

Copyright © 2011-2019 iXsystems

This Guide covers the installation and use of FreeNAS® 11.1.

The FreeNAS® User Guide is a work in progress and relies on the contributions of many individuals. If you are interested in helping us to improve the Guide, read the instructions in the README. IRC Freenode users are welcome to join the #freenas channel where you will find other FreeNAS® users.

The FreeNAS® User Guide is freely available for sharing and redistribution under the terms of the Creative Commons Attribution License. This means that you have permission to copy, distribute, translate, and adapt the work as long as you attribute iXsystems as the original source of the Guide.

FreeNAS® and the FreeNAS® logo are registered trademarks of iXsystems.

Active Directory® is a registered trademark or trademark of Microsoft Corporation in the United States and/or other countries.

Apple, Mac and Mac OS are trademarks of Apple Inc., registered in the U.S. and other countries.

Broadcom is a trademark of Broadcom Corporation.

Chelsio® is a registered trademark of Chelsio Communications.

Cisco® is a registered trademark or trademark of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.

Django® is a registered trademark of Django Software Foundation.

Facebook® is a registered trademark of Facebook Inc.

FreeBSD® and the FreeBSD® logo are registered trademarks of the FreeBSD Foundation®.

Intel, the Intel logo, Pentium Inside, and Pentium are trademarks of Intel Corporation in the U.S. and/or other countries.

LinkedIn® is a registered trademark of LinkedIn Corporation.

Linux® is a registered trademark of Linus Torvalds.

Oracle is a registered trademark of Oracle Corporation and/or its affiliates.

Twitter is a trademark of Twitter, Inc. in the United States and other countries.

UNIX® is a registered trademark of The Open Group.

VirtualBox® is a registered trademark of Oracle.

VMware® is a registered trademark of VMware, Inc.

Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.

Windows® is a registered trademark of Microsoft Corporation in the United States and other countries.

Typographic Conventions

The FreeNAS® 11.1 User Guide uses these typographic conventions:

Table 1 Text Format Examples
Item Visual Example
Graphical elements: buttons, icons, fields, columns, and boxes Click the Import CA button.
Menu selections Select System ‣ Information.
Commands Use the scp command.
File names and volume and dataset names Locate the /etc/rc.conf file.
Keyboard keys Press the Enter key.
Important points This is important.
Values entered into fields, or device names Enter 127.0.0.1 in the address field.

1. Introduction

FreeNAS® is an embedded open source network-attached storage (NAS) operating system based on FreeBSD and released under a 2-clause BSD license. A NAS has an operating system optimized for file storage and sharing.

FreeNAS® provides a browser-based, graphical configuration interface. The built-in networking protocols provide storage access to multiple operating systems. A plugin system is provided for extending the built-in features by installing additional software.

1.1. New Features in 11.1

FreeNAS® 11.1 is a feature release, which includes several new significant features, many improvements and bug fixes to existing features, and version updates to the operating system, base applications, and drivers. Users are encouraged to Update to this release in order to take advantage of these improvements and bug fixes.

These base applications and drivers have been updated or added:

  • The base operating system has been updated to FreeBSD 11.1-STABLE. This brings in many new features and drivers. Improvements have been made to the em(4), ixl(4), ixgbe(4), and mps(4) drivers. Additionally, the netmap(4) kernel module has been added to the build as some NIC drivers depend upon it.
  • There have been many improvements to OpenZFS, including a significant speed improvement when listing a large number of snapshots and deleting multiple snapshots or large files.
  • The algorithm used for scrubs and resilvers has received many improvements which are most noticeable on fragmented pools.
  • Samba 4.7 has been patched to address these security vulnerabilities.
  • The Dojo Toolkit has been updated to version 1.12.2.
  • OpenVPN has been updated to version 2.4.3.
  • Iperf version 3.2 has been added. To use this version, specify iperf3 instead of iperf.
  • Iocage has been updated to version 0.9.10.
  • The new middleware now uses Python asyncio which simplifies asynchronous code and makes it more readable.
  • The SNMP MIB has many improvements, including the ability to send SNMP traps for new alerts.
  • The system now sends an email when a scrub finishes.
  • mmv has been added. It can be used from the command line to safely move or copy multiple files using patterns, without any unexpected deletion of files due to target name collisions.
  • s3cmd has been added back as a CLI alternative to S3.
  • The zfs-stats CLI utility has been added. Type zfs-stats to see command usage.
  • The hardware watchdog has been reenabled for recent firmware versions of AsrockRack C2750D4I. The BMC bug which required the watchdog to be disabled is resolved with the 00.35.00 or newer BMC firmware version.
  • The system issues an alert if the system reboots itself.

These major features are new in this version:

  • Scrubs can be paused and resumed from the command line. Scrub pause state and progress are periodically synced to disk. If the system is restarted or the pool is exported during a paused scrub, the scrub remains paused until it is resumed. When resumed, the scrub picks up from the place where it was last checkpointed to disk. Paused scrubs can be resumed with zpool scrub. Scrubs can be paused manually with zpool scrub -p. A future version of FreeNAS® will add a button to the UI to resume or pause a scrub.
  • Cloud Credentials has been added to System. This can be used to provide a secure connection to a cloud service providers. Supported services include Amazon S3, Azure Blob Storage, Backblaze B2, and Google Cloud Storage.
  • Cloud Sync has been added to Tasks and can be used to synchronize files or directories to remote cloud storage providers with a specified transfer mode.
  • The Server Side Encryption drop-down menu appears on Tasks ‣ Cloud Sync ‣ Add Cloud Sync when an S3 provider is selected.
  • Resilver Priority has been added to Storage. This provides the ability to run resilvering at a higher priority at configurable times and days of the week.
  • The Netdata real-time performance and monitoring system has been added to Services.
  • Virtual Machines have received significant improvements, including:
    • support for non-US keyboards.
    • the ability to restart a VM and to clone a VM.
    • the ability to specify the NIC used by the VM as well as the MAC address for the VM NIC. These options can be set with VMs ‣ Devices ‣ Network Interface.
    • the ability to specify the sector size used by the emulated disk has been added to VMs ‣ Devices ‣ Disk.
    • the ability to edit the VNC screen resolution, select the IP address to bind to, set the VNC password, and select the option to use the Web version of VNC. These options can be set with VMs ‣ Devices ‣ VNC.

These screens have changed:

  • The Change E-mail button has been removed from Account ‣ Users.
  • Each device in a mirrored boot pool now displays a Detach button in System ‣ Boot ‣ Status. This can be used to remove a device from the boot pool.
  • The Enable Console Menu in System ‣ Advanced has been renamed to Show Text Console Without Password Prompt.
  • The Report CPU usage in percentage checkbox has been added to System ‣ Advanced.
  • The FreeNAS-11-Nightlies-SDK train has been added and the FreeNAS-9.3-STABLE train has been removed from System ‣ Update.
  • The Send Test Alert button has been added to System ‣ Alert Services ‣ Edit.
  • The Subject Alternate Names field has been added to System ‣ CAs ‣ Create Internal CA, System ‣ CAs ‣ Create Intermediate CA, System ‣ Certificates ‣ Create Internal Certificate, and System ‣ Certificates ‣ Create Certificate Signing Request screens.
  • The Sign CSR button has been added to System ‣ CAs.
  • The ability to edit an existing certificate’s Name, Certificate, and Private Key fields has been added to System ‣ Certificates ‣ View.
  • An Enabled checkbox has been added to Tasks ‣ Init/Shutdown Scripts.
  • The Additional domains field has been added to Network ‣ Global Configuration. This allows up to six additional DNS search domains with the caveat that adding more domains may negatively impact DNS lookup time.
  • The Identify Light button has been added to Network ‣ IPMI to make it easier to identify a system in a rack by flashing its IPMI LED light.
  • The Priority Code Point (CoS) field has been added to Network ‣ VLANs ‣ Add VLAN. This can be useful in datacenter environments to classify storage traffic on a given VLAN interface using IEEE 802.1p Class of Service (COS).
  • The Read-Only drop-down menu has been added to Storage ‣ Datasets ‣ Add Dataset ‣ Advanced Mode.
  • The Promote Dataset button has been added to Storage ‣ Volumes.
  • The Replication column has been removed from Storage ‣ Snapshots.
  • The Time Machine Quota checkbox has been added to Sharing ‣ Apple (AFP) Shares ‣ Add Apple (AFP) Share.
  • The Access Based Share Enumeration checkbox has been added to Sharing ‣ SMB (Windows) Shares ‣ Add SMB (Windows) Share.
  • The Home Share Time Machine checkbox has been added to Services ‣ AFP.
  • The CheckIP Server SSL, CheckIP Server, CheckIP Path, and Use SSL fields have been added to Services ‣ DDNS. The Forced update period and Auxiliary parameters fields have been removed. In addition, several dozen DDNS providers have been added to the Provider drop-down menu.
  • The Certificate drop-down menu has been added to Services ‣ S3 in order to configure encrypted S3 connections.
  • The Server minimum protocol and Server maximum protocol fields have been removed from Services ‣ SMB.
  • The Log Level drop-down menu has been added to Services ‣ SNMP. It defaults to the Error log level.
  • The No Communication Warning Time field has been added to Services ‣ UPS. This can be used to configure the frequency of email notifications during the loss of UPS communications.
  • The No Authentication choice has been added to the Services ‣ WebDAV ‣ HTTP Authentication drop-down menu.

1.2. Changes Since 11.1

FreeNAS® uses a “rolling release” model instead of point releases. The Update mechanism makes it easy to keep up-to-date with the latest security fixes, bug fixes, and new features. Some updates affect the user interface, so this section lists any functional changes that have occurred since 11.1 was released.

Note

The screenshots in this documentation assume that the system has been fully updated to the latest STABLE version of FreeNAS® 11.1-U7. If a screen on the system is not the same as shown in this guide, make sure that all updates have been applied.

1.2.1. U1

  • RancherOS has been updated to version 1.1.3.
  • Smartmontools has been updated to 6.6.
  • The Factory Restore button in System ‣ General has been renamed to Reset Configuration to Defaults.

1.2.2. U2

  • System Update has been removed from the Console Setup Menu.
  • System -> Information shows the system serial number on systems supplied or certified by iXsystems.
  • ZFS Sync write synchronization settings have been added to Create Dataset and Create zvol.
  • Midnight Commander has been updated to version 4.8.20.

1.2.3. U3

1.2.4. U5

  • Adds preliminary support for Self-Encrypting Drives.
  • The ATA Security User and SED Password fields have been added to System ‣ Advanced.
  • The Password for SED field has been added to Storage ‣ Volumes ‣ View Disks.

1.2.5. U6

  • The base operating system has been patched to address FreeBSD-SA-18:08.tcp and FreeBSD-SA-18:10.ip.
  • Samba 4.7 has been patched to address recent CVEs.
  • The em, igb, ixgbe, and ixl Intel drivers have been patched to resolve a performance degradation issue that occurs when the MTU is set to 9000 (9k jumbo clusters). Before configuring 9k jumbo clusters for cxgbe create a Tunables with a Variable of hw.cxgbe.largest_rx_cluster, a Type of Loader, and a Value of 4096. The cxgb driver does not support jumbo clusters and should not use an MTU greater than 4096.
  • The Endpoint URL has been added to System -> Cloud Credentials -> Add Cloud Credential but only appears when Amazon S3 is selected as the Provider. This can be used to configure a connection to another S3-compatible service, such as Wasabi.
  • The User Base and Group Base fields have been removed from Directory Services ‣ Active Directory ‣ Advanced Mode.

1.2.6. U6.3

1.2.7. U7

  • NFS has been patched to address FreeBSD-SA-18:13.nfs.
  • bhyve has been patched to address CVE-2018-17160.
  • Samba 4.7 has been patched to address recent CVEs.
  • The ixnas VFS module has been added to and the aio_pthread VFS module has been removed from Sharing ‣ Windows (SMB) ‣ VFS Objects.
  • The Enable SMB1 support checkbox has been added to Services ‣ SMB. However, users are cautioned to not use SMB1.

1.3. Path and Name Lengths

Names of files, directories, and devices are subject to some limits imposed by the FreeBSD operating system. The limits shown here are for names using plain-text characters that each occupy one byte of space. Some UTF-8 characters take more than a single byte of space, and using those characters reduces these limits proportionally. System overhead can also reduce the length of these limits by one or more bytes.

Table 1.3.1 Path and Name Lengths
Type Maximum Length Description
File Paths 1024 bytes

Total file path length (PATH_MAX). The full path includes directory separator slash characters, subdirectory names, and the name of the file itself. For example, the path /mnt/tank/mydataset/mydirectory/myfile.txt is 42 bytes long.

Using very long file or directory names can be problematic. A complete path with long directory and file names can exceed the 1024-byte limit, preventing direct access to that file until the directory names or filename are shortened or the file is moved into a directory with a shorter total path length.

File and Directory Names 255 bytes Individual directory or file name length (NAME_MAX).
Mounted Filesystem Paths 88 bytes Mounted filesystem path length (MNAMELEN). Longer paths can prevent a device from being mounted or data from being accessible.
Device Filesystem Paths 63 bytes devfs(8) device path lengths (SPECNAMELEN). Longer paths can prevent a device from being created.

Note

88 bytes is equal to 88 ASCII characters. The number of characters will vary when using Unicode.

Warning

If the mounted path length for a snapshot exceeds 88 bytes the data in the snapshot will be safe but inaccessible. When the mounted path length of the snapshot is less than the 88 byte limit, the data will be accessible again.

The 88 byte limit affects automatic and manual snapshot mounts in slightly different ways:

  • Automatic mount: ZFS temporarily mounts a snapshot whenever a user attempts to view or search the files within the snapshot. The mountpoint used will be in the hidden directory .zfs/snapshot/name within the same ZFS dataset. For example, the snapshot mypool/dataset/snap1@snap2 is mounted at /mnt/mypool/dataset/.zfs/snapshot/snap2/. If the length of this path exceeds 88 bytes the snapshot will not be automatically mounted by ZFS and the snapshot contents will not be visible or searchable. This can be resolved by renaming the ZFS pool or dataset containing the snapshot to shorter names (mypool or dataset), or by shortening the second part of the snapshot name (snap2), so that the total mounted path length does not exceed 88 bytes. ZFS will automatically perform any necessary unmount or remount of the file system as part of the rename operation. After renaming, the snapshot data will be visible and searchable again.
  • Manual mount: If the same example snapshot is mounted manually from the CLI, using mount -t zfs mypool/dataset/snap1@snap2 /mnt/mymountpoint the path /mnt/mountpoint/ must not exceed 88 bytes, but the length of the snapshot name will be irrelevant. When renaming a manual mountpoint, any object mounted on the mountpoint must be manually unmounted (using the umount command in the CLI) before renaming the mountpoint and can be remounted afterwards.

Note

A snapshot that cannot be mounted automatically by ZFS, can still be mounted manually from the CLI using a shorter mountpoint path. This makes it possible to mount and access snapshots that cannot be accessed automatically in other ways, such as from the GUI or from features such as “File History” or “Versions”.

1.4. Hardware Recommendations

FreeNAS® 11.1 is based on FreeBSD 11.1 and supports the same hardware found in the FreeBSD Hardware Compatibility List. Supported processors are listed in section 2.1 amd64. FreeNAS® is only available for 64-bit processors. This architecture is called amd64 by AMD and Intel 64 by Intel.

Note

FreeNAS® boots from a GPT partition. This means that the system BIOS must be able to boot using either the legacy BIOS firmware interface or EFI.

Actual hardware requirements vary depending on the usage of the FreeNAS® system. This section provides some starter guidelines. The FreeNAS® Hardware Forum has performance tips from FreeNAS® users and is a place to post questions regarding the hardware best suited to meet specific requirements. Hardware Recommendations gives detailed recommendations for system components, with the FreeNAS® Quick Hardware Guide providing short lists of components for various configurations. Building, Burn-In, and Testing your FreeNAS® system has detailed instructions on testing new hardware.

1.4.1. RAM

The best way to get the most out of a FreeNAS® system is to install as much RAM as possible. More RAM allows ZFS to provide better performance. The FreeNAS® Forums provide anecdotal evidence from users on how much performance can be gained by adding more RAM.

General guidelines for RAM:

  • A minimum of 8 GB of RAM is required.

    Additional features require additional RAM, and large amounts of storage require more RAM for cache. An old, somewhat overstated guideline is 1 GB of RAM per terabyte of disk capacity.

  • To use Active Directory with many users, add an additional 2 GB of RAM for the winbind internal cache.

  • For iSCSI, install at least 16 GB of RAM if performance is not critical, or at least 32 GB of RAM if good performance is a requirement.

  • Jails are very memory-efficient, but can still use memory that would otherwise be available for ZFS. If the system will be running many jails, or a few resource-intensive jails, adding 1 to 4 additional gigabytes of RAM can be helpful. This memory is shared by the host and will be used for ZFS when not being used by jails.

  • Virtual Machines require additional RAM beyond any amounts listed here. Memory used by virtual machines is not available to the host while the VM is running, and is not included in the amounts described above. For example, a system that will be running two VMs that each need 1 GB of RAM requires an additional 2 GB of RAM.

  • When installing FreeNAS® on a headless system, disable the shared memory settings for the video card in the BIOS.

  • For ZFS deduplication, ensure the system has at least 5 GB of RAM per terabyte of storage to be deduplicated.

If the hardware supports it, install ECC RAM. While more expensive, ECC RAM is highly recommended as it prevents in-flight corruption of data before the error-correcting properties of ZFS come into play, thus providing consistency for the checksumming and parity calculations performed by ZFS. If your data is important, use ECC RAM. This Case Study describes the risks associated with memory corruption.

Do not use FreeNAS® to store data without at least 8 GB of RAM. Many users expect FreeNAS® to function with less memory, just at reduced performance. The bottom line is that these minimums are based on feedback from many users. Requests for help in the forums or IRC are sometimes ignored when the installed system does not have at least 8 GB of RAM because of the abundance of information that FreeNAS® may not behave properly with less memory.

1.4.2. The Operating System Device

The FreeNAS® operating system is installed to at least one device that is separate from the storage disks. The device can be a SSD, USB memory stick, or DOM (Disk on Module). Installation to a hard drive is discouraged as that drive is then not available for data storage.

Note

To write the installation file to a USB stick, two USB ports are needed, each with an inserted USB device. One USB stick contains the installer, while the other USB stick is the destination for the FreeNAS® installation. Be careful to select the correct USB device for the FreeNAS® installation. FreeNAS® cannot be installed onto the same device that contains the installer. After installation, remove the installer USB stick. It might also be necessary to adjust the BIOS configuration to boot from the new FreeNAS® boot device.

When determining the type and size of the target device where FreeNAS® is to be installed, keep these points in mind:

  • The absolute bare minimum size is 8 GB. That does not provide much room. The recommended minimum is 16 GB. This provides room for the operating system and several boot environments created by updates. More space provides room for more boot environments and 32 GB or more is preferred.

  • SSDs (Solid State Disks) are fast and reliable, and make very good FreeNAS® operating system devices. Their one disadvantage is that they require a disk connection which might be needed for storage disks.

    Even a relatively large SSD (120 or 128 GB) is useful as a boot device. While it might appear that the unused space is wasted, that space is instead used internally by the SSD for wear leveling. This makes the SSD last longer and provides greater reliability.

  • When planning to add your own boot environments, budget about 1 GB of storage per boot environment. Consider deleting older boot environments after making sure they are no longer needed. Boot environments can be created and deleted using System ‣ Boot.

  • Use quality, name-brand USB sticks, as ZFS will quickly reveal errors on cheap, poorly-made sticks.

  • For a more reliable boot disk, use two identical devices and select them both during the installation. This will create a mirrored boot device.

Note

Current versions of FreeNAS® run directly from the operating system device. Early versions of FreeNAS® ran from RAM, but that has not been the case for years.

1.4.3. Storage Disks and Controllers

The Disk section of the FreeBSD Hardware List lists the supported disk controllers. In addition, support for 3ware 6 Gbps RAID controllers has been added along with the CLI utility tw_cli for managing 3ware RAID controllers.

FreeNAS® supports hot pluggable drives. Using this feature requires enabling AHCI in the BIOS.

Reliable disk alerting and immediate reporting of a failed drive can be obtained by using an HBA such as an Broadcom MegaRAID controller or a 3Ware twa-compatible controller.

Note

Upgrading the firmware of Broadcom SAS HBAs to the latest version is recommended.

Some Highpoint RAID controllers do not support pass-through of S.M.A.R.T. data or other disk information, potentially including disk serial numbers. It is best to use a different disk controller with FreeNAS®.

Note

The system is configured to prefer the mrsas(4) driver for controller cards like the Dell PERC H330 and H730 which are supported by several drivers. Although not recommended, the mfi(4) driver can be used instead by removing the loader Tunable: hw.mfi.mrsas_enable or setting the Value to 0.

Suggestions for testing disks before adding them to a RAID array can be found in this forum post. Additionally, badblocks is installed with FreeNAS® for testing disks.

If the budget allows optimization of the disk subsystem, consider the read/write needs and RAID requirements:

  • For steady, non-contiguous writes, use disks with low seek times. Examples are 10K or 15K SAS drives which cost about $1/GB. An example configuration would be six 600 GB 15K SAS drives in a RAID 10 which would yield 1.8 TB of usable space, or eight 600 GB 15K SAS drives in a RAID 10 which would yield 2.4 TB of usable space.

For ZFS, Disk Space Requirements for ZFS Storage Pools recommends a minimum of 16 GB of disk space. Due to the way that ZFS creates swap, it is not possible to format less than 3 GB of space with ZFS. However, on a drive that is below the minimum recommended size, a fair amount of storage space is lost to swap: for example, on a 4 GB drive, 2 GB will be reserved for swap.

Users new to ZFS who are purchasing hardware should read through ZFS Storage Pools Recommendations first.

ZFS vdevs, groups of disks that act like a single device, can be created using disks of different sizes. However, the capacity available on each disk is limited to the same capacity as the smallest disk in the group. For example, a vdev with one 2 TB and two 4 TB disks will only be able to use 2 TB of space on each disk. In general, use disks that are the same size for the best space usage and performance.

The ZFS Drive Size and Cost Comparison spreadsheet is available to compare usable space provided by different quantities and sizes of disks.

1.4.4. Network Interfaces

The Ethernet section of the FreeBSD Hardware Notes indicates which interfaces are supported by each driver. While many interfaces are supported, FreeNAS® users have seen the best performance from Intel and Chelsio interfaces, so consider these brands when purchasing a new NIC. Realtek cards often perform poorly under CPU load as interfaces with these chipsets do not provide their own processors.

At a minimum, a GigE interface is recommended. While GigE interfaces and switches are affordable for home use, modern disks can easily saturate their 110 MB/s throughput. For higher network throughput, multiple GigE cards can be bonded together using the LACP type of Link Aggregations. The Ethernet switch must support LACP, which means a more expensive managed switch is required.

When network performance is a requirement and there is some money to spend, use 10 GigE interfaces and a managed switch. Managed switches with support for LACP and jumbo frames are preferred, as both can be used to increase network throughput. Refer to the 10 Gig Networking Primer for more information.

Note

At present, these are not supported: InfiniBand, FibreChannel over Ethernet, or wireless interfaces.

Both hardware and the type of shares can affect network performance. On the same hardware, SMB is slower than FTP or NFS because Samba is single-threaded. So a fast CPU can help with SMB performance.

Wake on LAN (WOL) support depends on the FreeBSD driver for the interface. If the driver supports WOL, it can be enabled using ifconfig(8). To determine if WOL is supported on a particular interface, use the interface name with the following command. In this example, the capabilities line indicates that WOL is supported for the re0 interface:

ifconfig -m re0
re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=42098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO>
        capabilities=5399b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_UCAST,WOL_MCAST, WOL_MAGIC,VLAN_HWFILTER,VLAN_H WTSO>

If WOL support is shown but not working for a particular interface, create a bug report using the instructions in Support.

1.5. Getting Started with ZFS

Readers new to ZFS should take a moment to read the ZFS Primer.