System drive for FreeNAS

[deed02392]

Hi guys,

I've read about people installing FreeNAS on SSDs, yet I've also read that apparently FreeNAS runs as a live image, creating two 1 GiB partitions with any remaining capacity unused. In which case installing on an SSD, still a relatively valuable storage medium, is pretty pointless (and I may as well use it for L2ARC instead).

I am wondering if you can't partition a drive yourself so that FreeNAS has its own partitions and the rest CAN be added to a storage array or used for ZIL etc, though perhaps the reason you can't do this is that you have to add devices rather than volumes to pools?

Or should I just use a spare 2 GiB USB stick I have?

Thanks a lot :)

George

 

[Joshua Parker Ruehlig]

Hi guys,

I've read about people installing FreeNAS on SSDs, yet I've also read that apparently FreeNAS runs as a live image, creating two 1 GiB partitions with any remaining capacity unused. In which case installing on an SSD, still a relatively valuable storage medium, is pretty pointless (and I may as well use it for L2ARC instead).

I am wondering if you can't partition a drive yourself so that FreeNAS has its own partitions and the rest CAN be added to a storage array or used for ZIL etc, though perhaps the reason you can't do this is that you have to add devices rather than volumes to pools?

Or should I just use a spare 2 GiB USB stick I have?

Thanks a lot :)

George

You should just use a spare USB stick. you can even just backup the config so it the USB stick dies, you just image another one, restore the config, and you have the same exact system you had before.

 

[deed02392]

Sounds good. Set the system up with the USB to boot and the SSD as the ZFS cache disk.

 

[toddos]

Fair warning -- a 2GB stick is not going to be big enough. You need a 4GB stick.

Blame manufacturers and their insistence on using base-10 numbering. FreeNAS requires a true base-2 2GiB of free space (2 * 2^30 = 2147483648 bytes), but most (all?) 2GB USB drives are base-10 2GB (2 * 10^9 = 2000000000 bytes = 1.86GiB). The solution is to go one step up and use a 4GB (4000000000 bytes = 3.73GiB) drive.

 

[ramius]

I keep reading everywhere that the system drive for Freenas should be a USB stick. I really don't like USB, maybe it's a working experience, but this type of drive breaks too often. Aside from the waste of space when using a harddrive,is there some other issue not to use a hard drive as system drive. I was searching in eBay and found some really cheap 2.5" SAS 10k 36Gb hard drives. Is there any issue in using this drive to install Freenas?

 

[pirateghost]

I keep reading everywhere that the system drive for Freenas should be a USB stick. I really don't like USB, maybe it's a working experience, but this type of drive breaks too often. Aside from the waste of space when using a harddrive,is there some other issue not to use a hard drive as system drive. I was searching in eBay and found some really cheap 2.5" SAS 10k 36Gb hard drives. Is there any issue in using this drive to install Freenas?

a SAS drive will require a SAS controller.

why do you think USB thumbdrive is bad for this implementation? its DESIGNED for such. this isnt like installing ubuntu on a usb drive and having it go tits up from lots of read/writes...

 

[Yell]

Use this drive for dedicated zil drive and run FreeNas from a USB tumbdrive.

 

[ramius]

why do you think USB thumbdrive is bad for this implementation? its DESIGNED for such. this isnt like installing ubuntu on a usb drive and having it go tits up from lots of read/writes...

Don't take it the wrong way, I like the implementation made in Freenas. It's, under my point of view, a very good way to protect the life os a USB thumbdrive by keeping the writes to a minimum. My issue is with the reliability of the thumbdrive itself. A cheap USB doesn't have any kind of protection agains voltage spikes or data corruption and they are also build with very sensible electronic components. At work I have a box full of brocken thumbdrives. Maybe it's just a phobia.

About the SAS controller, I found the drive very cheep at Ebay (<$15, shipping included), witch would have been very close to the price of a halfway reliable USB stick, and like lot of people i'am already using a SAS controller in the system.

My concern still is, is there any disadvantage or issue, aside from the lost space and the power consumption?

 

[ramius]

Use this drive for dedicated zil drive and run FreeNas from a USB tumbdrive.

From what I can understand reading the user manual, it's very risky to use a single ZIL drive. On the other side, this drives are very reliable, but they don't have a very high writing speed. I think the main drive pool is faster than a single 2.5" SAS drive and adding this drive as a ZIL drive would slow the whole system down.

 

[pirateghost]

Don't take it the wrong way, I like the implementation made in Freenas. It's, under my point of view, a very good way to protect the life os a USB thumbdrive by keeping the writes to a minimum. My issue is with the reliability of the thumbdrive itself. A cheap USB doesn't have any kind of protection agains voltage spikes or data corruption and they are also build with very sensible electronic components. At work I have a box full of brocken thumbdrives. Maybe it's just a phobia.

About the SAS controller, I found the drive very cheep at Ebay (<$15, shipping included), witch would have been very close to the price of a halfway reliable USB stick, and like lot of people i'am already using a SAS controller in the system.

My concern still is, is there any disadvantage or issue, aside from the lost space and the power consumption?

keep a couple of spare USB thumbdrives, make config backups regularly, and if anything bad happens you can be up and running in about 15 minutes...to me, the extra SAS port and slot in the server is more valuable than a couple of thumbdrives or worrying about their longevity. i have plenty of USB drives, and it only takes minutes to get my server back online in the event the usb drive dies

 

[Yell]

From what I can understand reading the user manual, it's very risky to use a single ZIL drive. On the other side, this drives are very reliable, but they don't have a very high writing speed. I think the main drive pool is faster than a single 2.5" SAS drive and adding this drive as a ZIL drive would slow the whole system down.

10k SAS > consumer drive HDDs
forgot to mention use ZFS version 22+ (?) to be able to remove the ZIL from the pool without datalose

 

[cyberjock]

You are correct Yell. But also keep in mind that without mirroring the ZIL, a failure of the ZIL drive will result in a loss of whatever data is in the ZIL and not committed to the zpool.

I also have mixed thoughts on a 10k SAS drive as a ZIL. While random seek time is lower for a 10k drive, my RAIDZ2 zpool can achieve more than 400MB/sec on dd tests. There isn't a 10k SAS drive in existence that can hit those speeds. ZIL benefits really are meant for seek times, and if you are REALLY wanting to see the maximum benefit of a ZIL, an SSD is the way to go. Also, an 8GB ZIL is pretty big for non enterprise functions.

 

[Yell]

You are correct Yell. But also keep in mind that without mirroring the ZIL, a failure of the ZIL drive will result in a loss of whatever data is in the ZIL and not committed to the zpool.

I also have mixed thoughts on a 10k SAS drive as a ZIL. While random seek time is lower for a 10k drive, my RAIDZ2 zpool can achieve more than 400MB/sec on dd tests. There isn't a 10k SAS drive in existence that can hit those speeds. ZIL benefits really are meant for seek times, and if you are REALLY wanting to see the maximum benefit of a ZIL, an SSD is the way to go. Also, an 8GB ZIL is pretty big for non enterprise functions.

http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide

In current releases, if an unmirrored log device fails during operation, the system reverts to the default behavior, using blocks from the main storage pool for the ZIL, just as if the log device had been gracefully removed via the "zpool remove" command.

(i think this is for pool at or above version 19-22)

Even if you raidz2 can archiv 400MB/s, a dedicated ZIL with high IOPS is a good idea. (since you disk heads keep seeking over the drive finding data AND ZIL blocks)

Try this:

1. Boot 3.1
2. create a empty pool / dataset
3. share with NFS (sync writes) and mount at a gigabit connected client
4. DD a large file (dd if=/dev/zero of=/mnt/NFS/somelargefile.bak count=2048k bs=10k)
5. temp disable ZIL (zfs set sync=disabled tank/dataset )
6. DD again
7. attach a single spare drive sata drive
8. DD again
9. compare

should look like this

1. no ZIL
2. Single satadrive
3. normal setup

as posted in a other thread, i'm sure you read it ;)

With disabled ZIL my write speed jumps from 59 MB/s to 107MB/s writing a 21 GB file (dd if=/dev/zero of=/mnt/NFS/somelargefile.bak count=2048k bs=10k)
That's nearly 100% increase (since i don't have a SSD to spare for ZIL)

(this is fine you dont mind http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide#Disabling_the_ZIL_.28Don.27t.29 and have no critical data)