FreeNAS: a Migration Story

}

April 26, 2013

 

In 2010 I was employed as system administrator, and one of my aims was to administer the file sharing service. After having tried a few different solutions, all based on Linux and well known protocols (CIFS and Netatalk), I decided to switch to FreeNAS. And while it was an old version, and therefore without all the today’s gadgets, the choice was the right one and even years after I left the company, the other admins are able to run the machine without any problems.

This article briefly summarizes the migration process and what advantages the usage of FreeNAS provided. This is not a technical article, it is just a “tale” of how I’ve managed the migration. Please note, that the way described here could not be the best one or the one, that applies best to other scenarios.

The Context

The company was running several Linux physical servers, and in particular one for the ERP, one for the file sharing, one for the e-mail, one for the external services (FTP, Web Server) and one as gateway/proxy/firewall. There were around 200 workstations, with the majority of them made of Microsoft Windows PCs and 45 Apple Macs (as graphical workstations). All the network was wired as Gigabit Ethernet and there were three access points used by the selling agents who, in turn, were using Microsoft Windows laptops.

While the majority of the clients were sharing small-to-medium files, the graphical workstations were sharing files ranging from 50 MB to 600 MB each. Moreover, the Microsoft Windows PCs were doing frequent accesses to a lot of documents, meaning that the users were usually opening a file several times a day to add/modify the content and save it back, while graphical workstations were doing low frequency accesses to a small subset of the shared files.

While it was true that the PCs were seldomly accessing graphical files, the graphical workstations had to access (often) file shared by PCs.

Finally, all file accesses had to be authorized and granted on a per-user and a per-office policy. Luckily, all the users and their roles were already enumerated via an OpenLDAP server (the machine running the ERP).

Due to the above requirements, I decided to implement a NAS solution that had to:

  • connect to the OpenLDAP server in order to authenticate the users;
  • provide a per-office share as user(s) workspace;;
  • provide a way to do automatic backups at the fastest speed as possible;
  • provide a way to specify exactly which user(s) can access which document(s);
  • be as much as reliable as possible;
  • had an easy way to add extra disk space on demand.

The motivation that pushed me to choose a NAS dedicated system were that as time passed by, the sharing service got more and more complex with the need to deal with different operating systems and versions, as well as different platforms, file formats, and so on. While Open Source software like Samba and AppleTalk served well the scope, I was also looking for a more integrated solution and, most notably, something easier to use, without having to give away flexibility and stability. The “ease to use” point was motivated by the fact that I needed to share the NAS configuration with other adminitration profiles, and while I felt (and still feel) comfortable with the command line, others do not.

Migrating to FreeNAS

The choice was, as readers can imagine, to adopt FreeNAS. The reason behind that was that the company had already migrated some of the servers from Linux to FreeBSD-based solutions: the first was, in fact, the gateway/firewall appliance that was running pfSense; then it came the external server that migrated to FreeBSD to provide both Web and FTP; then a few internal workstations (mainly by the IT employees). Therefore, choosing a FreeBSD system for the NAS seemed a natural step, and since I wanted to have something dedicated and integrated, the FreeNAS project seemed to me the right one.

Of course, the decision was lead to several factor related to FreeBSD and FreeNAS, and in particular:

  • a low startup cost;
  • the stability of the system;
  • the availability of ZFS as file system;
  • the compatibility with the main sharing technologies (CIFS, AppleTalk);
  • an easy to use interface for administration.

In the following all the above will be motivated.

A Low Startup Cost

Luckily, the company I was employed for, was already used to the Open Source solutions, and did not want to get locked into vendors’ products if possible. And luckily, FreeNAS is a solution that does not impose a vendor lock-in, is free (that is at zero cost) and does not require a lot of resources to run. This allowed me to implement a NAS solution exploiting good, but cheap, hardware and with a no-cost at all for the software.

The Stability of the System

One thing I noted while working with both Linux and FreeBSD machines was that the latters were running smoothly for a longer time. Usually, keeping up-to-date a FreeBSD machine resulted in a simpler task than keeping up-to-date a Linux one, and this was particularly true also because the company was running different Linux distributions, each one with its own update policies, release schedules, and internal mechanisms. In other words, the FreeBSD machines were all behaving coherently. Finally, FreeBSD had proven to run longer without hanging or needing reboots than Linux on the same (old) hardware. Of course, while this strictly depends on both the Linux distributions, versions, the hardware and the administrator(s) skills, the above was true in such context.

The Availability of ZFS as File System

Two features of ZFS were particularly attracting to me for this job: the availability of snapshotting and the integrated volume manager.

On the other Linux machines we had to play with partitions and mount point each time a new disk was added, and that was due to the error of not setting up a volume manager from the beginning. With ZFS the problem disappeared.

Moreover, the idea of snapshotting was appealing to me because it allowed me to implement an automated backup up system transparent to the user. In fact, it often happened that a user started modifying a large graphical file just to discover it has made a wrong job and has to start over. Forcing the users to learn a revision control system was too complicated, therefore I decided to made snapshots on a regular schedule just to help the users to recover “damaged” files. On the other hand, the users were informed that after a certain amount of time snapshots would have been made persistent, that means no history was more available.

Just for the sake of clarity, beside the snapshotting, there was a regular backup technique.

The Compatibility with the Main Sharing Technologies

Having a heterogeneous environment and having to grant simultaneous access to Unix/Linux machines, Apple OSX computers and Microsoft Windows PCs required to have deep support for several sharing protocols and technologies. While one of my aims was to reduce protocol overlapping choosing, if possible, a single one to ease of administration, the fact that FreeNAS does support CIFS, AppleTalk and NFS allowed me to be prepared to switch to the best protocol available depending on the other side machine.

An Easy to Use Administration UI

Usually Unix administrators are tied to the Command Line Interface (CLI), and to some extent they also believe that using a GUI is a childhood proof. However, not all administrators are Unix ones, and this was the case: the company had one Microsoft Windows administrator and an Apple (Junior) one, both with no or little Unix skills. I decided that apart from the need for them to acquire some Unix skills, they had to be enabled to participate in the NAS management to both reduce the my overhead and make a better usage of the resources (in this case, people) available.

Being FreeNAS shipped with an excellent Web UI, I was able to let my colleagues to take part in the configuration phase and to grow up to the point they were able to create and manage shares, crontab and backup scripts, and users’ privileges without any further help. In this way, I needed mainly for the harder tasks and low level configuration, while they were able to do day-to-day maintanance.

The Migration

There was around 1 TB of data to be migrated from one Linux server, and another 400 GB to be consolidated to the FreeNAS installation. In fact, while the main project was to move only the Linux file shared part (the former), reviewing the sharing status reveled that other stuff need to be placed on the FreeNAS system (like, files and folders user have shared among their workstations without informing us, like, the latter).

Being the total size of data not that huge, I was able to migrate everything within hours, and in particular, during night. However, I did not run a single migration, but proceeded with different steps.

In the beginning, I set up the FreeNAS machine using an USB stick to boot, to not waste even a single byte on the hard disks. After the initial required setup (network interfaces, admin password, etc.) I connected the FreeNAS to the OpenLDAP server to get out of the box all the accounts. And this was really simple thanks to the great tools FreeNAS provides.

After that, I decided to remotely mount the already available shares into the FreeNAS, and to “republish” them on the network, so that the FreeNAS was effectively doing a proxy for the shares. Thanks to the fact that the FreeNAS and the original server were accessing the same account database, I did not have any problem about the permissions. My choice at this point was to republish shares using only the CIFS protocol, in order to have a simpler situation than using a per-client protocol.

I ran few tests for a week to ensure that everything was working fine, even if performances were not good due to the fact that the FreeNAS machine was not servicing local data.

After that, I implemented a local-based backup, that was a set of scripts to do regular backups of the FreeNAS and its content over a part of the FreeNAS itself, as well as to the central backup machine. This was the only phase were I needed to get to the FreeNAS CLI, since we had to deal with shell script development. More in detail, I had to do some “tricks” to place scripts on the USB stick and to be able to install a few other tools out of the ports. However, as soon as the scripts were in place, I took back the Web UI to place them in the crontab table.

Having the FreeNAS machine acting as a proxy, and the backup ready to work, I did start the migration of the clients so that they were redirected to the FreeNAS server instead of the original Linux NAS one.

I deployed a ZFS file system with roughly one per office/user file system, setting up quotas and assigning ownership and permissions.

Finally, I migrated all the data, using rsync on a regular base for a couple of pass, so to reduce the total copy overload. The migration was performed during a weekend to reduce the problems with live data and to avoid having to disconnect the users.

So Far, So Good!

While the above migration could seem even too long for such a mid amount of data, I preferred to do it in that way because it was my first experience with FreeNAS in an enterprise environment. It is worth saying I did not have any particular problem to complete the migration, and the system was very stable and rock-solid.

During the time I added a few extra features to our FreeNAS machine, like the aggregation of the two on-board network cards, as well as regular updates of the whole system, without incorring in any downtime.

The last time I had the opportunity to check the FreeNAS machine it was running from a year without any particular problem, and remember I’m talking about a machine tused in day-by-day work but that us, the administrators, were forgetting to have!

The only problem I ran into was tied to the USB stick I used as root filesystem: one day I was physically moving the machine from one place to another just to discover that the USB stick was unable to boot the machine anymore. It was, however, a ‘no pain situation’ since I had another ‘clone’ USB stick from which I started the machine.

Conclusions

My experience with FreeNAS is nothing but good on every aspect. It is worth noting that the effort done by the FreeNAS team to provide a clean, usable and easy to understand Web UI is really important in my opinion: while “real” Unix system administrator will always be able to fire up a console and see what it is happening under the hood, having a good UI allows other people to jump into the admin side. Of course this does not mean that everyone is automatically skilled as an administrator, but means that the admin burden can be scattered among different people with different skills. This was my experience indeed: thanks to the UI other not-Unix administrators were able to be tought quickly and perform many routinely tasks on the FreeNAS configuration.

Today, even years after I left the company, the other admins are still able to keep the FreeNAS box up-to-date and to run it without any particular problem, and this is the proof of the stability of the system itself.

Physical or Virtual?

In the last years the virtualization has been strongly advocated, and today is quite common to find whole servers virtualized onto a single hardware machine. It may sound strange that the described FreeNAS server was implemented as physical instead of virtual, but the reason for that was to keep the total cost as low as possible. As described in the “A Low Startup Cost” section, the company was already in the mood of the Open Source and commodity hardware mindset, and therefore instead of investing money in a huge super cluster, the company preferred to buy and assemble “small” and cheap servers. This also granted the company to implement replication solutions quite easily.

About the author

Luca Ferrari lives in Italy with his wife and son. He is an Adjunct Professor at Nipissing University, Canada, a co-founder and the vice-president of the Italian PostgreSQL Users’ Group (ITPUG). He simply loves the Open Source culture and refuses to log-in to non-Unix systems. He can be reached on line at http://fluca1978.blogspot.com

Share On Social: