The goal of this article is to introduce you to the products, technologies and protocols which will be used in setting up an office server. The article will describe them in detail in order to clarify their interaction with the server and with the Internet at large.
The FreeBSD operating system is the layer of software which exists between applications and the hardware. Each low-level operation issued by the application (such as reading or writing a file) will be transformed by the operating system’s kernel into a series of instructions to the hardware (such as finding the correct place on the hard drive to read or write and programming the correct hardware controllers to perform the operation).
There are many features of a modern computer environment which users take for granted, but which are only possible because of advanced algorithms implemented in the operating system’s kernel, such as: multitasking and switching between applications, security isolation between users and between applications, network communication between distant systems, memory management and swapping.
As traditional for modern BSD operating systems, FreeBSD is divided into at least three distinct parts: the kernel, the userland utilities, and the third party application collection. The kernel is the really low-level interface to the hardware. The only way an application can run is by being started by the kernel and by using kernel services to do all of the non-trivial tasks, like communicating with the user (either graphically or textually), reading or writing files, or using the network.
New applications are started when existing applications request that the kernel load and execute specific files, and the kernel cleans up the resources used by the application after it is done executing.
UserLand
The term “userland” refers to all the applications running on top of a kernel. The userland utilities of the operating system are present for convenience to the system administrator and its users. They are in fact a common, more-or-less standardised way of configuring and managing a system. For example: though the kernel has a concept of different users which may or may not be forbidden to read each other’s files, there needs to be a set of applications which manage a list of these users.
The kernel knows how to read and write data to and from files, but it needs to be externally configured with which devices contain which file systems and where to mount these. As another example: the kernel knows how to send, receive and route network packets, but thereneeds to be a way to tell the kernel that certain network interfaces (or “network cards”) have certain IP addresses, what the default gateway is, and which networks, if any, are connected to this one.
All of these tasks, and plenty more, are performed by the operating systems’ userland utilities. The FreeBSD base system contains enough utilities to make it a complete application development environment for the operating system itself, including a compiler and a text editor.
There is a sharp distinction between program code that is a part of the kernel and that which is a part of a regular application (or “userland”). The userland code cannot access the hardware directly but must rely on the kernel to provide its services, but because of this the applications can focus on performing some high-level tasks useful to the users, and can grow quite large.
The kernel code exists mostly to provide services to userland applications so it must be fast and compact, but at the same time since it is “close” to the hardware it is bogged down by the fact that it needs to perform many very low-level operations by itself.
In contrast to some operating systems (most notably Linux), the BSD userland utilities are maintained by the same group of people and in the same project package as the kernel.
This makes the utilities leaner and allows deeper integration between the features provided by the kernel and the utilities. In FreeBSD (and most other BSDs), the collection of the userland utilities shipped with the operating system is sometimes called the “base system” or the “world”.
The collection of third party applications is what makes an operating system generally usable. While basic administration tools such as those to manage users, files, file systems, and the network are enough to have a running, working system, it is the third party applications which usually provide useful user-visible features to this system. Those can be web servers, databases, e-mail servers, desktop environments, games and multimedia applications.
The manual pages
FreeBSD comes with an extensive set of system manual pages. These cover all aspects of the system: the kernel and its facilities, the userland utilities and even the programming APIs and libraries from which the system is built.
These manual pages are grouped into sections identified by their number:
• General commands – such as ls, mkdir, top, ps, ssh, etc.
• System calls (the kernel API) – such as open(),
close(), read(), write(), socket(), etc.
• C library functions – such as strcpy(), err(), strtod(),
malloc(), pthread _ create(), etc.
• Device drivers and kernel interfaces – such as ahci,
ehci, vga, ipfw, tun, ipsec, etc.
• Configuration files and file formats – such as rc.conf,
jail.conf, crontab, fstab, etc.
• “Games” (non-essential software) – such as fortune,
morse, ppt, etc.
• “Miscellaneous information” (documentation) – such
as hier, ports, tuning, etc.
• System management utilities – such as adduser, fsck,
newfs, syslogd, gmirror, etc.
• Kernel internal API (“KPI”) – such as mbuf, signal,
wakeup, rtalloc, etc.
There is no logical relation between the number and the section content – the assignment was historical and evolutionary, not intelligently designed.
A manual page can be viewed in the terminal by issuing
a command such as:
> man pkg
The manual viewer (“man”) will find and display the named manual page from the lowest-numbered section which contains the page with the given name. In case there are manual pages with the same name in different sections (for example: ipfw is both a kernel interface and a userland utility to manage this kernel interface), you can force the choice of the section by adding it before
the page name:
> man 8 ipfw
In official FreeBSD documents, “man page” references are usually written with the page name followed by the section number in parenthesis, such as “ipfw(8)”.
Ports and packages
The traditional way third party applications are introduced into a BSD operating system is via a variation of a concept called the “ports tree.”
This is a structure of directories (“folders”) containing scripts and other utilities which compile and install specific pieces of software.
This was an efficient and agile method of making software available to the users because only this directory tree needed to be distributed – the actual application source code was automatically fetched just before it was compiled. It enabled quick adoption of new software and new versions of existing applications.
However, the principle of having to compile source code for every application which needs to be installed or updated is not very scalable at the present time where the applications required on single systems are more numerous and time is at a premium. Compiling large desktop software such as LibreOffice can take hours, or even days on low-end hardware. On the other hand, only by compiling software from scratch can the users have direct control over various optional features of such software.
Since FreeBSD 10, the preferred method of installing and maintaining software is with the “pkg” utility, which deals with binary (pre-compiled) software packages. The ports tree is still present and it is the source from which the binary packages are built, but installing software from both the ports tree and the binary packages can result in problems.
The package manager
The “pkg” package manager shares much of its design with similar software on Linux and other operating systems. It maintains databases of installed and available application packages, and enables the user to search, install, remove or query packages.
On the command line, pkg is invoked by passing a command and optional command arguments to the main program, for example:
> pkg search vim-lite # pkg install vim-lite > pkg info -f vim-lite # pkg remove vim-lite
A freshly installed FreeBSD system only comes with a small bootstrap program instead of the full package manager.
This program will install the package manager on first invocation of the pkg command.
Installing software
The first step for installing software usually involves finding it. The “search” argument to the “pkg” command followed by a string of letters will result in a print-out of all the packages available in the package repositories whose names contain the specified string. For example:
> pkg search dovecot
currently lists the following packages:
dovecot-1.2.17_4 dovecot-antispam-1.3_2,1 dovecot-managesieve-0.11.13_1 dovecot-pigeonhole-0.4.3_1 dovecot-sieve-1.2+0.1.19_1 dovecot2-2.2.13_3 dovecot2-antispam-plugin-20130429_7
There are often multiple versions of the same applications offered in the repository. It is up to you to select the required version. After choosing the correct software (for example, in this case, let’s say it’s dovecot2), the software can be installed by issuing a command such as:
> pkg install dovecot2
More than one name can be added after the “install” argument, and the package manager will find and install all the packages and their dependencies in the same operation.
De-installing (removing) software
Removing packages also usually starts with finding a package to remove. In this case we will utilize the “info” argument to the “pkg” command with the “-g” switch to find all packages matching a pattern in the “shell glob” format:
> pkg info -g ‘php5*’
Note: Don’t forget to add the single quotes (the apostrophes) around the shell glob expression, to prevent your current shell from expanding the glob expression itself.
The above command will list all packages beginning with ‘php5?. An example of its output is:
php5-5.4.23 php5-bz2-5.4.21 php5-calendar-5.4.21 php5-ctype-5.4.21 php5-curl-5.4.21_1 php5-dom-5.4.21 php5-exif-5.4.21 php5-filter-5.4.23 php5-ftp-5.4.21 php5-gd-5.4.21 php5-hash-5.4.21 php5-iconv-5.4.21 php5-json-5.4.21
The “delete” argument to “pkg” will remove the named package:
> pkg delete php5-ftp
The “pkg” command implements a large number of arguments dealing with specific situations, and its manual pages are fairly complete, so it can be informative to read through them.
Web servers and application servers
One of the first steps after installing FreeBSD is to install Apache and PHP. Apache is a web server, one of many existing today. It is a piece of software which implements the HTTP protocol which is used by web browsers to access web pages.
By default, software such as Apache only serves ordinary files from the file system, so called “static files”. These can include complex HTML pages but also images, CSS and JavaScript files, MP3 files, WebM videos, etc. In order to have a complex web application, an additional piece of software is needed: an application server. This is software which runs server-side code to e.g. process content before it is served to the web browser, handle user logins, or generate new, dynamic content.
One of the most popular application servers is PHP, which will also be used in this tutorial to run some specific services.
The web server and the application need to communicate in some way, and one of the standard ways to do this is via the FastCGI protocol.
This protocol enables the use of PHP as a proper application server in a separate process, which is beneficial from the aspects of performance and security. Our specific setup will use Apache version 2.4 and the mod_fcgid module to interface it with PHP.
An additional component now required in non-trivial web server setups: the SSL / TLS encryption layer (“SSL” is an obsolete name for the set of protocols now called “TLS”).
This is a protocol which lays below HTTP and ensures that all communication between the web browser and the web server is encrypted and secured. An important aspect of SSL is that it requires a type of server credentials: there needs to be a way for the server to introduce itself to the client (the browser) and for the client to trust this introduction.
This is accomplished by using so-called “SSL certificates” which use public-key cryptography to implement a trust model where a third party (the “certificate authority”) vouches that it has issued a certificate for a specific server and knows about it.
The browsers carry a list of trusted certificate authorities which they use to check certificates sent by the servers as an introduction. Modern web servers support SSL / TLS out of the box, but they need to be configured with the valid SSL certificate and associated data in order to function. The combination of SSL / TLS and the HTTP protocol is called HTTPS.
File sharing in the local network
A common goal when connecting computers directly in a network is to enable their users to share files. The file sharing service of networks is so important that there are a large number of different protocols and ways to accomplish this, but the most common ones today are SMB / CIFS and NFS.
The names “SMB” and “CIFS” usually refer to the same protocol, though technically CIFS is a subset of SMB. It is a protocol developed and updated by Microsoft which gained popularity together with its Windows operating system, and is now available in most other operating systems, including OS X and Linux. Its biggest advantage is that it integrates seamlessly with Windows workstations.
The Samba project delivers a robust implementation of a SMB / CIFS server which can even act as a Domain Controller or an Active Directory controller. Its disadvantage is that its security model and the way users and groups are handled is at odds with how they are handled in Unix-like operating systems, so there are limitations and compatibility tweaks needed to maintain appropriate levels of interoperability.
If the computers involved in file sharing are Unixlike, they can use the NFS protocol. It is a light-weight protocol developed specifically for Unix-like systems so the interoperability is much better.
It can be used natively with FreeBSD, OS X and Linux, but has a slight disadvantage in available security settings, since it is made for completely trusted networks where computers can be permitted to access each-other’s files based on nothing but having a certain IP address (no usernames or passwords are exchanged).
Despite this, it is very common and the de-facto standard for local file sharing on Unix-like systems.
The complexities of e-mail
E-mail is a hugely popular Internet service, but it is also one of its most complex. A part of the reason for this complexity is that it is one of the earliest services which has survived essentially intact in the modern era. The backbone of the e-mail service is the SMTP protocol, used simply to pass e-mail messages from server to server.
The protocol itself and the messages it carries contain next to no security provisions, making e-mail spam and forgery an everyday occurrence.
In addition to this, a completely different type of protocol, like IMAP, is required to retrieve the messages from the servers.
In order for e-mail to function, the SMTP server (which commonly contains the users’ mailboxes) should have a fixed IP address and an appropriate entry in the DNS system needs to be created to point to it.
Next, the SMTP server needs to be properly configured to only accept messages addressed to a certain domain, to be resistant to various attacks, and to perform at least some kind of spam checking of the messages before allowing them to be passed through to the users. A separate IMAP server needs to be configured to enable users to retrieve their messages from the server’s mailboxes.
All these protocols should be secured with SSL, similarly to the web servers.
Conclusions
Generally, we use Postfix 2.11 as the SMTP server and Dovecot 2.2 for the IMAP server. POP3 will not be configured, as it is an example of a very obsolete protocol. In addition to those, Spamassassin will be used to perform spam analysis and the RoundCube application will be used to provide web-based access to e-mail.
If you want to learn more, I invite you to my workshop on Deploying an office / workgroup server on FreeBSD.
IVAN VORAS
Ivan Voras is a FreeBSD developer and a long-time user, starting with FreeBSD 4.3 and throughout all the versions since. In real life he is a researcher, system administrator and a developer, as opportunity presents itself, with a wide range of experience from hardware hacking to cloud computing. He is currently employed at the University of Zagreb Faculty of Electrical Engineering and Computing and lives in Zagreb, Croatia. You can follow him on his blog in English at http://ivoras.net/ blog or in Croatian at http://hrblog.ivoras.net/, as well as on Google+ at https://plus.google.com/+IvanVoras.
