How to upgrade the ports tree (including nut) and customize FreeNAS 8.0.2!

Status
Not open for further replies.

bryanburke

Dabbler
Joined
Nov 11, 2011
Messages
11
THIS HOW-TO IS NOW OUTDATED. PLEASE USE THE 8.0.3 VERSION

IMPORTANT: Please read the guide completely to avoid confusion and prevent errors!

Good day, everyone! I've been poking around with the FreeNAS 8.0.2 source trying to upgrade nut for use with my newer model UPS. Along the way, I found out how to upgrade the ports tree (almost) as a whole and even add ports like transmission-daemon to the FreeNAS image. This guide requires the ability to build FreeNAS 8.0.2 from source using the FreeBSD command-line; if you have no idea how to do this, consult joeschmuck‎'s excellent guide in this same forum.

Note: This modification to the FreeNAS 8.0.2 source is NOT sponsored or supported by the development team. Do NOT send them any questions about this guide!

Note: I am NOT responsible for any damage that may occur to your data as a result of this method. If you are concerned about the safety of the following instructions, MAKE A BACKUP FIRST!

Note: I can only certify that this method works with the 8.0.2 source. Don't bug me with questions about older versions or trunk!

Note: Read all notes...

Step 1: Get the Sources

If you already have the 8.0.2 sources, skip to step 2.

We use svn just like we would for trunk, except we are pulling the 8.0.2 branch instead:

Code:
svn co https://freenas.svn.sourceforge.net/svnroot/freenas/branches/8.0.2


The new folder created on your local machine (or virtual machine) is called "8.0.2" and is the root directory during the FreeNAS build. From here on I will simply call it FREENAS_ROOT.

Step 2: Patch the Build Script

Here is my unified diff which you can use to patch the FreeNAS build script:

Code:
--- build/do_build.sh	2011-11-10 12:55:31.000000000 -0600
+++ build/do_build.sh	2011-11-11 17:47:53.000000000 -0600
@@ -26,6 +26,8 @@
         exit 1
     fi
     echo "Checking out tree from ${FREEBSD_CVSUP_HOST}..."
+
+# Fetch CURRENT ports tree here
     cat <<EOF > FreeBSD/supfile
 *default host=${FREEBSD_CVSUP_HOST}
 *default base=${root}/FreeBSD/sup
@@ -34,7 +36,7 @@
 *default delete use-rel-suffix
 
 src-all tag=RELENG_8_2
-ports-all date=2011.07.17.00.00.00
+ports-all date=.
 EOF
     csup -L 1 ${root}/FreeBSD/supfile
 # cvsup fixes any changes we make, it seems.  Repatch
@@ -42,6 +44,42 @@
     rm -f ${root}/FreeBSD/ports-patches
 fi
 
+# Added from trunk build script to prevent build errors
+        # Nuke the newly created files to avoid build errors, as
+        # patch(1) will automatically append to the previously
+        # non-existent file.
+        for file in $(find ${root}/FreeBSD/ -name '*.orig' -size 0); do
+                rm -f "$(echo "$file" | sed -e 's/.orig//')"
+        done
+
+# Create a new supfile to backport only the ports we need to patch
+    cat <<EOF > FreeBSD/supfile_backports
+*default host=${FREEBSD_CVSUP_HOST}
+*default base=${root}/FreeBSD/sup
+*default prefix=${root}/FreeBSD
+*default release=cvs
+*default delete use-rel-suffix
+
+ports-all date=2011.07.17.00.00.00
+EOF
+
+# Create a file containing the categories and names of all
+# ports that will be patched (and therefore must be backported)
+    cat <<EOF > ${root}/ports.topatch
+sysutils/ataidle
+dns/inadyn
+net/istgt
+net/netatalk
+www/py-django
+www/py-dojango
+net/samba35
+EOF
+
+# Now backport the ports listed above
+for port in $(cd ${root} && cat ports.topatch); do
+    csup -L 1 -i "ports/$port" ${root}/FreeBSD/supfile_backports
+done
+
 # Make sure that all the patches are applied
 touch ${root}/FreeBSD/src-patches
 for i in $(cd ${root}/patches && echo freebsd-*.patch); do
@@ -64,6 +102,18 @@
     exit
 fi
 
+# Added from trunk build script to prevent build errors
+# Probably not even necessary, just a precaution
+#
+# HACK: chmod +x the script because:
+# 1. It's not in FreeBSD proper, so it will always be touched.
+# 2. The mode is 0644 by default, and using a pattern like ${SHELL}
+#    in the Makefile snippet won't work with csh users because the
+#    script uses /bin/sh constructs.
+if [ -f "${root}/FreeBSD/src/include/mk-osreldate.sh.orig" ]; then
+        chmod +x ${root}/FreeBSD/src/include/mk-osreldate.sh
+fi
+
 # OK, now we can build
 cd FreeBSD/src
 args="-c ../../nanobsd/freenas-common"


Copy that code and place it in a file called "do_build.sh.diff" in FREENAS_ROOT. Then run the patch command to apply it:

Code:
patch < do_build.sh.diff


Note: The diff file MUST be in FREENAS_ROOT for the above command to work as it is.

So, what does this patch actually do? Well, the original build script downloads the ports tree as it existed on July 17, 2011. The development team froze FreeNAS 8's ports tree on that date to apply custom patches to several ports. On the downside, though, important ports like nut are frozen at older versions. So, to update the ports tree while still allowing the FreeNAS patches to be applied, the above patch to the build script does the following:

1) Download a current, up-to-date ports tree initially.
2) Regress only the few ports that have custom patches for FreeNAS back to their July 17, 2011 snapshots.
3) Apply a couple precautionary measures taken from the build script in trunk.

That's really all there is to it!

Step 3: Customize FreeNAS with More Ports (Optional)

Use your favorite command-line text editor (I use nano) and open up the file "freenas-common" under the "nanobsd" directory.

Scroll down until you see a bunch of lines that start with "add_port" and go to the bottom of the list, which ends in "add_port benchmarks/iperf".

Now simply use the same syntax to add additional ports with the formula:

Code:
add_port <category>/<port_name>


For example, to add transmission-daemon, the code would be like this:

Code:
add_port net-p2p/transmission-daemon


Step 4: Build FreeNAS

If you have previously built FreeNAS, you might have the "FreeBSD" directory in FREENAS_ROOT. If you do, type the following commands:

Code:
cd FreeBSD/
rm -rf supfile sup/ src-patches ports-patches
cd ../


Those commands will ensure that the patched build script updates everything.

Now from FREENAS_ROOT run the following commands (yes, dropping into sh first can prevent build errors):

Code:
sh
export FREEBSD_CVSUP_HOST=cvsup10.freebsd.org
build/do_build.sh


You can, of course substitute any CVSUP host for the one above.

When the build finishes, you should have your custom images waiting for you in the "obj.***" directory, where the *** is either i386 or amd64.

If you don't know how to install the custom image, refer to the official FreeNAS documentation.

Step 5: Make Your Changes Persist Between Reboots (Optional)

Once your FreeNAS installation has booted and you have configured it to your liking (or restored your previous configuration), you can modify "/etc/rc.conf" to start ports you added in step 3 at boot (like transmission-daemon) and pass arguments to their RC scripts (such as configuration directories, etc.). To do this, use the following commands from the FreeNAS command-line (must be logged in as root):

Code:
mount -uw /
cd /conf/base/etc/
nano -w rc.conf


Make your changes to the file as required. For example, I have the following inserted for transmission-daemon:

Code:
transmission_enable="YES"
transmission_conf_dir="/mnt/zfs_volume/config/transmission"
transmission_user="root"


Refer to each port's documentation for applicable RC options.

Now reboot your FreeNAS installation and your changes will be made!

--------------------------------------------

Thanks for reading! Let me know if there are any mistakes or if anything is missing.
 

ProtoSD

MVP
Joined
Jul 1, 2011
Messages
3,348
Bryan, I don't know if you realized it, but @gcooper, one of the developers has already included a version of transmission in a special build of 8.02. There's a thread about how to enable it here somewhere.

Otherwise, thanks for the nice HowTo and welcome to the FreeNAS 8 forums!
 

bryanburke

Dabbler
Joined
Nov 11, 2011
Messages
11
Thank you for all the kind words. I'm just glad I could contribute something to the FreeNAS community. I've been utterly fascinated with FreeBSD since I made the switch to it from Linux. I love how polished and well-tested (and well-documented) it is, and FreeNAS is of course no exception. I'm really shocked as well that my patch to upgrade the ports tree didn't break anything (or at least anything I've tested).

I also was not aware of the build of 8.0.2 with transmission included. But I suppose anyway that the "customization" part of my guide will become moot when the developers get the PBI package system up and running.
 

ProtoSD

MVP
Joined
Jul 1, 2011
Messages
3,348
Bryan,

I wondered if you could clarify this:

Note: The diff file MUST be in ROOT for the above command to work as it is.

Do you mean, the root of the source tree (svnroot), "/", or the root user's home folder?
 

ProtoSD

MVP
Joined
Jul 1, 2011
Messages
3,348
Sorry, my bad, just noticed this:

The new folder created on your local machine (or virtual machine) is called "8.0.2" and is the root directory during the FreeNAS build. From here on I will simply call it ROOT.
 

bryanburke

Dabbler
Joined
Nov 11, 2011
Messages
11
You're right. ROOT is quite ambiguous given its many uses under Unix. I've edited the guide for clarification. Thanks for pointing that out to me.
 

ProtoSD

MVP
Joined
Jul 1, 2011
Messages
3,348
Now from FREENAS_ROOT run the following commands (yes, dropping into sh first can prevent build errors):

Discovered that even dropping into sh from the command line when your login shell is something different, your environment variable SHELL will still show your login shell, which seems to be causing build problems that I didn't have before. I changed my login shell and logged in again, now I am waiting to see what happens... weird. Just thought I'd share in case anyone else was having trouble. I'm not sure changing my login shell will help, but its better than waiting for another build to fail.
 

bryanburke

Dabbler
Joined
Nov 11, 2011
Messages
11
I also got shell-related build errors with the vanilla 8.0.2 source, without even applying my patch. They seem to have fixed those errors in trunk, which is why I included the two precautionary methods from the trunk build script. However, I do not know if those methods actually do anything in the 8.0.2 source tree. Dropping into sh actually solved the problems for me, both with and without the ports tree patch. Out of curiosity, what is your login shell? My login shell is csh. Also, what's the output of uname -a for you?
 

ProtoSD

MVP
Joined
Jul 1, 2011
Messages
3,348
Out of curiosity, what is your login shell? My login shell is csh. Also, what's the output of uname -a for you?

My login shell also WAS csh, I changed it to sh so the environment variable was set right.

Here's my uname -a

FreeBSD freebsd64 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Thu Feb 17 02:41:51 UTC 2011 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64

I'm using the latest build of 8.02 RELEASE (branch), but with buildworld it would fail creating 'texinfo', however if I cd'd to the source folder and ran make directely there it worked great. For the moment I managed to get past it, but it was a long night and I don't remember exactly what I did other than change the shell. I saw a post here with someone else having the same problem with texinfo and mention of a ticket and a fix HERE, but I'm not sure that fix made it to the 8.02 branch or which file it was.

I'm kind of going off the path / using your build patch to do an experiment, so it might not be related to anything you posted, but if it works I couldn't have done it without your post. :)
 

bryanburke

Dabbler
Joined
Nov 11, 2011
Messages
11
Well, the failure at texinfo actually has nothing to do with texinfo at all. The log shows the errors right after the texinfo build because the nanoBSD build script uses several concurrent jobs (NANO_PMAKE="make -j8"), so everything in the log gets overlapped to a degree. The only way to get the real error written to the build log is to change that line in "nanobsd/common" to NANO_PMAKE="make" and re-run the build until it fails.

Also, I notice that you are building on FreeBSD 8.2-RELEASE. The FreeBSD source tree is now on 8.2-RELEASE-p4, which is the patched release I am building on. You can update your release either by updating the source tree in "/usr/src" and doing a buildworld, mergemaster -p, installworld, mergemaster or by using the freebsd-update utility to apply binary patches. Trying to build the p4 source on p0 could theoretically cause errors.

Refer to Chapter 25 of the official FreeBSD Handbook for instructions and caveats on updating your world.
 

donairb

Dabbler
Joined
Jan 5, 2012
Messages
19
Step 1: Get the Sources

If you already have the 8.0.2 sources, skip to step 2.

We use svn just like we would for trunk, except we are pulling the 8.0.2 branch instead:

Code:
svn co https://freenas.svn.sourceforge.net/svnroot/freenas/branches/8.0.2


Step 2: Patch the Build Script

Code:
patch < do_build.sh.diff
Sorry to be somewhat noob-y here, but does this need to be done from FreeBSD/FreeNAS, or can it be done in another OS? I have tried in both Linux (Ubuntu) and FreeBSD (in a VM in Win7), and am getting different errors.

In FreeBSD, I get 'svn command not found'. I had first run "pkg_add -r subversion" beacuse I thought that's where the svn command came from.

In Ubuntu, I can get the source just fine, but when I run the patch command, I get an error that there is no file to patch on line 3. (It gives me the first two lines of the diff file in the error message, so I think I have that part right.)

Thanks

Brian
 

donairb

Dabbler
Joined
Jan 5, 2012
Messages
19
D'Oh!

Didn't reboot after adding subversion.

Will check back in a little while.

Sorry.

Brian
 
Status
Not open for further replies.
Top