tonyschuite
Dabbler
- Joined
- Oct 8, 2012
- Messages
- 12
OK, so working with Sickbeard, Couchpotato, sabnzbd and headphones is a dream. However, Headphones has one major problem: the musicbrainz scraping.
Installing a musicbrainz mirror on Ubuntu is a pain but doable. This is because the musicbrainz team develops on Ubuntu or similar.
Now, you could install a mirror on Ubuntu itself (I am still working on it, but I am getting there). This has the disadvantage that your ubuntu client must be running if the Freenas system is scraping.
I was fed up with the official musicbrainz mirror, so I decided it <u>should be possible</u> to compile on FreeBSD / Freenas.
Below is how far I got. The dependencies is the biggest pain, but so far it is mostly time consuming.
I am posting this in order to create a Howto, as n00b friendly as I can make it. Beware, this contains some advanced shell commands. I myself have never done anything on FreeBSD/freenas, so a lot of this could be done easier. Please tell me so I can change that in the Howto. Also the Musicbrainz server will take A LOT of space! around 8GB (make sure your Jail is big enough).
When I get this running, I will look into the ability to create a clean Database and a script that Headphones can connect to so as to update the needed Artist / Release.
This has all been done on FreeNAS-9.1.1-RELEASE-x64 (a752d35)
TIP: use 'tmux new -s session_name' to start a detached ssh session. these sessions can be resumed any time (provided your freenas box doesn't crash) using the command 'tmux attach -t session_name'.
Let's Begin:
// ___________________________________________
# First we need to do a portsnap inside the jail so we can compile existing ports for FreeBSD
# in ssh
# in jail (jexec 1 csh)
# TAKES AGES!!
# if already done with a different plugin --> portsnap fetch update
# Ports tree is already up to date.
# you can use vi as editor but i prefer nano
# exit jail and re enter for it to work
##let's install postgres
# use default selected options when prompted
#grab a cup of coffee!
#add
#go to WebUI
system-->sysctls-->add Sysctl
#add (not sure if the values are the same with every system, but for me this worked)
Variable = kern.ipc.shmall value = 32768
Variable = kern.ipc.shmmax value = 134217728
Variable = kern.ipc.semmap value = 256
#in ssh
#exit jail
#add
security.jail.sysvipc_allowed=1
#exit with :qw
#look for jail id
#give it a password, give it no SU priviliges
#no message means SUCCESS!!
#check https://www.wunki.org/posts/2011-05-07-solid-freebasd-server-postgresql-90.html for more options (encryption and remote connection)
# That is where I got this from.
##postgres done
##Next up is the prerequisites of MB
##GIT
##PhantomJS (not listed in official install guide but it gave me a headache during build)
##PERL no need (preinstalled, perl -v to recheck)
##memcached
#add
memcached_enable="YES"
## ICU
## FINALLY!! MusicBrainz! (sadly we are nowhere near done )
# change MB_SERVER_ROOT to your desired location (I left it on default)
# add password, host and port to READWRITE, READONLY and SYSTEM
# in my case (local server) server and port are 127.0.0.1 and 5432
# change WEB_SERVER to "127.0.0.1:5000" (we will only be running a local mirror)
# change REPLICATION_TYPE to RT_SLAVE
# change DB_STAGING_SERVER to 0
# run the following to install dependencies:
#take another cup of coffee!
# in most instances this should complete. if not, edit the cpanfile with the contents of the following:
#
IF! The following three libraries failed to install, install them manually:
# remove the # before listen_adresses='localhost'
#quit using :wq
# comment all lines that are uncommented by putting # in front of it
now add
# Sadly we need a full musicbrainz dump (PUN also intended) to be able to sync changes to the database.
# in a browser, go to http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/LATEST
# copy the code and insert in the following commands
# more Coffee!
This takes a while.
after that you can run the server with
refer to https://github.com/metabrainz/musicbrainz-server/blob/master/INSTALL.md
for further instructions on auto starting the server and loading changes.
carton exec -- ./admin/replication/LoadReplicationChanges
##___________________________________________
Installing a musicbrainz mirror on Ubuntu is a pain but doable. This is because the musicbrainz team develops on Ubuntu or similar.
Now, you could install a mirror on Ubuntu itself (I am still working on it, but I am getting there). This has the disadvantage that your ubuntu client must be running if the Freenas system is scraping.
I was fed up with the official musicbrainz mirror, so I decided it <u>should be possible</u> to compile on FreeBSD / Freenas.
Below is how far I got. The dependencies is the biggest pain, but so far it is mostly time consuming.
I am posting this in order to create a Howto, as n00b friendly as I can make it. Beware, this contains some advanced shell commands. I myself have never done anything on FreeBSD/freenas, so a lot of this could be done easier. Please tell me so I can change that in the Howto. Also the Musicbrainz server will take A LOT of space! around 8GB (make sure your Jail is big enough).
When I get this running, I will look into the ability to create a clean Database and a script that Headphones can connect to so as to update the needed Artist / Release.
This has all been done on FreeNAS-9.1.1-RELEASE-x64 (a752d35)
TIP: use 'tmux new -s session_name' to start a detached ssh session. these sessions can be resumed any time (provided your freenas box doesn't crash) using the command 'tmux attach -t session_name'.
Let's Begin:
// ___________________________________________
# First we need to do a portsnap inside the jail so we can compile existing ports for FreeBSD
# in ssh
# in jail (jexec 1 csh)
Code:
portsnap fetch extract
# TAKES AGES!!
# if already done with a different plugin --> portsnap fetch update
# Ports tree is already up to date.
# you can use vi as editor but i prefer nano
Code:
cd /usr/ports/editors/nano[/FONT][/COLOR][/FONT][/COLOR]
make install clean
# exit jail and re enter for it to work
##let's install postgres
# use default selected options when prompted
Code:
cd /usr/ports/databases/postgresql92-server make install clean cd /usr/ports/databases/postgresql92-contrib make install clean
#grab a cup of coffee!
Code:
nano /etc/rc.conf
#add
Code:
postgresql_enable="YES"
#go to WebUI
system-->sysctls-->add Sysctl
#add (not sure if the values are the same with every system, but for me this worked)
Variable = kern.ipc.shmall value = 32768
Variable = kern.ipc.shmmax value = 134217728
Variable = kern.ipc.semmap value = 256
#in ssh
#exit jail
Code:
exit nano /etc/sysctl.conf
#add
security.jail.sysvipc_allowed=1
#exit with :qw
Code:
jls
#look for jail id
Code:
jail -m jid=<insert jailID here> allow.sysvipc=1 jexec <insert jailID here> csh /usr/local/etc/rc.d/postgresql initdb /usr/local/etc/rc.d/postgresql start #creating test database su pgsql createuser -P pg_test
#give it a password, give it no SU priviliges
Code:
/usr/local/bin/createdb db_test -O pg_test --encoding=UNICODE
#no message means SUCCESS!!
Code:
su pgsql createuser -P -s postgres # give it a password exit
#check https://www.wunki.org/posts/2011-05-07-solid-freebasd-server-postgresql-90.html for more options (encryption and remote connection)
# That is where I got this from.
##postgres done
##Next up is the prerequisites of MB
##GIT
Code:
pkg_add -r git
##PhantomJS (not listed in official install guide but it gave me a headache during build)
Code:
cd /usr/ports/lang/phantomjs make install clean
##PERL no need (preinstalled, perl -v to recheck)
##memcached
Code:
pkg_add -r memcached nano /etc/rc.conf
#add
memcached_enable="YES"
Code:
/usr/local/bin/memcached -d -u nobody
## ICU
Code:
cd /tmp fetch http://download.icu-project.org/files/icu4c/4.0/icu4c-4_0-src.tgz tar -zxvf icu4c-4_0-src.tgz cd icu/source/ ./runConfigureICU FreeBSD gmake gmake install echo "/usr/local/lib" >> /etc/ld.so.conf ldconfig
## FINALLY!! MusicBrainz! (sadly we are nowhere near done )
Code:
mkdir /home/musicbrainz/ cd /home/musicbrainz rehash git clone --recursive git://github.com/metabrainz/musicbrainz-server.git cd musicbrainz-server cp lib/DBDefs.pm.sample lib/DBDefs.pm nano lib/DBDefs.pm
# change MB_SERVER_ROOT to your desired location (I left it on default)
# add password, host and port to READWRITE, READONLY and SYSTEM
# in my case (local server) server and port are 127.0.0.1 and 5432
# change WEB_SERVER to "127.0.0.1:5000" (we will only be running a local mirror)
# change REPLICATION_TYPE to RT_SLAVE
# change DB_STAGING_SERVER to 0
# run the following to install dependencies:
Code:
su cpan App::cpanminus cpanm Carton cpanm --uninst-shadows ExtUtils::MakeMaker cpanm .
#take another cup of coffee!
Code:
pkg_add -r libxml2 cat Makefile.PL | grep ^requires > cpanfile carton install --deployment
# in most instances this should complete. if not, edit the cpanfile with the contents of the following:
#
IF! The following three libraries failed to install, install them manually:
Code:
cpanm Hash::Merge cpanm JSON::Syck cpanm Net::CoverArtArchive::CoverArt
Code:
cd /home/musicbrainz/musicbrainz-server/postgresql-musicbrainz-unaccent gmake gmake install cd ../ cd postgresql-musicbrainz-collate gmake gmake install cd ../ vi /usr/local/pgsql/data/postgresql.conf
# remove the # before listen_adresses='localhost'
#quit using :wq
Code:
vi /usr/local/pgsql/data/postgresql.conf
# comment all lines that are uncommented by putting # in front of it
now add
Code:
local all all trust
# Sadly we need a full musicbrainz dump (PUN also intended) to be able to sync changes to the database.
# in a browser, go to http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/LATEST
# copy the code and insert in the following commands
Code:
cd /tmp mkdir dumps cd dumps fetch http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/<insert code here>/mbdump-cdstubs.tar.bz2 && fetch http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/<insert code here>/mbdump-cover-art-archive.tar.bz2 && fetch http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/<insert code here>/mbdump-derived.tar.bz2 && fetch http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/<insert code here>/mbdump-edit.tar.bz2 && fetch http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/<insert code here>/mbdump-editor.tar.bz2 && fetch http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/<insert code here>/mbdump-stats.tar.bz2 && fetch http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/<insert code here>/mbdump.tar.bz2
# more Coffee!
Code:
cd /home/musicbrainz/musicbrainz-server/ ./admin/InitDb.pl --createdb --import /tmp/dumps/mbdump*.tar.bz2 --echo
This takes a while.
after that you can run the server with
Code:
plackup -Ilib -r
refer to https://github.com/metabrainz/musicbrainz-server/blob/master/INSTALL.md
for further instructions on auto starting the server and loading changes.
carton exec -- ./admin/replication/LoadReplicationChanges
##___________________________________________