Scripted installation of Nextcloud 28 in iocage jail 2018-03-23

[danb35]

Is there anything in the logs? Either /var/log/messages or some specific mysql logs?

 

[Elo]

Is there anything in the logs? Either /var/log/messages or some specific mysql logs?

No can not find any Mysql LOG in /var/log ??
Can not find anything or relevance in messages

I am starting to think that it would be just as effective to make a fresh installation. I also have a need to clean up the Pool structure and define new datasets for the data. I just need to be sure that I retain the data and the users information although the later can be recreated. The data is intact outside the jail as pr. the links shown bellow.

Update: Looking further into the details I have a suspicion of what can be the problem. Some time ago this FreeNAS server was moved into our main domain. During this a lot of file permissions and ownerships were changed. I see that changes has been made to files under the POOL_PATH which should not have been touched.I would still like to make a fresh installation in a new jail.

I deleted the old Jail and used the script for a fresh instal!

 

[Elo]

I have tried to make a fresh install. It seems to go OK but there was some error during the running of the script but is say that Nextcloud was successfully installed at the end!..

When I try to connect the connection is denied.

OS Version: FreeNAS-11.2-U6 (Build Date: Sep 17, 2019 0:16)

My config file is as follows (HOST_NAME is set to the name of my registered domain.

JAIL_IP="192.168.111.15"
DEFAULT_GW_IP="192.168.111.1"
INTERFACE="igb0"
VNET="off"
POOL_PATH="/mnt/Main_Pool/Jails/Nextcloud"
JAIL_NAME="nextcloud"
TIME_ZONE="Europe/Oslo" # See http://php.net/manual/en/timezones.php
HOST_NAME="vabxxx.com"
STANDALONE_CERT=0
DNS_CERT=1
SELFSIGNED_CERT=1
NO_CERT=0
TEST_CERT="--test"
DB_PATH="/mnt/Main_Pool/db"
FILES_PATH="/mnt/Main_Pool/files"
DNS_PLUGIN="dyn"
DNS_ENV="DYN_CUSTOMER_NAME=ola normann DYN_USER_NAME=onorman DYN_PASSWORD=password"


I see in the messages file the last line:

Nov 6 18:02:57 nextcloud root: /etc/rc: WARNING: failed to start php_fpm

and in php-fpm.log:

[06-Nov-2019 18:02:57] NOTICE: configuration file /usr/local/etc/php-fpm.conf test is successful
[06-Nov-2019 18:02:57] ERROR: no data have been read from pipe

Can anyone tip on how to debug this as I am lost?

There are some errors reported by the script which states that Nexcloud was successfully installed at the end:

mysql_enable: -> YES
redis_enable: -> YES
php_fpm_enable: -> YES

===> Cleaning for php73-pecl-redis-5.0.2
===> Cleaning for php71-pecl-redis-5.0.2
===> Cleaning for php72-pecl-redis-5.0.2
===> Cleaning for php74-pecl-redis-5.0.2
/!\ ERROR: /!\

Ports Collection support for your FreeBSD version has ended, and no ports are
guaranteed to build on this system. Please upgrade to a supported release.

No support will be provided if you silence this message by defining
ALLOW_UNSUPPORTED_SYSTEM.

*** Error code 1

Stop.
make: stopped in /usr/ports/databases/pecl-redis

===> Cleaning for php73-pecl-APCu-5.1.17
===> Cleaning for php71-pecl-APCu-5.1.17
===> Cleaning for php72-pecl-APCu-5.1.17
/!\ ERROR: /!\

Ports Collection support for your FreeBSD version has ended, and no ports are
guaranteed to build on this system. Please upgrade to a supported release.

No support will be provided if you silence this message by defining
ALLOW_UNSUPPORTED_SYSTEM.

*** Error code 1
Stop.
make: stopped in /usr/ports/databases/pecl-redis

Copying Caddyfile for self-signed cert
caddy_enable: -> YES
caddy_cert_email: ->
caddy_env: -> DYN_CUSTOMER_NAME=aaaaa DYN_USER_NAME=bbbbb DYN_PASSWORD=cccccc
* Stopping nextcloud
+ Running prestop OK
+ Stopping services OK
+ Removing devfs_ruleset: 5 OK
+ Removing jail process OK
+ Running poststop OK
* Starting nextcloud
+ Started OK
+ Starting services OK

Nextcloud was successfully installed

System config value mysql.utf8mb4 set to boolean true
System config value logtimezone set to string Europe/Oslo
System config value log_type set to string file
System config value logfile set to string /var/log/nextcloud.log
System config value loglevel set to string 2
System config value logrotate_size set to string 104847600
System config value memcache.local set to string \OC\Memcache\APCu
System config value redis => host set to string /tmp/redis.sock
System config value redis => port set to integer 0
System config value memcache.locking set to string \OC\Memcache\Redis
System config value overwrite.cli.url set to string https://xxxxxxxxxxxxx/
System config value htaccess.RewriteBase set to string /.htaccess has been updated
System config value trusted_domains => 1 set to string xxxxxxxxxxxxxx
System config value trusted_domains => 2 set to string 192.168.111.15
encryption enabled
Encryption enabled
Default module: OC_DEFAULT_MODULE
Encryption disabled
Set mode for background jobs to 'cron'

Successfully removed mount from nextcloud's fstab
Installation complete!
Using your web browser, go to https://xxxxxxxx.com to log in
Default user is admin, password is yyyyyyyy

Database Information
--------------------
Database user = nextcloud
Database password = zzzzzzzzzz==
The MariaDB root password is xxxxxxxxx==

All passwords are saved in /root/nextcloud_db_password.txt

You have obtained your Let's Encrypt certificate using the staging server.
This certificate will not be trusted by your browser and will cause SSL errors
when you connect. Once you've verified that everything else is working
correctly, you should issue a trusted certificate. To do this, run:
iocage exec nextcloud /root/remove-staging.sh

 

[danb35]

Ports Collection support for your FreeBSD version has ended, and no ports are
guaranteed to build on this system. Please upgrade to a supported release.

Here we go again. The problem is that iX consistently can't get a new release of FreeNAS out before the FreeBSD release on which the current version is based goes EOL. We had that problem with (at least) 11.0 and 11.1, and now it looks like it's cropping up with 11.2. A couple of possible ways around it, but it may take me a couple of days to get the script fixed.

Inside the jail, can you do pkg install php73-pecl-redis and pkg install php73-pecl-APCu?

 

[Elo]

Here we go again. The problem is that iX consistently can't get a new release of FreeNAS out before the FreeBSD release on which the current version is based goes EOL. We had that problem with (at least) 11.0 and 11.1, and now it looks like it's cropping up with 11.2. A couple of possible ways around it, but it may take me a couple of days to get the script fixed.

Inside the jail, can you do pkg install php73-pecl-redis and pkg install php73-pecl-APCu?

Hi Dan. First THANK YOU for taking the time and for producing this script that helps many people to get Nextcloud going., as it is not a trivial install!!!!
I did read the "discussion" about the issues you are mentioning in the 11.3 thread. If I understand you correctly the FreeBSD version of the Jail is already EOL..

I installed both inside the Jail but not successful. After rebooting my NAS the jail did not started and the following messages occurred. I think perhaps it is better that you fix the script instead of helping me here now. I can wait for the fixed script and then do a reinstall:


RuntimeError
mount_nullfs: /mnt/Main_Pool/Jails/Nextcloud/portsnap/db: No such file or directory jail: /sbin/mount -t nullfs -o rw /mnt/Main_Pool/Jails/Nextcloud/portsnap/db /mnt/Main_Pool/iocage/jails/nextcloud/root/var/db/portsnap: failed

There is NO db in /mnt/Main_Pool/Jails/Nextcloud/portsnap/? (perhaps because the make was terminated during the script?)

This is my mount points (fstab nextcloud):


+-------+-------------------------------------------------------------------------------------------------------------------------------------------+
| INDEX | FSTAB ENTRY |
+=======+===========================================================================================================================================+
| 0 | /mnt/Main_Pool/Jails/Nextcloud/portsnap/db /mnt/Main_Pool/iocage/jails/nextcloud/root/var/db/portsnap nullfs rw 0 0 |
+-------+-------------------------------------------------------------------------------------------------------------------------------------------+
| 1 | /mnt/Main_Pool/files /mnt/Main_Pool/iocage/jails/nextcloud/root/mnt/files nullfs rw 0 0 |
+-------+-------------------------------------------------------------------------------------------------------------------------------------------+
| 2 | /mnt/Main_Pool/db /mnt/Main_Pool/iocage/jails/nextcloud/root/var/db/mysql nullfs rw 0 0 |
+-------+-------------------------------------------------------------------------------------------------------------------------------------------+




Details:

Error: concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/local/lib/python3.6/concurrent/futures/process.py", line 175, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/worker.py", line 133, in main_worker
res = loop.run_until_complete(coro)
File "/usr/local/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete
return future.result()
File "/usr/local/lib/python3.6/site-packages/middlewared/worker.py", line 93, in _run
return await self._call(name, serviceobj, methodobj, params=args, job=job)
File "/usr/local/lib/python3.6/site-packages/middlewared/worker.py", line 85, in _call
return methodobj(*params)
File "/usr/local/lib/python3.6/site-packages/middlewared/worker.py", line 85, in _call
return methodobj(*params)
File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 668, in nf
return f(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/jail.py", line 623, in start
iocage.start()
File "/usr/local/lib/python3.6/site-packages/iocage_lib/iocage.py", line 1653, in start
callback=self.callback
File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_start.py", line 67, in __init__
self.__start_jail__()
File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_start.py", line 413, in __start_jail__
silent=self.silent)
File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_common.py", line 90, in logit
_callback(content, exception)
File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_common.py", line 64, in callback
raise callback_exception(message)
RuntimeError: mount_nullfs: /mnt/Main_Pool/Jails/Nextcloud/portsnap/db: No such file or directory
jail: /sbin/mount -t nullfs -o rw /mnt/Main_Pool/Jails/Nextcloud/portsnap/db /mnt/Main_Pool/iocage/jails/nextcloud/root/var/db/portsnap: failed

"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 165, in call_method
result = await self.middleware.call_method(self, message)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1096, in call_method
return await self._call(message['method'], serviceobj, methodobj, params, app=app, io_thread=False)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1041, in _call
return await self._call_worker(name, *args)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1061, in _call_worker
return await self.run_in_proc(main_worker, name, args, job)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 995, in run_in_proc
return await self.run_in_executor(self.__procpool, method, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 971, in run_in_executor
return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
RuntimeError: mount_nullfs: /mnt/Main_Pool/Jails/Nextcloud/portsnap/db: No such file or directory
jail: /sbin/mount -t nullfs -o rw /mnt/Main_Pool/Jails/Nextcloud/portsnap/db /mnt/Main_Pool/iocage/jails/nextcloud/root/var/db/portsnap: failed

 

[notspam]

A few days ago my SSL cert expired or got lost for reasons I have not figured out. As part of working through this and FreeNAS maintenance, I made the mistake of upgrading FreeNAS from 11.2-U3 to 11.2-U6 which messed up my SMB set up. Now that SMB is stable enough to move back to my original problem of losing my SSL cert, I went and tried getting a cert using SSLForFree. I have not installed the new cert yet. While trying to fiddle around I looked at the NextCloud jail configs and may have touched something. Although I retraced my steps I am not sure what could have caused the problem, but now when I navigate to the domain name I get a blank screen. Since I am at my wits end here, I tried setting up a NextCloud using the plugin and it appears to work (have not configured it to be on my own domain nor with my shares or users). The plugin is on the latest version of NextCloud (v17). I thought that was the reason I used the scripted install to begin with. Now I am asking myself, why would I want to use the scripted jail install vs the plugin. It was a year or so ago, when I decided to go the scripted install and I did not document my reasons to recall in this moment.
So I figured I would ask here:
What is the advantage of the scripted install vs the plugin before I invest more time in the plugin or trying to reassert a new SSL cert and resussitate the scripted install jail?
If there are compelling pros to the scripted install jail, then I will have follow up questions around resussitating the existing install that was working (although with an expired cert) before I touched it.

 

[danb35]

What is the advantage of the scripted install

• It includes SSL out of the box, with a pretty decent configuration, and should automatically renew it if you're using Let's Encrypt
• It stores its database and user data outside the jail
• You can upgrade packages using a simple pkg upgrade; you aren't waiting for iX to update the plugin
• It actually gives you a complete installation; you don't need to manually configure the database with credentials the system doesn't give you

Probably others; these are what come to my mind at the moment.

 

[notspam]

• It includes SSL out of the box, with a pretty decent configuration, and should automatically renew it if you're using Let's Encrypt
• It stores its database and user data outside the jail
• You can upgrade packages using a simple pkg upgrade; you aren't waiting for iX to update the plugin
• It actually gives you a complete installation; you don't need to manually configure the database with credentials the system doesn't give you

Probably others; these are what come to my mind at the moment.

Sounds compelling. Given that my current install is non-functional, would it be possible to reinstall in the same jail or do I need to reconstruct a new one?

 

[Apollo]

@notspam, one of my nextcloud install did actually had its certificate expire just an hour or so ago, and while trying to update the certifiacte, it was left waiting indefinitely without success.
I did a quick search and found the issue was caused by my version of acme to be out of date.
The command suggested to be used to update acme is as follow:

acme.sh --upgrade -b dev
Once done, I was able to successfully run acme with the --renew option.

 

[danb35]

Given that my current install is non-functional, would it be possible to reinstall in the same jail or do I need to reconstruct a new one?

The better course would be to fix the installation that's there, but of course for that to happen we need to figure out what broke. First, let's try to take care of the cert. Run these commands:

Code:

iocage console nextcloud
acme.sh --upgrade
acme.sh --cron

...and see what happens from there.

 

[Apollo]

The better course would be to fix the installation that's there, but of course for that to happen we need to figure out what broke. First, let's try to take care of the cert. Run these commands:

Code:

iocage console nextcloud
acme.sh --upgrade
acme.sh --cron

...and see what happens from there.

If the jail failed to run after performing some modification to it, it is easy enough to disable it and create a new jail by creating the db, files, portsnap and install datasets into a new location. Then the "install" dataset content of the failed jail can be copied to the new "install" dataset and it will be easy enough to run the install from there.
Upon successful completion, a complitlely new Nextcloud server environment should present itself. If this is the case, then you can stop the jail, modify the "fstab" so that your personnal server "db" and "files" dataset are mounted instead and then copy your old jail "config.php" file located under the old "/usr/etc/local/apache24/data/www/Nextcloud /config" or what ever location the "config" folder resides.
restart the jail and your old Nextcloud server should be up and running.
If none of the above works, then soemthing specific to your environemnt (network, firewall) issue or simply some Let's encrypt recent update are causing the issue.
A note of caution, you should update the new jail to use the same version of Nextcloud your server was on before you ty to handle the old data. If the update fails, then that would mean some apache or other packages are missing or too old to support the newest version of Nextcloud. Updating or adding them up to the point you are able to make the jail work should be enough for you to regain access to your own server.

 

[leshric]

Thanks for the script! I'm have an issue accessing Nextcloud, 502 Bad Gateway error. No errors during installation. I'm using Google domain with dynamic dns, I port forwarded 80 and 443 on my router and I used the STANDALONE_CERT option. I did notice that the redis service was not running initially because it had incorrect permissions on the redis.conf file.

Any ideas? I have FreeNAS 11.2-U6

 

[danb35]

Any ideas?

Probably the same as other recent issues--FreeBSD 11.2 is EOL, and ports won't build without an additional flag. I haven't yet gotten around to testing if those packages could be installed in a different way. When 11.2-U7 releases (which should be soon), you'll be able to build 11.3 jails, which will avoid the problem, but then the issue is one of setting the script up correctly to do that.

 

[leshric]

Probably the same as other recent issues--FreeBSD 11.2 is EOL, and ports won't build without an additional flag. I haven't yet gotten around to testing if those packages could be installed in a different way. When 11.2-U7 releases (which should be soon), you'll be able to build 11.3 jails, which will avoid the problem, but then the issue is one of setting the script up correctly to do that.

If that's the case, I'm glad it's not something I did wrong! I can wait a bit. Thanks!

 

[notspam]

The better course would be to fix the installation that's there, but of course for that to happen we need to figure out what broke. First, let's try to take care of the cert. Run these commands:

Code:

iocage console nextcloud
acme.sh --upgrade
acme.sh --cron

...and see what happens from there.

I did that and no joy. I first ran the last two commands from the jail console. The upgrade and cron seemed to perform as expected, but navigating to the domain did not. To confirm the cert is working I added a simple noname html with "Hi" in it and navigated to it. That worked. So I know my cert works. Although I never actually installed the one I got from sslforfree. So I am not sure why it did not tell me a bad cert was there.
I tried navigating to index.php and get a blank page still. Where do I look for a log? Actually I found what I think was a log a couple of days ago and it said something about inability to connect to db... I may have found it in /var/log/httpd-error.log. The most recent are DB, Doctrine and file not found errors in the nextcloud directory structure.
I will continue to pursue, but I am guessing that I may just have to reconstruct the jail and assign its IP with old one. From the config.php, it looks like my Nextcloud version is 14 and the latest is 17. So the earlier recommendation of swapping referenced to db and files may not work.
WAIT I THINK I may have it.
While trying to work through the 11.2-U3 to U6 upgrade issues and losing my SMB shares, I tried experimenting with either of the db or files shares forgetting that they were related to the Nextcloud jail until now. Oops. Let me see if I can get them back to their right state.
I retraced my apparent mistakes and looked in the script to see what might help to repair and tried adding the files to a unix share. That did not work. I think I will have to create a new jail and set it all up again.

 

[Apollo]

@notspam, what was the version of Nextcloud when it worked the last time? Was it 14 or 17?
You can updated the original script to point to the specific version of the tar file rather than using the "latest".
If you did perform an upgrade as part of the last action from your working Nextcloud, that leads me to think you may still have your upgrade under maintenance mode.
See if you can bring the Nextcloud out of maintenance. One other thing is that you could run the database upgrade command.
All the above should be run under CLI inside the jail.

 

[danb35]

So I am not sure why it did not tell me a bad cert was there.

Because you renewed the cert with the acme.sh --cron command. At least I guess you did; you didn't tell me that you got any error running that command.

something about inability to connect to db

That could be relevant. See if the database server is running: service mysql status. If it isn't, try starting it: service mysql start

My concern, and the reason I'm not as comfortable as @Apollo with "do a fresh install and then mount the old datasets to the new jail", is that I haven't done that before and I'm not sure it will work. And if it doesn't, you run the risk of losing the data in your Nextcloud installation.

See if you can bring the Nextcloud out of maintenance.

What makes you think it's in maintenance mode?

 

[danb35]

I've updated the script so you should be able to install under 11.2. If you've previously downloaded it and had trouble due to 11.2 being EOL, change to the directory to which you downloaded it and run git pull to download the updated script. You should then be ready to go.

 

[Apollo]

What makes you think it's in maintenance mode?

Sometimes, when I perform the Nextcloud update using the WEB GUI, it gets stuck and I cannot access the Nextcloud page any longer. The reason is that the database is still in maintenance mode. using the occ commands to "upgrade" and then bring the database out of "maintenance" is all that is needed.
I have no idea if this is the issue here, but I am just adding this as a possible culprit.

 

[danb35]

Sometimes, when I perform the Nextcloud update using the WEB GUI, it gets stuck and I cannot access the Nextcloud page any longer.

But doesn't it then show a page indicating that it's in maintenance mode? Or does it just give a blank page?