NextCloud

Jail Creation

bastille create nextcloud 14.2-RELEASE 10.0.0.20/8 bastille0

Installation

Database

We use mariadb as the database for NextCloud since it is the best supported option.

bastille pkg nextcloud install mariadb1011-server
bastille sysrc nextcloud mysql_enable=YES
bastille service nextcloud mysql-server start

To set up the database we connect to the mariadb instance in the jail. Use an empty password if promted:

bastille cmd nextcloud mysql -u root

Then, in the shell that opens, enter the following SQL commands. Replace changeme with a password of your choice. Be sure to use the ip address of the nextcloud jail as host for the nextcloud use (the part after the @).

CREATE DATABASE nextcloud;
CREATE USER 'nextcloud'@'10.0.0.20' IDENTIFIED by 'changeme';
GRANT ALL PRIVILEGES ON nextcloud. * TO 'nextcloud'@'10.0.0.20';
GRANT ALL PRIVILEGES ON *.* TO 'nextcloud'@'10.0.1.20' IDENTIFIED BY 'changeme' WITH GRANT OPTION;
--- exit the shell
exit

Nextcloud

Thankfully, we have a nextcloud package for FreeBSD which installs all the dependencies and puts all application files at /usr/local/www/nextcloud. We still need to install the SQlite PDO adapter, since this is required by Nextcloud Collectives.

bastille pkg nextcloud install nextcloud-php83 php83-pdo_sqlite php83-pecl-redis

Webserver

Now we need a way to serve the Nextcloud files. Apache is often used for this, but we opt again for Caddy, since using it with PHP-FPM is dead simple.

bastille pkg nextcloud install caddy

Edit the Caddyfile of the nextcloud jail:

micro /usr/local/bastille/jails/nextcloud/root/usr/local/etc/caddy/Caddyfile

Use the following Configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

# Because this is the second caddy server our clients connect to, we
# specify trusted upstream proxies as a global option
{
        servers {
                trusted_proxies static 10.0.0.1/24
                client_ip_headers X-Forwarded-For X-Real-IP
        }
}

# Our Reverse Proxy takes care of tls.
# Internally, we can deal with HTTP, so only expose the server on port 80
:80 {
  # Define the installation directory of Nextcloud as our root dir.
	root * /usr/local/www/nextcloud

	# Enable the static file server:
	file_server

  # Enable php by connecting to php_fpm
	php_fastcgi localhost:9000

	# Enable logging:
	log {
		output file /var/log/caddy/access.log
		# Caddy's structured log format:
		format json
	}
}

Enable the caddy service

bastille sysrc nextcloud caddy_enable=YES

PHP Configuration

Use the production template as a starting point:

bastille cmd nextcloud cp /usr/local/etc/php.ini-production /usr/local/etc/php.ini

Change the following values in the newly created php.ini

micro /usr/local/bastille/jails/nextcloud/root/usr/local/etc/php.ini

 0
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

upload_max_filesize = 16G
post_max_size = 16G
max_input_time = 3600
max_execution_time = 3600
upload_tmp_dir = /var/nextcloud_temp/
output_buffering = 0
memory_limit = 2048M

### OPCache Settings
opcache.enable=1
opcache.enable_cli=1
opcache.interned_strigs_buffer=32
opcache.revalidate_freq = 60
opcache.validate_timestamps = 0
opcache.save_comments = 1
opcache.jit = 1255
opcache.jit_buffer_size = 8M

Create the temp-dir we just specified and make it writeable by the www user:

mkdir /usr/local/bastille/jails/nextcloud/root/var/nextcloud_temp
chown www:www /usr/local/bastille/jails/nextcloud/root/var/nextcloud_temp

Lets enable php_fpm on the nextcloud jail

bastille sysrc nextcloud php_fpm_enable=YES

Uncomment and adapt the following values in the php_fpm pool configuration

clear_env = no
env[HOSTNAME] = $HOSTNAME
env[PATH] = /usr/local/bin:/usr/bin:/bin:/usr/sbin
env[TMP] = /tmp
env[TMPDIR] = /tmp
env[TEMP] = /tmp

Reverse-Proxy

Edit the Caddyfile of our reverse-proxy and a block to forward traffic to nextcloud:

micro /usr/local/bastille/jails/caddy/root/usr/local/etc/caddy/Caddyfile

Replace ezdk.org with your own domain.

nextcloud.ezdk.org {
    redir /.well-known/carddav /remote.php/dav/ 301
    redir /.well-known/caldav /remote.php/dav/ 301
	reverse_proxy 10.0.0.20:80 {
		trusted_proxies 10.0.0.1/32
	}
}

Reload the config

bastille service caddy caddy reload

Nextcloud Cronjobs

Nextcloud needs to run some background jobs periodically which are run with the cron.php script. Add the following line to the crontab file to have the www user run this script every 5 minutes :

micro /usr/local/bastille/jails/nextcloud/etc/crontab

# Nextclouid Cronjobs
*/5  *  *  *  *  www  /usr/local/bin/php -f /usr/local/www/nextcloud/cron.php

NextCloud Configuration

For good measure, lets restart the nextcloud jail, making sure all our settings are applied.

bastille restart nextcloud

Setup

Go to nextcloud.<yourdomain.tld> to start the nextcloud setup. Enter a Username for the admin account and a password. Under Storage & Database, change the db type to MariaDB, enter nextcloud for the username and Database name and enter the password you’ve used to create the user. Be sure to change the Database Host from localhost to 127.0.0.1:3306.

Setup will work for a while and then present you with a screen to install recommended apps. Only select the Calendar and Contacts apps for now. We will deal with the others later.

Afterwards, you should get logged in into your new nextcloud instance as an administrator.

LDAP Integration

Navigate to the Apps Menu through the Nextcloud Menu on the top-right. Go to Featured Apps and locate the app LDAP user and group backend in the list. Enable or install it. Open the Administrative Settings and navigate to “LDAP/AD integration”. Now follow the guide from lldap on how to integrate nextcloud, or simply adopt the settings from the following screenshots:

Log out as the Administrator and log in with the user you have created earlier in lldap. If everything works well, log back in as an administrator, go to the Features Apps list and install Team Folders

Caching

The FreeBSD Nextcloud package allready set up APCu for local caching. We want to improve on it by using redis.

First, install redis, enable and start it:

bastille pkg nextcloud install redis
bastille sysrc nextcloud redis_enable=YES
bastille service nextcloud redis start

Due to Localhost Pitfalls in Jails, we need to bind redis to the ip address of the jail. Edit the following two values in /usr/local/etc/redis.conf

micro /usr/local/bastille/jails/nextcloud/root/usr/local/etc/redis.conf

0
1
2
3

# Bind to internal ip address
bind 10.0.0.20
# Also disable protected mode
protected-mode no

Now we are ready to adapt nextclouds caching settings in /usr/local/www/nextcloud/config/config.php. Change the following values in this file:

micro /usr/local/bastille/jails/nextcloud/root/usr/local/www/nextcloud/config/config.php

0
1
2
3
4
5
6
7
8

'default_phone_region' => 'CH', 
'maintenance_window_start' => 1,
'memcache.local' => '\OC\Memcache\APCu',
'memcache.distributed' => '\OC\Memcache\Redis',
'memcache.locking' => '\OC\Memcache\Redis',
'redis' => [
     'host' => '10.0.0.20',
     'port' => 6379,
],

for good measure, lets restart the nextcloud jail:

bastille restart nextcloud

Document Backend

Without a Document Backend, users won’t be able to edit office documents in Nextcloud. They would only be able to Up- and Download files. There are currently two available document backends: Collabora and OnlyOffice. Collabora is based on LibreOffice and the default choice for Nextcloud. But Collabora has no plans of supporting FreeBSD and attemps to port it to FreeBSD are currently stuck. OnlyOffice on the other hand, provides native FreeBSD packages. So we choose OnlyOffice.

Jail Setup

The document backend is a separate application intended to run on a separate host or container. So we are creating a jail for it:

bastille create onlyoffice 14.2-RELEASE 10.0.0.21/8 bastille0

# for PostgreSQL we need to set an additional option for the jail
bastille config onlyoffice set allow.sysvipc 1

# restart it for good measure
bastille restart onlyoffice

Lets install the packages required for this container and enable all the services these will install (we will also install micro to have an editor ready in the jail):

bastille pkg onlyoffice install onlyoffice-documentserver py311-importlib-resources py311-setuptools postgresql16-server micro

bastille sysrc onlyoffice postgresql_enable=YES
bastille sysrc onlyoffice nginx_enable=YES
bastille sysrc onlyoffice rabbitmq_enable=YES
bastille sysrc onlyoffice supervisord_enable=YES

Database

We will use Postgresql as the Database for OnlyOffice. Lets set it up now. We will have to execute quite a few commands and edit multiple files. It will be easier if we perform these steps directly in the jail. So let’s enter it:

bastille console onlyoffice

From here, lets initialize the database:

service postgresql initdb

We need to tell postgres to listen on the ip of the jail. Edit the file /var/db/postgres/data16/pg_hba.conf and add the folowing at the end of the file. This will tell postgres to trust every connection coming from this address (the local jail).

1

host    all             all             10.0.0.21/32            trust

We also need to edit /var/db/postgres/data16/postgresql.conf to tell postgres to listen on this interface. Edit the file and change the line containing listen_addresses to the following:

1

listen_addresses = 'localhost,10.0.0.21'

Now we are ready to create and configure the database for onlyoffice:

service postgresql start
psql -U postgres -c "CREATE DATABASE onlyoffice;"
psql -U postgres -c "CREATE USER onlyoffice WITH password 'onlyoffice';"
psql -U postgres -c "GRANT ALL privileges ON DATABASE onlyoffice TO onlyoffice;"
psql -U postgres -c "ALTER DATABASE onlyoffice OWNER to onlyoffice;"
psql -h10.0.0.21 -Uonlyoffice -d onlyoffice -f /usr/local/www/onlyoffice/documentserver/server/schema/postgresql/createdb.sql

RabbitMQ

RabbitMQ provides a Message Queue which OnlyOffice depends on. We configure it as follows:

service rabbitmq start
rabbitmqctl --erlang-cookie `cat /var/db/rabbitmq/.erlang.cookie` add_user onlyoffice onlyoffice # usr=onlyoffice, pw=onlyoffice
rabbitmqctl --erlang-cookie `cat /var/db/rabbitmq/.erlang.cookie` set_user_tags onlyoffice administrator
rabbitmqctl --erlang-cookie `cat /var/db/rabbitmq/.erlang.cookie` set_permissions -p / onlyoffice ".*" ".*" ".*"

Documentserver

Now we are finally ready to configure the OnlyOffice Documentserver. We need to add our database and rabbitmq credentials to the configuratioon and add a specific block to set request-filtering-agent to allow requests from private ip addresses. You can edit the configuration with micro /usr/local/etc/onlyoffice/documentserver/local.json. Following is the content of this file, after adaption.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

{
  "services": {
    "CoAuthoring": {
      "request-filtering-agent": {
      	"allowPrivateIPAddress": true,
      	"allowMetaIPAddress": true
      },
      "sql": {
        "type": "postgres",
        "dbHost": "10.0.0.21",
        "dbPort": "5432",
        "dbName": "onlyoffice",
        "dbUser": "onlyoffice",
        "dbPass": "onlyoffice"
      },
      "token": {
        "enable": {
          "request": {
            "inbox": false,
            "outbox": false
          },
          "browser": false
        },
        "inbox": {
          "header": "Authorization"
        },
        "outbox": {
          "header": "Authorization"
        }
      },
      "secret": {
        "inbox": {
          "string": "secret"
        },
        "outbox": {
          "string": "secret"
        },
        "session": {
          "string": "secret"
        }
      }
    }
  },
  "rabbitmq": {
    "url": "amqp://onlyoffice:onlyoffice@localhost"
  },
  "storage": {
    "fs": {
      "secretString": "N8ogZ0VEXV4IcXdSXDPx"
    }
  }
 }

Open the supervisord configuration and add an include statement to include the onlyoffice supervisord files:

micro /usr/local/etc/supervisord.conf

1
2

[include]
files = /usr/local/etc/onlyoffice/documentserver/supervisor/*.conf

Now we can start supervisord

service supervisord start

nginx

OnlyOffice comes with pre-made configuration files for nginx. We will use this to expose it internally (but we still use our caddy reverse-proxy to expose it externally).

Lets edit /usr/local/etc/nginx/nginx.conf and on the line after http { enter the following include statement

1

include /usr/local/etc/onlyoffice/documentserver/nginx/ds.conf;

There is already a server {} block in this file (inside the http { block). Remove the whole block. Now run the following script:

documentserver-update-securelink.sh

After the script has run, open /usr/local/etc/onlyoffice/documentserver/nginx/ds.conf and remove the line listen [::]:80 default_server;. Our jail does not have a ipv6 stack and this line will cause an error. After that, rerun the previous script

documentserver-update-securelink.sh

Now the output should state that the config is sane and we are ready to start the nginx service

service nginx start

We now have everything set up in our onlyoffice-jail. Exit the shell to return to our hosts shell:

exit

Caddy Configuration

To expose the documentserver open the Caddyfile of our reverse-proxy jail with micro /usr/local/bastille/jails/caddy/root/usr/local/etc/caddy/Caddyfile and add the following block:

1
2
3

docs.<domain> {
	reverse_proxy 10.0.0.21:80
}

And restart caddy:

bastille service caddy caddy reload

Nextcloud Configuration

Log in to Nextcloud as an admin and install the ONLYOFFICE app. Open the administrator settings and in the OnlyOffice settings add https://docs.<domain> as the Docs address. Input http://10.0.0.21 as the address for internal request and http://nextcloud.<domain.tld> as the Nextcloud address for internal requests. (in the /etc/hosts file of the onlyoffice jail, nextcloud.domain.tld needs to point to the internal nextcloud ip)

Talk Backend

For voice and video communication, we also need a separate backend for which we will create a jail

bastille create talk 14.2-RELEASE 10.0.0.22/8 bastille0

The talk backend depends on NATS and we’ll need to compile the signaling server ourselves. So let’s get all dependencies first.

bastille pkg talk install git go gmake nats-server protobuf3

Enable and start the nats service

bastille sysrc talk nats_enable=YES
bastille service talk nats start

Now on to build the spreed signaling server

# switch to the jail itself
bastille console talk

# clone the repo. Replace the branch with the current stable version
git clone --depth 1 --branch v2.0.2 https://github.com/strukturag/nextcloud-spreed-signaling.git

# cd into the repo
cd nextcloud-spreed-signaling

# build the signaling server
gmake build

# and check if it runs
./bin/signaling -h

# Copy the and binary and config file to /usr/local
cp ./bin/signaling /usr/local/bin/signaling
cp server.conf.in /usr/local/etc/signaling.conf

# exit the jail
exit

Install and enable janus

bastille pkg talk install janus
bastille sysrc talk janus_enable=YES

Then edit the janus websocket configuration

micro /usr/local/bastille/jails/talk/root/usr/local/etc/janus/janus.transport.websockets.jcfg

In this file, uncomment the ws_ip line and change it to ws_ip = "10.0.0.22"

And start janus

bastille service talk janus start

For our signaling configuration, we need one random 64byte and three 32byte strings to use as keys. Generate them with the following commands and note them down.

cat /dev/urandom | env LC_CTYPE=C tr -dc a-zA-Z0-9 | head -c 64; echo
cat /dev/urandom | env LC_CTYPE=C tr -dc a-zA-Z0-9 | head -c 32; echo #execute three times for three different keys

Edit the signaling config file

micro /usr/local/bastille/jails/talk/root/usr/local/etc/signaling.conf

and change the following values in the file accordingly:

[http]
listen = 10.0.0.22:8443
#Set to the jails ip and remove the hashtag in front

[app]
trustedproxies = 10.0.0.1
#IP of the caddy jail

[sessions]
hashkey = <your-64byte-random-string>
blockkey = <a-32byte-random-string>

[clients]
internalsecret = <a-32byte-random-string>

[backend]
backendtype = static
# uncomment the line
secret = <a-32byte-random-string>
# we need to add a common secret here - this is anoter 32byte random string
backends = nextcloud
# string does not matter, we define it later
allowed = nextcloud.ezdk.org, 10.0.0.1, 10.0.0.20
# nextcloud domain  must be addded. This value needs to be newly added to the [backend] block
connectionsperhost = 128

[nextcloud]
# add this after the [backend-id] block
url = https://nextcloud.ezdk.org
secret =
#we leave this empty to apply the common shared secret
sessionlimit = 32

[nats]
url = nats://10.0.0.22:4222
# Change to the jails ip address

[mcu]
type = janus
url = ws://10.0.0.22:8188

Edit the /etc/hosts file of the jail

micro /usr/local/bastille/jails/talk/root/etc/hosts

and add the following entry. So that internally the services will send requests for nextcloud to the reverse proxy.

10.0.0.1 nextcloud.ezdk.org

We still need a service for the signaling server. Lets make one by creating a file at /usr/local/etc/rc.d/signaling in the talk jail

micro /usr/local/bastille/jails/talk/root/usr/local/etc/rc.d/signaling

and fill it with the following content

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

#!/bin/sh
#
# PROVIDE: signaling
# REQUIRE: networking
# KEYWORD:
#
# To enable, add "signaling_enable="YES"" to /etc/rc.conf

. /etc/rc.subr

name=signaling
rcvar=signaling_enable
desc="Nextcloud Spreed Signaling High Performnce Talk Backend"

load_rc_config $name
: ${signaling_enable:="NO"}

signaling_bin="/usr/local/bin/signaling"
signaling_config="/usr/local/etc/signaling.conf"
signaling_args="--config ${signaling_config}"
pidfile="/var/run/${name}.pid"
signaling_cmd="${signaling_bin} ${signaling_args} 1>/dev/null 2>&1"

command="/usr/sbin/daemon"
command_args="-P ${pidfile} ${signaling_cmd}"

run_rc_command "$1"

We need to make that script executable and then we can enable it as a service

chmod +x /usr/local/bastille/jails/talk/root/usr/local/etc/rc.d/signaling

bastille sysrc talk signaling_enable=YES
bastille service talk signaling start

We can test if everything worked by reading the welcome message from the api

bastille cmd talk curl localhost:8080/api/v1/welcome

You should get the message {"nextcloud-spreed-signaling":"Welcome","version":"unreleased"}

Edit the /etc/hosts file of the nextcloud jail

micro /usr/local/bastille/jails/nextcloud/root/etc/hosts

And add the following line to it so that the nextcloud instance will make requests for the signaling server to the reverse-proxy

10.0.0.1    signaling.ezdk.org

Now we only need to add a new subdomain to our main caddy configuration

micro /usr/local/bastille/jails/caddy/root/usr/local/etc/caddy/Caddyfile

signaling.ezdk.org {
	reverse_proxy 10.0.0.22:8080 {
			header_up X-Real_IP {remote-host}
	}
}

And reload the config as usual

bastille service caddy caddy reload

Talk App Setup

Login as an Admin to the nextcloud instance. Go to the Apps settings and enable the talk app.

Go to the administrative settings and on the Talk page press the + Add High-performance backend server button.

Enter https://signaling.ezdk.org as the URL for the backend and enter string you set as secret in the signaling configuratin as the shared secret.