Self-hosting with Docker Compose: Migrating

In my recent post, I talked about leaving YunoHost and switching to Docker Compose. A large driver of the switch was to be able to move some of my important services offsite to a hosted VPS. I was able to set up my new self-hosting setup with one giant Docker Compose file locally. Once I had every service working the way I like, I split the Docker Compose file into two. One for the local server and one for the cloud server. Because all the data is separate from the system I was able to simply move the data for the cloud services to a new location and be up and running in no time at all. I was also able to move some services back locally with the same ease. Simply shut down the Docker container on one server and start it up again on the other server.

For a Docker Compose setup, the configuration is all contained in one spot. With Yunohost I had to track down various bits of data and configurations for each app. Combining this with a system automation framework like Ansible and you have a completely reproducible build.

I did run into a few snags while migrating. Most of these issues probably don’t apply to most situations. The biggest problem that I faced was changing the authentication methods. Since Yunohost had most apps integrated into their LDAP server, I had to do some manual database intervention to get some apps working again. Nextcloud handles LDAP users separately from regular users. To overcome this I had to recreate the users in the database with the same names to transfer the data. Here is a forum post over at the Nextcloud forum detailing the migration steps that I followed.

One piece of advice I would give to anyone that is starting on a self-hosting journey, stick to one type of database as much as possible. My first setup had both PostgreSQL and MySQL databases. Unfortunately, the 1GB VPS that I was running couldn’t handle both databases and the apps that I wanted to run. I imagine this would be similar for memory-constrained systems like some models of Raspberry Pis. Luckily many apps support several different database types. So limiting myself to one database was just a matter of migrating the data. I ended up using a manual migration, but I encountered pgloader as a tool to migrate from different databases to PostgreSQL.