../../_images/docker.png

Docker Compose

Introduction

As of 3.0.0, L7|ESP is distributed as a container image. A container runtime, such as Docker, is the only requirement to run L7|ESP on any operating system. Cloud-based PaaS (platform as a service) solutions may also be used to run L7|ESP containers. In addition, container orchestration systems such as Kubernetes or Docker Swarm may be used to manage L7|ESP containers at scale. This particular guide assumes you are installing L7|ESP on a Linux server using the Docker container runtime. If this is not the case, please request documentation for your intended deployment approach.

Configure Linux

The simplest way to get started with L7|ESP is to use a Linux server with your operating system of choice. The first thing you will want to do is create a non-privileged Linux user that will be used to run the L7|ESP container and be the owner of any L7|ESP related files.

Optional L7|ESP User Account:

The following command should create a user called l7esp with home folder /home/l7esp:

 $ adduser l7esp

The following command can be used to switch to this new user at any time:

 $ sudo su - l7esp

Note

To get back to the root or previous, privileged user, simply type exit or press Ctrl+D.

Installing Docker

It is recommended that you follow the official installation documentation for your operating system: https://docs.docker.com/engine/install/

At the time of writing, Docker provides a convenience script at https://get.docker.com/ that we will be using in this guide.

 $ curl -fsSL https://get.docker.com -o get-docker.sh
 $ sudo sh get-docker.sh

Follow any on-screen instructions to complete the Docker Engine installation. It is recommended to also review the post-installation instructions:

Optional, you can add the l7esp user to the docker group so it can run containers without requiring root or sudo privileges:

 $ sudo usermod --append --groups docker l7esp

Note

For the Linux group change to take effect you will need to log out of your SSH session and back in again.

Installing Docker Compose

Docker Compose simplifies the running of containers by allowing you to specify all the container configuration in a YAML file, instead of running individual docker commands to create, configure and run containers.

It is recommended that you follow the official installation documentation for your operating system: https://docs.docker.com/compose/install/

At the time of writing, Docker Compose can be installed on Linux simply by downloading the executable:

 $ sudo curl -L "https://github.com/docker/compose/releases/download/v2.3.3/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
 $ sudo chmod +x /usr/local/bin/docker-compose

Note

Replace the version v2.3.3 with the latest version available at the time of installation.

Installing L7|ESP Server

L7|ESP Deployment Bundle

Download and extract the archive provided to your by the L7 Informatics implementation team using roughly the following commands, but with the correct values where the dollar-signs are:

 $ wget https://$download_link
 $ tar xf $project-$version.tar.gz

L7|ESP Container Configuration

In the Deployment Bundle, there will be a Docker Compose configuration file that looks similar to the following:

 $ cat docker-compose.yml
 version: "3.9"
 services:
 server:
     image: l7esp/server:3.0.0-rc.1
     environment:
     L7ESP_PASSWORD: "${L7ESP_PASSWORD?Set password in .env file}"
     L7ESP_LICENSE_FILE: "/opt/l7esp/data/project/conf/esp.license"
     ports:
     - published: "${L7ESP_PORT?Set port in .env file}"
         target: 8002
         protocol: tcp
         mode: host
     volumes:
     - type: volume
         source: data
         target: /opt/l7esp/data
     - type: bind
         source: .
         target: /opt/l7esp/data/project
 volumes:
 data:

Things to note in the above configuration:

  • The L7ESP_PASSWORD and L7ESP_PORT environment variables are required and will be read from the .env file automatically. Make sure the .env file contains the initial administrator password you wish to use and that the port is set to 8002.

  • The exact image name will be provided to you by L7 Informatics who will be able to set you up with access to pull the required image. This will require you to run the docker login command on the server. (Alternatively, if you have already downloaded an archive containing the image to use, you may use the docker load command.)

  • You should replace L7ESP_PASSWORD environment variable value with the initial administrator password you wish to use. This will be the admin@localhost user unless you override the username with the L7ESP_USER environment variable.

  • The published port 8002 will be the publicly exposed port that the L7|ESP web interface will be accessible on. It is recommended to set up a proxy server that provides TLS offloading before forwarding traffic to L7|ESP.

  • The data volume is where L7|ESP will persist all data created at runtime and should be a part of your backup plan. You can determine the exact location of this volume by using the docker inspect command on the running container.

  • The project bind-mount source path should point to the directory that contains your L7|ESP environment’s project configuration and content. This will be provided to you by L7 Informatics either as a tarball that can be extracted or as a Git repository that can be cloned, depending on the use-case.

If you wish to make changes to the docker-compose.yml file, please communicate these changes with the L7 Informatics implementation team so they can incorporate these into the next archive you receive, such that the changes are permanent and well tested. Alternatively, you may create your own copy of the docker-compose.yml file in a different directory if you wish to maintain this yourself, provided you adjust the project bind-mount source path to point to the archive directory.

Starting L7|ESP

From the same directory as the Docker Compose configuration file (docker-compose.yml), run the following command to start the L7|ESP container in the background:

 $ docker-compose up --detach

Next, run the following command to watch the logs as the container starts to ensure there are no errors:

 $ docker-compose logs --follow

Once the log output has stopped, you can issue the following command to check if the L7|ESP web interface is healthy:

 $ curl localhost:8002/health
 PASS

Finally, visit L7|ESP in your web browser of choice (Chrome is recommended):

http://SERVER_IP:8002

You should now be able to log in with the following credentials:

  • Username: admin@localhost (or L7ESP_USERNAME value)

  • Password: Password12! (or L7ESP_PASSWORD value)

Installing L7|ESP Content

So far, you should have the core L7|ESP server software running successfully. However, you will likely notice that the software is not licensed or otherwise configured correctly at this point.

The next step is to install your L7|ESP environment’s project configuration and content, provided to you by the L7 Informatics implementation team.

First, get a shell into the L7|ESP container using the following command:

 $ docker-compose exec server bash

Next, issue the the following commands to install the configuration and content:

 $ make install

You will now see the a series of installation logs as the L7|ESP SDK, written mostly in Ansible, performs a series of task, including but not limited to:

Synchronize source files with the corresponding target directories in the container Install your L7|ESP software license Install any curated L7|Hub content bundles Import custom configuration into the L7|ESP Config app Import custom content into the L7|ESP Builders section Hook any server extensions your configuration requires and reload services

Appendix

Migrating Data

If you are upgrading from an L7|ESP 2.x instance that was not containerized or otherwise need to restore the data volume from a backup, you can start the L7|ESP container while overriding the entrypoint command. This will disable the initialization process that usually runs, allowing you to perform the restoration.

Note

Replace server_base with installation directory where the previous L7|ESP server resides, which can be found by running the following command from the 2.x tarball directory:

 $ grep server_base roles/prod.yml

  • First, stop the existing L7|ESP 2.x server that you are planning on migration to containerized L7|ESP 3.x platform by running the following l7 stop command:

 $ /server_base/Lab7_ESP/current/bin/l7 stop $(/server_base/Lab7_ESP/current/bin/l7 status | grep -v database | grep RUNNING | awk '{print $1}')

  • Second, once the L7|ESP 2.x server has stopped, create a backup of the database and temporary copy of the Data directory that will be used during the restore process:

 $ /server_base/Lab7_ESP/current/sys/bin/pg_dump \
     --host localhost \
     --port 1487 \
     --dbname lab7 \
     --clean \
     --if-exists \
     --format=c > /tmp/backup/esp_db_$date.pgdump

 $ rsync \
     --verbose \
     --archive \
     --exclude 'log/' \
     --exclude 'run/' \
     --exclude 'database/' \
     --exclude 'conf/' \
     /server_base/Lab7_ESP/Data/ \
     /tmp/backup

  • Next, extract the L7|ESP 3.x deployment bundle tarball and change into the directory it unpacked (where docker-compose.yml resides) before running the rest of the commands:

 $ wget https://$download_link
 $ tar xf $project-$version.tar.gz
 $ cd $project/

Note

You will need to log into Docker Hub with the appropriate credentials before proceeding using the docker login command. Please contact L7 Informatics for more information.

The following command will launch a one-off L7|ESP 3.x container with the configuration specified in docker-compose.yml (mainly, there will be a Docker volume created at /opt/l7esp/data) but with the following settings overridden by command-line flags:

  • The entrypoint process will be overridden to the Bash shell, such that a) L7|ESP won’t automatically initialize itself and run the server processes, and b) instead you will be granted a shell session where you can perform the restoration process

  • The container will be run as the root user (UID 1) such that you don’t run into Linux permission issues given that the files mounted in from the host may be owned by user(s) not present inside the container and you will later have to adjust these permissions.

  • The container will be automatically removed when you leave the Bash shell such that it doesn’t remain in the stopped state on the host after the restoration process, as we only need this container during the restore process and any files you move into the data volume will remain since the volume will be persisted.

  • The backup taken earlier to /tmp/backup on the host machine will be bind-mounted to /tmp/restore in the container file system such that the files will be available for you to restore from once you have a shell session inside the container.

 $ docker-compose run \
     --entrypoint bash \
     --user root \
     --rm \
     --volume /tmp/backup:/tmp/restore server

Copy the backup you mounted into this container to the location you just freed up:

 $ rsync -avh /tmp/restore/* /opt/l7esp/data

Modify the Linux file ownership of the restored files to match the container’s default user:

 $ chown -Rfv l7esp:l7esp /opt/l7esp/data

The data has now been restored and you may now exit the one-off container. The container will be automatically removed but the volume with the restored data will remain:

 $ exit

Now you can exit the one-off container and it will be automatically removed. The data volume will be kept, and will now contain the data you restored from the previous L7|ESP instance.

$ docker-compose up --detach && docker-compose logs --follow

Once the logs have finished initialization login to the container console to complete the database restore.

$ docker-compose exec server bash

Use the following commands to stop the current ESP instance and perform the DB restore

$ l7 stop $(l7 status | grep -v database | grep RUNNING | awk '{print $1}')

$ psql -h localhost -p 1487 -d postgres << EOF
    DROP DATABASE lab7;
    CREATE DATABASE lab7;
    CREATE ROLE esp;
    GRANT ALL PRIVILEGES ON DATABASE lab7 TO esp;
EOF

$ pg_restore \
    --host localhost \
    --port 1487 \
    --dbname lab7 \
    --clean \
    --if-exists \
    --no-owner \
    /opt/l7esp/data/backup/esp_db_$date.pgdump

$ l7 init --yes

$ pg_esp_docker_migrate $server_base

$ exit

Note

Replace localhost and 1487 with the appropriate hostname and PGSQL port when using a different ESP

See the Starting L7|ESP section above for commands to run to start up L7|ESP 3.0. The database you restored from 2.x will be automatically migrated to the latest 3.0 schema upon startup and if there are any errors you will see those in the container logs.