Project Structure

The sections below detail an overview of the structure and usage patterns of the L7|ESP SDK. For information about the L7|ESP Python client, please see the documentation section. For an in-depth example of creating content using the L7|ESP SDK, see the Tutorial section.

At a high level, the L7|ESP SDK was designed to speed up and simplify the process of building content for the L7|ESP platform. It provides a set of tools that enable rapid content development and testing, and also provides a contract by which content developers, internal and external to L7 Informatics, can communicate ideas about content.

Files and Directories

To start understanding and using the L7|ESP SDK, let’s first go over the structure of the repository and important components:

Makefile - A file used primarily for administration of L7|ESP. With the Makefile, you can start, stop, and reset the application. You can also use it to run testing commands.
docker-compose.yml - A Docker Compose configuration file that creates a containerized development environment. This file will use Docker Compose for provisioning a development system to mirror the production environment as closely as possible.
roles/ - Custom Ansible roles for project deployments. General purpose roles will be defined in app/esp-content/roles, but roles specific to each project’s deployment can be included here. Also, this folder contains development (dev.yml) and production (prod.yml) playbooks to manage development/deployment environments.
conf/ - All L7|ESP configuration files and data sources (lab7.conf).
content/ – All content related to this project installation, including all importable configuration files and scripts.
extensions/ – All extension points installed in the software. These points are reserved for custom expressions and custom API endpoints.
patches/ – Any patches that need applied to this project installation. This can patch the core L7|ESP codebase or any configuration files.
tests/ – All client-side and server-side integration tests specific to a project’s workflows and configuration.

Organizing Content and Tests

Now that we’ve gone over some of the components at a high level, let’s dive into some detail about the internal structure of some of these subdirectories. Here’s a high-level layout of how the L7|ESP SDK directory structure might look for an L7|ESP SDK with a couple of L7|ESP Workflows and Protocols already defined:

sdk/
├── README.md
├── Makefile
├── docker-compose.yml
├── requirements.txt
├── pyproject.toml
├── docs/
├── conf/
│   └── lab7.conf
├── content/
│   ├── admin/
│   │   ├── Users.yml
│   ├── workflows/
│   │   ├── QC-Workflow.yml
│   ├── protocols/
│   │   ├── QC-Quantification.yml
│   │   └── QC-Report.yml
│   ├── pipelines/
│   │   └── QC-Report-Pipeline.yml
│   ├── tasks/
│   │   └── generate_qc_report.py
│   └── inventory/
│       ├── Sample-Types.yml
│       └── Container-Types.yml
├── extensions/
│   ├── server/
│   |   ├── expressions.py
│   |   ├── invokables.py
│   |   └── requirements.txt
│   └── client/
│       └── expressions.js
├── patches/
|   ├── conf/
|   │   └── logs.patch
|   └── esp/
|       └── sso.patch
├── roles/
│   └── container.yml
└── tests/
    ├── __init__.py
    ├── conftest.py
    ├── resources/
    │   └── EXP001.txt
    ├── test_status.py
    └── test_content.py

We’ll go over details for specific files later in this documentation.

Note

How you organize your tests and content directories is up to you, but the remainder of the repository must use the above structure (the L7|ESP SDK assumes a specific file structure).

Notes on Specific Files

Let’s go over some of the more important files in more detail. We’ve already covered the purpose of the Makefile and docker-compose.yml in the root directory of the repository, so let’s continue with others.

Configuration

pyproject.toml - A configuration file used for specifying high-level metadata about the Python project. In the [tool.pytest.ini_options] section, you can change the default options used by pytest. Usually, this file won’t need to be changed, and will only need updates if you wish to adjust how certain Python tools fuction. Here’s an example pyproject.toml file:
[tool.pytest.ini_options] addopts = "--log-level=INFO -v -s -p no:warnings" testpaths = [ tests ]
conf/lab7.conf - A file containing all L7|ESP configuration needed to run the application. For more information on this file, please see documentation for the L7|ESP platform.

Ansible Roles

roles/container.yml - An Ansible playbook used for configuring the container environment with common content used for seeding the application in both development and production environments. Here’s an example container playbook:

---
- hosts: localhost
  connection: local
  vars_prompt:
    - name: "l7esp_password"
      prompt: "Password for L7|ESP superuser: `admin@localhost`?"
      default: password
      private: yes
  roles:
    - role: l7esp_sdk
      vars:
        env: dev # This will allow database access from outside.
        db_archive: true # Whether to archive the database during installs.
      seed:
        - 'roles/seed/content.yaml' # List of seed files (from `roles/packages.yml`)
  tasks: [] # Any additional pre/post tasks you may have (optional)

Information about the seed section of this config is included in the Content section later in the documentation.

Testing

tests/conftest.py, tests/__init__.py - Configuration files used for testing content within the L7|ESP SDK. These files generally won’t need to be altered throughout L7|ESP SDK use.