Create Test

Test-driven development is highly recommended as good practice when using the L7|ESP SDK, so we’ll start our tutorial with testing. As a gentle introduction, let’s write a simple test to verify that L7|ESP is connected and accepting requests.

To do this task, create a test file in the ``tests`` directory named ``test_status.py``. The contents of that file should look something like:

# -*- coding: utf-8 -*-

import pytest

def test_status():
    import esp
    running = esp.status()
    assert running, 'Could not connect to L7|ESP via Python client!'

Now that you’ve written a test, run it using ``pytest``:

~$ pytest tests/test_status.py b
==================================== test session starts ====================================
platform darwin -- Python 3.7.1, pytest-3.10.0, py-1.7.0, pluggy-0.8.0 -- /usr/local/opt/python/bin/python3.7
cachedir: .pytest_cache
collected 1 item

tests/test_status.py::test_status Connection established!

PASSED

================================= 1 passed in 0.68 seconds ==================================

If L7|ESP was not running and the tests failed, you’d see the following:

~$ pytest tests/test_status.py b
==================================== test session starts ====================================
platform darwin -- Python 3.7.1, pytest-3.10.0, py-1.7.0, pluggy-0.8.0 -- /usr/local/opt/python/bin/python3.7
cachedir: .pytest_cache
collected 1 item

tests/test_status.py::test_status FAILED

================================= 1 failed in 0.68 seconds ==================================

Later in this tutorial, we’ll talk about different ways of structuring tests for various types of content definitions.

Step 1: Create test file for importing data

Now that we have our content defined, we can use L7|ESP SDK tools to write a simple test that will import these files and verify that they exist in the system. This test also enables us to verify that there are no syntax errors in our Workflow definitions.

Create the test file, tests/test_qc_workflow.py, to look like the following:

# -*- coding: utf-8 -*-

# imports
import os
import unittest

from . import CONFIG, RESOURCES, CONTENT
from esp.testing import ModelLoaderMixin

# tests
class TestQCWorkflows(ModelLoaderMixin, unittest.TestCase):
    IMPORT = dict(
        Workflow=[
            os.path.join(CONTENT, 'workflows', 'QC-Workflow.yml'),
        ],
        Protocol=[
            os.path.join(CONTENT, 'protocols', 'QC-Quantification.yml'),
            os.path.join(CONTENT, 'protocols', 'QC-Report.yml'),
        ],
        Pipeline=[
            os.path.join(CONTENT, 'pipelines', 'QC-Report-Pipeline.yml'),
        ]
    )

Run this test to get the following output:

~$ pytest tests/test_qc_workflow.py

================================== test session starts ===================================
platform darwin -- Python 3.7.1, pytest-4.1.0, py-1.7.0, pluggy-0.8.0 -- /usr/local/opt/python/bin/python3.7
cachedir: .pytest_cache
collected 1 item

tests/test_qc_workflow.py::TestQCWorkflows::test__content
INFO:root:Clearing existing content from database.
INFO:root:Creating Task: Generate QC Report
INFO:root:Creating Pipeline: QC Report Pipeline
INFO:root:Creating PipelineReport: QC Report
INFO:root:Creating Protocol: QC Quantification
INFO:root:Creating Protocol: QC Report
INFO:root:Creating Workflow: QC Workflow
Successfully imported config data.
PASSED

================================ 1 passed in 3.47 seconds ================================

From this test output, we can see that all of our content was imported correctly, and references within the files were properly defined.

Step 2: Create Experiment Configs for Testing

Along with simply testing Workflow definitions, we can also test that the Workflow works properly when used in an Experiment. To do this, we can use the same format as above, but add a DATA attribute to our test class for what Projects and Experiments need to be created. For this example, we’ll need to create a Project and two Experiments for testing out the Workflow.

Create the following config files for our two Experiments:

# Contents of tests/resources/QC-Test-1.yml

QC Experiment 1:
  submit: True
  project: QC Project
  workflow: QC Workflow

  samples:
    - ESP001
    - ESP002

  protocols:
    - QC Quantification:
        data:
          ESP001:
            Type: DNA
            Ratio: 1.8
          ESP002:
            Type: RNA
            Ratio: 1.9

# Contents of tests/resources/QC-Test-2.yml

QC Experiment 2:
  submit: True
  project: QC Project
  workflow: QC Workflow

  samples:
    - ESP003
    - ESP004

  protocols:
    - QC Quantification:
        data:
          ESP003:
            Type: DNA
            Ratio: 1.7
          ESP004:
            Type: RNA
            Ratio: 2.0

Step 3: Update Tests to Include Experiments

After creating these two configs, we can update our tests to run them after Workflows are created.

Update the original test file to look as follows:

# -*- coding: utf-8 -*-

# imports
import os
import unittest

from . import CONFIG, RESOURCES, CONTENT
from esp.testing import ModelLoaderMixin

# tests
class TestQCWorkflows(ModelLoaderMixin, unittest.TestCase):
    IMPORT = dict(
        Workflow=[
            os.path.join(CONTENT, 'workflows', 'QC-Workflow.yml'),
        ],
        Protocol=[
            os.path.join(CONTENT, 'protocols', 'QC-Quantification.yml'),
            os.path.join(CONTENT, 'protocols', 'QC-Report.yml'),
        ],
        Pipeline=[
            os.path.join(CONTENT, 'pipelines', 'QC-Report-Pipeline.yml'),
        ]
    )
    DATA = dict(
        Project=[
            {'name': 'QC Project'}
        ],
        Experiment=[
            os.path.join(RESOURCES, 'QC-Test-1.yml'),
            os.path.join(RESOURCES, 'QC-Test-2.yml'),
        ]
    )

Finally, run the updated test to get the following output:

~$ pytest tests/test_qc_workflow.py

================================== test session starts ===================================
platform darwin -- Python 3.7.1, pytest-4.1.0, py-1.7.0, pluggy-0.8.0 -- /usr/local/opt/python/bin/python3.7
cachedir: .pytest_cache
collected 1 item

tests/test_demos/test_quickstart.py::TestQuickstart::test__content
INFO:root:Clearing existing content from database.
INFO:root:Creating Task: Generate QC Report
INFO:root:Creating Pipeline: QC Report Pipeline
INFO:root:Creating PipelineReport: QC Report
INFO:root:Creating Protocol: QC Quantification
INFO:root:Creating Protocol: QC Report
INFO:root:Creating Workflow: QC Workflow
INFO:root:Creating Project: QC Project
INFO:root:Creating Sample: ESP001
INFO:root:Creating Sample: ESP002
INFO:root:Creating Experiment: QC Experiment 1
INFO:root:Submitting Experiment: QC Experiment 1
INFO:root:Creating SampleSheet: QC Experiment 1
INFO:root:Creating Sample: ESP003
INFO:root:Creating Sample: ESP004
INFO:root:Creating Experiment: QC Experiment 2
INFO:root:Submitting Experiment: QC Experiment 2
INFO:root:Creating SampleSheet: QC Experiment 2
Successfully imported config data.
PASSED

================================ 1 passed in 8.54 seconds ================================

From this test output, we can see that all of our testing data was defined correctly (along with our content).