Skip to main content

Examples

Miscellaneous Examples

Below detail code-heavy examples for how to use the client API. They’re not meant to be comprehensive, but rather give users a feel for how to traverse and interact with the API, particularly for filling data in LIMS workflows.

Create and Submit Experiment

Similarly to how you can create and submit an experiment via YAML configuration (see below), you can also control how experiments are managed in pure python.

from esp.models import Experiment

# create experiment with 2 new illumina library samples
exp = Experiment.create(
  name='My Experiment',
  workflow='Example Workflow',
  project='My Project',
  samples={
    'count': 3,
    'type': 'Illumina Library'
  }
)

# submit experiment and create sample sheet
exp.submit(sheet=True)

To submit a sample sheet with a specific name, you can use:

exp.subit(sheet='My Sheet Name')

Update Data in a SampleSheet

You can also use the client to update data in a sample sheet programmatically. For instance, here’s how you fill in data for a specific column in a sample sheet from an experiment.

from esp.models import Experiment

# get submitted experiment
exp = Experiment('My Experiment')

# get first protocol from experiment
sheet = exp.protocols[0]

# you can alternatively query via protocol name
sheet = exp.protocol('My First Protocol')

# show first few rows of data
sheet.head()

# set all values for a column in a worksheet
sheet['Column 1'] = 'test'

# save values in the application
sheet.save()

Create a Sample Sheet from Multiple Experiments Programmatically

In some contexts, it’s useful to combine experiments into a single sample sheet for data recording. You can do this in the client using the SampleSheet object.

from esp.models import Experiment, SampleSheet

# get experiment objects
exp1 = Experiment('Experiment 1')
exp2 = Experiment('Experiment 2')

# create sample sheet with experiments
sheet = SampleSheet.create(name='Combined Sheet', experiments=[exp1, exp2])

Change the Name of a Sample and add Tags

from esp.models import Sample

# get sample and make sure it exists
sample = Sample('ESP000001')
if not sample.exists():
  raise AssertionError('Sample does not exist!')

# change name and add tags
sample.name = 'ESP-1'
sample.tags.extend(['foo', 'bar'])

# save updates
sample.push()

Complete a Workflow Programmatically

In contexts where data entry can be automated, you can use the client to populate data for workflows. This can be particularly useful if users are migrating data from a legacy system and want to maintain data structures created throughout workflows and workflow chains. Here is a YAML config example for creating an experiment and filling out data:

name: MS001
workflow: Illumina Sequencing
project: Miseq Sequencing
samples:
    count: 2
    type: Extract
protocols:
  - Set Samples:
      complete: true
      data:
        Attachment: ${CWD}/runsheet.txt
        Note: ['Ready to go!', 'Ready!']

  - Create Illumina Library:
      complete: true
      load: ${CWD}/spreadsheet.txt'

  - Analyze Sequencing Results:
      run: true
      complete: true
      data:
        Container: 'MiSeq: Lane 1'
        Reference: ['GRCh38', 'GRCh37']

You can run this config with the client using the python client CLI:

~$ esp import experiment path/to/experiment.yaml

Or, you can run it in a python shell using Experiment.create():

from esp.models import Experiment

exp = Experiment.create('path/to/experiment.yaml')

You can also use the python client to control the process programmatically in a configuration-free way. Here is matching Python code to fill data in data via code:

from esp.models import Experiment

# create experiment with samples
exp = Experiment.create(
  name='MS001',
  workflow='Illumina Sequencing',
  project='Miseq Sequencing',
  samples={
    'count': 2,
    'type': 'Extract'
  }
)

# submit experiment to sheet
exp.submit()

# fill in data for first protocol
sheet = exp.protocol('Set Samples')
sheet['Attachment'] = 'runsheet.txt'
sheet.loc[:, 'Note'] = ['Ready to go!', 'Ready!']

# save and complete sheet
sheet.save()
sheet.complete()

# fill in data for second protocol
sheet = exp.protocol('Create Illumina Library')
sheet.load('spreadsheet.txt', sep='\t')
sheet.save()
sheet.complete()

# fill in data for final (pipeline) protocol
sheet = exp.protocol('Analyze Sequencing Results')
sheet['Container'] = 'Miseq: Lane 1'
sheet.loc[:, 'Reference'] = ['GRCh38', 'GRCh37']
sheet.save()

# run pipeline for sheet
sheet.run()

# complete sheet
sheet.complete()

Complete a Workflow Chain Programmatically

Just like with workflows, it’s possible to complete a workflow chain programmatically or with configuration. Here is an example config for filling workflow chain data:

name: MF Chain 1
chain: Manufacturing Chain
project: Manufacturing
samples:
    count: 3
    type: Tissue
workflows:
    - Initiate Manufacturing
        submit: true
        transition: true
        protocols:
            - Initiate Manufacturing:
                complete: true
                data:
                  Start Date: 01Jan18
                  Signature: true
    - Finish Manufacturing:
        submit: true
        protocols:
            - Process A:
                data:
                  Completion Date: 01Jan18
                  Signature: true

You can run this config with the client using the python client CLI:

~$ esp import experiment_chain path/to/experiment_chain.yaml

Or, you can run it in a python shell using ExperimentChain.create():

from esp.models import ExperimentChain

exp = ExperimentChain.create('path/to/experiment_chain.yaml')

You can also use the python client to control the process programmatically in a configuration-free way. Here is matching Python code to fill data in data via code:

from esp.models import ExperimentChain

# create experiment with samples
chain = ExperimentChain.create(
  name='MF Chain 1',
  workflow=' Manufacturing Chain',
  project='Manufacturing',
  samples={
    'count': 3,
    'type': 'Tissue'
  }
)

# get first node in chain and submit
exp = chain.experiments[0]
exp.submit()

# fill data for first experiment sample sheet
sheet = exp.protocols[0]
sheet['Start Date'] = '2018-01-01'
sheet['Signature'] = True

# submit sheet and transition experiment chain
sheet.save()
sheet.complete()
sheet.transition()

# get second node in chain and submit
exp = chain.experiments[1]
exp.submit()

# fill in data for second node in chain
sheet = exp.protocols[0]
sheet['Completion Date'] = '2018-01-01'
sheet['Signature'] = True
sheet.save()
sheet.complete()

Ingest Data from an External Service

The following code details how to ingest data from an arbitrary process into ESP via the LIMS application.

# Make sure protocol and workflow exist (ESP requirement for adding data)
pc = Protocol.create(name='Ingest Protocol')
wf = Workflow.create(name='Ingest Workflow', protocols=[pc])

# Read Ingest data from File
import pandas
df = pandas.read_csv('iris.txt', sep='\t')

# Configure ingest protocol to take new columns
# (accounting for column differences across files)
pc.variables = [{'name': col, 'var_type': 'text'} for col in df.columns]
pc.push()

# Create experiment and push data
Experiment.create(
    name='My Ingest',
    workflow=wf,
    samples=dict(count=len(df)),
    submit=True,
    protocols=[dict(name='Ingest Protocol', data=df)])

Find and print the creation date/creator of all samples with the tag “test”

Sometimes it’s useful to query all samples in the application and filter by some criteria. Here’s how to do that using Sample tags:

for sample in Sample.all():
    if 'test' in sample.tags:
        print('{}: {} by {}'.format(sample.name, sample.created_at, sample.owner.name))

Alternatively, you can search by tag using the Sample.search classmethod:

samples = Sample.search(tags=['test'])

Note

The .search() method works for other models as well.

Find and Submit all Experiments with >10 Samples from a Specific Project

You can also use the client for managing project data. For instance, here’s an example of searching for experiments using specific criteria, and then submitting them:

from exp.models import Project
obj = Project('My Project')
for exp in obj.experiments:
    if len(exp.samples) > 10 and not exp.submitted:
        exp.submit()

Do a PCA with data from a specific Experiment

Here’s a more comprehensive example of how one might use the client for data science applications. Below, the client is used to pull all metadata within a project to perform a Principal Component Analysis:

# imports
import pandas as pd
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.pipeline import make_pipeline

# load the project
project = Project('My Project')
project.experiments

# get the data
df = pd.concat(map(
    lambda x: x.protocols[0].df,
    project.experiments
)).drop(['Complete'], axis=1)

# fit model on the data
X, Y = df.drop(['Species'], axis=1).values, df.Species.values
model = make_pipeline(
    StandardScaler(),
    PCA(n_components=3),
)
tX = model.fit_transform(X)

Note

More examples will be included in future versions of the documentation.

Model Configuration

User

class esp.models. User ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with users from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Create user:name: New User email: user@localhost password: passwordCreate user with roles:name: New Role User email: user-role@localhost password: password roles: - Lab Manager - AdminExamples>>> from esp.models import User >>> user = User('New User') >>> user.name, user.created_at ('User', '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> user.roles [<Role(name=Lab Manager)>, <Role(name=Admin)>] >>> user.workgroups [<Workgroup(name=Lab A)>] >>> # log in as specific user >>> User('New User').login(password='password') >>> # get current user >>> user = User.current()Parameters:ident (str) – Name or uuid for object.

Role

class esp.models. Role ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with users from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Create role:name: my-role desc: My Role permissions: Project: [create, update] Sample: [create, read, update, delete]Create role with default users:name: my-role desc: My Role members: - User 1 - User 2 applications: - admin - limsCreate role with specific app permissions:name: my-role desc: My Role applications: - admin - limsConfiguration Notes:Default role permissions are access to all actions for each type of model. Permissions need to be overridden to restrict access.Examples>>> from esp.models import Role >>> role = Role('my-role') >>> role.name, role.created_at ('my-role, '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> role.users [<User(name=User 1)>, <User(name=User 2)>]Parameters:ident (str) – Name or uuid for object.

ParamGroup

class esp.models. ParamGroup ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with ParamGroups from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Create param group:illumina: miseq-1: /path/to/miseq-1 miseq-2: /path/to/miseq-2Create multiple param groups:illumina: miseq-1: /path/to/miseq-1 miseq-2: /path/to/miseq-2 integrations: clarity: http://clarity-url:1000 internal: http://internal-serverExamples>>> from esp.models import ParamGroup >>> pg = ParamGroup('apps') >>> pg.name, group.created_at ('apps, '2019-06-21T16:04:01.199076Z') >>> # show params >>> pg.params.client '/path/to/client/bin/esp'Parameters:ident (str) – Name or uuid for object.

Workgroup

class esp.models. Workgroup ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with workgroups from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Create workgroup:name: Lab A desc: Workgroup for all users in Lab ACreate workgroup with default users:name: Lab A desc: Workgroup for all users in Lab A members: - User 1 - User 2Examples>>> from esp.models import Workgroup >>> group = Workgroup('Lab A') >>> group.name, group.created_at ('Lab A, '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> group.users [<User(name=User 1)>, <User(name=User 2)>]Parameters:ident (str) – Name or uuid for object.

Config

class esp.models. Config (*args, **kwargs)

Object for interacting with instance configuration within ESP. This model allows developers to directly interact with the /api/config endpoint in ESP.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.That is, this configuraiton is always updaitng a DB row, never truly creating it, but uses the same interface as all other models for consistency.This model is deprecated and will be removed in the future. Use Configuration instead.

Configuration

class esp.models. Configuration ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with database-backed instance configuration within ESP. This model allows developers to directly interact with the /api/configuration endpoint in ESP.Model to interact with ESP’s DB-backed configuration system.A configuration consists of two top-level dictionaries: one for “public” configuration and one for “secret” configuration. Generally, this model masks the distinction and makes the subkeys of both available as top-level properties, but also supports direct access of the public and secret dictionaries.Examples>>> from esp.models import Configuration >>> conf = Configuration('esp') # get the ESP system config. >>> conf.executor.num_workers 1 >>> conf = Configuration.create(name='CustomConfig', key1='value1', key2={'subkey1': 'value1.1', 'subkey2': 'value2.2'}) >>> conf.key2 {'subkey1': 'value2.1', 'subkey2': 'value2.2'} >>> conf.key2.subkey1 'value2.1' >>> conf.config {'key1': 'value2', 'key2': {'subkey1': 'value2.1', 'subkey2': 'value2.2'}} >>> conf.add_secret('secretkey', 'this value is secret') >>> conf.secrets {'secretkey': 'this value is secret'} >>> conf.push() >>> conf.refresh() >>> conf.secrets {'secretkey': 'XXXXXXX'} >>> conf.secretkey 'XXXXXXX' >>> conf.newkey = 'new value' >>> conf.config {'key1': 'value2', 'key2': {'subkey1': 'value2.1', 'subkey2': 'value2.2'}, 'newkey': 'new value'} >>> conf.add_secret('newkey', 'conflicting secret') >>> conf.newkey 'conflicting secret' >>> conf.config.newkey 'newkey'

File

class esp.models. File ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with files from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Create file:name: My Protocol SOP desc: An SOP file tags: [sop, protocol] uri: relative/path/to/instructions.pdf upload: true type: PDFCreate file object with reference to existing file:name: Large Sequencing File desc: An SOP file tags: [sop, protocol] uri: file:///path/to/local/file/instructions.bam upload: true type: rawCreate file object and register as task output:name: SampleSheet.csv desc: Illumina Run Sheet task_file: Illumina Runsheet uri: /path/to/SampleSheet.csv upload: false type: csvConfiguration Notes:Due to current backend limitations, uri inputs can only take the formats: “relative/path”, “/absoluate/path”, and “file:///absolute/path”.If upload is true, uri must resolve to an existing file on the local file system, otherwise an error will be raised. If upload is False, file will only be registered. Upload defaults to false.The task_file parameter is primarily useful when registering pipeline output files (since you can pass in the task instance UUID).“type” is used internally by ESP and is not necessary a mime type. For instance, the mime type of a bam file is generally application/gzip or application/octet-stream, but the type in ESP might normally be “bam”.mime-type is an acceptable key for specifying the mime-type. If no mime-type is provided, the client will attempt to guess the type and pass that type along as the content type of the uploaded file.Examples>>> from esp.models import File >>> fi = File('My Protocol SOP') >>> fi.name, fi.created_at ('My Protocol SOP', '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> fi.path '/path/to/Lab7_ESP/Data/files/0000/instructions.pdf.6a523a74-6703-4474-a24e-5a463b9d9770' >>> fi.contents '... raw file contents ...' >>> fi.download('local-copy.pdf')Parameters:ident (str) – Name or uuid for object.

Report

class esp.models. Report ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with Reports from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Create report with embedded html:name: My Report desc: An example html report. tags: [html, demo] contents: <h1>My Report</h1>Create report with local html file:name: My Report desc: An example html report. tags: [html, demo] contents: $LAB7DATA/contents/reports/my-report.htmlCreate applet from local html file:name: My Report desc: An example html applet. tags: ['esp:applet'] contents: $LAB7DATA/contents/reports/my-report.htmlConfiguration Notes:To promote an ESP report to an ESP applet, include esp:applet in the set of report tags.The contents parameter can either take a local file or raw html contents to use as the report data. If no file is found from the string, the data are assumed to be raw html data to include as contents.Examples>>> from esp.models import Report >>> report = Report('My Report') >>> report.name, report.created_at ('My Report', '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> report.contents '<h1>My Report</h1>'Parameters:ident (str) – Name or uuid for object.

Pipeline

class esp.models. Pipeline ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with Pipelines from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Create pipeline with single task:name: Generate Illumina Runsheet desc: Run script to generate illumina runsheet and generate runsheet report tasks: - Create Illumina RunsheetCreate pipeline with nested tasks:name: Generate Illumina Runsheet desc: Run script to generate illumina runsheet and generate runsheet report tasks: - Create Illumina Runsheet: desc: Run script to generate illumina runsheet. cmd: /path/to/script.py {{project}} > runsheet.xml files: - Illumina Runsheet: file_type: xml filename_template: '{{ "runsheet.xml" }}'Create pipeline with task dependencies:name: Generate Illumina Runsheet desc: Run script to generate illumina runsheet and generate runsheet report tasks: - Create Illumina Runsheet - Upload Illumina Runsheet deps: Upload Illumina Runsheet: Create Illumina RunsheetCreate pipeline with report:name: Generate Illumina Runsheet desc: Run script to generate illumina runsheet and generate runsheet report report: Runsheet Report: desc: Report showing details of runsheet generation elements - - type: file_details depends: file: Illumina Runsheet tasknumber:1 - [] tasks: - Create Illumina Runsheet - Upload Illumina Runsheet deps: Upload Illumina Runsheet: Create Illumina RunsheetCreate pipeline with report and failure report:name: Generate Illumina Runsheet desc: Run script to generate illumina runsheet and generate runsheet report report: Runsheet Report: desc: Report showing details of runsheet generation elements - - type: file_details depends: file: Illumina Runsheet tasknumber:1 - [] failure_report: Runsheet Failure Report: desc: Report showing details of failed runsheet generation elements - - type: file_details depends: file: stderr tasknumber:1 - [] tasks: - Create Illumina Runsheet - Upload Illumina Runsheet deps: Upload Illumina Runsheet: Create Illumina RunsheetConfiguration Notes:Current implementation requires special syntax around creating report elements - - and - []. This is due to payload requirements and will be updated in future versions to be more user-friendly.Examples>>> from esp.models import Pipeline >>> pipe = Pipeline('Generate Illumina Runsheet') >>> pipe.name, pipe.created_at ('Generate Illumina Runsheet', '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> pipe.tasks [<Task(name=Create Illumina Runsheet)>, <Task(name=Upload Illumina Runsheet)>] >>> pipe.report <ReportTemplate(name=Runsheet Report)> >>> # run pipeline (wait to finish) >>> analysis = pipe.run(project='My Project') >>> analysis.task[0].stdout 'stdout from the first analysis task' >>> analysis.task[0].files [<File(name=Illumina Runsheet)>] >>> # run pipeline (in background) >>> analysis = pipe.run(project='My Project', background=True) >>> analysis.pause() >>> analysis.restart() >>> analysis.kill()Parameters:ident (str) – Name or uuid for object.

Task

class esp.models. Task ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with Tasks from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Simple task (no variables):name: Ping Internal Server desc: Run command to ping internal server. cmd: curl http://internal-serverSimple task with variables:name: Ping Internal Server desc: Run command to ping internal server. cmd: curl '{{ server }}'Create task and track outputs:name: Create Illumina Runsheet desc: Run script to generate illumina runsheet. cmd: /path/to/generate_runsheet.py {{project}} > runsheet.xml files: - Illumina Runsheet: file_type: xml filename_template: '{{ "runsheet.xml" }}'Create task with inline code:name: Report DNA Type desc: Run bash script to generate simple report. cmd: |+ # Simple Task that determines if the specified 'ratio' is in the proper range if [ "{{ type }}" = "DNA" ]; then echo "<b>This is a <font color='green'>DNA</font> sample</b>" > result.html elif "{{ type }}" = "DNA" ]; then echo "<b>This is a <font color='green'>RNA</font> sample</b>" > result.html fi files: - Type Report: file_type: html filename_template: '{{ "report.html" }}'Examples>>> from esp.models import Task >>> task = Task('Create Illumina Runsheet') >>> task.name, task.created_at ('Create Illumina Runsheet', '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> task.variables ['project'] >>> task.cmd '/path/to/generate_runsheet.py {{project}} > runsheet.xml'Parameters:ident (str) – Name or uuid for object.

PipelineReport

class esp.models. PipelineReport ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with PipelineReports from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Create pipeline with report:name: Generate Illumina Runsheet desc: Run script to generate illumina runsheet and generate runsheet report report: Runsheet Report: desc: Report showing details of runsheet generation elements - - type: file_details depends: file: Illumina Runsheet tasknumber:1 - [] tasks: - Create Illumina RunsheetCreate multi-step pipeline with report:name: Generate Illumina Runsheet desc: Run script to generate illumina runsheet and generate runsheet report report: Runsheet Report: desc: Report showing details of runsheet generation elements: - - type: file_details depends: file: Illumina Runsheet tasknumber:1 - - type: raw_file depends: file: Runsheet Upload Report tasknumber:2 - - type: html contents: |+ <h1>Report Header</h1> <p>Report Body</p> - [] tasks: - Create Illumina Runsheet - Upload Illumina Runsheet deps: Upload Illumina Runsheet: Create Illumina RunsheetConfiguration Notes:Report configuration will generally happen in the context of a pipeline, Accordingly, this documentation references report generation in that context.Current implementation requires special syntax around creating report elements - - and - []. This is due to payload requirements and will be updated in future versions to be more user-friendly.Examples>>> from esp.models import Pipeline >>> pipe = Pipeline('Create Illumina Runsheet') >>> pipe.name, pipe.created_at ('Generate Illumina Runsheet', '2019-06-21T16:04:01.199076Z') >>> pipe.report.name, pipe.created_at ('Runsheet Report', '2019-06-21T16:04:01.199076Z') >>> pipe.report.pipeline.name, pipe.report.pipeline.created_at ('Generate Illumina Runsheet', '2019-06-21T16:04:01.199076Z')Parameters:ident (str) – Name or uuid for object.

Analysis

class esp.models. Analysis ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with analyses from the ESP database.Examples>>> from esp.models import Pipeline >>> pipe = Pipeline('Generate Illumina Runsheet') >>> # run pipeline to return analysis object >>> analysis = pipe.run(project='My Project') >>> analysis.task[0].stdout 'stdout from the first analysis task' >>> analysis.task[0].files [<File(name=Illumina Runsheet)>] >>> # run pipeline (in background) >>> analysis = pipe.run(project='My Project', background=True) >>> analysis.pause() >>> analysis.restart() >>> analysis.kill() >>> # get current analysis (within script) >>> analysis = Analysis.current() >>> analysis.pause()Parameters:ident (str) – Name or uuid for object.

AnalysisTask

class esp.models. AnalysisTask ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with task instances from the ESP database.Examples>>> from esp.models import Pipeline >>> pipe = Pipeline('Generate Illumina Runsheet') >>> # run pipeline to return analysis object >>> analysis = pipe.run(project='My Project') >>> # get first task object from analysis >>> atask = analysis.task[0] >>> atask.stdout 'stdout from the first analysis task' >>> atask.stderr 'stderr from the first analysis task' >>> atask.files [<File(name=Illumina Runsheet)>]Parameters:ident (str) – Name or uuid for object.

AnalysisPool

class esp.models. AnalysisPool ( analyses )

Object for interacting with analyses from the ESP database.Examples>>> from esp.models import Pipeline >>> pipe = Pipeline('Generate Illumina Runsheet') >>> # run multiple scripts to gather in pool >>> foo_analysis = pipe.run(project='Foo Project') >>> bar_analysis = pipe.run(project='Bar Project') >>> # create analysis pool to control execution >>> pool = AnalysisPool([foo_analysis, bar_analysis]) >>> pool.state 'running' >>> # wait for all analyses to finish in pool >>> pool.join() >>> pool.state 'done'Parameters:analyses (str) – List of analyses to include in pool.

ContainerType

class esp.models. ContainerType ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with container types from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Single-element container:name: 96-Well Plate desc: 96-Well Plate for sample aliquots tags: [plate, lab] label_format: '%s%s' slot_capacity: element axes: - Rows: [A, B, C, D, E, F, G, H] - Cols: [01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11, 12] contains: - SampleMulti-element container:name: Freezer desc: Freezer for samples and racks tags: [freezer, lab] label_format: 'Shelf %s' slot_capacity: list axes: - Shelf: [1, 2, 3, 4, 5] contains: - Sample - ContainerType with default containers:name: Freezer desc: Freezer for samples and racks tags: [freezer, lab] label_format: 'Shelf %s' slot_capacity: list axes: - Shelf: [1, 2, 3, 4, 5] contains: - Sample - Container create: - Freezer 1 - Freezer 2Nested config with containers and samples to fill:name: Freezer desc: Freezer for samples and racks tags: [freezer, lab] label_format: 'Shelf %s' slot_capacity: list axes: - Shelf: [1, 2, 3, 4, 5] contains: - Sample - Container create: - Freezer 1 - name: Freezer 2 barcode: 1234 fill: Shelf 1: [ESP0001, ESP0002]Examples>>> from esp.models import ContainerType >>> ct = ContainerType('Freezer') >>> ct.name, ct.created_at ('Freezer', '2019-06-21T16:04:01.199076Z') >>> ct.containers [<Container(name=Freezer 1)>, <Container(name=Freezer 2)>]Parameters:ident (str) – Name or uuid for object.

Container

class esp.models. Container ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with containers from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Create container:name: Freezer 1 desc: Freezer in lab A tags: [freezer, lab] type: Freezer barcode: 12345Container with samples to fill:name: Freezer 1 desc: Freezer in lab A tags: [freezer, lab] type: Freezer barcode: 12345 fill: Shelf 1: - ESP0001 - name: ESP0002 desc: special new sample tags: [one, two]Configuration Notes:Upon creation, samples can be used to fill container slots via the fill parameter.Default container values can be set for samples using the variables parameter.Examples>>> from esp.models import Container >>> ct = Container('Freezer 1') >>> ct.name, ct.created_at ('Freezer 1', '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> ct.slots ['Shelf 1', 'Shelf 2', 'Shelf 3', 'Shelf 4', 'Shelf 5'] >>> ct.samples [<Sample(name=ESP0001)>, <Sample(name=ESP0002)>] >>> ct.slot_samples { 'Shelf 1': [<Sample(name=ESP0001)>, <Sample(name=ESP0002)>], 'Shelf 2': [], 'Shelf 3': [], ... } >>> # put items in specific slots >>> ct.put_items_in_slots( >>> items=[Sample('ESP0001'), Sample('ESP0002')], >>> slots=['Shelf 1', 'Shelf 2'] >>> )Parameters:ident (str) – Name or uuid for object.

Customer

class esp.models. Customer ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with Customers from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Simple customer:name: Customer 1 desc: Customer 1 from the North Pole. tags: [demo, north] billing_address: street1: 1234 Northword Lane street2: Suite # 123 city: North Pole state: North Pole country: North Pole postal_code: North PoleCustomer without address:name: Customer 2 desc: Customer 2 without address. tags: [demo, north]Customer with billing and shipping address:name: Customer 3 desc: Customer 3 from the North Pole tags: [demo, north] billing_address: street1: 1234 Northword Lane street2: Suite # 123 city: North Pole state: North Pole country: North Pole postal_code: North Pole shipping_address: street1: 1234 Northword Lane street2: Suite # 123 city: North Pole state: North Pole country: North Pole postal_code: North PoleConfiguration Notes:Customer addresses are optional, and the minimal config for a customer is one with only a specified name parameter.Parameters:ident (str) – Name or uuid for object.

ItemType

class esp.models. ItemType ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with ItemTypes from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Item type for stock reagent:name: Demo Reagent consumption_type: stock reagent units: uL reorder_threshold_qty: 100 reorder_amount_qty: 50 vendors: - AgilentItem type for consumable:name: Demo Consumable consumption_type: consumable units: uL reorder_threshold_qty: 100 reorder_amount_qty: 50 vendors: - AgilentItem type for kit component:name: Demo Kit Component consumption_type: kit component units: uL reorder_threshold_qty: 100 reorder_amount_qty: 50 vendors: - AgilentCreate item type with associated items:name: Consumable consumption_type: consumable units: uL reorder_threshold_qty: 100 reorder_amount_qty: 50 vendors: - Agilent create: - CNS001 - CNS002Examples>>> from esp.models import ItemType >>> it = ItemType('Demo Consumable') >>> it.name, it.created_at ('Demo Consumable', '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> it.vendors [<Vendor(name=Agilent)>]Parameters:ident (str) – Name or uuid for object.

Item

class esp.models. Item ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with ItemTypes from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Simple item:name: Example Item tags: [agilent, perishable] barcode: 45262341 barcode_type: QR item_type: Demo Consumable vendor: Agilent lot_id: Lot # 1 serial_id: 12345 initial_qty: 10 expires: 12/25/2019 status: QuarantinedItem with different barcode type:name: Example Item tags: [agilent, perishable] barcode: 1234 barcode_type: 1D item_type: Demo Consumable vendor: Agilent lot_id: ABCExamples>>> from esp.models import Item >>> item = Item('Example Item') >>> item.name, item.created_at ('Example Item', '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> it.vendor <Vendor(name=Agilent)> >>> # useful functions >>> item.expired() False >>> item.consumed() True >>> item.consumed() True >>> # interacting with item >>> item.add_quantity(10) >>> item.remove_quantity(5) >>> item.empty() >>> # interacting with variables >>> item = Item('Example Item', params='["resource_vals"]') >>> item.variables >>> item.variables['Example Variable'] >>> # adding item with variables >>> item_to_add = { >>> 'name': 'The Name of the Item', >>> 'description': 'Item Description', >>> 'item_type': 'Item Type Name', >>> 'initial_qty': 22, >>> 'status': 'Quarantined', >>> 'resource_vals': [{'name': 'The name of custom field', 'value': 'The Value'}] >>> } >>> Item.create(item_to_add)Parameters:ident (str) – Name or uuid for object.

ServiceType

class esp.models. ServiceType ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with ServiceTypes from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Sample service type:name: Sample Service Type desc: Service type for a demo service. tags: [service, demo] price_type: sample base_price: 350Time service type:name: Time Service Type desc: Service type for a time service. tags: [service, demo] price_type: time base_price: 350 scale_price: 25 scale_units: minutesConfiguration Notes:Available options for ServiceType price_type parameters are: sample, protocol, workflow, time, or volume. However, only sample and time are currently supported in the UI.Available options for ServiceType scale_units parameters are: hour, minutes, and seconds.Parameters:ident (str) – Name or uuid for object.

Service

class esp.models. Service ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with Service from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Simple service:name: My Service service_type: My Service Type customer: Lab1Service with embedded type and customer:name: My Service service_type: My Service Type: price_type: time base_price: 350 scale_price: 25 scale_units: minutes customer: Lab1: billing_address: street1: 1234 Northword Lane street2: Suite # 123 city: North Pole state: North Pole country: North Pole postal_code: North PoleExamples>>> from esp.models import Service >>> service = Service('My Protocol SOP') >>> service.name, service.created_at ('My Service', '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> service.service_type.name 'My Service Type' >>> service.customer.name 'Customer 1'Parameters:ident (str) – Name or uuid for object.

Vendor

class esp.models. Vendor ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with Vendors from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Simple vendor:name: Vendor 1 desc: Vendor 1 from the North Pole. tags: [demo, north] billing_address: street1: 1234 Northword Lane street2: Suite # 123 city: North Pole state: North Pole country: North Pole postal_code: North PoleVendor without address:name: Vendor 2 desc: Vendor 2 without address. tags: [demo, north]Vendor with billing and shipping address:name: Vendor 3 desc: Vendor 3 from the North Pole tags: [demo, north] billing_address: street1: 1234 Northword Lane street2: Suite # 123 city: North Pole state: North Pole country: North Pole postal_code: North Pole shipping_address: street1: 1234 Northword Lane street2: Suite # 123 city: North Pole state: North Pole country: North Pole postal_code: North PoleConfiguration Notes:Vendor addresses are optional, and the minimal config for a customer is one with only a specified name parameter.Parameters:ident (str) – Name or uuid for object.

Project

class esp.models. Project ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with Projects from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Simple project:name: My Project desc: Project for a series of experiments. tags: [demo, test]Project with nested experiments:name: My Project desc: Project for a series of experiments. tags: [demo, test] experiments: - My Experiment 1: workflow: Workflow 1 submit: true samples: [ESP001, ESP002]Project with nested experiment chains:name: My Project desc: Project for a series of experiments. tags: [demo, test] chains: - My Experiment Chain 1: chain: Workflow Chain 1 samples: [ESP001, ESP002]Configuration Notes:Experiments and Experiment Chains can be nested within project configuration, which is particularly useful for seeding data in bulk or creating test resources.Examples>>> from esp.models import Project >>> project = Project('Project 1') >>> project.name, project.created_at ('Project 1', '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> project.experiments [<Experiment(name=Experiment 1)>, <Experiment(name=Experiment 2)>] >>> project.samples [<Sample(name=ESP001)>, <Sample(name=ESP001)>, ...]Parameters:ident (str) – Name or uuid for object.

ExperimentChain

class esp.models. ExperimentChain ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with ExperimentChains from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Create simple chain:name: MF Chain 1 chain: Manufacturing Chain project: Manufacturing samples: count: 3 type: TissueCreate chain with nested experiments:name: MF Chain 1 chain: Manufacturing Chain project: Manufacturing samples: count: 3 type: Tissue workflows: - Initiate Manufacturing submit: true transition: true protocols: - Initiate Manufacturing: complete: true data: Start Date: 01Jan18 Signature: true - Finish Manufacturing: submit: true protocols: - Process A: data: Completion Date: 01Jan18 Signature: trueConfiguration Notes:The workflows parameter in the config definition references nested configuration for specific Experiment objects.To automatically transition across workflows via configuration, nodes with possible transitions need to include the transition: true configuration (seen above).Experiment Chain configuration must include a referenced project, per constraints on Experiment Chains in ESP.Examples>>> from esp.models import ExperimentChain >>> chain = ExperimentChain('MF Chain 1') >>> chain.name, chain.created_at ('MF Chain 1', '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> chain.samples [<Sample(name=ESP001)>, <Sample(name=ESP001)>, ...] >>> chain.project <Project(name=Project 1)> >>> chain.experiments [<Experiment(name=Experiment 1)>, <Experiment(name=Experiment 2)>] >>> chain.chain <WorkflowChain(name=Manufacturing Chain)> >>> chain.workflows [<Workflow(name=Initiate Manufacturing)>, <Workflow(name=Finish Manufacturing>] >>> # helper methods >>> chain.active True >>> chain.submit() # submit workflow chain >>> chain.fill(...) # fill with workflows config aboveParameters:ident (str) – Name or uuid for object.

Experiment

class esp.models. Experiment ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with Experiments from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Create simple experiment:name: MS001 workflow: Illumina Sequencing project: Miseq Sequencing samples: count: 3 type: ExtractCreate experiment and fill in data:name: MS001 workflow: Illumina Sequencing project: Miseq Sequencing samples: count: 2 type: Extract protocols: - Set Samples: complete: true data: Attachment: ${CWD}/runsheet.txt Note: ['Ready to go!', 'Ready!'] - Create Illumina Library: complete: true load: ${CWD}/spreadsheet.txt' - Analyze Sequencing Results: run: true complete: true data: Container: 'MiSeq: Lane 1' Reference: ['GRCh38', 'GRCh37']Create experiment and fill in data with verification:name: MS001 workflow: Illumina Sequencing project: Miseq Sequencing samples: count: 2 type: Extract protocols: - Set Samples: complete: true before: verify: Attachment: '{{ value is None }}' data: Attachment: ${CWD}/runsheet.txt Note: ['Ready to go!', 'Ready!'] after: verify: Attachment: '{{ value is not None }}' - Create Illumina Library: complete: true cols: - Index Type - I7 Index ID - I5 Index ID actions: - data: - ['TruSeq Amplicon', 'A701', 'A501'] - ['TruSeq Amplicon', 'A702', 'A502'] verify: Index Type: 'TruSeq Amplicon' - Analyze Sequencing Results: run: true complete: true data: Amount: 10 Reference: ['GRCh38', 'GRCh37'] verify: Amount: '{{ value >= 10 }}' Reference: ['GRCh38', 'GRCh37']Configuration Notes:There are several hooks for entering data in workflows, and each of them are useful in different contexts. At a high-level, data are entered in order of the block they’re specified in. An overview of the blocks are as follows:before - Perform actions before any other actions are perform.actions - List of action blocks to perform (as a list).after - Perform actions after any other actions are perform.If no before, actions, or after blocks are specified, then the parameters included in the config will be run in a single action. See the examples above for context.Available actions that can be performed when filling in data are:data - Fill in data specified in configuration. Data can takethe format of a dictionary of values (fill in all rows with the specified value), or a dictionary of lists (use the list indices to fill in individual rows).load - Load tabular file into worksheet, matching file headerswith protocol columns.user/password - Log in as specific user before entering data.verify - Run assertions on data after data are entered intoa worksheet. This action is particularly useful for verifying that expressions work as expected.run - Start a pipeline from a pipeline protocol. This option willrun all pipelines specified for the worksheet protocol.wait - Wait a number of seconds before performing the next action.To automatically create Worksheets from experiments, use the submit: true parameter.To complete a protocol at the end of filling in data, use the complete: true parameter.Examples>>> from esp.models import Experiment >>> exp = Experiment('MS001') >>> exp.name, exp.created_at ('MS001', '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> exp.samples [<Sample(name=ESP001)>, <Sample(name=ESP001)>, ...] >>> exp.project <Project(name=Project 1)> >>> exp.workflow <Workflow(name=Initiate Manufacturing)> >>> # accessing worksheet data >>> exp.protocols[0] Sample ID Column 1 Column 2 ESP001 1.2 foo ESP002 1.3 bar >>> exp.protocol('Set Samples')['Column 1'] [1.2, 1.3] >>> # helper methods >>> exp.active True >>> exp.submit() # submit workflow >>> exp.fill(...) # fill with workflow config above >>> exp.archive()Parameters:ident (str) – Name or uuid for object.

SampleSheet

class esp.models. SampleSheet ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with SampleSheets from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Create sample sheet with submitted experiments:name: Sheet 001 desc: Sample sheet with experiments experiments: - Experiment 1 - Experiment 2Create sample sheet nested experiments:name: Sheet 001 desc: Sample sheet with experiments experiments: - Experiment 1 - Experiment 2: samples: count 2 submit: True sheet: FalseConfiguration Notes:It’s recommended to use Experiment objects for accessing worksheet data. This model is primarily useful for creating multi-experiment sample sheets.Examples>>> from esp.models import SampleSheet >>> sheet = SampleSheet('Sheet 001') >>> sheet.name, sheet.created_at ('Sheet 001', '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> sheet.samples [<Sample(name=ESP001)>, <Sample(name=ESP001)>, ...] >>> sheet.experiments [<Experiment(name=Experiment 1)>, <Sample(name=Experiment 2)>] >>> sheet.project <Project(name=My Project)>Parameters:ident (str) – Name or uuid for object.

Worksheet

class esp.models. Worksheet ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for managing sample sheets within ESP. This object is not configurable, and operates like a pandas DataFrame for each protocol in a workflow. You can access these worksheet objects from an experiment via:>>> from esp.models import Experiment >>> exp = Experiment('My Experiment') >>> ws = exp.protocol('My Protocol') >>> ws = exp.protocols[0]Also, since the Worksheet object acts as a pandas DataFrame, you can interact with it like a data frame:>>> ws.columns ['Column 1', 'Column 2'] >>> ws['Column 1'] = ws['Column 1'].apply(lambda x: x.replace('foo', 'bar')) >>> ws.melt(...)After changing the data in the worksheet object, you can save it back to the L7 database and proceed to the next worksheet using:>>> ws.save() >>> ws.complete() >>> ws.next()Parameters:ident (str) – UUID for worksheet instance.experiment (Experiment) – Experiment object related to worksheet.

SampleType

esp.models. SampleType

alias of EntityType

WorkflowableClass

class esp.models. WorkflowableClass ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with WorkflowableClass from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Simple workflowable type:name: Patient desc: A Human Patient tags: [patient, demo], plural_name: Patients view_template: <h1>My View Template</h1> list_view_template: <h1>My View Template</h1> detail_panel_template: <h1>My View Template</h1> schedulability: 'on' icon_svg: <svg xmlns=“http://www.w3.org/2000/svg” width=“24" height=“24” viewBox=“0 0 24 24"><g fill=“none” fill-rule=“evenodd”><path d=“M0 0h24v24H0z”/><path fill=“#550185" fill-rule=“nonzero” d=“M6 6v14h12V6h-3v1H9V6H6zm4-1a2 2 0 1 1 4 0h5v16H5V5h5zm1 5h5v1h-5v-1zm0 3h5v1h-5v-1zm-3-3h2v1H8v-1zm0 3h2v1H8v-1zm0 3h2v1H8v-1zm3 0h5v1h-5v-1z”/>Examples>>> from esp.models import WorkflowableClass >>> wf = WorkflowableClass('Patient') >>> wf.name, wf.created_at ('Library', '2019-06-21T16:04:01.199076Z')Parameters:ident (str) – Name or uuid for object.

Sample

esp.models. Sample

alias of Entity

WorkflowChain

class esp.models. WorkflowChain ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with WorkflowChains from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Simple workflow chain:name: Manufacturing Chain desc: Workflow chain for manufacturing process. tags: [demo, test] chain: - Initiate Manufacturing: to: - Process A - Process B - Process A: head: false workflow: My Process virtual: true - Process B: head: false workflow: My Process resubmit: TrueWorkflow chain with conditionals and strategies:name: Manufacturing Chain desc: Workflow chain for manufacturing process. tags: [demo, test] chain: - Initiate Manufacturing: to: - Process A: if: '{{ column_value("Column 1") == "foo" }}' sample_type: Specimen - Process A: head: false to: Process B workflow: My Process strategy: named_strategy - Process B: head: false workflow: My ProcessWorkflow chain with node positioning and swimlanes:Demo WFC 3: chain: - Node 1: head: true position: x: 290.75 y: 100 startable: true swimlane: sl-1641460171013 to: - Checkbox Test: if: '{{ True }}' - Checkbox Test: position: x: 570.8214285714287 y: 300 startable: true swimlane: sl-1641460181862 to: - Approval Instructions: if: '' swimlanes: - color: '#3d85c6' height: 200 id: sl-1641460171013 label: lane 1 y: 0 - color: '#ff9900' height: 200 id: sl-1641460181862 label: lane 2 y: 200Configuration Notes:See the ESP documentation for information about how to configure workflow transition strategies.See the conditional transition logic above for an example of how to send subsets of samples to downstream nodes in a workflow chain.More information on types of workflow configuration can be found in the Workflow object documentation.Nesting configuration for workflows is usually an easier way of defining configuration for workflows that won’t be used across workflow chains.Note the on the backend, rules are evaluated for transitions in the context of the worksheet that triggers the transition, except there is no “current”/”active” protocol, so expressions like column_value must specify both the column and the protocol.Examples>>> from esp.models import WorkflowChain >>> chain = WorkflowChain('Illumina Sequencing') >>> chain.name, chain.created_at ('Illumina Sequencing', '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> chain.workflows [<Workflow(name=Initiate Manufacturing)>, <Workflow(name=Process A)>, <Workflow(name=Process B)>]Parameters:ident (str) – Name or uuid for object.

Workflow

class esp.models. Workflow ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with Workflows from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Simple workflow:name: Illumina Sequencing desc: Workflow for filling in data during illumina sequencing. tags: [demo, test] protocols: - Set Samples - Create Illumina Library - Analyze Sequencing ResultsWorkflow with embedded protocols:name: Illumina Sequencing desc: Workflow for filling in data during illumina sequencing. tags: [demo, test] protocols: - Set Samples: protocol: standard variables: - Attachment: rule: attachment - Note: rule: string - Create Illumina Library: desc: Create Illumina library and assign index codes. protocol: sample sample: Illumina Library relation: 1-to-1 - Analyze Sequencing Results: protocol: pipeline group: All pipeline: Miseq Analysis PipelineWorkflow with sample types (embedded and referenced):name: Illumina Sequencing desc: Workflow for filling in data during illumina sequencing. tags: [demo, test] sample_types: - Generic sample - Illumina Library: desc: Sample type for illumina library. sequences: - ESP SEQUENCE protocols: - Set Samples - Create Illumina Library - Analyze Sequencing ResultsWorkflow with data links and sample groupings:name: Illumina Sequencing desc: Workflow for filling in data during illumina sequencing. tags: [demo, test] links: Analyze Sequencing Results: Instrument: '{{ column_value("Instrument", "Create Illumina Library") }}' groups: Analyze Sequencing Results: Set Samples sample_types: - Illumina Library protocols: - Set Samples - Create Illumina Library - Analyze Sequencing ResultsConfiguration Notes:See the ESP documentation for information about data linking, sample groups, and available protocol configuration options.More information on types of protocol configuration can be found in the Protocol object documentation.Nesting configuration for protocols is usually an easier way of defining configuration for protocols that won’t be used across workflows.Examples>>> from esp.models import Workflow >>> wf = Workflow('Illumina Sequencing') >>> wf.name, wf.created_at ('Illumina Sequencing', '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> wf.experiments [<Experiment(name=Experiment 1)>, <Experiment(name=Experiment 2)>] >>> wf.protocols [<Protocol(name=Set Samples)>, <Protocol(name=Create Illumina Library)>, <Protocol(name=Analyze Sequencing Results)>] >>> wf.sample_types [<SampleType(name=Illumina Library)>]Parameters:ident (str) – Name or uuid for object.

Protocol

class esp.models. Protocol ( ident , data=None, version=None, snapshot=None, **kwargs)

Object for interacting with Protocols from the ESP database.See the Usage and Examples pages of the documentation for more context and comprehensive examples of how to create and use this type of objects.Configuration:Standard protocol:name: Set Samples protocol: standard desc: Example standard protocol with a few columns. tags: - illumina - sequencing variables: - Attachment Column: rule: attachment - Container Column: rule: location container: Freezer - Dropdown Column: rule: dropdown dropdown: - foo - bar - Date Column: rule: date - Note: rule: stringSample protocol with expressions:name: Create Illumina Library desc: Example sample protocol with expression columns. protocol: sample sample: Illumina Library relation: 1-to-1 variables: - Sample Type: rule: string onchange: |+ alert('Changed Sample Type') - Instrument: rule: string value: MiSeq, HiSeq 2000/2500 - Index Type: rule: dropdown required: true dropdown: '{{ ilmn_kits() }}' - I7 Index ID: rule: dropdown required: true dropdown: '{{ ilmn_adapter_names(column_value("Index Type"), position="i7") }}' - I7 Index: rule: string read_only: true value: '{{ ilmn_adapter_seq(column_value("Index Type"), column_value("I7 Index ID")) }}' - I5 Index ID: rule: dropdown required: true dropdown: '{{ ilmn_adapter_names(column_value("Index Type"), position="i5") }}' - I5 Index: rule: string read_only: true value: '{{ ilmn_adapter_seq(column_value("Index Type"), column_value("I5 Index ID"), position="i5") }}'Pipeline protocol with grouping:name: Analyze Sequencing Results protocol: pipeline desc: Run bioinformatics pipelines to analyze sequencing data. group: Reference pipeline: Miseq Analysis Pipeline variables: - Sample: rule: string visible: false pipeline_param: true value: '{{ sample["name"] }}' - Reference: rule: dropdown pipeline_param: true default: GRCh38 dropdown: - GRCh37 - GRCh38Pipeline protocol with embedded pipeline:name: Analyze Sequencing Results protocol: pipeline desc: Run bioinformatics pipelines to analyze sequencing data. group: All pipeline: Miseq Analysis Pipeline: desc: Analyze sequenced library with MiSeq analysis pipeline. tasks: - BWA Align: desc: Align fastq files to reference. cmd: "echo 'This is a bam created with the command: bwa mem {{ Sample }}.fq.gz' > {{ Sample }}.bam" files: - Aligned BAM: file_type: text filename_template: "{{ Sample }}.bam" - GATK Unified Genotyper: desc: Use GATK to genotype bam file. cmd: "echo 'This is a vcf created with the command: gatk UnifiedGenotyper -I {{ Sample }}.bam -R /references/{{ Reference }}.fa' > {{ Sample }}.vcf" files: - Genotype VCF: file_type: text filename_template: "{{ Sample }}.vcf" deps: GATK Unified Genotyper: BWA AlignProtocol with inheritance:name: Set Samples Override inherit: Set Samples desc: Inherited protocol with extra columns. variables: - Extra Column: rule: numericConfiguration Notes:See the ESP documentation for information about type types of columns and associated metadata available to use as protocol variables.Using the group parameter on protocol definitions allows developers to group columns by a specified column or value.The sample protocol relation property can take either 1-to-1, fan-out, or fan-in.Protocol inheritance is useful for creating slightly modified copies of existing protocols without needing to re-define variable configuration. All parameters are copied over from the inherited protocol.Nesting configuration for pipelines is usually an easier way of defining configuration for pipelines that won’t be used across protocols.Examples>>> from esp.models import Protocol >>> pc = Protocol('Analyze Sequencing Results') >>> pc.name, pc.created_at ('Analyze Sequencing Results', '2019-06-21T16:04:01.199076Z') >>> # show relationships >>> pc.pipeline <Pipeline(name=Analyze Sequencing Results)>Parameters:ident (str) – Name or uuid for object.

In this section: