Skip to main content

Performance Considerations when Developing

System Overview

  • System Layers:

ESP Client

→

REST API

Backend

→

Queries

Database

Customization

Custom endpoint

Custom expression

Custom protocol action

Custom transition strategy

Custom query

  • Consider performance as the data in the system grows

    • What is being queried?

    • What is actually needed?

  • Load in large dataset to flag potential slowdowns

    • Monitor times and resource usage

  • 503s (ESP won’t start)

    • l7 status and l7 restart

    • server logs data/logs/ – web workers will be most useful (http.*)

Pipelines (ESP Client)

Debugging and Development Tips

  • Test ESP Client in a standard Python terminal

  • Write to stdout i.e. print() (ok) or stderr i.e. sys.stderr.write (better)

    • Look at the stderr and stdout associated with a failing task

  • ‘Uncomplete’ and ‘Resubmit Pipeline’ after making changes to a pipeline script

  • Execute pipeline script in the client python environment (after supplying the context yourself)

    • No LimsAnalysis.current() → Worksheet(<UUID>)

  • Create Experiment yaml files to speed up testing

  • Testing pipelines can be automated:

    • as part of an Experiment yaml file using run: true in the pipeline protocol

    • in a pytest

      def test_pipeline():
  from esp.models import Pipeline
  analysis = Pipeline(<pipeline name>).run(<pipeline parameters>)
  assert analysis.tasks[0].files[0].exists()

  • REST API calls made by the client can be printed

    • ESP python client:

      import logging, sys
logger = logging.getLogger('esp')
logger.setLevel('DEBUG')
logger.addHandler(logging.StreamHandler(sys.stdout))

    • ESP CLI: esp --log-level DEBUG <command>

Performance

  • Be careful using functions such as Sample.all()

  • Use bulk calls

  • from esp.models import Sample
sample_uuids = [<list of uuids>]
samples = Sample.items_for_uuids(sample_uuids) 
sample_names = []
for sample in samples:
    sample_names.append('Duplicate' + sample.name)
new_samples = Sample.create({'names':sample_names})

    rather than api calls in a loop

    from esp.models import Sample
sample_uuids = [<list of uuids>]
samples = [Sample(uuid) for uuid in sample_uuids]
new_samples = []
for sample in samples:
    new_samples.append(Sample.create(name= 'Duplicate' + sample.name))

  • When searching by names and uuids use

    • esp.models.<model>.items_for_names()

    • esp.models.<model>.items_for_uuids()

respectively rather than esp.models.<model>.search()

  • If a query returns data as a dictionary, use

    • esp.models.<model>.from_data()

to return an object rather than looping through esp.models.<model>(<uuid or name>)

  • When saving worksheets, use the evaluate and refresh parameters since the pipeline will automatically save and reevaluate expressions after it is completed

    sheet.protocol(<name of protocol>).save(evaluate=False, refresh=False)

    • Set evaluate=False in all cases unless reevaluated expressions are needed in other parts of the pipeline

    • Set refresh=False in all cases unless updated sheet values are needed in other parts of the pipeline

  • REST API endpoints can be called directly from esp.base.SESSION

    • esp.base.SESSION.get()

    • esp.base.SESSION.post()

    • esp.base.SESSION.put()

    • esp.base.SESSION.delete()

Client side and REST APIs

Debugging and Development Tips

  • General tips (including applets and dashboard reports)

    • use debugger (make sure dev tools is open)

    • check network tab for failing API calls

    • Disable browser cache in devtools while debugging (avoid needing to hard-reload)

  • onRender/onChange

    • use invokables.js to iterate quickly on your code

  • flexview

    • test things in a generic renderer

    • use experiment yaml to execute workflow

    • also use invokables.js (must manually load file)

  • Use API Utility or /main/static/util.html to test

  • Use REST API client to test

    • Can get API key from User Settings → Security

Performance

  • Be careful when hitting an endpoint without uuids like /api/samples

  • Use params to limit data returned

    /api/samples?tags=[<list of tags>]&params=["name"]&limit=10

  • Use bulk queries

    /api/samples?uuids=[<list of uuids]

Server-side Extensions

Debugging and Development Tips

  • Test back-end APIs with l7 console

    • agent is admin in backend api calls

  • data/logs for server logs (look at web worker-related services http.*)

  • Turn off the celery queue (via Config App → feature_flags-> async_calls)

  • Use pdb or an integrated debugger (turn off web workers, start up and attach to 1 web worker)

  • Check the Extensions Applet to ensure everything loaded correctly

  • Test custom endpoints at /api/invoke/<custom endpoint> using API utility/python client/pytest

  • Test custom expressions:

    • Use REST API /api/extensions/eval_expression

    • esp.utils.eval_expression in the python client/pytest

    • Blank free-text field in LIMS will evaluate expressions on save

    • Use the verify block in Experiment yaml files to check that expressions in LIMS worksheets evaluated correctly

  • Use Experiment Chain yaml files to test custom protocol actions (complete: true)

  • Use Experiment Chain yaml files to test custom transitions (transition: true)

  • All server side extensions can be tested through Experiment yaml files or pytests

  • To show the SQL Alchemy logging:

    from lab7.db import Session # sessionmaker
del session
import logging
import sys
logger = logging.getLogger('sqlalchemy.engine')
logger.setLevel(logging.INFO)
session = Session() # Session must be initialized after setting the log level above

    • Note:

      The SQLAlchemy Engine conserves Python function call overhead by only emitting log statements when the current logging level is detected as logging.INFO or logging.DEBUG. It only checks this level when a new connection is procured from the connection pool. Therefore when changing the logging configuration for an already-running application, any Connection that's currently active, or more commonly a Session object that's active in a transaction, won't log any SQL according to the new configuration until a new Connection is procured (in the case of Session, this is after the current transaction ends and a new one begins).

Performance

  • Use return_dict=False for backend api calls which have it as a parameter

  • Use bulk calls

  • import lab7.sample.api as sapi
sample_uuids = [<list of uuids>]
samples = sapi.query_samples({"uuids":sample_uuids},return_dict=False,agent=agent,session=session)
sample_data = []
for sample in samples:
  sample_spec = {"name": "Duplicate" + sample.name,"sample_type":"Generic sample"}
  sample_data.append(sample_spec)
new_samples = sapi.create_samples(sample_data, return_dict=False, agent=agent, session=session)

    rather than api calls in a loop

    import lab7.sample.api as sapi
sample_uuids = [<list of uuids>]
samples = [sapi.get_sample(uuid, agent=agent, session=session) for uuid in sample_uuids]
new_samples = []
for sample in samples:
  sample_spec = {"name": "Duplicate" + sample["name"],"sample_type":"Generic sample"}
  new_samples.append(sapi.create_sample(**sample_spec, agent=agent, session=session))

  • Use filters when querying, the filters are largely the same as what is available through the REST API aside from params

    sapi.query_samples({"name":<entity name>,"limit":1},return_dict=False,agent=agent,session=session)

Custom Queries

Debugging and Development Tips

  • Test in some kind of database tool like DBeaver

  • Look at examples queries in server/queries

  • Check registered queries at /api/v2/queries/list

  • Test custom queries at /api/v2/queries/<custom query name>

In this section: