PICSL Histology Annotation Service Quick Start

This quick start shows you how to get started with the PHAS server using a sample dataset.

Prerequisites

A Linux/MacOS machine with Python3 (or Docker)
An open port on this machine. We will assume port number 8888.

Install OpenSlide

The OpenSlide library is required and must be installed sepately from the Python dependencies. This is not required for Docker-based installations.

On Debian/Ubuntu:
```
apt-get install -y libopenslide-dev
```
On MacOS:
```
brew install openslide
```

Install PHAS

Three installation options are available in this tutorial:

pip: This is the simplest option and is recommended for most users. PHAS will be installed as a Python package in a virtual environment.
git: This is the recommended option for those who would like to use the bleeding edge version of PHAS or to modify the source code.
Docker: Running PHAS in a Docker container that we provide is a good solution for Windows users. Another advantage of the Docker container is that it is already set up for running PHAS in production mode with the nginx web server.

Install using `pip`

Create a directory where you will store all the files for your PHAS installation. This directory will contain your configuration, your database, and other important files. In this tutorial we will assume your Linux username is foo and that your installation directory is /home/foo/phas. This command will create this directory:

# Change to match your username and preferred location
mkdir -p /home/foo/phas

Next, create a virtual environment that will contain PHAS and all the Python modules on which it depends:

cd /home/foo/phas
python3 -m venv .venv
source .venv/bin/activate

Now, install the PHAS package:

pip install phas

Finally, create the instance directory. This directory will contain your PHAS database, configuration scripts, and other important files:

mkdir -p instance

The full path to the instance directory will be /home/foo/phas/instance.

Install using `git`

In this tutorial we will assume your Linux username is foo and that your installation directory is /home/foo/phas. This command will clone the PHAS repository in this directory:

git clone https://github.com/pyushkevich/histoannot.git /home/foo/phas

Next, enter this directory and create a virtual environment that will contain all the Python modules on which PHAS depends:

cd /home/foo/phas
python3 -m venv .venv
source .venv/bin/activate

Now, install all the PHAS dependencies:

pip install -r phas/requirements.txt

Finally, create the instance directory. This directory will contain your PHAS database, configuration scripts, and other important files:

mkdir -p instance

The full path to the instance directory will be /home/foo/phas/instance.

Install using Docker

To run the Docker container, you need to create an instance directory somewhere on your computer. In this tutorial we assume that you created an instance directory called /home/foo/phas/instance. Replace this with the actual name of your instance directory. For example, on Windows, this could be C:\Users\foo\phas\instance.

You also need to point the container to the directory with your histology slides. In this tutorial, we assume that your data are organized in the folder /data/archive, and you should replace this with your own path. Finally, you may want to replace 8888 with the port number at which you would like to be able to access the PHAS web application.

To launch the docker container, execute:

docker run -d -p 8888:80 --name phas \
    -v /data/archive:/data \
    -v /home/foo/phas/instance:/instance \
    pyushkevich/phas:latest

After running the command above, you should be able to access the web application by pointing your browser to http://localhost:8888. At this point, you should see a login page.

To restart the container (after making changes to your configuration below), you can run:

docker restart phas

To interact with the application using flask ... commands (which are introduced below and used to configure users, projects, and tasks) use the following commands:

docker exec -it phas /bin/bash
source env.sh

Configuration and Launching

These instructions are slightly different for the native (pip or git) and Docker-based installations.

Setup the Environment

Skip this step for Docker-based installations.

Create a shell script env.sh in the directory /home/foo/phas with the contents below, modified to fit your installation.

#!/bin/bash
source .venv/bin/activate

# Name of the FLASK application
export FLASK_APP=phas

# Path to the instance directory
export FLASK_INSTANCE_PATH=/home/foo/phas/instance

# On Mac, if using homebrew to install openslide, set this to the location of the openslide library
export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/opt/homebrew/lib

Before executing the “flask” commands below, run once per terminal session:

source env.sh

Create Flask Configuration File

In your instance directory (/home/foo/phas/instance), create a file config.py and add the lines below, customizing them for your installation. Replace the secret key with your own.

# Replace with your own random sequence
SECRET_KEY="cf55754542254a76fbd839970ddd55fee4088ed594511c90ea3976428a851374"

# Name of your server printed on the landing page
HISTOANNOT_PUBLIC_NAME="My Test PHAS Server"

# URL for your PHAS installation
HISTOANNOT_PUBLIC_URL="http://127.0.0.1:8888"

# Optionally, enable a guest user account (comment out if not needed)
HISTOANNOT_GUEST_USER="guest"

Database Creation

Run this command to create the sqlite3 database structure for the first time:

flask init-db

Test Your Configuration

If successful, the command below will print the configuration settings you provided above and list the tables in the database (about 20):

flask info

Start the Web Application

Skip this step for Docker-based installations. The web application is launched automatically when you run the Docker container.

To test-drive PHAS, you can use the command below to launch the web application. However, when in production, you should use nginx and uwsgi to launch your application instead, as described in Running in a Production Environment.

flask run --debug --port 8888

You will see this output:

* Serving Flask app 'histoannot'
* Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
* Running on http://127.0.0.1:8888
Press CTRL+C to quit

Navigate to the URL provided (http://127.0.0.1:8888) and you should see the login page.

Start the Slide Server Process

Skip this step for Docker-based installations. The slide server is launched automatically when you run the Docker container.

In addition to running the main web application with flask run, you need to launch the slide server process, which manages the interface between the web application and the histology images. The server process should be run in a separate terminal window.

Open a new terminal window and navigate to the /home/foo/phas directory:

source env.sh
flask slide-server-run

Creating Users, Projects and Tasks

Open a separate terminal window or tab so that you can interact with the server while it is running. In the terminal go to your /home/foo/phas directory and run, as before:

source env.sh

For Docker installations, open a terminal window and run:

docker exec -it phas /bin/bash
source env.sh

Create Admin User Account

Create a user (replace testuser with your own id) and provide them administrator privileges:

flask users-add -e testuser@gmail.com testuser
flask users-set-site-admin test user

This will print a URL. Navigate to this URL and set up the password for your account. Now you should see the landing page with the message that you have not been added to any projects yet.

You can click on your username on the top right of the web application to change your profile and manage other users on the server.

Download Sample Dataset

The easiest way to get started with PHAS is to download a sample dataset. It contains some blockface images of brain tissue prior to cryosectioning. Download the dataset histoannot_sample_data.zip from https://upenn.box.com/v/phas-sample-data and unpack it into a folder separate from your main PHAS install. Let’s suppose you called this folder /data/archive/histoannot_sample_data.

Create a directory where you will keep the json descriptor files used to configure projects and tasks:

mkdir instance/json

Create a json descriptor file for the project you downloaded, called instance/json/example_project.json, with the contents below:

{
    "base_url": "/data/archive/histoannot_sample_data",
    "disp_name": "Example Project",
    "desc": "Example project with some blockface images",
    "manifest_mode": "individual_json",
    "url_schema": {
        "pattern": {
            "raw": "{specimen}/raw/{slide_name}.{slide_ext}",
            "thumb": "{specimen}/proc/{slide_name}_thumb.png",
            "metadata": "{specimen}/proc/{slide_name}_metadata.json"
        },
        "raw_slide_ext": [ "tiff" ]
    }
}

For Docker installations, base-url should be relative to the directory /data, which is the path on the container to which you mapped your data directory when calling docker run. For example, if you placed the sample dataset in /home/foo/phas/histology_data/histoannot_sample_data, then base_url should be set to /data/histoannot_sample_data.

The commands below configure the project and add your username to it as administrator:

flask project-add example instance/json/example_project.json
flask users-set-access-level -p example admin testuser

The commands below import slides from the sample project into the database. You should run this command every time that new slides are added to your data folder:

flask refresh-slides example

If you edit the .json file later, you need to run the command below for your edits to take effect:

flask project-update example instance/json/example_project.json

Configure Browse and Annotation Tasks

If you browse to your PHAS URL, you will see that there is a project with one specimen and four slides. However, you cannot view these slides yet because we have not yet set up any tasks. Tasks are specific ways of interacting with histology images, and they include browsing, annotation, placing boxes for training classifiers, and placing sampling regions.

Each task is specified by creating a json configuration file.

Create file instance/json/example_browse.json for the browsing task with contents:

{
    "restrict-access": false,
    "mode": "browse",
    "name": "Browse",
    "desc": "Browse the slide collection"
}

And create file instance/json/example_annot.json for the annotation task with contents:

{
    "restrict-access": true,
    "mode": "annot",
    "name": "Anatomical Labeling",
    "desc": "Labeling anatomical boundaries and regions"
}

The commands below will intialize these tasks and rebuild the slide index for the tasks:

flask tasks-add example instance/json/example_browse.json
flask tasks-add example instance/json/example_annot.json
flask rebuild-task-slide-index example

You will be able to see the Browse task immediately. To see the Annotation task, go to the “manage users” menu option under your username and give yourself write access to the task. Alternatively, you can use the flask users-set-access-level command with -t flag to give yourself write access to the newly created task.

Configure a Classification Training Task

To create a classifier training task, we first need to create a set of classification labels. Create the file instance/json/blockface_labels.json with contents:

[
    { "name" : "gray matter", "color" : "#18b497", "description" : "Gray Matter" },
    { "name" : "white matter", "color" : "#2816ba", "description" : "White Matter" },
    { "name" : "background", "color" : "#f97a8a", "description" : "Ice/Background" }
]

Then add this labelset to the server:

flask labelset-add example blockface_tissue_types instance/json/blockface_labels.json

The labelset should be available for editing under the dropdown menus on the project menu in the web interface.

Then create a task descriptor for generating training patches in file instance/json/example_training.json with contents:

{
    "restrict-access": false,
    "name": "Tissue Class Training",
    "stains": [
        "blockface"
    ],
    "dltrain": {
        "labelset": "blockface_tissue_types",
        "min-size": 128,
        "max-size": 128,
        "display-patch-size": 128
    },
    "mode": "dltrain",
    "desc": "Training a deep learning classifier to segment blockface images"
}

Then add the task to the server:

flask tasks-add example instance/json/example_training.json
flask rebuild-task-slide-index example

Now the task will be available in the web interface.

Configure a Sampling ROI Task

A sampling ROI task allows you to define sampling ROIs from which quantitative measures can be derived. To set up this task we also first have to define labels.

Create the file instance/json/sampling_labels.json with contents:

[
    {
        "name": "Hipp",
        "color": "#ff3300",
        "description": "Hipppocampus"
    },
    {
        "name": "PHG",
        "color": "#ff6600",
        "description": "Parahippocampal Gyrus"
    },
    {
        "name": "FuG",
        "color": "#ff6699",
        "description": "Fusiform Gyrus"
    }
]

And create the task descriptor file instance/json/example_sroi.json with contents:

{
    "restrict-access": false,
    "name": "Sampling ROI Placement",
    "desc": "Placement of Sampling ROIs for Quantification",
    "mode": "sampling",
    "sampling": {
        "labelset": "blockface_srois"
    }
}

Then add the labelset and task to the server:

flask labelset-add example blockface_srois instance/json/sampling_labels.json
flask tasks-add example instance/json/example_sroi.json
flask rebuild-task-slide-index example