Load Modules

Historically, two modules served the same functionality as the current project structure: LoadImages and LoadData. While the approach described above supercedes these modules in part, old pipelines loaded into CellProfiler that contain these modules will provide the option of preserving them; these pipelines will operate exactly as before.

Alternately, the user can choose to convert these modules into the project equivalent as closely as possible. Both modules remain accesible via the "Add module" and   button at the bottom of the pipeline panel. The section details information relevant for users who would like to continue using these modules. Please note, however, that these modules are deprcated and may be removed in the future.

Associating metadata with images

Metadata (i.e., additional data about image data) is sometimes available for input images. This information can be:

  1. Used by CellProfiler to group images with common metadata identifiers (or "tags") together for particular steps in a pipeline;
  2. Stored in the output file along with CellProfiler-measured features for annotation or sample-tracking purposes;
  3. Used to name additional input/output files.

Two sources of metadata are:

Details for the metadata-specific help is given next to the appropriate settings in LoadImages and LoadData, as well the specific settings in other modules which can make use of metadata. However, here is an overview of how metadata is obtained and used.

In LoadImages, metadata can be extracted from the filename and/or folder location using regular expression, a specialized syntax used for text pattern-matching. These regular expressions can be used to identify different parts of the filename / folder. The syntax (?P<fieldname>expr) will extract whatever matches expr and assign it to the image's fieldname measurement. A regular expression tool is available which will allow you to check the accuracy of your regular expression.

For instance, say a researcher has folder names with the date and subfolders containing the images with the run ID (e.g., ./2009_10_02/1234/). The following regular expression will capture the plate, well and site in the fields Date and Run:
.*[\\/](?P<Date>.*)[\\/](?P<Run>.*)$
.*[\\/]Skip characters at the beginning of the pathname until either a slash (/) or backslash (\) is encountered (depending on the OS). The extra slash for the backslash is used as an escape sequence.
(?P<Date>Name the captured field Date
.*Capture as many characters that follow
[\\/]Discard the slash/backslash character
(?P<Run>Name the captured field Run
.*Capture as many characters as follow
$The Run field must be at the end of the path string, i.e. the last folder on the path. This also means that the Date field contains the parent folder of the Date folder.

In LoadData, metadata is extracted from a CSV (comma-separated) file (a spreadsheet). Columns whose name begins with "Metadata" can be used to group files loaded by LoadData that are associated with a common metadata value. The files thus grouped together are then processed as a distinct image set.

For instance, an experiment might require that images created on the same day use an illumination correction function calculated from all images from that day, and furthermore, that the date be captured in the file names for the individual image sets and in a CSV file specifying the illumination correction functions.

In this case, if the illumination correction images are loaded with the LoadData module, the file should have a "Metadata_Date" column which contains the date metadata tags. Similarly, if the individual images are loaded using the LoadImages module, LoadImages should be set to extract the metadata tag from the file names. The pipeline will then match the individual images with their corresponding illumination correction functions based on matching "Metadata_Date" fields.

Using image grouping

To use grouping, you must define the relevant metadata for each image. This can be done using regular expressions in LoadImages or having them pre-defined in a CSV file for use in LoadData.

To use image grouping in LoadImages, please note the following: