Output & Results

Command Line Output

When running the workflow you should see output similar to:

Launching `https://github.com/mriffle/nf-openmod-dda` [deadly_swanson] DSL2 - revision: cedbe21973 [main]
executor >  awsbatch (10)
[7d/ff54be] process > wf_magnum_combined_percolator:MSCONVERT (2)                 [100%] 2 of 2 ✔
[f6/f9330c] process > wf_magnum_combined_percolator:ADD_PARAMS_TO_MAGNUM_CONF (2) [100%] 2 of 2 ✔
[d2/d2ab30] process > wf_magnum_combined_percolator:MAGNUM (2)                    [100%] 2 of 2 ✔
[6e/1c901f] process > wf_magnum_combined_percolator:COMBINE_PIN_FILES             [100%] 1 of 1 ✔
[86/898518] process > wf_magnum_combined_percolator:PERCOLATOR                    [100%] 1 of 1 ✔
[2d/af2776] process > wf_magnum_combined_percolator:CONVERT_TO_LIMELIGHT_XML      [100%] 1 of 1 ✔
[48/11a8d6] process > wf_magnum_combined_percolator:UPLOAD_TO_LIMELIGHT           [100%] 1 of 1 ✔
Completed at: 15-Jun-2023 18:38:42
Duration    : 3h 38m 13s
CPU hours   : 133.8
Succeeded   : 10

The first line shows the version of Nextflow you are running. The second line shows the version of the workflow you are running. The third line shows the executor you are using. An executor in Nextflow describes the actual system the steps of the workflow are running on. In this case the AWS Batch executor was used. The next several lines show the actual steps of the workflow as they are running. If a particular step is run multiple times (e.g., converting many RAW files to mzML using msconvert), the percent complete shows the percentage of the RAW files that have been converted. The final four lines appear when the workflow completes, showing the completion time, how long it took, and the number of steps that succeeded.

Workflow Log

The log file called .nextflow.log will appear in the directory in which the workflow was run. It can be helpful for determining the cause of any problems. A log file will also be generated for each task executed by the workflow, which will be described below.

Workflow Results

All results will be output to the results/nf-openmod-dda subdirectory in the directory in which the workflow was run. In this directory is a subdirectory for each program that was run as part of the workflow. A full description of output files can be found below.

Output Files

Below are the subdirectories created in results/nf-openmod-dda and a description of the files found in each.

Note

Most outputs are organized into a per-sample subdirectory named after the base name of each input scan file. In these descriptions <sample> stands for that base name.

The two processing modes (set by process_separately) affect the layout of the percolator and limelight directories:

  • Combined mode (process_separately = false, the default): the per-sample results from all scan files are merged, and Percolator and Limelight run once over the pooled data. The combined results are written under a single subdirectory literally named combined (so <sample> is combined).

  • Separate mode (process_separately = true): Percolator and Limelight run once per scan file, producing one <sample> subdirectory per input file.

panorama Subdirectory

This directory is only created when inputs are retrieved from PanoramaWeb (any path beginning with https://). It holds the logs from downloading files from Panorama, plus any FASTA or Magnum configuration file that was fetched from Panorama:

  • panorama-get-*.stdout - The standard command line output generated while listing and downloading files from Panorama.

  • panorama-get-*.stderr - Any error messages generated while listing and downloading files from Panorama.

(Downloaded RAW files are not published here; they are stored in the panorama_cache_directory.)

yarp Subdirectory

This directory is only created when generate_decoys = true. It holds the decoy-augmented FASTA produced by YARP:

  • <fasta-basename>.plus-decoys.fasta - A copy of your input FASTA with generated decoy sequences appended. This is the database actually used by Magnum. <fasta-basename> is the input FASTA’s name with its extension removed (e.g. human.fasta produces human.plus-decoys.fasta).

magnum Subdirectory

This directory holds logs and output related to running Magnum, organized into a <sample> subdirectory for each input scan file. Each magnum/<sample>/ directory contains:

  • <sample>.conf - The Magnum configuration file for this scan file, generated from your Magnum.conf with the FASTA and scan-file paths automatically substituted in.

  • <sample>.add-params.stderr - Any errors generated while substituting those paths into the configuration file.

  • <sample>.magnum.stderr - Any errors generated by running Magnum on this scan file.

  • <sample>.magnum.stdout - The standard output generated by Magnum for this scan file.

  • *.pep.xml - The results, in PepXML format, generated by Magnum for this scan file.

  • *.perc.txt - The Percolator input file generated by Magnum for this scan file.

  • *.diag.xml - An optional Magnum diagnostic file, present only if Magnum is configured to produce one.

percolator Subdirectory

This directory holds logs and output related to running Percolator.

In combined mode, the merged Percolator input is written directly in this directory:

  • combined.filtered.pin - The result of combining the Percolator input files from every scan file into a single .pin file.

  • combine-pin.stderr - Any error messages output while combining the .pin files.

(These two files are not produced in separate mode, where no combining step runs.)

The Percolator results for each <sample> (the literal combined in combined mode, or one per scan file in separate mode) are written under a percolator/<sample>/ subdirectory:

  • <sample>.pout.xml - The final results, in XML format, of running Percolator.

  • <sample>.percolator.stdout - The command line output generated by Percolator.

  • <sample>.percolator.stderr - Any error messages generated by Percolator.

  • <sample>.columns_filtered.pin - Present only when percolator_pin_columns_to_remove is set. This is the .pin file after removing the requested columns (the input Percolator actually ran on).

  • <sample>.filter-pin.stderr - Present only when percolator_pin_columns_to_remove is set. Any error messages from the column-filtering step.

limelight Subdirectory

This directory holds logs related to Limelight XML conversion and uploading to Limelight, and is only created when limelight_upload = true. In combined mode the files below are written directly in this directory; in separate mode they are written under a limelight/<sample>/ subdirectory per scan file (and the Limelight XML file is named <sample>.limelight.xml).

  • results.limelight.xml - The Limelight XML file generated by converting the search results. This format is necessary for uploading data to Limelight. (Named <sample>.limelight.xml in separate mode.)

  • limelight-xml-convert.stdout - The standard command line output generated by the program that converts search results to Limelight XML.

  • limelight-xml-convert.stderr - Any error messages generated by converting the search results to Limelight XML.

  • limelight-submit-upload.stdout - The standard command line output generated by the program that uploads data to Limelight.

  • limelight-submit-upload.stderr - Any error messages generated by uploading the search results to Limelight.

Execution Reports

In addition to the results above, every run writes Nextflow execution reports to the reports/nf-openmod-dda directory (configurable via the report_dir parameter). These are overwritten on each run and are useful for inspecting resource usage and timing:

  • execution_report.html - An HTML report summarizing each task, its resource usage, and its exit status.

  • execution_timeline.html - An HTML timeline showing when each task ran and how long it took.

  • execution_trace.txt - A tab-delimited trace of every task executed during the run.