=================================== Output & Results =================================== Command Line Output =================== When running the workflow you should see output similar to: .. code-block:: console Launching `https://github.com/mriffle/nf-openmod-dda` [deadly_swanson] DSL2 - revision: cedbe21973 [main] executor > awsbatch (10) [7d/ff54be] process > wf_magnum_combined_percolator:MSCONVERT (2) [100%] 2 of 2 ✔ [f6/f9330c] process > wf_magnum_combined_percolator:ADD_PARAMS_TO_MAGNUM_CONF (2) [100%] 2 of 2 ✔ [d2/d2ab30] process > wf_magnum_combined_percolator:MAGNUM (2) [100%] 2 of 2 ✔ [6e/1c901f] process > wf_magnum_combined_percolator:COMBINE_PIN_FILES [100%] 1 of 1 ✔ [86/898518] process > wf_magnum_combined_percolator:PERCOLATOR [100%] 1 of 1 ✔ [2d/af2776] process > wf_magnum_combined_percolator:CONVERT_TO_LIMELIGHT_XML [100%] 1 of 1 ✔ [48/11a8d6] process > wf_magnum_combined_percolator:UPLOAD_TO_LIMELIGHT [100%] 1 of 1 ✔ Completed at: 15-Jun-2023 18:38:42 Duration : 3h 38m 13s CPU hours : 133.8 Succeeded : 10 The first line shows the version of Nextflow you are running. The second line shows the version of the workflow you are running. The third line shows the executor you are using. An executor in Nextflow describes the actual system the steps of the workflow are running on. In this case the *AWS Batch* executor was used. The next several lines show the actual steps of the workflow as they are running. If a particular step is run multiple times (e.g., converting many RAW files to mzML using msconvert), the percent complete shows the percentage of the RAW files that have been converted. The final four lines appear when the workflow completes, showing the completion time, how long it took, and the number of steps that succeeded. Workflow Log ============ The log file called ``.nextflow.log`` will appear in the directory in which the workflow was run. It can be helpful for determining the cause of any problems. A log file will also be generated for each task executed by the workflow, which will be described below. Workflow Results ================ All results will be output to the ``results/nf-openmod-dda`` subdirectory in the directory in which the workflow was run. In this directory is a subdirectory for each program that was run as part of the workflow. A full description of output files can be found below. Output Files ============ Below are the subdirectories created in ``results/nf-openmod-dda`` and a description of the files found in each. .. note:: Most outputs are organized into a **per-sample subdirectory** named after the base name of each input scan file. In these descriptions ```` stands for that base name. The two processing modes (set by ``process_separately``) affect the layout of the ``percolator`` and ``limelight`` directories: - **Combined mode** (``process_separately = false``, the default): the per-sample results from all scan files are merged, and Percolator and Limelight run once over the pooled data. The combined results are written under a single subdirectory literally named ``combined`` (so ```` is ``combined``). - **Separate mode** (``process_separately = true``): Percolator and Limelight run once per scan file, producing one ```` subdirectory per input file. ``panorama`` Subdirectory ^^^^^^^^^^^^^^^^^^^^^^^^^^ This directory is only created when inputs are retrieved from **PanoramaWeb** (any path beginning with ``https://``). It holds the logs from downloading files from Panorama, plus any FASTA or Magnum configuration file that was fetched from Panorama: - ``panorama-get-*.stdout`` - The standard command line output generated while listing and downloading files from Panorama. - ``panorama-get-*.stderr`` - Any error messages generated while listing and downloading files from Panorama. (Downloaded RAW files are not published here; they are stored in the ``panorama_cache_directory``.) ``yarp`` Subdirectory ^^^^^^^^^^^^^^^^^^^^^^ This directory is only created when ``generate_decoys = true``. It holds the decoy-augmented FASTA produced by YARP: - ``.plus-decoys.fasta`` - A copy of your input FASTA with generated decoy sequences appended. This is the database actually used by Magnum. ```` is the input FASTA's name with its extension removed (e.g. ``human.fasta`` produces ``human.plus-decoys.fasta``). ``magnum`` Subdirectory ^^^^^^^^^^^^^^^^^^^^^^^^^ This directory holds logs and output related to running Magnum, organized into a ```` subdirectory for each input scan file. Each ``magnum//`` directory contains: - ``.conf`` - The Magnum configuration file for this scan file, generated from your ``Magnum.conf`` with the FASTA and scan-file paths automatically substituted in. - ``.add-params.stderr`` - Any errors generated while substituting those paths into the configuration file. - ``.magnum.stderr`` - Any errors generated by running Magnum on this scan file. - ``.magnum.stdout`` - The standard output generated by Magnum for this scan file. - ``*.pep.xml`` - The results, in PepXML format, generated by Magnum for this scan file. - ``*.perc.txt`` - The Percolator input file generated by Magnum for this scan file. - ``*.diag.xml`` - An optional Magnum diagnostic file, present only if Magnum is configured to produce one. ``percolator`` Subdirectory ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This directory holds logs and output related to running Percolator. In **combined mode**, the merged Percolator input is written directly in this directory: - ``combined.filtered.pin`` - The result of combining the Percolator input files from every scan file into a single ``.pin`` file. - ``combine-pin.stderr`` - Any error messages output while combining the ``.pin`` files. (These two files are not produced in separate mode, where no combining step runs.) The Percolator results for each ```` (the literal ``combined`` in combined mode, or one per scan file in separate mode) are written under a ``percolator//`` subdirectory: - ``.pout.xml`` - The final results, in XML format, of running Percolator. - ``.percolator.stdout`` - The command line output generated by Percolator. - ``.percolator.stderr`` - Any error messages generated by Percolator. - ``.columns_filtered.pin`` - Present only when ``percolator_pin_columns_to_remove`` is set. This is the ``.pin`` file after removing the requested columns (the input Percolator actually ran on). - ``.filter-pin.stderr`` - Present only when ``percolator_pin_columns_to_remove`` is set. Any error messages from the column-filtering step. ``limelight`` Subdirectory ^^^^^^^^^^^^^^^^^^^^^^^^^^^ This directory holds logs related to Limelight XML conversion and uploading to Limelight, and is only created when ``limelight_upload = true``. In combined mode the files below are written directly in this directory; in separate mode they are written under a ``limelight//`` subdirectory per scan file (and the Limelight XML file is named ``.limelight.xml``). - ``results.limelight.xml`` - The Limelight XML file generated by converting the search results. This format is necessary for uploading data to Limelight. (Named ``.limelight.xml`` in separate mode.) - ``limelight-xml-convert.stdout`` - The standard command line output generated by the program that converts search results to Limelight XML. - ``limelight-xml-convert.stderr`` - Any error messages generated by converting the search results to Limelight XML. - ``limelight-submit-upload.stdout`` - The standard command line output generated by the program that uploads data to Limelight. - ``limelight-submit-upload.stderr`` - Any error messages generated by uploading the search results to Limelight. Execution Reports ================= In addition to the results above, every run writes Nextflow execution reports to the ``reports/nf-openmod-dda`` directory (configurable via the ``report_dir`` parameter). These are overwritten on each run and are useful for inspecting resource usage and timing: - ``execution_report.html`` - An HTML report summarizing each task, its resource usage, and its exit status. - ``execution_timeline.html`` - An HTML timeline showing when each task ran and how long it took. - ``execution_trace.txt`` - A tab-delimited trace of every task executed during the run.