Skip to content

How to return an output file

Questions

  • Where does the output of a command go?

  • How can I save the output of a command?

Objectives

  • Learn how to describe and handle outputs from a tool.

Explanation

The outputs of a tool is a list of output parameters that should be returned after running the tool. Each parameter has an id for the name of parameter, and type describing what types of values are valid for that parameter.

When a tool runs under CWL, the starting working directory is the designated output directory. The underlying tool or script must record its results in the form of files created in the output directory. The output parameters returned by the CWL tool are either the output files themselves, or come from examining the content of those files.

The following example demonstrates how to return a file generated by gdal_translate.

Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
cwlVersion: v1.0

$graph:
- class: Workflow
  id: main
  inputs:
    tif: 
      type: string
  outputs:
    preview: 
      outputSource: node_gdal/preview
      type: File
  steps:
    node_gdal:
      in: 
        tif: tif
      out: 
      - preview
      run:
        "#gdal"

- class: CommandLineTool
  id: gdal
  requirements:
    InlineJavascriptRequirement: {}
    EnvVarRequirement:
      envDef:
        PROJ_LIB: /srv/conda/envs/notebook/share/proj
  hints:
    DockerRequirement: 
      dockerPull: docker.io/osgeo/gdal  
  baseCommand: gdal_translate
  arguments: 
  - -of
  - PNG
  - -ot 
  - Byte
  - valueFrom: |
        ${ if (inputs.tif.startsWith("http")) {
             return "/vsicurl/" + inputs.tif; 
           } else { 
             return inputs.tif;
           } 
        }
  - preview.png
  inputs:
    tif:
      type: string
  outputs:
    preview: 
      type: File
      outputBinding:
        glob: preview.png

Parameters

The parameters file return-output-file.yml contains:

tif: https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/53/H/PA/2021/7/S2B_53HPA_20210723_0_L2A/B8A.tif

Run this how-to on Code Server

  1. Run me on Code Server
  2. Open a new terminal
  3. Execute
 cwltool --no-container return-output-file.cwl return-output-file.yml

This will return:

INFO /srv/conda/envs/notebook/bin/cwltool 3.1.20220607081835
INFO Resolved 'return-output-file.cwl' to 'file:///home/jovyan/cwl-how-to.git/01-output/return-output-file.cwl'
INFO [workflow ] start
INFO [workflow ] starting step node_gdal
INFO [step node_gdal] start
INFO [job node_gdal] /tmp/dm_vv1x5$ gdal_translate \
    -of \
    PNG \
    -ot \
    Byte \
    /vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/53/H/PA/2021/7/S2B_53HPA_20210723_0_L2A/B8A.tif \
    preview.png
Input file size is 5490, 5490
0...10...20...30...40...50...60...70...80...90...100 - done.
INFO [job node_gdal] Max memory used: 53MiB
INFO [job node_gdal] completed success
INFO [step node_gdal] completed success
INFO [workflow ] completed success
{
    "preview": {
        "location": "file:///home/jovyan/cwl-how-to.git/01-output/preview.png",
        "basename": "preview.png",
        "class": "File",
        "checksum": "sha1$042f63c30a8a0ddb0d83fd8f3e84301c18957daf",
        "size": 9217085,
        "path": "/home/jovyan/cwl-how-to.git/01-output/preview.png"
    }
}
INFO Final process status is success

Key points

  • Outputs are described in the outputs section of a CWL description.

  • The field outputBinding describes how to to set the value of each output parameter.

  • Wildcards are allowed in the glob field.