Taskcluster Mach commands¶

A number of mach subcommands are available aside from mach taskgraph decision to make this complex system more accessible to those trying to understand or modify it. They allow you to run portions of the graph-generation process and output the results.

mach taskgraph tasks: Get the full task set
mach taskgraph full: Get the full task graph
mach taskgraph target: Get the target task set
mach taskgraph target-graph: Get the target task graph
mach taskgraph optimized: Get the optimized task graph
mach taskgraph morphed: Get the morphed task graph

See How Tos for further practical tips on debugging task-graph mechanics locally.

Parameters¶

Each of these commands takes an optional --parameters argument giving a file with parameters to guide the graph generation. The decision task helpfully produces such a file on every run, and that is generally the easiest way to get a parameter file. The parameter keys and values are described in Parameters; using that information, you may modify an existing parameters.yml or create your own. The --parameters option can also take the following forms:

project=<project>: Fetch the parameters from the latest push on that project
task-id=<task-id>: Fetch the parameters from the given decision task id

If not specified, parameters will default to project=mozilla-central.

Taskgraph JSON Format¶

By default, the above commands will only output a list of tasks. Use -J flag to output full task definitions. For example:

$ ./mach taskgraph optimized -J

Task graphs – both the graph artifacts produced by the decision task and those output by the --json option to the mach taskgraph commands – are JSON objects, keyed by label, or for optimized task graphs, by taskId. For convenience, the decision task also writes out label-to-taskid.json containing a mapping from label to taskId. Each task in the graph is represented as a JSON object.

Each task has the following properties:

kind: The name of this task’s kind
task_id: The task’s taskId (only for optimized task graphs)
label: The task’s label
attributes: The task’s attributes
dependencies: The task’s in-graph dependencies, represented as an object mapping dependency name to label (or to taskId for optimized task graphs)
optimizations: The optimizations to be applied to this task
task: The task’s TaskCluster task definition.

The results from each command are in the same format, but with some differences in the content:

The tasks and target subcommands both return graphs with no edges. That is, just collections of tasks without any dependencies indicated.
The optimized subcommand returns tasks that have been assigned taskIds. The dependencies array, too, contains taskIds instead of labels, with dependencies on optimized tasks omitted. However, the task.dependencies array is populated with the full list of dependency taskIds. All task references are resolved in the optimized graph.

The output of the mach taskgraph commands are suitable for processing with the jq utility. For example, to extract all tasks’ labels and their dependencies:

jq 'to_entries | map({label: .value.label, dependencies: .value.dependencies})'

An alternate way of searching the output of mach taskgraph is gron, which converts json into a format that’s easily searched with grep

gron taskgraph.json | grep -E 'test.*machine.platform = "linux64";'
./mach taskgraph --json | gron | grep ...

Diffing Taskgraphs¶

A common use case for running ./mach taskgraph is to examine what changed in the taskgraph from one revision to the next. The --diff flag can facilitate this by automatically updating to the base revision of your stack (or a specified revision) and generating the taskgraph there. It will then update back to the current revision, generate the taskgraph again and produce a diff of the two. It can work with both labels and JSON (-J). All flags (such as --target-kind, --parameters or --target-tasks-filter) will apply to both graphs so can therefore be used in conjunction with --diff.

For example:

./mach taskgraph target --target-kind test --fast -J --diff

The above will display a diff of the current taskgraph compared to the base revision of your stack. To compare against arbitrary revisions you can use --diff <specifier>. E.g:

./mach taskgraph target --diff .~1    # hg
./mach taskgraph target --dif HEAD~1  # git

The command used to perform the diff can be set via the taskgraph.diffcmd setting. For example, add the following to your ~/.mozbuild/machrc file:

[taskgraph]
diffcmd = diff --side-by-side