gecko_taskgraph.loader package¶

Submodules¶

gecko_taskgraph.loader.multi_dep module¶

gecko_taskgraph.loader.multi_dep.assert_unique_members(kinds, error_msg=None)¶

gecko_taskgraph.loader.multi_dep.chunk_locale_grouping(config, tasks)¶

Split by a chunk_locale (but also by platform, build-type, product)

This grouping is written for mac signing with notarization, but might also be useful elsewhere.

gecko_taskgraph.loader.multi_dep.get_primary_dep(config, dep_tasks)¶

Find the dependent task to inherit attributes from.

If primary-dependency is defined in kind.yml and is a string, then find the first dep with that task kind and return it. If it is defined and is a list, the first kind in that list with a matching dep is the primary dependency. If it’s undefined, return the first dep.

gecko_taskgraph.loader.multi_dep.group_by(name)¶

gecko_taskgraph.loader.multi_dep.group_tasks(config, tasks)¶

gecko_taskgraph.loader.multi_dep.loader(kind, path, config, params, loaded_tasks)¶

Load tasks based on the jobs dependant kinds, designed for use as multiple-dependent needs.

Required group-by-fn is used to define how we coalesce the multiple deps together to pass to transforms, e.g. all kinds specified get collapsed by platform with platform

Optional primary-dependency (ordered list or string) is used to determine which upstream kind to inherit attrs from. See get_primary_dep.

The only-for-build-platforms kind configuration, if specified, will limit the build platforms for which a job will be created. Alternatively there is ‘not-for-build-platforms’ kind configuration which will be consulted only after ‘only-for-build-platforms’ is checked (if present), and omit any jobs where the build platform matches.

Optional job-template kind configuration value, if specified, will be used to pass configuration down to the specified transforms used.

gecko_taskgraph.loader.multi_dep.partner_repack_ids_grouping(config, tasks)¶

Split by partner_repack_ids (but also by platform, build-type, product)

This grouping is written for release-{eme-free,partner}-repack-signing.

gecko_taskgraph.loader.multi_dep.platform_grouping(config, tasks)¶

gecko_taskgraph.loader.multi_dep.single_locale_grouping(config, tasks)¶

Split by a single locale (but also by platform, build-type, product)

The locale can be None (en-US build/signing/repackage), a single locale, or multiple locales per task, e.g. for l10n chunking. In the case of a task with, say, five locales, the task will show up in all five locale groupings.

This grouping is written for non-partner-repack beetmover, but might also be useful elsewhere.

gecko_taskgraph.loader.multi_dep.skip_only_or_not(config, task)¶: Return True if we should skip this task based on only_ or not_ config.

gecko_taskgraph.loader.single_dep module¶

gecko_taskgraph.loader.single_dep.loader(kind, path, config, params, loaded_tasks)¶

Load tasks based on the jobs dependant kinds.

The only-for-build-platforms kind configuration, if specified, will limit the build platforms for which a job will be created. Alternatively there is ‘not-for-build-platforms’ kind configuration which will be consulted only after ‘only-for-build-platforms’ is checked (if present), and omit any jobs where the build platform matches.

Optional only-for-attributes kind configuration, if specified, will limit the jobs chosen to ones which have the specified attribute, with the specified value.

Optional job-template kind configuration value, if specified, will be used to pass configuration down to the specified transforms used.

gecko_taskgraph.loader.test module¶

gecko_taskgraph.loader.test.expand_tests(test_sets_cfg, test_platforms)¶: Expand the test sets in test_platforms out to sets of test names. Returns a dictionary like get_test_platforms, with an additional test-names key for each test platform, containing a set of test names.

gecko_taskgraph.loader.test.get_builds_by_platform(dep_kind, loaded_tasks)¶: Find the build tasks on which tests will depend, keyed by platform/type. Returns a dictionary mapping build platform to task.

gecko_taskgraph.loader.test.get_test_platforms(test_platforms_cfg, builds_by_platform, signed_builds_by_platform={})¶: Get the test platforms for which test tasks should be generated, based on the available build platforms. Returns a dictionary mapping test platform to {test-set, build-platform, build-label}.

gecko_taskgraph.loader.test.loader(kind, path, config, params, loaded_tasks)¶: Generate tasks implementing Gecko tests.

gecko_taskgraph.loader.transform module¶

gecko_taskgraph.loader.transform.loader(kind, path, config, params, loaded_tasks)¶

Get the input elements that will be transformed into tasks in a generic way. The elements themselves are free-form, and become the input to the first transform.

By default, this reads jobs from the jobs key, or from yaml files named by jobs-from. The entities are read from mappings, and the keys to those mappings are added in the name key of each entity.

If there is a job-defaults config, then every job is merged with it. This provides a simple way to set default values for all jobs of a kind. The job-defaults key can also be specified in a yaml file pointed to by jobs-from. In this case it will only apply to tasks defined in the same file.

Other kind implementations can use a different loader function to produce inputs and hand them to transform_inputs.