gecko_taskgraph.transforms package

Subpackages

Submodules

gecko_taskgraph.transforms.artifact module

gecko_taskgraph.transforms.attribution module

gecko_taskgraph.transforms.attribution.attribution_keyed_by(config, jobs)
gecko_taskgraph.transforms.attribution.mac_attribution(config, jobs)

Adds padding to the mac attribution data. Implicitly assumes that the attribution data is the last thing in job.run.command

gecko_taskgraph.transforms.attribution.remove_attributes(config, jobs)

Remove attributes from parent task that aren’t necessary.

gecko_taskgraph.transforms.attribution.set_locale_label(config, jobs)
gecko_taskgraph.transforms.attribution.set_treeherder(config, jobs)
gecko_taskgraph.transforms.attribution.stub_installer(config, jobs)

Not all windows builds come with a stub installer (only win32, and not on esr), so conditionally add it here based on our dependency’s stub-installer attribute.

gecko_taskgraph.transforms.balrog_submit module

gecko_taskgraph.transforms.balrog_toplevel module

Transform the beetmover task into an actual task description.

gecko_taskgraph.transforms.balrog_toplevel.generate_update_line(config, jobs)

Resolve fields that can be keyed by platform, etc.

gecko_taskgraph.transforms.beetmover module

gecko_taskgraph.transforms.beetmover_apt module

gecko_taskgraph.transforms.beetmover_apt.beetmover_apt(config, tasks)
gecko_taskgraph.transforms.beetmover_apt.filter_beetmover_apt_dep(task)
gecko_taskgraph.transforms.beetmover_apt.filter_out_extra_langpacks(config, tasks)

Langpacks are architecture-independent so we only upload them from the x86_64 tasks

gecko_taskgraph.transforms.beetmover_apt.is_not_l10n_task(task)
gecko_taskgraph.transforms.beetmover_apt.is_x86_64_l10n_task(task)

gecko_taskgraph.transforms.beetmover_checksums module

gecko_taskgraph.transforms.beetmover_emefree_checksums module

gecko_taskgraph.transforms.beetmover_geckoview module

gecko_taskgraph.transforms.beetmover_integration module

gecko_taskgraph.transforms.beetmover_langpack_checksums module

gecko_taskgraph.transforms.beetmover_push_to_release module

gecko_taskgraph.transforms.beetmover_repackage module

gecko_taskgraph.transforms.beetmover_repackage_l10n module

Transform the signing task into an actual task description.

gecko_taskgraph.transforms.beetmover_repackage_l10n.make_beetmover_description(config, jobs)

gecko_taskgraph.transforms.beetmover_repackage_partner module

gecko_taskgraph.transforms.beetmover_repackage_rpm module

gecko_taskgraph.transforms.beetmover_rpm module

gecko_taskgraph.transforms.beetmover_rpm.beetmover_rpm(config, tasks)

gecko_taskgraph.transforms.beetmover_source module

Transform the beetmover-source task to also append build as dependency

gecko_taskgraph.transforms.beetmover_source.remove_build_dependency_in_beetmover_source(config, jobs)

gecko_taskgraph.transforms.beetmover_source_checksums module

gecko_taskgraph.transforms.bootstrap module

gecko_taskgraph.transforms.bouncer_aliases module

gecko_taskgraph.transforms.bouncer_check module

gecko_taskgraph.transforms.bouncer_check.add_command(config, jobs)
gecko_taskgraph.transforms.bouncer_check.add_previous_versions(config, jobs)
gecko_taskgraph.transforms.bouncer_check.command_to_string(config, jobs)

Convert command to string to make it work properly with run-task

gecko_taskgraph.transforms.bouncer_check.handle_keyed_by(config, jobs)

Resolve fields that can be keyed by project, etc.

gecko_taskgraph.transforms.bouncer_locations module

gecko_taskgraph.transforms.bouncer_locations.make_task_worker(config, jobs)

gecko_taskgraph.transforms.bouncer_submission module

gecko_taskgraph.transforms.bouncer_submission_partners module

gecko_taskgraph.transforms.build module

Apply some defaults and minor modifications to the jobs defined in the build kind.

gecko_taskgraph.transforms.build.add_signing_artifacts(config, jobs)

Add signing artifacts to macOS build jobs.

gecko_taskgraph.transforms.build.enable_full_crashsymbols(config, jobs)

Enable full crashsymbols on jobs with ‘enable-full-crashsymbols’ set to True and on release branches, or on try

gecko_taskgraph.transforms.build.mozconfig(config, jobs)
gecko_taskgraph.transforms.build.resolve_keys(config, jobs)
gecko_taskgraph.transforms.build.resolve_shipping_product(config, jobs)
gecko_taskgraph.transforms.build.set_ccov_attribute(config, jobs)
gecko_taskgraph.transforms.build.set_defaults(config, jobs)

Set defaults, including those that differ per worker implementation

gecko_taskgraph.transforms.build.set_expiry(config, jobs)
gecko_taskgraph.transforms.build.stub_installer(config, jobs)
gecko_taskgraph.transforms.build.update_channel(config, jobs)
gecko_taskgraph.transforms.build.use_artifact(config, jobs)
gecko_taskgraph.transforms.build.use_profile_data(config, jobs)

gecko_taskgraph.transforms.build_attrs module

gecko_taskgraph.transforms.build_attrs.set_build_attributes(config, jobs)

Set the build_platform and build_type attributes based on the job name. Although not all jobs using this transform are actual “builds”, the try option syntax treats them as such, and this arranges the attributes appropriately for that purpose.

gecko_taskgraph.transforms.build_lints module

Apply some defaults and minor modifications to the jobs defined in the build kind.

gecko_taskgraph.transforms.build_lints.check_mozharness_perfherder_options(config, jobs)

Verify that multiple jobs don’t use the same perfherder bucket.

Build jobs record perfherder metrics by default. Perfherder metrics go to a bucket derived by the platform by default. The name can further be customized by the presence of “extra options” either defined in mozharness sub-configs or in an environment variable.

This linter tries to verify that no 2 jobs will send Perfherder metrics to the same bucket by looking for jobs not defining extra options when their platform or mozharness config are otherwise similar.

gecko_taskgraph.transforms.build_schedules module

gecko_taskgraph.transforms.build_schedules.set_build_schedules_optimization(config, tasks)

Set the build optimization based on the build platform.

gecko_taskgraph.transforms.build_signing module

Transform the signing task into an actual task description.

gecko_taskgraph.transforms.build_signing.add_signed_routes(config, jobs)

Add routes corresponding to the routes of the build task this corresponds to, with .signed inserted, for all gecko.v2 routes

gecko_taskgraph.transforms.build_signing.define_upstream_artifacts(config, jobs)

gecko_taskgraph.transforms.cached_tasks module

gecko_taskgraph.transforms.cached_tasks.bump_priority(config, tasks)

Bump priority of cached tasks on autoland from low to medium to avoid breakage for developers

gecko_taskgraph.transforms.cached_tasks.cache_task(config, tasks)
gecko_taskgraph.transforms.cached_tasks.format_task_digest(cached_task)
gecko_taskgraph.transforms.cached_tasks.order_tasks(config, tasks)

Iterate image tasks in an order where parent tasks come first.

gecko_taskgraph.transforms.chunk_partners module

Chunk the partner repack tasks by subpartner and locale

gecko_taskgraph.transforms.chunk_partners.chunk_partners(config, jobs)

gecko_taskgraph.transforms.code_review module

Add soft dependencies and configuration to code-review tasks.

gecko_taskgraph.transforms.code_review.add_dependencies(config, jobs)
gecko_taskgraph.transforms.code_review.add_github_config(config, jobs)
gecko_taskgraph.transforms.code_review.add_phabricator_config(config, jobs)

gecko_taskgraph.transforms.condprof module

gecko_taskgraph.transforms.copy_cached_dep module

gecko_taskgraph.transforms.copy_cached_dep.copy_cached_dep(config, tasks)

Ensure this task is replaced anytime the primary dep is.

gecko_taskgraph.transforms.custom_car module

gecko_taskgraph.transforms.custom_car.add_custom_car_optimization(config, tasks)

gecko_taskgraph.transforms.diffoscope module

gecko_taskgraph.transforms.docker_image module

gecko_taskgraph.transforms.fetch module

gecko_taskgraph.transforms.filter_notarization module

Filter notarization tasks to only run at level 3.

gecko_taskgraph.transforms.filter_notarization.only_level_3_notarization(config, jobs)

Filter out any notarization jobs that are not level 3

gecko_taskgraph.transforms.final_verify module

Transform the beetmover task into an actual task description.

gecko_taskgraph.transforms.final_verify.add_command(config, tasks)

gecko_taskgraph.transforms.geckodriver_signing module

gecko_taskgraph.transforms.hardened_signing module

Transform the signing task into an actual task description.

gecko_taskgraph.transforms.hardened_signing.add_hardened_sign_config(config, jobs)
gecko_taskgraph.transforms.hardened_signing.add_provisioning_profile_config(config, jobs)
gecko_taskgraph.transforms.hardened_signing.add_upstream_signing_resources(config, jobs)

Add the upstream signing resources to the job payload

gecko_taskgraph.transforms.l10n module

gecko_taskgraph.transforms.mac_dummy module

Add dependencies to dummy macosx64 tasks.

gecko_taskgraph.transforms.mac_dummy.add_dependencies(config, jobs)

gecko_taskgraph.transforms.mac_notarization module

gecko_taskgraph.transforms.mar_signing module

Transform the {partials,mar}-signing task into an actual task description.

gecko_taskgraph.transforms.mar_signing.generate_complete_artifacts(job, kind)
gecko_taskgraph.transforms.mar_signing.generate_partials_artifacts(job, release_history, platform, upstream_kind, locale=None)
gecko_taskgraph.transforms.mar_signing.make_task_description(config, jobs)

gecko_taskgraph.transforms.mark_as_merged module

gecko_taskgraph.transforms.mark_as_merged.make_task_description(config, jobs)

gecko_taskgraph.transforms.maybe_release module

gecko_taskgraph.transforms.maybe_release.make_task_description(config, jobs)

gecko_taskgraph.transforms.merge_automation module

Transform the update generation task into an actual task description.

gecko_taskgraph.transforms.merge_automation.add_payload_config(config, tasks)
gecko_taskgraph.transforms.merge_automation.handle_keyed_by(config, tasks)

Resolve fields that can be keyed by platform, etc.

gecko_taskgraph.transforms.merge_automation.update_labels(config, tasks)

gecko_taskgraph.transforms.name_sanity module

Generate labels for tasks without names, consistently. Uses attributes from primary dependency.

gecko_taskgraph.transforms.name_sanity.make_label(config, jobs)

Generate a sane label for a new task constructed from a dependency Using attributes from the dependent job and the current task kind

gecko_taskgraph.transforms.openh264 module

gecko_taskgraph.transforms.openh264.set_openh264_version(config, jobs)

gecko_taskgraph.transforms.openh264_signing module

gecko_taskgraph.transforms.openh264_symbol_upload module

Transform the openh264-symbol-upload task into an actual task description.

gecko_taskgraph.transforms.openh264_symbol_upload.make_symbol_upload_description(config, jobs)

gecko_taskgraph.transforms.partials module

Transform the partials task into an actual task description.

gecko_taskgraph.transforms.partials.identify_desired_signing_keys(project, product)
gecko_taskgraph.transforms.partials.make_task_description(config, jobs)

gecko_taskgraph.transforms.partials_zucchini module

Transform the partials task into an actual task description.

gecko_taskgraph.transforms.partials_zucchini.find_enus_partials_zucchini_task(config, build_platform, build_type)

Find the matching enUS partials-zucchini task for an l10n task by iterating kind_dependencies_tasks and filtering on attributes.

gecko_taskgraph.transforms.partials_zucchini.identify_desired_signing_keys(config)

Determine the signing certificate type to use for MAR validation. It identifies the appropriate signing type based on the release product and configuration parameters, handling special logic for devedition releases.

Parameters:

config – The task graph configuration object containing:

Returns:

The signing type (defined in util/scriptworker.py)

Return type:

str

gecko_taskgraph.transforms.partials_zucchini.make_task_description(config, tasks)

gecko_taskgraph.transforms.partner_attribution module

Transform the partner attribution task into an actual task description.

gecko_taskgraph.transforms.partner_attribution.add_command_arguments(config, tasks)

gecko_taskgraph.transforms.partner_attribution_beetmover module

gecko_taskgraph.transforms.partner_repack module

Transform the partner repack task into an actual task description.

gecko_taskgraph.transforms.partner_repack.add_command_arguments(config, tasks)
gecko_taskgraph.transforms.partner_repack.add_macos_signing_artifacts(config, tasks)
gecko_taskgraph.transforms.partner_repack.make_label(config, tasks)
gecko_taskgraph.transforms.partner_repack.populate_repack_manifests_url(config, tasks)
gecko_taskgraph.transforms.partner_repack.remove_mac_dependency(config, tasks)

Remove mac dependency depending on current level to accomodate for mac notarization not running on level 1

gecko_taskgraph.transforms.partner_repack.skip_unnecessary_platforms(config, tasks)

gecko_taskgraph.transforms.partner_signing module

Transform the signing task into an actual task description.

gecko_taskgraph.transforms.partner_signing.define_upstream_artifacts(config, jobs)
gecko_taskgraph.transforms.partner_signing.set_mac_label(config, jobs)

gecko_taskgraph.transforms.per_platform_dummy module

Transform the repackage task into an actual task description.

gecko_taskgraph.transforms.per_platform_dummy.one_task_per_product_and_platform(config, jobs)

gecko_taskgraph.transforms.perftest module

gecko_taskgraph.transforms.python_update module

Transform the repo-update task into an actual task description.

gecko_taskgraph.transforms.python_update.resolve_keys(config, tasks)

gecko_taskgraph.transforms.release module

Transforms for release tasks

gecko_taskgraph.transforms.release.run_on_releases(config, jobs)

Filter out jobs with run-on-releases set, and that don’t match the release_type paramater.

gecko_taskgraph.transforms.release_beetmover_signed_addons module

gecko_taskgraph.transforms.release_deps module

Add dependencies to release tasks.

gecko_taskgraph.transforms.release_deps.add_dependencies(config, jobs)

gecko_taskgraph.transforms.release_flatpak_push module

gecko_taskgraph.transforms.release_generate_checksums module

Transform the checksums task into an actual task description.

gecko_taskgraph.transforms.release_generate_checksums.handle_keyed_by(config, jobs)

Resolve fields that can be keyed by project, etc.

gecko_taskgraph.transforms.release_generate_checksums.interpolate(config, jobs)

gecko_taskgraph.transforms.release_generate_checksums_beetmover module

gecko_taskgraph.transforms.release_generate_checksums_signing module

gecko_taskgraph.transforms.release_mark_as_shipped module

gecko_taskgraph.transforms.release_mark_as_shipped.make_task_description(config, jobs)

gecko_taskgraph.transforms.release_msix_push module

gecko_taskgraph.transforms.release_notifications module

Add notifications via taskcluster-notify for release tasks

class gecko_taskgraph.transforms.release_notifications.TitleCaseFormatter

Bases: Formatter

Support title formatter for strings

convert_field(value, conversion)
gecko_taskgraph.transforms.release_notifications.add_notifications(config, jobs)

gecko_taskgraph.transforms.release_sign_and_push_langpacks module

gecko_taskgraph.transforms.release_snap_repackage module

gecko_taskgraph.transforms.release_snap_repackage.format(config, tasks)

Apply format substitution to worker.env and worker.command.

gecko_taskgraph.transforms.release_started module

Add notifications via taskcluster-notify for release tasks

gecko_taskgraph.transforms.release_started.add_notifications(config, jobs)

gecko_taskgraph.transforms.release_update_product_channel_version module

gecko_taskgraph.transforms.release_update_product_channel_version.make_task_worker(config, jobs)

gecko_taskgraph.transforms.repackage module

gecko_taskgraph.transforms.repackage_l10n module

Transform the repackage task into an actual task description.

gecko_taskgraph.transforms.repackage_l10n.split_locales(config, jobs)

gecko_taskgraph.transforms.repackage_partner module

gecko_taskgraph.transforms.repackage_pkg_notarization module

gecko_taskgraph.transforms.repackage_pkg_notarization.define_upstream_artifacts(config, jobs)
gecko_taskgraph.transforms.repackage_pkg_notarization.treeherder(config, jobs)

gecko_taskgraph.transforms.repackage_pkg_signing module

gecko_taskgraph.transforms.repackage_routes module

Add indexes to repackage kinds

gecko_taskgraph.transforms.repackage_routes.add_indexes(config, jobs)

gecko_taskgraph.transforms.repackage_rpm_signing module

Transform the repackage-rpm-signing task into an actual task description.

gecko_taskgraph.transforms.repackage_rpm_signing.make_task_description(config, jobs)

gecko_taskgraph.transforms.repackage_set_upstream_mac_kind module

Transform mac notarization tasks

gecko_taskgraph.transforms.repackage_set_upstream_mac_kind.repackage_set_upstream_mac_kind(config, tasks)

Notarization only runs on level 3 If level < 3 then repackage the mac-signing task artifact Exception for debug builds, which will use signed build on level 3

gecko_taskgraph.transforms.repackage_signing module

gecko_taskgraph.transforms.repackage_signing_partner module

gecko_taskgraph.transforms.repo_update module

Transform the repo-update task into an actual task description.

gecko_taskgraph.transforms.repo_update.resolve_keys(config, tasks)

gecko_taskgraph.transforms.reprocess_symbols module

Transform the reprocess-symbols task description template, taskcluster/kinds/reprocess-symbols/job-template.yml into an actual task description.

gecko_taskgraph.transforms.reprocess_symbols.fill_template(config, tasks)

gecko_taskgraph.transforms.resolve_landoscript_keyed_by module

Transform the update generation task into an actual task description.

gecko_taskgraph.transforms.resolve_landoscript_keyed_by.handle_keyed_by(config, tasks)

Resolve fields that can be keyed by platform, etc.

gecko_taskgraph.transforms.reverse_chunk_deps module

Adjust dependencies to not exceed MAX_DEPENDENCIES

gecko_taskgraph.transforms.reverse_chunk_deps.add_dependencies(config, jobs)
gecko_taskgraph.transforms.reverse_chunk_deps.yield_job(orig_job, deps, count)

gecko_taskgraph.transforms.run_pgo_profile module

Apply some defaults and minor modifications to the pgo jobs.

gecko_taskgraph.transforms.run_pgo_profile.run_profile_data(config, jobs)

gecko_taskgraph.transforms.scriptworker module

Transforms for adding appropriate scopes to scriptworker tasks.

gecko_taskgraph.transforms.scriptworker.add_balrog_scopes(config, jobs)

gecko_taskgraph.transforms.scriptworker_canary module

Build a command to run mach release push-scriptworker-canaries.

gecko_taskgraph.transforms.scriptworker_canary.build_command(config, jobs)

gecko_taskgraph.transforms.sentry module

gecko_taskgraph.transforms.sentry.sentry(config, tasks)

Do transforms specific to github-sync tasks.

gecko_taskgraph.transforms.shippable_l10n_signing module

Transform the signing task into an actual task description.

gecko_taskgraph.transforms.shippable_l10n_signing.define_upstream_artifacts(config, jobs)
gecko_taskgraph.transforms.shippable_l10n_signing.make_signing_description(config, jobs)

gecko_taskgraph.transforms.signing module

gecko_taskgraph.transforms.snap_test module

gecko_taskgraph.transforms.snap_test.fill_template(config, tasks)

gecko_taskgraph.transforms.source_checksums_signing module

gecko_taskgraph.transforms.source_test module

gecko_taskgraph.transforms.spidermonkey module

gecko_taskgraph.transforms.spidermonkey.handle_keyed_by(config, jobs)

Resolve fields that can be keyed by platform, etc.

gecko_taskgraph.transforms.split_by_locale module

gecko_taskgraph.transforms.startup_test module

gecko_taskgraph.transforms.startup_test.add_command(config, jobs)
gecko_taskgraph.transforms.startup_test.job_priority(config, jobs)

Override default job priority if set in the yaml

gecko_taskgraph.transforms.task module

gecko_taskgraph.transforms.test_apk module

Apply some defaults and minor modifications to the jobs defined in the test kinds.

gecko_taskgraph.transforms.test_apk.resolve_keys(config, tasks)

gecko_taskgraph.transforms.trigger_comm_central module

Resolve keys for the jobs defined in the trigger-comm-central kind.

gecko_taskgraph.transforms.trigger_comm_central.resolve_keys(config, jobs)

gecko_taskgraph.transforms.try_job module

gecko_taskgraph.transforms.try_job.set_job_try_name(config, jobs)

For a task which is governed by -j in try syntax, set the job_try_name attribute based on the job name.

gecko_taskgraph.transforms.ui_test_apk module

UI-test-apk-specific transforms. Build the Slack notification payload from the task’s shipping-product, and build the test-lab.py invocation from the task’s flank-config / artifact-type / no-test-apk fields.

gecko_taskgraph.transforms.ui_test_apk.build_slack_notification(config, tasks)
gecko_taskgraph.transforms.ui_test_apk.build_test_lab_command(config, tasks)

gecko_taskgraph.transforms.update_integrity module

gecko_taskgraph.transforms.update_integrity.add_additional_fetches_and_command(config, jobs)

Adds fetch entries for the “from” installers and partial MARs.

gecko_taskgraph.transforms.update_integrity.add_build_target(config, jobs)
gecko_taskgraph.transforms.update_integrity.add_to_installer(config, jobs)

Adds fetch entries for the “to” installer to fetches.

gecko_taskgraph.transforms.update_integrity.resolve_keys(config, jobs)
gecko_taskgraph.transforms.update_integrity.set_treeherder(config, jobs)
gecko_taskgraph.transforms.update_integrity.skip_for_new_locales_and_platforms(config, jobs)

Don’t generate any jobs for newly added locales or platforms that don’t have from releases to test.

gecko_taskgraph.transforms.update_integrity.skip_for_non_nightly(config, jobs)

Don’t generate any jobs unless running as a nightly. Other code in this transform depends on nightly-specific parameters being set.

gecko_taskgraph.transforms.update_test module

Transform the update-test suite to parametrize by locale, source version, machine

class gecko_taskgraph.transforms.update_test.ReleaseType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

Release type

beta = 1
esr = 2
other = 3
release = 0
gecko_taskgraph.transforms.update_test.get_build_platform(platform)
gecko_taskgraph.transforms.update_test.infix_treeherder_symbol(symbol, infix)
gecko_taskgraph.transforms.update_test.parametrize_by_locale(config, tasks)
gecko_taskgraph.transforms.update_test.parametrize_by_source_version(config, tasks)
gecko_taskgraph.transforms.update_test.set_task_configuration(config, tasks)

gecko_taskgraph.transforms.update_verify module

Transform the beetmover task into an actual task description.

gecko_taskgraph.transforms.update_verify.add_command(config, tasks)

gecko_taskgraph.transforms.update_verify_config module

gecko_taskgraph.transforms.upload_generated_sources module

Transform the upload-generated-files task description template, taskcluster/kinds/upload-generated-sources/kind.yml, into an actual task description.

gecko_taskgraph.transforms.upload_generated_sources.add_task_info(config, jobs)

gecko_taskgraph.transforms.upload_symbols module

Transform the upload-symbols task description template, taskcluster/kinds/upload-symbols/job-template.yml into an actual task description.

gecko_taskgraph.transforms.upload_symbols.check_nightlies(config, tasks)

Ensure that we upload symbols for all shippable builds, so that crash-stats can resolve any reports sent to it. Try may enable full symbols but not upload them.

Putting this check here (instead of the transforms for the build kind) lets us leverage the any not-for-build-platforms set in the update-symbols kind.

gecko_taskgraph.transforms.upload_symbols.fill_template(config, tasks)

gecko_taskgraph.transforms.zucchini_partial_rollout module

gecko_taskgraph.transforms.zucchini_partial_rollout.filter_partials_by_project(config, tasks)

Control the rollout of partials-zucchini across release channels.

This transform manages the gradual transition from the legacy “partials” implementation to the new “partials-zucchini” implementation. It ensures that partials-zucchini is only used on nightly builds, allowing thorough testing before the implementation rides the train to beta, release, and ESR channels.

The transform filters tasks based on their primary dependency (partials or partials-zucchini) and the current project/channel, ensuring the appropriate implementation is used for each release channel.

Module contents