This is a half-baked thought I had while thinking about #52. I'm writing it down to capture the idea only. I don't know if it's feasible, or even desirable, but there's likely some interesting stuff that could come out of thinking more deeply about it!
When a user triggers an action, the client (e.g Treeherder) calls the Taskcluster triggerHook API for that action. The hook's template is the contents of .taskcluster.yml, so the action task itself is defined in-tree. This is why anytime we modify .taskcluster.yml, we need to re-generate hooks, so the new action hooks exist.
Now all the context for the action (which callback to invoke, which arguments to pass in, etc) are all provided via the hook's trigger input. So the action task itself is really just the stuff outside around it.. what pool to use, what docker-image to load, the taskgraph command to invoke, etc.
The Idea
Instead of defining the action task in .taskcluster.yml, what if we defined it in fxci-config? E.g like we do for some other hooks:
https://github.com/mozilla-releng/fxci-config/blob/main/hooks/project-relman/code-coverage-crontrigger-production.yml
For the most part, we want the action task definition to be standardized across projects as much as possible anyway. In this world, instead of the hook's payload being the contents of .taskcluster.yml, it would simply be the static one defined in fxci-config. The obvious benefit here, is that modifying .taskcluster.yml would no longer invalidate the action hooks. But it would also simplify the .taskcluster.yml files and mean we have a single standard way of calling into actions.
There are things defined in the .taskcluster.yml file that should be different across repos. Things like the trust domain, level and Taskgraph version to use. But all of this context could be passed into the action task definition via the trigger input, alongside the callback name and other arguments. Treeherder already downloads actions.json from the Decision task, so we could conceivably just stuff any additional context required in there, and have Treeherder pass it along in the triggerHook request.
There are probably downsides and chain of trust implications to doing this so, more thought is needed!
Getting Rid of .taskcluster.yml Entirely
If moving action task definitions out of the tree is a good idea, then why not also consider Decision tasks! All the same logic applies: we want them to be consistent across repos and they all do the same things with the exception of a few differences that are easily parametrized away.
The idea is the same:
Define a standard decision task definition in fxci-config as a hook, then trigger the hook and render it with only the context necessary. The difference is that instead of Treeherder triggering the hook, it would be the Taskcluster Github service. It could render it with all the same properties as it currently renders .taskcluster.yml. Though some things, like which image to use, which Taskcluster version to use, etc.. Would need to be fetched from the project repo somehow. Unlike with actions, we wouldn't have the benefit of a prior Decision task to pull artifacts from. Perhaps this extra context could continue living in .taskcluster.yml, but it would be in a dedicated context top-level key (we discussed similar ideas in the past as part of taskcluster/taskcluster-rfcs#182). But crucially, there'd be no tasks key, so the .taskcluster.yml would be very bare-bones. Maybe we could even specify which hook to trigger in here somewhere.
I'm a bit less clear on the benefits of doing this. Not needing to maintain N different .taskcluster.yml files would be nice, and removing them would likely be less confusing for folks who happen to stumble into them.. And generally, Taskcluster Github being able to use triggerHook instead of only being limited to rendering a .tc.yml seems like a nice feature to have..
But otherwise I still need to think through what this might mean for us and how we might be able to benefit. Thoughts and comments welcome!
This is a half-baked thought I had while thinking about #52. I'm writing it down to capture the idea only. I don't know if it's feasible, or even desirable, but there's likely some interesting stuff that could come out of thinking more deeply about it!
When a user triggers an action, the client (e.g Treeherder) calls the Taskcluster triggerHook API for that action. The hook's template is the contents of
.taskcluster.yml, so the action task itself is defined in-tree. This is why anytime we modify.taskcluster.yml, we need to re-generate hooks, so the new action hooks exist.Now all the context for the action (which callback to invoke, which arguments to pass in, etc) are all provided via the hook's trigger input. So the action task itself is really just the stuff outside around it.. what pool to use, what docker-image to load, the taskgraph command to invoke, etc.
The Idea
Instead of defining the action task in
.taskcluster.yml, what if we defined it infxci-config? E.g like we do for some other hooks:https://github.com/mozilla-releng/fxci-config/blob/main/hooks/project-relman/code-coverage-crontrigger-production.yml
For the most part, we want the action task definition to be standardized across projects as much as possible anyway. In this world, instead of the hook's payload being the contents of
.taskcluster.yml, it would simply be the static one defined infxci-config. The obvious benefit here, is that modifying.taskcluster.ymlwould no longer invalidate the action hooks. But it would also simplify the.taskcluster.ymlfiles and mean we have a single standard way of calling into actions.There are things defined in the
.taskcluster.ymlfile that should be different across repos. Things like the trust domain, level and Taskgraph version to use. But all of this context could be passed into the action task definition via the trigger input, alongside the callback name and other arguments. Treeherder already downloadsactions.jsonfrom the Decision task, so we could conceivably just stuff any additional context required in there, and have Treeherder pass it along in thetriggerHookrequest.There are probably downsides and chain of trust implications to doing this so, more thought is needed!
Getting Rid of
.taskcluster.ymlEntirelyIf moving action task definitions out of the tree is a good idea, then why not also consider Decision tasks! All the same logic applies: we want them to be consistent across repos and they all do the same things with the exception of a few differences that are easily parametrized away.
The idea is the same:
Define a standard decision task definition in
fxci-configas a hook, then trigger the hook and render it with only the context necessary. The difference is that instead of Treeherder triggering the hook, it would be the Taskcluster Github service. It could render it with all the same properties as it currently renders.taskcluster.yml. Though some things, like which image to use, which Taskcluster version to use, etc.. Would need to be fetched from the project repo somehow. Unlike with actions, we wouldn't have the benefit of a prior Decision task to pull artifacts from. Perhaps this extra context could continue living in.taskcluster.yml, but it would be in a dedicatedcontexttop-level key (we discussed similar ideas in the past as part of taskcluster/taskcluster-rfcs#182). But crucially, there'd be notaskskey, so the.taskcluster.ymlwould be very bare-bones. Maybe we could even specify which hook to trigger in here somewhere.I'm a bit less clear on the benefits of doing this. Not needing to maintain N different .taskcluster.yml files would be nice, and removing them would likely be less confusing for folks who happen to stumble into them.. And generally, Taskcluster Github being able to use
triggerHookinstead of only being limited to rendering a.tc.ymlseems like a nice feature to have..But otherwise I still need to think through what this might mean for us and how we might be able to benefit. Thoughts and comments welcome!