How controllers from industrial machinery can coordinate multitask machine learning

Summary

Multitask machine learning often suffers from gradient conflicts, drift, and drought when blending multiple objective gradients in single update steps. To address this in graph self-supervised learning, researchers developed ControlG, a framework that allocates discrete computational blocks to individual objectives over time. ControlG operates across three coupled loops: sensing objective difficulty via spectral demand and gradient interference, planning target compute allocations using log-hypervolume Pareto sensitivities, and executing the schedule via a proportional-integral-derivative controller. Evaluated across homophilic and heterophilic graph benchmarks, the approach outperforms traditional multitask gradient-manipulation baselines while incurring low computational overhead. The framework also produces interpretable training timelines and deficit traces.

Takeaways

1. 1

   Simply separating training objectives across time via random block scheduling beats several gradient-manipulation methods like AutoSSL, PCGrad, and CAGrad on node clustering.

2. 2

   ControlG uses spectral demand (measured via the Rayleigh quotient of per-node gradients) and multiple-gradient descent interference to estimate per-objective difficulty.

3. 3

   On the ogbn-arxiv benchmark, ControlG achieved 72.86% node classification accuracy, outperforming CAGrad while running substantially faster than AutoSSL and ParetoGNN.