Meta-scheduling
Technique for optimising workloads across multiple job schedulers
Meta-scheduling (also called super-scheduling) is a computer software technique for optimising computational workloads by coordinating multiple underlying job schedulers. A meta-scheduler sits above the individual schedulers within a distributed environment, such as a computing grid or a multi-site high-performance computing facility, and provides an aggregated view of available resources, allowing batch jobs to be directed to the most appropriate location for execution.
The term is also used in the context of embedded real-time systems for a related but distinct concept: generating schedules that cover multiple anticipated operating scenarios or modes, so the system can switch between pre-computed schedules at runtime.
01Grid computing context
In grid computing, organisations may run different job schedulers at different sites (for example PBS, SLURM, or LSF). A meta-scheduler provides a unified interface that abstracts these differences, enabling users to submit jobs without specifying which underlying system will execute them. The meta-scheduler selects the target based on factors such as current queue depth, resource availability, job requirements and policy constraints. This approach increases overall utilisation and simplifies access for users working across multiple administrative domains.
Implementations
The following is a partial list of open-source and commercial meta-schedulers used in grid and cluster computing:
- GridWay, developed under the Globus Alliance framework
- Community Scheduler Framework, by Platform Computing and Jilin University
- Moab Cluster Suite and Maui Cluster Scheduler, from Adaptive Computing
- Accelerator Plus, which routes jobs via host jobs in an underlying workload manager to achieve high throughput
- SynfiniWay's meta-scheduler
02Embedded systems context
In embedded real-time systems, particularly those using time-triggered architectures such as multi-core systems-on-chip (MPSoCs), meta-scheduling refers to the generation of a set of pre-computed schedules, one per anticipated operating scenario or mode, that the system can select between at runtime without incurring the overhead of dynamic scheduling.
This scenario-based meta-scheduling (SBMeS) approach is applicable to reconfigurable systems and those with variable workloads. By preparing schedules in advance for each expected mode of operation, the system can respond to environmental or workload changes by switching to a different pre-validated schedule rather than recomputing one dynamically. This reduces scheduling overhead and can improve fault recovery behaviour.
Sources and credits
This article is adapted from the Wikipedia article “Meta-scheduling”, written by its contributors and licensed under CC BY-SA 4.0. Fathomly has changed the layout, removed citation markers, navigation and maintenance notices, and adjusted punctuation. This adapted version is shared under the same license. For references, see the original article.
Fathomly is not affiliated with or endorsed by the Wikimedia Foundation. Spotted a problem? Tell us.