Load Balancing of Jobs with Chained Dependencies

Features

• Parameterized virtual memory constraints for each Job, optional guarantee of the in-RAM computations of all Jobs
• Chained rescheduling of the heavier Jobs with the same category to meet RAM limitation / timeout constraints
• Load balancing of the worker processes combined with jobs queue rescheduling, automatic reduction of the number of workers to compute heavier jobs withing the specified memory limit / in-RAM if jobs rescheduling does not help
• Unittests integrated

Fixes & Optimizations

• Forced termination of the job works fine even when SIG_TERM is ignored
• Lots of fixes and optimizations related to the scheduling

Known Bugs

• _LIMIT_WORKERS_RAM causes huge degrade of the rescheduling performance when the worker processes meet the specified constraint
• Jobs rescheduling with _CHAINED_CONSTRAINTS does not kill jobs related to the terminated origin if they are in the terminating state with requested restart or are rescheduled because of the group violation of the memory constraints