We have a setup with one Jenkins master and Zuul triggers the job through the
Jenkins Gearman plugin.
Sometimes no new jobs will be scheduled even though all slots are available.
A workaround for the slaves is to disconnect/connect the slave and then new
job would be scheduled again.
For the master the only way to get jobs to be scheduled again is to restart
the Jenkins service.
When this happens on one node jobs would still be scheduled on other nodes.
Attaching server thread log for gearman threads when no jobs are currently
running and jobs are scheduled in the queue.
Also attaching a trunkated jenkins.log by (grep -C 2 10.33.14.26_manager)
Let me know if you need more logs or other info, I would be happy to help