News from Azureland says that "rerun failed jobs" is finally fixed. I just checked, it works, so no more close and reopen issues.
They've also added Ubuntu 18.04 to the pool of VM agent images, which means that if we want we can stop generating the Dockerfile and have a bit more speed when compiling, even though it's also worth checking how much time it takes to prepare the VM each time with what we need.