I don't think they should delete themselves. We can just add a set of ids of oneshot queries that were already run.