@seph fwiw our testing found ATC working as expected: https://github.com/fleetdm/fleet/issues/18490#issuecomment-2075600610

Thanks for testing it. We’re still working through on our side.

I just tried to update to 5.12.1 and think we're also running into this issue when an extension that provides table plugins registers first. Let me know what you all find!

Micah dug pretty far. I’m sure it’s a bug, and #8233 was the wrong direction. See https://github.com/osquery/osquery/issues/8323

I'm inquiring further with our QA team because we may not have tested with the table plugin scenario: https://github.com/fleetdm/fleet/issues/18490#issuecomment-2088874873

Yeah I'm writing up a comment on the issue now - I think I've figured out why this happens too, and it is specifically triggered by table plugin registration

Awesome, thanks Brad!

Ok left the comment! (cc @Micah Sorenson)

Thank you for adding more context on the issue! I started poking around the database init code a little bit ago, but since you’re more familiar with what the fix would be I’ll wait to see your example PR.

Zack — no worries about your QA, I think you said y’all didn’t ship ATC tables, so I don’t know that we expected you to catch it

I requested that we specifically test ATC though so I want to figure out why we missed it.

Ah, yes. Well 🤷

OK curious what you all think of this idea: https://github.com/osquery/osquery/pull/8324

I’m not sure if Micah has had a chance to review yet.

Sorry for the delay, I left a comment on the PR just now in support of the solution. I tested it as well and it fixes all of the failure conditions I encountered while debugging. I really like the approach as it’s not changing anything around initialization.

Chatting about this in office hours, I think it’s the only pertinent thing. • We want to get a 0.12.2 cut soon. • doing a narrow revert of the problem PR is easy • But there’s an underlying race condition • So the question becomes whether we have confidence in Brad’s PR (#8324) to put it into a patch release Micah, Zach — Either of you want to review that enough to weigh on whether to put it in a patch release?

I feel mostly confident that #8324 will fix the issue without introducing any new ones. However since the initial problem was just that the ATC table attach would create error noise, it may be nice to not introduce this change for one release, while we work towards having the fix more throughly tested for the release after that. Personally I think the safety of not introducing potential new bugs outweighs fixing the ATC table error noise, but I also don’t think this PR would introduce any new bugs.

Sorry that I missed this. I do think we should cut the 5.12.2 with just the revert and then I would love to get Brad's work merged for 5.13. I see that you went ahead with this. Thank you!

No worries! Stefano and Micah were aligned, so it was easy

We are pushing this out to our

edge

channel.

Yes sounds good to me too, thanks all! I'll try rolling out 5.12.2 next week as well