is there a way to get launcher to load an osquery extension?

Not really. You could manually start the extension pointing at the socket path, but launcher doesn't expose hooks for this

hrm ok. I guess people just set up their own update server if they need to?

What kind of update server? What does that have to do with extensions?

like the update server that launcher downloads from

I'm not sure what that had to do with extensions. I know a bunch of people use kolide a update server. I've seen a couple people run their own.