Now to the problem: I have one windows machine (virtual box) that refuses to "talk" to fleet. Honestly I do not know where the error is. Fleet shows the machine (with some basic info but as "never fetched"). But I cannot run queries. This is also the same if I reset the osquery db on the machine. And even more curious I cannot delete the machine from fleet. The error of the HTTP request in chrome dev tools shows as follows:

{
  "message": "unsupported Scan, storing driver.Value type <nil> into type *time.Time",
  "errors": [
    {
      "name": "base",
      "reason": "unsupported Scan, storing driver.Value type <nil> into type *time.Time"
    }
  ]
}

Any ideas?

As for the error log above, it looks like a bug, we'll try to reproduce on our end and get back to you.

Yes, I've updated from 4.3.1. I think the upgrade logs are gone, but I did not remember anything special. When I start fleet the following shows up:

fleet_1          | {"component":"crons","cron":"cleanups","details":"looping through ids: running visitFunc for queries: getting user_time p50 for query 1: timestamp: 2021-11-22T13:45:38+01:00: Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'row_number, mm.* FROM (\n\t\tSELECT d.scheduled_query_id, d.user_time, d.executions' at line 4","err":"aggregating query stats","level":"error","ts":"2021-11-22T12:45:38.017855521Z"}
fleet_1          | {"component":"crons","cron":"cleanups","details":"looping through ids: running visitFunc for scheduled_queries: getting user_time p50 for scheduled_query 4: timestamp: 2021-11-22T13:45:38+01:00: Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'row_number, mm.* FROM (\n\t\tSELECT d.scheduled_query_id, d.user_time, d.executions' at line 4","err":"aggregating scheduled query stats","level":"error","ts":"2021-11-22T12:45:38.019472394Z"}

OK, did you run

fleet prepare db

before running the new version of fleet?

sure, that's run everytime the container starts

OK, let me check with the team

cool, thx

QQ: MySQL server and version?

@Lucas Rodriguez 8.0.26 - I'm seeing the same error now for another newly onboarded machine

Hi Martin! Another user hit the same error, we have an issue and are working on a fix: https://github.com/fleetdm/fleet/issues/3095

Thanks. I subscribed to the ticket. Will the clients resume to work normally after the fix? If there is need for manual intervention (which I have no problem with) it's okay to mention that in the ticket.

I think there are two issues, 1. One is the fleet side bug (unsupported Scan). 2. The other one is some issue on the osquery side, are you running osquery manually on the VM? (could you get logs?)

no, I run it as service (osquery MSI installer). The logs folder is empty and I did not find anything in win eventlog. Are there any other places to look for?

Hi Martin I just saw your comment, great timing, I've started working on the issue today.

Good to hear 🙂

There are a few related issues we are working on: 1. Proper Orbit logs on Windows 2. Allow setting a "platform" to policies.

we're not using orbit

Not sure, we do support vanilla osquery.

Cool, thanks.

Correct, we will allow configuring platform for policies and run them as live queries to test them out first (coming soon).

We're having this same issue with fleetdm/fleet:main image on k8s and osquery 5.0.1

Hi Flngen! We are working on a fix for this that we'll try to include in fleet 4.6.2. (ETA: some day this week, hopefully Thursday).

@Lucas Rodriguez I've updated to 6.4.2 just now. The hosts that were throwing this error still show the same error 🙂 will it take some time to get these updated? I also cannot delete these hosts, nor force-refresh these via the API (same error)

Hi Martin!, can you double check the version in top right ->`My Account` -> fleet version should show up in the bottom right.

OK, if possible, please run the following query

SELECT * FROM fleet.hosts h LEFT JOIN fleet.host_seen_times hst ON h.id = hst.host_id

(feel free to not include any sensitive data like hostnames)

See PM

Thanks, one more thing, could you also send us the server logs? (those surrounds a scan error you see on the browser)