I have a few basic schedules set for my environment, and unfortunately they are not running on the one active computer that I have enrolled. The device is checking in and is able to run packs that are assigned to it, however any of the Scheduled items are not working. I am not seeing them listed on the host screen, or being written to the logs on the end point.

Hey @benbass! Can you share your osquery flags? What do you see if you run this query against that host:

SELECT name, query, denylisted, executions from osquery_schedule

?

Hi Kathy! I am seeing the 3 queries that I have scheduled from the pack that I have working on that mac, and I am not seeing any of the ‘Scheduled’ queries.

Everything looks good there at a glance. Are you seeing any errors in the osquery status logs on the host?

The only error I am seeing is in regards to fleet asking about the mdm table - which I know I don’t have with the vanilla osquery.

Can your run osquery in verbose mode for a couple of hours (add the

--verbose

flag) and send over the status logs? I'd love to get a look at whether the config is being fetched as expected.

Do you want --tls_dump as well?

Can't hurt 🙂

Which logs would you like? When you run it in --verbose it rights to std out - is that what you would like?

--verbose

adds

INFO

level logs, which add some detail about what osquery is doing at any given time.

--tls_dump

includes https body content in tls related logs.

Sounds good. I have been running in --verbose and --tls_dump for about an hour. I am starting to wrap up my day, so I’ll PM you the std out and info logs.

I skipped right past this earlier, but I'm seeing now that you've got logging set to local filesystem:

--logger_plugin=filesystem

Can you check the osquery results logs on the host?

I am seeing the assigned pack results, not anything from the schedule. This was where I initially discovered this. We have our logs shipped to our SIEM and I was not seeing the results there.

Taking a look at the config that's being passed from Fleet, I'm not seeing any scheduled queries coming through there. Just to be sure, did you schedule these queries on the Fleet side, or as part of a local osquery configuration?

I went into the fleet GUI and scheduled them - from there - here is the advanced info from one of them.

What happens if you run one of those queries as a live query?

the live queries run just fine. I just ran that etc_hosts and got a reply back in seconds.

Thanks! I've reached out to the team for some ideas.

Another point of data - the ‘policies’ from the policies tab are running and working as expected.

Thanks for that additional info!

Hi @benbass! Could you share the osquery flags you are using? (What's the value for

--config_plugin

? Should be

tls

for Fleet to be able to configure the agent.)

Hey @Lucas Rodriguez, I've confirmed the tls config endpoint is set up, and the config is coming down from Fleet, but there isn't a Global schedule coming through. I'll send you those logs.

Taking a look.

@benbass Are you able to query the Fleet database? I’d love to see what you get back for

select * from packs

.

Yup, I should be able to.

Thank you! I am noticing that the Global pack has been disabled, which is… odd.

Not that I am aware of. I did disable all of the packs at one point via the GUI, as this is a dev/testing box.

Yes, 100%.

Very interesting. I will keep an eye on that when I upgrade my prod environment next week.

Sounds good! Things have tightened up quite a bit, but it looks like in versions prior to 4.2.0, it was possible to disable that through the REST API. That could be the culprit depending on your version/how long this has been happening. Otherwise, my gut instinct tells me that the database was likely modified manually at some point. I'll keep an ear to the ground in case this comes up again.

I doubt the DB was altered, as I am the only one who really has access to it, and I think these were the first queries I have run on that DB. 🙂. This is a very old install, as the DB goes back to 1.06 I think and has been upgraded through most, and had a rather large jump recently from 4.9.1 to 4.30 then to 4.32.

Had schedule been working recently, or might this be something that goes back that far?

This was the first version that schedule had been in, we jumped from an older version that pre-dated schedule.

Can you try adding a new scheduled query?

I created a new scheduled query about an hour ago and no dice. The data had been refetched from the endpoint about 7 minutes ago, so that info is up to date.

Can you send over a new batch of logs for me?

And make sure to double check the value of

disabled

for the packs? (Given that we don't know what caused it to be

1

)

Here is the output from the mysql db. I manually disabled the packs originally via the GUI, and if you note, the Global Pack is enabled.

mysql> select * from packs \G;
*************************** 1. row ***************************
     id: 3
 created_at: 2018-07-27 16:43:26
 updated_at: 2020-11-19 13:38:34
  disabled: 1
    name: hardware-monitoring
description: 
  platform: 
 pack_type: NULL
*************************** 2. row ***************************
     id: 5
 created_at: 2018-07-27 16:51:34
 updated_at: 2020-11-19 13:38:34
  disabled: 1
    name: incident-response
description: 
  platform: 
 pack_type: NULL
*************************** 3. row ***************************
     id: 6
 created_at: 2018-07-27 16:55:59
 updated_at: 2020-11-19 13:38:34
  disabled: 1
    name: it-compliance
description: 
  platform: 
 pack_type: NULL
*************************** 4. row ***************************
     id: 7
 created_at: 2018-07-27 16:56:08
 updated_at: 2020-11-19 13:38:34
  disabled: 1
    name: osquery-monitoring
description: 
  platform: 
 pack_type: NULL
*************************** 5. row ***************************
     id: 8
 created_at: 2018-07-27 16:56:15
 updated_at: 2020-11-19 13:38:34
  disabled: 1
    name: ossec-rootkit
description: 
  platform: 
 pack_type: NULL
*************************** 6. row ***************************
     id: 9
 created_at: 2018-07-27 16:56:25
 updated_at: 2020-11-19 13:38:34
  disabled: 1
    name: osx-attacks
description: 
  platform: 
 pack_type: NULL
*************************** 7. row ***************************
     id: 10
 created_at: 2018-07-27 16:56:32
 updated_at: 2020-11-19 13:38:34
  disabled: 1
    name: unwanted-chrome-extensions
description: 
  platform: 
 pack_type: NULL
*************************** 8. row ***************************
     id: 11
 created_at: 2018-07-27 16:56:38
 updated_at: 2020-11-19 13:38:34
  disabled: 1
    name: vuln-management
description: 
  platform: 
 pack_type: NULL
*************************** 9. row ***************************
     id: 12
 created_at: 2018-07-27 16:56:47
 updated_at: 2020-11-19 13:38:34
  disabled: 1
    name: windows-attacks
description: 
  platform: 
 pack_type: NULL
*************************** 10. row ***************************
     id: 13
 created_at: 2018-07-27 16:56:54
 updated_at: 2020-11-19 13:38:34
  disabled: 1
    name: windows-hardening
description: 
  platform: 
 pack_type: NULL
*************************** 11. row ***************************
     id: 24
 created_at: 2018-07-27 17:53:31
 updated_at: 2020-11-19 13:38:34
  disabled: 1
    name: mitre_att&ck
description: 
  platform: 
 pack_type: NULL
*************************** 12. row ***************************
     id: 31
 created_at: 2018-09-05 15:42:11
 updated_at: 2020-11-19 13:38:34
  disabled: 1
    name: APL_ops
description: 
  platform: 
 pack_type: NULL
*************************** 13. row ***************************
     id: 32
 created_at: 2018-09-05 15:48:48
 updated_at: 2020-11-19 13:38:34
  disabled: 1
    name: APL_threats
description: 
  platform: 
 pack_type: NULL
*************************** 14. row ***************************
     id: 35
 created_at: 2018-09-05 16:08:40
 updated_at: 2020-11-19 13:38:34
  disabled: 1
    name: APL_compliance
description: 
  platform: 
 pack_type: NULL
*************************** 15. row ***************************
     id: 41
 created_at: 2018-09-06 09:42:09
 updated_at: 2018-09-06 10:31:07
  disabled: 1
    name: APL_packless
description: Pack that contains queries currently not in a pack. Making export via yaml easier.
  platform: 
 pack_type: NULL
*************************** 16. row ***************************
     id: 42
 created_at: 2019-09-18 14:19:02
 updated_at: 2020-11-19 13:38:34
  disabled: 1
    name: Nicks_Dev_Pack
description: 
  platform: 
 pack_type: NULL
*************************** 17. row ***************************
     id: 43
 created_at: 2019-11-23 14:36:41
 updated_at: 2020-11-19 13:38:34
  disabled: 1
    name: Ali_Practice
description: A pack for practice queries 
  platform: 
 pack_type: NULL
*************************** 18. row ***************************
     id: 44
 created_at: 2020-08-04 14:42:23
 updated_at: 2023-05-02 10:13:13
  disabled: 0
    name: Test pack
description: Pack to ensure that logs are running locally when upgrading.
  platform: 
 pack_type: NULL
*************************** 19. row ***************************
     id: 45
 created_at: 2021-07-28 13:39:30
 updated_at: 2023-06-14 16:12:57
  disabled: 0
    name: Global
description: Global pack
  platform: 
 pack_type: global
*************************** 20. row ***************************
     id: 46
 created_at: 2023-03-06 15:16:03
 updated_at: 2023-03-06 15:16:03
  disabled: 0
    name: test_es_query_pack
description: 
  platform: 
 pack_type: NULL
20 rows in set (0.00 sec)

ERROR: 
No query specified