Title
#fleet
o

Ojas

04/25/2022, 10:07 AM
message has been deleted
11:36 AM
I have 600+ windows hosts and all are offline
Lucas Rodriguez

Lucas Rodriguez

04/25/2022, 11:54 AM
Hi Ojas, could you check if any of your Windows hosts can connect to tuf.fleetctl.com? (
ping <http://tuf.fleetctl.com|tuf.fleetctl.com>
and maybe
curl <https://tuf.fleetctl.com>
)
11:55 AM
Also, if you have access to any of such hosts, please run:
C:\Program Files\Orbit\bin\orbit.exe --version
o

Ojas

04/25/2022, 11:57 AM
i am able to ping and version is orbit 0.0.9
Lucas Rodriguez

Lucas Rodriguez

04/25/2022, 11:58 AM
How about the curl or wget command? Asking this due to the
lookup <http://tuf.fleetctl.com|tuf.fleetctl.com>: no such host
logs.
o

Ojas

04/25/2022, 11:59 AM
curl https://tuf.fleetctl.om does give me result
12:28 PM
@Lucas Rodriguez seems like it got screwed up after new updates maybe?
Lucas Rodriguez

Lucas Rodriguez

04/25/2022, 12:53 PM
Mhm... something to check: any AV or firewall configuration that could be preventing Orbit to connect to
<http://tuf.fleetctl.com|tuf.fleetctl.com>
?
o

Ojas

04/25/2022, 12:54 PM
but then how i am able to curl and ping to it?
Lucas Rodriguez

Lucas Rodriguez

04/25/2022, 12:55 PM
Not sure, but maybe something is preventing the Orbit process itself to making such connection or making the DNS request? (the error seems to suggest Orbit cannot even resolve DNS for tuf.fleetctl.com)
o

Ojas

04/25/2022, 1:02 PM
how can i debug it?
Lucas Rodriguez

Lucas Rodriguez

04/25/2022, 1:03 PM
You could try running Orbit manually in
cmd
and see if it provides more logs/information there.
1:04 PM
To do so, you can do the following:
1:05 PM
Click start, open "Services" as administrator, right click
fleet-osquery
and select
Properties
. And you will be able to copy/paste the command+arguments used to start Orbit.
o

Ojas

04/25/2022, 1:12 PM
this is what i get
Lucas Rodriguez

Lucas Rodriguez

04/25/2022, 1:13 PM
Error reading config: Cannot parse JSON: The document root must not be followed by other values. Offset: 4
Did you change the agents options in Fleet recently?
o

Ojas

04/25/2022, 1:14 PM
config: options: logger_plugin: filesystem filesystem: enable_log_rotation: true
1:14 PM
i added filesystem: enable_log_rotation: true
Lucas Rodriguez

Lucas Rodriguez

04/25/2022, 1:16 PM
Mhm... is that the
agent options
or the
fleet config
? By agent options I mean "Global agent options" from the settings:
o

Ojas

04/25/2022, 1:17 PM
Lucas Rodriguez

Lucas Rodriguez

04/25/2022, 1:17 PM
If it's the global agent options, then:
filesystem:
    enable_log_rotation: true
is not valid, as it's a fleet config, not an osquery config.
1:17 PM
Removing that should fix the issue.
o

Ojas

04/25/2022, 1:18 PM
where do i put the osquery flasgs then. I though it goes in global agent options only
Lucas Rodriguez

Lucas Rodriguez

04/25/2022, 1:18 PM
filesystem.enable_log_rotation
that configures log rotation for fleet logs, not osquery logs.
1:19 PM
By fleet logs I mean Fleet log files.
1:20 PM
Such config should be added via
fleetctl
, it doesn't affect agents, it affects fleet log files.
o

Ojas

04/25/2022, 1:20 PM
oh alright
1:34 PM
i have removed the config but still nothing.
Lucas Rodriguez

Lucas Rodriguez

04/25/2022, 2:05 PM
OK, let me check, osquery agents may be trying to use the old config...
2:10 PM
By executing Orbit manually in
cmd
as before do you still get the same error?
2:12 PM
Any reason to not use some of the default values (other than
logger_plugin
) for
Global agent settings
configuration that comes with fleet?
config:
  options:
    logger_plugin: tls
    pack_delimiter: /
    logger_tls_period: 10
    distributed_plugin: tls
    disable_distributed: false
    logger_tls_endpoint: /api/osquery/log
    distributed_interval: 10
    distributed_tls_max_attempts: 3
  decorators:
    load:
      - SELECT uuid AS host_uuid FROM system_info;
      - SELECT hostname AS hostname FROM system_info;
overrides: {}
o

Ojas

04/25/2022, 7:03 PM
It was working well without all these, so i removed them. Didnt have any issues till now.
7:03 PM
also yes i am getting the same error on executing orbit manually
Lucas Rodriguez

Lucas Rodriguez

04/25/2022, 7:16 PM
OK, so currently you have the following agent config?
config:
  options:
    logger_plugin: filesystem
o

Ojas

04/25/2022, 7:16 PM
nope i changed to the default one and set logger plugin to filesystem
7:17 PM
config: options: pack_delimiter: / logger_tls_period: 10 distributed_plugin: tls disable_distributed: false logger_tls_endpoint: /api/osquery/log distributed_interval: 10 distributed_tls_max_attempts: 3 decorators: load: - SELECT uuid AS host_uuid FROM system_info; - SELECT hostname AS hostname FROM system_info; overrides: {}
Lucas Rodriguez

Lucas Rodriguez

04/25/2022, 7:17 PM
OK, and you still get the
Error reading config: Cannot parse JSON: The document root must not be followed by other values. Offset: 4
when running Orbit manually in
cmd
?
o

Ojas

04/25/2022, 7:17 PM
yep yep
7:26 PM
any fix for it? Also how do i stop auto update in future?
Lucas Rodriguez

Lucas Rodriguez

04/25/2022, 8:38 PM
Mhm... that's odd, I'll check with the team, but the issue doesn't seem related to the Orbit upgrade.
8:39 PM
Also, Orbit now has a
--disable-updates
option (can be set in Orbit manually or you can define it when generating the packages (MSI, PKG, DEB, RPM) with
fleetctl package
).
d

Dawei Zhang

04/26/2022, 2:54 AM
hi, I got the exactly same problem
2:54 AM
W0426 02:47:01.457355 247643 init.cpp:616] Error reading config: Cannot parse JSON: The document root must not be followed by other values. Offset: 4 is this caused by orbit automatic upgrade?
2:55 AM
according last date, it seems this was working fine until April 22nd
o

Ojas

04/26/2022, 6:06 AM
@Lucas Rodriguez Is there an option in which we can revet to the older version where things were stable for us?
Lucas Rodriguez

Lucas Rodriguez

04/26/2022, 11:25 AM
Hi folks, I'll check with the team and get back to you.
12:01 PM
OK, I was able to reproduce the issue. We will be working on a fix today and release an auto-update as soon as we can. Issue: #5367.
o

Ojas

04/26/2022, 12:02 PM
Amazing 🔥 Thanks Lucas