https://github.com/osquery/osquery logo
Powered by
Title
m

Mystery Incorporated

06/23/2021, 1:41 AM
2021/06/23 11:41:36 http: TLS handshake error from x.x.x.x:52295: local error: tls: bad record MAC
z

zwass

06/23/2021, 1:44 AM
Let's please collect this up into a thread.
m

Mystery Incorporated

06/23/2021, 1:45 AM
ok, when I run from cmd all is fine, it enrols into fleet and no problem. When I start the osqueryd service, it floods that message server (fleet) side
z

zwass

06/23/2021, 1:45 AM
I see the other user filed an issue here: https://github.com/fleetdm/fleet/issues/1160. It seems that user finds the application still works and we don't yet have a good understanding of why it's happening in their case. We do address this exact error message in the FAQ (https://github.com/fleetdm/fleet/blob/main/docs/2-Deploying/FAQ.md#why-arent-my-osquery-agents-connecting-to-fleet).
I'm not sure if this could be an explanation for what you are seeing, but I'd advise looking into your certificate configuration.
I'd also advise making sure that osquery uses the exact same parameters when run as a service as from cmd. This can be a little tricky in Windows, and unfortunately the Windows Service system doesn't make any logs available.
m

Mystery Incorporated

06/23/2021, 1:47 AM
Yes it is using the exact same flag file
And it works no issues from CMD prompt
So why would it then bomb out when running asd a service? it makes no sense
"C:\Program Files\osquery\osqueryd\osqueryd.exe" --flagfile="C:\Program Files\osquery\osquery.flags"
That is path to binary for service
z

zwass

06/23/2021, 1:54 AM
I suspect you're having issues with the quoting. Try without quotes around the binary path? I'm not sure the exact combination off the top of my head and I'm not able to look it up at the moment as I need to sign off.
m

Mystery Incorporated

06/23/2021, 1:54 AM
@zwass that is exactly how the service was created by the osquery msi installer
This works from cmd prompt 
C:\Program Files\osquery>osqueryd.exe --flagfile="C:\Program Files\osquery\osquery.flags"
You can see I push the default osquery_info query top it from fleet and it executes fine no problem
So there is not an error with my config
I have changed the service executable as per your suggestion (although I had also done it earlier) it is now: 
C:\Program Files\osquery\osqueryd\osqueryd.exe --flagfile="C:\Program Files\osquery\osquery.flags"
Now service is running and the message about tls is not being spamed fleet side, but the host does not show as online and if I delete it, it does not re-enrol
No errors, nothing. Total madness. How can one even debug this case? Bonkers
c

Chris

06/23/2021, 1:41 PM
@zwass my issue was due to my own stupidity and failure to troubleshoot... I figured out what it was and closed #1160.
:ty: 1
@Mystery Incorporated what version of osquery and what version of Windows are you working with?
m

Martavis Parker

06/23/2021, 2:38 PM
@Mystery Incorporated Thank you for your patience and diligence. For clarity, are you saying that you found these three bugs? • Incorrect syntax for the installer • Host does not show as online • A deleted host does not re-enroll automatically
z

zwass

06/23/2021, 4:00 PM
More things to check: Are the cmd line and the Service both running osquery as the same user? It's possible there could be some permission issue. Are all paths in the flagfile absolute paths?
m

Mystery Incorporated

06/23/2021, 5:23 PM
@zwass OSquery MSI installs service as Local System account. I am running as a priviledged domain admin from cmd so definitely not the same.
c

Chris

06/23/2021, 5:41 PM
@Mystery Incorporated you had previously posted your flagfile, but what does it look like now?
m

Mystery Incorporated

06/23/2021, 6:34 PM
--enable_ntfs_event_publisher=true
--enable_windows_events_subscriber=true
--enable_windows_events_publisher=true
--enable_powershell_events_subscriber=true
--events_optimize=true
--events_max=200000
--events_expiry=86400
--disable_events=false
--windows_event_channels=System,Application,Setup,Security,Microsoft-Windows-Windows Firewall With Advanced Security/Firewall,Microsoft-Windows-Windows Firewall With Advanced Security/ConnectionSecurity

# Server
--tls_hostname=<http://mydomain.com:8881|mydomain.com:8881>
--tls_server_certs=fleet.pem

# Enrollment
--host_identifier=instance
--enroll_secret_path=secret.txt
--enroll_tls_endpoint=/api/v1/osquery/enroll

# Configuration
--config_plugin=tls
--config_tls_endpoint=/api/v1/osquery/config
--config_refresh=10

# Live query
--disable_distributed=false
--distributed_plugin=tls
--distributed_interval=10
--distributed_tls_max_attempts=3
--distributed_tls_read_endpoint=/api/v1/osquery/distributed/read
--distributed_tls_write_endpoint=/api/v1/osquery/distributed/write

# Logging
--logger_plugin=tls
--logger_tls_endpoint=/api/v1/osquery/log
--logger_tls_period=10

# File carving
--disable_carver=false
--carver_start_endpoint=/api/v1/osquery/carve/begin
--carver_continue_endpoint=/api/v1/osquery/carve/block
--carver_block_size=2000000
Works when osqueryd ran from cmd. secret.txt and fleet.pem reside with osqueryd.exe
@Martavis Parker not really, nothing gives me enough information to say a great deal of anything with confidence. The only error I have is the TLS handshake one in fleet so I haven't been able to come to any conclusion from that. I suspect that perhaps the certificate isn't being grabbed from the correct path, but that's only a suspicion.
m

Martavis Parker

06/23/2021, 9:18 PM
I'm sorry to hear that. Unfortunately, using Windows services has its setbacks, logging being one of them. We think you're on the right track but we won't be able to pinpoint the issue on our end as it's not a large part of our operation. If you are able to figure it out, we would love more insight into your process so we can document it for future users if you don't mind.
m

Mystery Incorporated

06/25/2021, 4:11 AM
@Martavis Parker yeh we figured it out, have to use absolute paths with widows service. No quotes for the path even if there are spaces in the file path. Basically throw conventional practice out the window and it works.
m

Martavis Parker

06/25/2021, 3:10 PM
That's great to hear. We will note this for future users. Thank you again for your diligence in figuring it out.
5 Views
#fleetJoin Slack
Powered by