Hi Team - have you seen any previous issues where a Windows Fleet Agent is being blocked/flagged by Cloudflare as "Command injection - Common Attack Commands"? This does not appear to be a rule we have written, but an automatic CF detection. Agent package creation was done only with --fleet-url and --enroll-secret. The issue is not being seen on Mac or Linux agents.

Hi @Mike S. - Are you deploying fleetd (orbit) or plain osquery?

Hi Zay - I am generating the package in Fleet (community version) - I believe that's orbit. Apologies, I'm still very much a novice. It's not fully blocked. The deployment completes, and Fleet can sometimes pull the vitals, but no queries can be run. It ends up looking like this:

@Kathy Satterlee Can you take a look at this?

While it’s definitely odd that it’s only blocking those Windows devices, this seems like something that would need to be managed on the CloudFlare side. That being said, I’m going to reach out to the team to see if there’s a substantial difference in the way osquery returns data from Windows.

Hi @Kathy Satterlee agreed, we are looking at how to properly manage that on the CF side. It was just odd that Windows was the only agent doing this.

100% agree that it’s odd!

I do have debug logs if those would help.

I’m wondering if it’s a Window’s specific detail query that’s triggering CF.

Looks like the agent doesn't respond to the queries.

@Mike S. We are having the team look into this, but we may need to get back to you on Monday. I hope this doesn't stall you on any progress with Fleet! We have a reminder to get back to you next week

I will, thanks all! No worries on getting back on Monday.

Windows Fleet Agent is being blocked/flagged by Cloudflare as "Command injection - Common Attack Commands"?

Hey, quick question: Is a cloudflare component running in the host blocking Orbit? Or is Cloudflare blocking network connections to/from the Windows agent?

Hi Lucas, it is blocking network connections to/from the agent.

this is at the WAF, right ?

Yes, this is at the WAF

So the WAF is decrypting the Fleet comms and then re-encrypting and sending it along to the Fleet server? Is there a possibility of allow-listing the osqueryd traffic? There is potentially a bunch of sensitive data that could be queried.

Is there any chance that your non-Windows hosts share something in common that might explain why they’re not getting caught up? For example, location (they’re all connected to an internal network that has different rules), VPN, etc?

Hi @defensivedepth - That's something we are looking into. I noticed that there's an osquery user-agent we may be able to allow-list. That could be a quick win to get that traffic through. Hi @Kathy Satterlee - sorry, I'm not sure what you mean. Our non-Windows hosts appear to be working properly at the moment. Sorry if I'm misunderstanding here. 🙂

I was wondering if there might something different between the two groups other than just OS that could explain why some requests are making it though and others won’t.

Ok, I was misunderstanding! So the the Mac systems are all laptops, on-site and remote. The Linux system is an EC2 instance. The Windows systems are also laptops that can be on-site/remote - Win10/11. They all use the same security stack, and have the same agents installed.

No, they're both using the same API endpoints and the same general request body.

So far I'm not seeing any differences on the Mac side. I'll keep looking.

Interesting, it flagged the

distributed/write

request, which means it flagged some part of the query results as command injection. I wonder what exactly caused the flagging, but I can see "machine learning" there so it's kind of a black box?...

Similar issue for my usecase. I was working on Windows enrollment today (orbit+osquery on Windows, packaged by fleetctl). Mac agents already running with no issues. Going from endpoint -> cloudflare (WAF) -> fleet server. Cloudflare let through windows host enrollment, but queries are getting blocked on path

/api/v1/osquery/distributed/write

UA

osquery/5.8.2

CF WAF rule

Command Injection - Common Attack Commands

. Very odd that from CF WAF logic Mac queries are safe while Windows is not. I'll be doing some researching, and probably also look for a way to specifically exclude the osquery user-agent/path from cloudflare WAF on cloudflare side.

Yeah that is really odd. We are using the FortiWeb Cloud product as a WAF and noticed SQL injection being flagged -- but by all platforms, not just Windows. I had to remove those checks from certain endpoints.

Plot twist. I was using the "old" CloudFlare WAF rules, they started rolling out the next generation of the rules engine last year. After upgrading to the new WAF model today, and without any additional changes/exceptions, the Windows agent is now able to pass CF WAF and send back telemetry. TL;DR possible "fix" to this: upgrade to new CF WAF rules?

Thats really interesting @Ando! Please keep us posted if this resolves the issue.

Thanks for posting this @Ando! I will bring this up with our Cloud team and see if this is an option for us. If not, we'll probably just look for a bypass rule for the user-agent or something to that effect.

I'd say updating to the new WAF rule engine has resolved the issue. Been running with this for a week with no problems. Also, cloudflare recently sent an e-mail announcing the old rules engine will stop working on May 2024 (so migration is mandatory within a year).