https://github.com/osquery/osquery logo
Join Slack
Powered by
Hi, how did you deploy osctrl in Ubuntu? Directly ...
# osctrl
j
Hi, how did you deploy osctrl in Ubuntu? Directly in the host or using docker?
v
Deployed via docker and docker-compose (
sudo make docker-dev
) after configuring the .env and generating the certificates following the instructions in the README.md inside the 
deploy/docker
directory
Actually, I'm able to load the GUI and even login. But no nodes are showing up there because of the TLS certificate verify error
This is feedback for the enroll script: it should not claim that the node has been enrolled until it is actually enrolled in osctrl-dev...
Copy code
ubuntu@node09ob48:~$ curl -sk <https://172.27.48.89/2f8ad0c8-b6a2-46d9-ad4d-bcfcdfd5d42c/371vf45IPt91USm0LmflI1IQ3nT/enroll.sh> | sh
[+] _OS=linux
[+] _SECRET_FILE=/etc/osquery/osctrl-dev.secret
[+] _FLAGS=/etc/osquery/osquery.flags
[+] _CERT=/etc/osquery/certs/osctrl-dev.crt
[+] IMPORTANT! If osquery is not installed, it will be installed.
/usr/bin/osqueryi
[+] Installed version of osquery is 5.20.0
[+] Stopping osqueryd
[+] Preparing osquery secret in /etc/osquery/osctrl-dev.secret
zaT4rizZiD83ccvPBvPCuLOc4qLzj12cE7zGMDc_S3d8ICne9G88pq8Bd8VuFDMr
[+] Preparing osquery flags in /etc/osquery/osquery.flags
[+] Preparing osquery certificate in /etc/osquery/certs/osctrl-dev.crt
[+] Starting osqueryd
Synchronizing state of osqueryd.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
Executing: /usr/lib/systemd/systemd-sysv-install enable osqueryd
Created symlink /etc/systemd/system/multi-user.target.wants/osqueryd.service → /usr/lib/systemd/system/osqueryd.service.
[+] Congratulations! The node has been enrolled in osctrl-dev
[+] REMINDER! osqueryd has been started and enabled.
ubuntu@node09ob48:~$ 
ubuntu@node09ob48:~$ 
ubuntu@node09ob48:~$ sudo systemctl status osqueryd.service 
● osqueryd.service - The osquery Daemon
     Loaded: loaded (/usr/lib/systemd/system/osqueryd.service; enabled; preset: enabled)
     Active: active (running) since Thu 2025-12-18 20:43:54 UTC; 4s ago
   Main PID: 145725 (osqueryd)
      Tasks: 12 (limit: 18941)
     Memory: 14.3M (peak: 14.9M)
        CPU: 91ms
     CGroup: /system.slice/osqueryd.service
             ├─145725 /opt/osquery/bin/osqueryd --flagfile /etc/osquery/osquery.flags --config_path /etc/osquery/osquery.conf
             └─145770 /opt/osquery/bin/osqueryd

Dec 18 20:43:54 node09ob48 systemd[1]: Starting osqueryd.service - The osquery Daemon...
Dec 18 20:43:54 node09ob48 systemd[1]: Started osqueryd.service - The osquery Daemon.
Dec 18 20:43:54 node09ob48 osqueryd[145725]: osqueryd started [version=5.20.0]
Dec 18 20:43:54 node09ob48 osqueryd[145770]: W1218 20:43:54.660670 145770 tls_enroll.cpp:101] Failed enrollment request to <https://osctrl-nginx/2f8ad0c8-b6a2-46d9-ad4d>>
Dec 18 20:43:55 node09ob48 osqueryd[145770]: W1218 20:43:55.692766 145770 tls_enroll.cpp:101] Failed enrollment request to <https://osctrl-nginx/2f8ad0c8-b6a2-46d9-ad4d>>
ubuntu@node09ob48:~$
j
How did you generate the certificate?
check with the command
Copy code
openssl x509 -in certfile -noout -text
to see what alt names you have
I would generate a new one with the IP address you have there, or put the ip address in the 
/etc/hosts
using one of the domains in the cert
see what is in 
/etc/osquery/osquery.flags
for the tls hostname to be used
osquery is a bit finicky with certificates
also make sure there is connectivity from the osquery clients to 
<https://osctrl-nginx>
👍 1
✅ 1
v
I put the output I got upon running `hostname -f`on the machine. And I can verify that is the same value in the cert that got generated too
j
but you are spinning those containers within a different network
v
For reference:
Copy code
ubuntu@node10ob48:~/osctrl$ hostname -f
node10ob48.maas
ubuntu@node10ob48:~/osctrl$ openssl x509 -noout -in deploy/docker/conf/tls/osctrl.crt -text | grep -A 1 -E "Subject Alternative | CN"
        Issuer: emailAddress = dev@node10ob48.maas, CN = node10ob48.maas
        Validity
--
        Subject: emailAddress = dev@node10ob48.maas, CN = node10ob48.maas
        Subject Public Key Info:
--
            X509v3 Subject Alternative Name:
                DNS:node10ob48.maas, DNS:osctrl-nginx, DNS:nginx, DNS:osctrl
j
now check 
/etc/osquery/osquery.flags
in the osquery containers
to see what is the machine to connect to
v
Okay, if I have use IP address on the certificate, which one should I use? I assume it's the Ubuntu machine's network on which other nodes can access the osctr-dev endpoint?
j
and see if you can run a 
curl -k <https://osctrl-nginx>
you are spinning those containers using docker, so no
v
curl works
Copy code
ubuntu@node10ob48:~/osctrl$ curl -k <https://osctrl-nginx>
<a href="/login">Found</a>.
Hmm, okay, now I see your point. I was actually checking the certs and everything only inside my ubuntu machine
I think I need to check it on the containers?
j
yeah
the ubuntu machine is the hypervisor
but the osquery nodes are containers
v
Hmm, why do we have three osquery containers in the deployment?
j
the docker environment is the development environment
it is intended to be used as dev playground
if you need to deploy it differently, make changes to the docker compose and the docker files
v
Right, I initially started with the system service deployment. But I'm sorry to say I was unsuccessful on getting it to work or to even access it. I think the env vars and other configs weren't done properly and I struggled with the little documentation that was available
j
Yeah and also I just merged this yesterday
it breaks all the configuration
I am in the middle of a big refactor, and fixingthe deploy scripts
great timing you got 😅
v
Ooh, wrong timing from my end then haha
This is actually quite cool, kudos on the effort and for maintaining this one!
j
the plan is to release 
0.4.9
before the xmas break
🎯 1
and it will be fully operational
v
I would really appreciate some kind of a tutorial for newbies like me to actually just get started...
j
okay, more people have asked me about it
maybe I need to finally write something
what are you using osctrl for?
v
Haha, I'm a Field Engineer myself, I can help you with writing a tutorial if I can get it to work 😉
So, the use case is pretty interesting. We chose osctrl as the primary fleet management tool to fetch OS-level details on large virtualized environments.
j
cool, yeah it can do that
v
The exact queries can change, but I likethe architecture of this tool to aggregate everything in a single location. And also simplyfying the process of setting up the osquery service on the target nodes
j
I have optimized the UI in the past few months, so it can take up to several thousands nodes
🙌 1
v
I'm almost there! Just want the nodes to enrol... But no luck...
j
give me few days to fix the 
deploy.sh
script
that should get you going without docker
v
I actually tried the provision.sh script.
That took me almost to the end... I even saw something like this:
Copy code
[+] Congratulations! The node has been enrolled in osctrl-dev
[+] REMINDER! osqueryd has been started and enabled.

[+] Your osctrl is ready 👌🏽

[+]  -> <https://osctrl.dev:8443>


[+]  -> <https://172.27.48.89:8443>
[+]  -> 🔐 Credentials: admin / c98fca81beaada60d2d49a29f74cd4d8


[+] Completed in 4 minutes and 23 seconds
But the problem is I don't know how to access the setup, lol
j
the 
8443
is the port for the admin ui
v
Which is why I took the docker approach. Coz you have the README there which was mighty helpful
Yeah well, that didn't work for me. Also, I didn't know how to use those other nice CLI tools you got in the bin directory too.
j
yeah, I got a bunch
sorry about the lack of documentation!
v
I've currently stopped those services to allow the docker deplyoment to work well (port conflicts otherwise)
But if you could help me with some guidance there, I can switch back to the deployment and give it another shot. And BTW, how to start from a clean slate? We def. need an "unprovision.sh" script for repeatability...
j
if you can wait 2-3 days, I swear release 
0.4.9
will be in good shape
❤️ 1
v
I'm working against time here, haha. Need to get this to work by EOD tomorrow and document the process 😅
j
and the issue is osquery not enrolling?
v
But I really appreciate your taking the time to engage with me so far... You see, github issues wouldn't have been as much helpful
Yes, but let me tear down the docker deployment and start the services on the machine to share the exact issue.
j
yeah, the docker as it is is only meant to be for dev purposes
if you use the 
provision.sh
script, and install all necessary in Ubuntu, it should have autoenrolled the machine
if the node does not show up, I would check connectivity to the 
osctrl-tls
component, which should be running in the 
443
port
v
I see, checking one moment. Killed docker 😄
j
the next step is to check the certificate validity for osquery, and confirm they are TLS/SSL issues
when you deploy this in production, and use a cert from let's encrypt, there is no need to roll certificates to osquery, it is much easier
👍 1
but it depends on the infra
v
I agree. But I'm actually experimenting with it rn
Will definitely have to switch to non-self-signed-certs later on
So, I just started the three services: osctrl-admin/api/tls
And they are all running
But I'm seeing messages like Backend not ready... Maybe I should give it some time to load?
Wow, so it actually failed to connect to backend
j
if you are using 
main
, all configuration is broken
v
I actually pull the code like 48 hours back
Haven't pulled it again
Or when did the breaking changes get merged?
j
yesterday 🤣
v
Nice, so caught in bad times again, lol. I did the pull like 45 hours back, so I believe I should be good
Looks like a mostly clean setup:
Copy code
ubuntu@node10ob48:~/osctrl$ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   deploy/lib.sh
        modified:   docker-compose-dev.yml

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        certs.pem
        go1.25.5.linux-amd64.tar.gz

no changes added to commit (use "git add" and/or "git commit -a")
ubuntu@node10ob48:~/osctrl$ git log -1
commit 42f2de2bca36564a4ed7a0d5f04930b10a98ea88 (HEAD -> main, origin/main, origin/HEAD)
Merge: a83a2906 158aa00e
Author: Javier Marcos <1271349+javuto@users.noreply.github.com>
Date:   Sun Nov 30 15:38:02 2025 +0100

    Merge pull request #753 from jmpsec/dependencies-update-17

    Dependencies upgrade
ubuntu@node10ob48:~/osctrl$
j
yeah, that should be good
despite the go install missing
v
But unfortunately the setup on this machine seems borked. Which is why I'm looking for an unprovision.sh script, haha
How do I start from a clean slate???
Or maybe I can try this on a new machine. Shouldn't be a challenge.
j
from the cloned repo, you can 
make install
and it will compile and deploy each component
you need to make sure the postgres and redis details and valid in the 
/opt/osctrl/config
directory
💡 1
but I will keep in mind the 
unfuck
button for a script
😅 1
v
Nice nomenclature 🤣
j
is this a project for a company?
just out of curiosity
v
Hmm, so tell me more about the configurations to be done on the /opt/osctrl/config dir. Doesn't it happen automatically?
j
the 
provision.sh
script do generate the configuration in 
/opt/osctrl/config
look for 
db.json
and 
redis.json
👍 1
those are the values you need to make sure are valid
each service will use those files
v
Yeah, we're all in the same team 🙂 I work for Canonical, and we're trying to evaluate osquery for MAAS -- the open source metal as a service hardware provisioning tool, and other virtualization platforms like openstack.
👍 1
j
(with the unified configuration in a single YAML file, it will be easier and simpler...)
🤞 1
oh nice, yeah osquery is a good agent to collect data
💯 1
and the SQL abstraction is handy
v
Hmm, nice. I did find those files. But not sure how to trigger them to the "correct" configurations...
j
you can use the 
osctrl-cli
to check db connectivity
let me get you the command
v
I have it!
But not getting what I should be getting lol
Copy code
ubuntu@node10ob48:~/osctrl$ ./bin/osctrl-cli --api-file /opt/osctrl/config/api.json settings show
2025-12-17T02:47:58Z FTL main.go:2151 > ❌ loadAPIConfiguration - JSON key osctrl not found in /opt/osctrl/config/api.json
1 ubuntu@node10ob48:~/osctrl$ less /opt/osctrl/config/api.json
ubuntu@node10ob48:~/osctrl$ ./bin/osctrl-cli --api-file /opt/osctrl/config/api.json check-db

2025/12/17 02:49:47 /home/ubuntu/osctrl/pkg/backend/backend.go:104
[error] failed to initialize database, got error failed to connect to `user=postgres database=osctrl`: 127.0.0.1:5432 (127.0.0.1): failed SASL auth: FATAL: password authentication failed for user "postgres" (SQLSTATE 28P01)
2025-12-17T02:49:47Z FTL main.go:2151 > ❌ failed to create backend - Failed to get DB - failed to connect to `user=postgres database=osctrl`: 127.0.0.1:5432 (127.0.0.1): failed SASL auth: FATAL: password authentication failed for user "postgres" (SQLSTATE 28P01)
Which is understandable because nothing's listening on port 5432
Now I'm wondering how the postgresql service can claim it's running when the no process is listening on port 5432
j
osctrl-cli check-db -h
you want to use the local handler
but I think the provision, didn't provision the DB
v
Yeah, that's my guess too. But the output really gave me a DONE! feeling, haha
I ran the provision script like this:
Copy code
./deploy/provision.sh -m dev --nginx --postgres --redis -p all --all-hostname "172.27.48.89" -E
j
it should have installed it all
v
Yeah, so that's what. What would you suggest I do next?
I can actually deploy a new machine (ubuntu) and try the steps as you recommend it
j
this is Ubuntu 22 or 24?
v
This one's jammy (22.04)
j
it should not matter, I believe I used it in both
v
And yeah, I did have to tweak your script a little because the postgresql client was not available
Some 14 vs. 16 issue I think
j
oh interesting
v
I think I still have all those output. Let me find it
j
let me fire up my local dev VM
v
May I know which TZ you're in? I'm seeing upwards of 1 AM on your profile, so don't want to bother you this late
j
I am in CET, yeah a bit late
😢 1
are you in PST?
v
CST (PST+2, GMT-6)
j
do you want to sync up tomorrow and maybe jump in a call to see where we at?
I don't mind helping you
that also gives me time to make sure all the provision works as expected in both 22 and 24
👌 1
v
Yeah sure! Really appreciate your gesture at this point. Let's sync up tomorrow for sure. I'll probably try spinning up a new machine and trying the fresh install (
make install
) that you recommended, just as a fallback
Okay, so which version of ubuntu do you recommend using with the "working" main branch?
I can deploy any of focal, jammy, or noble
j
IIRC jammy is the official one, but I've also got it working in 24.04
v
Noted, let me try on both then later tonight
31 Views
Open in Slack
PreviousNext
Powered by