12/18/2020, 10:20 PM
fleet carving feedback <thread>
10:21 PM
when I run
fleetctl get carves
, I don't have any context on: • which host executed the query • what the query was So I can basically see... "kolide_distributed_query_19" and when the carve was returned. But I don't know which machine returned the file.
10:22 PM
Also - when thinking about "more granular user permissions", one thing that clearly stands out is whether someone can run file carving queries... This is like a data exfil dream.
10:23 PM
(to expand on more granular user permissions): • RBAC, with custom defined roles • be able to limit who can edit options, saml, editing labels, additional host data queries, etc. • limit who can run distributed queries, where they can run them (static list, label, etc) • limit who can run file carving queries
10:27 PM
The feedback from fleetctl when running the query is strange. I see:
  "host": "<hostname>",
  "rows": []
Even if files are returned... I see empty rows. It would be useful to see something like how many files were returned maybe
10:30 PM
I'm not sure if it's not supported by fleet or if I just don't understand file carving (I haven't used it at all prior to fleet's support)
10:31 PM
But... I can't seem to get multiple files. e.g.:
SELECT * FROM carves WHERE path LIKE '/some/path/%' AND carve=1;
SELECT carve(path) FROM file WHERE directory LIKE '/some/path/%' AND mode='0755' AND type='regular';
10:32 PM
10:37 PM
(feedback is from fleet version 3.5.0)


12/19/2020, 12:19 AM
Hey Brendan, thanks for the feedback. Some of this is due to limitations of the carving implementation in osquery, though most of it can be addressed in Fleet (and/or osquery itself).
12:22 AM
(no rows) for example is what osquery returns when a carve is initiated via live query, and the carve API doesn't allow us to pass any metadata through with the carve results besides for whatever is in the query name. I think what we are going to have to do is make a higher level carves concept in Fleet that will shuffle around some of the metadata on the backend and display more useful feedback to the user.
12:25 AM
IIRC the first query you have above there (or something like it) should get all the files in the directory. I know I was able to carve multiple files when I was testing out this feature. Perhaps you need to use a subquery to expand the glob? Like
SELECT * FROM carves WHERE carve=1 AND path IN (SELECT path FROM file WHERE path LIKE '/some/path/%');


12/19/2020, 4:03 AM
$ cat /tmp/carve.txt
This is a super secret file
$ cat /tmp/carve2.txt
the most super secret carve file #2

Using a virtual database. Need help, type '.help'
osquery> select path from file where path LIKE "/tmp/carve%";
| path            |
| /tmp/carve.txt  |
| /tmp/carve/     |
| /tmp/carve2.txt |
| /tmp/carve3.txt |

$ fleetctl query --hosts "myhost" --query 'SELECT * FROM carves WHERE carve = 1 AND path IN (SELECT path FROM file WHERE path LIKE "/tmp/carve%";' --context default 
 100% responded (100% online) | 1/1 targeted hosts (1/1 online)

$ fleetctl get carves --context default
No carves found

$ fleetctl query --hosts "myhost" --query 'SELECT * FROM carves WHERE carve = 1 AND path = "/tmp/carve.txt";' --context default

$ fleetctl get carves --context default
| ID |          CREATED AT           |         REQUEST ID         | CARVE SIZE | COMPLETION |
|  1 | 2020-12-18 22:01:49 -0600 CST | kolide_distributed_query_8 |       2048 | 100%       |
4:03 AM
🤷 just tried again with a few variations
4:04 AM
I can get single files but can't seem to get multiple files
4:04 AM
also - first time trying
fleetctl preview
... this experiences is very nice 😄
4:05 AM
(and when I got the new carve, it says carve size is 2048, but the file is only 28 bytes, which is interesting? 🤷 edit: seems that's just tar overhead nvm