6190 needs a review and merge had to make a change to update

Question

6190 needs a review and merge had to make a change to update with changes to master

thor · Answer

Considering this is quite the sizeable change do you mind if this is held off until 4 2 1

farfella · Answer

Oh let s wait I apologize I thought we were releasing 4 2 1 since theo had wanted me to merge rebase master haha

thor · Answer

I m definitely open to discussing it more I saw that < seph> tagged it as 4 2 1 and considering it s a relatively broad change I d prefer to push it to another release as we re already changing so much in 4 2 0 but maybe that s not the right way to think about it considering 4 2 0 may be a stable candidate this fix would be nice to have slightly smiling face What does the rest of TSC think < zwass> < seph> < Stefano Bonicatti> < alessandrogario> ^^

zwass · Answer

I d like to get the release out ASAP so whatever will enable us to do that quicker Then let s work on increasing the cadence so we can get other new things out without waiting months

Stefano Bonicatti · Answer

A bit late but on that note I think that for a CVE having a release with only that fix and nothing else might be better for several reasons We can focus on the fix and the user supposedly have to retest only the code around the fix

farfella · Answer

I agree with < Stefano Bonicatti> a x y z release where z is patch for CVEs critical issues would be better for users who do not want new features yet

thor · Answer

I don t disagree but so much of the testing needed for this bug was baked into quite a few PRs that shipped and it somewhat goes against the release cadence style we ve commonly used I d say that s a really good discussion we should have at an OH but given how close we are to already tagging 4 2 0 can this be something we explore for future releases

thor · Answer

To the concept that folks might be less inclined to adopt this release and get the fix we already have quite a good slew of folks who don t upgrade to the latest cuts and are running very far back on releases the issue tracker is a good testament to this as we see lots of 3 X issues brought up

Stefano Bonicatti · Answer

ah sure my comment was meant for the future

zwass · Answer

I do agree we should do so for security fixes in the future

seph · Answer

The milestone tagging should be seen as a loose guideline not a prescriptive assignment For 4 2 0 specifically I move ~everythign to 4 2 1 to clear it for the SSL stuff

zwass · Answer

It sounds like 4 2 1 should probably be 4 3 0

alessandrogario · Answer

< thor> I ve handed the PR to Stefano I initially thought it was easier to just close my PR and add that missing line to Teddy s PR so I marked it as do not merge as I didn t even test it Stefano can push to that branch if keep the PR open and remove the do not merge tag

thor · Answer

Ok yeah so perhaps we can update the vernacular here <https github com osquery osquery vulnerabilities> to reflect that for CVEs we ll strive to have minor tagged releases to highlight such things

seph · Answer

I don t have strong feelings about whether 4 2 1 is called 4 3 0 or not

alessandrogario · Answer

I think osquery has seen a single commit fixex for big stuff but in this case it had dependencies on stuff that got disabled in the past

seph · Answer

We won t always be able to roll a patch release Like we d already merged too much to cut a 4 1 3 Thus 4 2 0 I don t think we should get really angsty about any of

zwass · Answer

We could always cut a branch from the latest stable release and cherry pick only the relevant changes This is I think what Stefano is suggesting for the future

Stefano Bonicatti · Answer

Yes it s a very special case anyway hopefully stuck out tongue

thor · Answer

I agree with trying that approach but I think we ll always inevitably hit similar situations to this one where the fix may not just have one simple PR to ship If it s the case we can cherry pick on top of the most recent stable I think it s a good idea but I think as < seph> eludes having flexibility in this will be good

Stefano Bonicatti · Answer

But it s not our flexibility I m concerned about though ^^

farfella · Answer

So Q folks using 3 x what prevents them from upgrading to 4 x Are there any state persistent files that need to be upgraded

farfella · Answer

That is is there a case where user upgrades to 4 x from 3 x and osqueryd does not run anymore as it depends on some file that is different between versions for example

Stefano Bonicatti · Answer

For persistent stuff there s the configuration and event db

Stefano Bonicatti · Answer

about the reasons I can think of some 1 Customizations private forks 2 If it ain t broke don t fix it 3 Extensions need a couple of changes in the build system and the code

samuel · Answer

i think there s a sizable number of people sitting at 3 3 2 ish since 4 x hasn t really been stable until 4 1 2

seph · Answer

What wasn t stable about the 4 0 line

samuel · Answer

aws was broken for a bit and i think others were advised to wait for a bit

seph · Answer

I don t remember that There have totally been unstable releases oops but I don t think 4 1 fixed any major stability issues People may have biases against 0 releases

seph · Answer

I sorta disagree that we should backport and cherry pick fixes It seems nice in theory but I don t think we have the time people to do it well and it doesn t feel like a thing we need AFAIK we don t generally make big breaking changes

seph · Answer

I think if people keep feeling that we should ditch semver

farfella · Answer

Agree let s not backport

farfella · Answer

We just need to ensure future where user can always safely upgrade from any old to new

farfella · Answer

It should be a guarantee

seph · Answer

I think that s always a goal I m leery if making strong guarantees Agility is important And trying toi exhaustively test corner cases is hard

samuel · Answer

in this case i don t think there should be a backport because there s not really anything which would make sense to backport to

seph · Answer

I could argue backporting to 3 x or 4 1 x But I am always in favor of rolling forward

samuel · Answer

i don t think anyone who is running 4 x already is going to protest not having a backport

farfella · Answer

Like we need to get folks on 3 X out of 3 X D

samuel · Answer

for what we have left on 3 3 2 those will jump straight to this release we were basically just waiting for a stable enough 4 x and then everyone took off for thanksgiving and stuff

seph · Answer

It s kinda concerning to me if people think of the 4 1 line as unstable

samuel · Answer

right now we have a mix of 3 3 2 a weird build of 4 0 2 and some 4 1 2 nothing stops the update now

samuel · Answer

i don t think 4 1 2 is unstable but building 4 x was is difficult on some platforms

seph · Answer

Ah yeah The build constraints I can sympathize with disappointed

samuel · Answer

there were some issues with deps and things which weren t documented yet

alessandrogario · Answer

< samuel> can you show us where it s hard to build osquery on maybe there s something we can do about it

alessandrogario · Answer

on the latest ish Ubuntu macOS and Windows it should be fairly easy

samuel · Answer

its been a while but in particular it has been hard to build a working nupkg from scratch on windows

alessandrogario · Answer

if there is a specific platform where it s troublesome we can take a look at it

alessandrogario · Answer

ah i see it s packaging

alessandrogario · Answer

I have to look at how it worked before cause I don t know much about this specific format

alessandrogario · Answer

<https cmake org cmake help latest cpack gen nuget html cpack gen CPack 20NuGet 20Generator> seems like it is supported though

samuel · Answer

its always been rough but maybe it is better now

samuel · Answer

like if you had a space in your username it didn t work previously

alessandrogario · Answer

I think we have solved the space issue on our side but on Windows when we use MSBuild we are constrained to its own limitations

alessandrogario · Answer

like the 256 char limits on paths

alessandrogario · Answer

and there is not much we can do about it

alessandrogario · Answer

but yeah if you feel like trying out the new build process we can tag along

alessandrogario · Answer

if there s any issue with it

alessandrogario · Answer

about nupkg unless Stefano added it it s not there yet

seph · Answer

TBH I don t want to support nupkg Or brew Or other things that feel like weird third party targets

alessandrogario · Answer

It seems like we can just append NUPKG to the list of generators

alessandrogario · Answer

I agree it would be better to use more standard formats

alessandrogario · Answer

but if it just requires a single line we can try it out

alessandrogario · Answer

nupkg is for chocolatey right

seph · Answer

yeah

alessandrogario · Answer

i d like it more if it didn t involve powershell scripts joy

seph · Answer

That too

samuel · Answer

Pretty sure nupkg is officially supported by Microsoft now and chocolatey is now just a third party wrapper

Stefano Bonicatti · Answer

Yeah visual studio has the integration plus `Install Package` in powershell

farfella · Answer

imo we don t have to support nupkg or chocolatey msi is acceptable and widespread I d want to reduce different vectors users can use to install to just one msi doesn t require any other dependencies on the system and handles upgrades nicely

farfella · Answer

For users they d want the official signed msi vs building it from scratch no

alessandrogario · Answer

there may be pre existing deployment strategies based on nupkg since it was originally supported

alessandrogario · Answer

i think it s fine not to provide new additional package formats

alessandrogario · Answer

but it would be nice if nupkg would still work for people who were relying on it

alessandrogario · Answer

without necessarily investing all the time on it

alessandrogario · Answer

we can test the nupkg format on CMake without making the format official i e we can make it work without publishing the package on the website

alessandrogario · Answer

what do you think seph

thor · Answer

We did chocolatey originally because that s how FB does package distribution internally The MSI came later as external folks wanted it but I don t see any reason we can t move away from nupkg based things as we ve changed how internaly distro works so we re not using the external choco packaegs

thor · Answer

That being said I do personally enjoy being able to `choco install yf osquery` on my home boxes

farfella · Answer

Is it set up so choco upgrade works if you re in 3 X to move up to 4 X

thor · Answer

It s supposed to be but its been some time since I ve done that upgrade path

thor · Answer

When I ve done testing for package releases the things we ship upstream which I build using the powershell scripts in the repo are tested for upgrades That being said going backwards might not always work so well due to database changes

farfella · Answer

Yeah only one way