15:30:09 <fao89> #startmeeting Pulp Triage 2021-01-26
15:30:09 <fao89> #info fao89 has joined triage
15:30:09 <fao89> !start
15:30:09 <pulpbot> Meeting started Tue Jan 26 15:30:09 2021 UTC.  The chair is fao89. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:30:09 <pulpbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:30:09 <pulpbot> The meeting name has been set to 'pulp_triage_2021-01-26'
15:30:09 <pulpbot> fao89: fao89 has joined triage
15:30:19 <fao89> Open Floor!
15:31:07 <ggainey> #info ggainey has joined triage
15:31:07 <ggainey> !here
15:31:07 <pulpbot> ggainey: ggainey has joined triage
15:31:07 <mikedep333> #info mikedep333 has joined triage
15:31:07 <mikedep333> !here
15:31:08 <pulpbot> mikedep333: mikedep333 has joined triage
15:31:12 <daviddavis> #info daviddavis has joined triage
15:31:12 <daviddavis> !here
15:31:12 <pulpbot> daviddavis: daviddavis has joined triage
15:31:14 <gerrod> #info gerrod has joined triage
15:31:14 <gerrod> !here
15:31:15 <pulpbot> gerrod: gerrod has joined triage
15:31:20 <ppicka> #info ppicka has joined triage
15:31:20 <ppicka> !here
15:31:20 <pulpbot> ppicka: ppicka has joined triage
15:31:22 <bmbouter> #info bmbouter has joined triage
15:31:22 <bmbouter> !here
15:31:22 <pulpbot> bmbouter: bmbouter has joined triage
15:31:24 <dalley> #info dalley has joined triage
15:31:24 <dalley> !here
15:31:24 <pulpbot> dalley: dalley has joined triage
15:31:28 <ttereshc> #info ttereshc has joined triage
15:31:28 <ttereshc> !here
15:31:28 <pulpbot> ttereshc: ttereshc has joined triage
15:31:31 <dkliban> #info dkliban has joined triage
15:31:31 <dkliban> !here
15:31:31 <pulpbot> dkliban: dkliban has joined triage
15:31:31 <fao89> topic: Pulp-NEXT query
15:31:49 <mikedep333> No bugs, just open floor?
15:32:17 <fao89> the triage will happen after the open floor
15:32:31 <ggainey> yus
15:32:41 <fao89> mikedep333: these are the bugs https://pulp.plan.io/issues?query_id=143
15:32:54 <bmbouter> oh I never made this query
15:32:57 <bmbouter> let me make this query real quick
15:33:26 <mikedep333> I want to have a discussion about which of the (roughly 87 min long) FIPS / Vagrant tests to run on PRs, and which on cronjobs, and which on tagged release jobs. https://github.com/pulp/pulp_installer/pull/503
15:33:43 <mikedep333> The tradeoffs being time vs coverage.
15:33:46 <ttereshc> bmbouter, I think we have an example with "work that needs some love" or something like that
15:33:53 <daviddavis> mikedep333: can oyu add it to the open floor agenda https://hackmd.io/SVCMjpwXTfOMqF2OeyyLRw?edit
15:34:05 <mikedep333> I'm thinking of just narrowing down the best schedule in a separate commit / issue.
15:34:07 <mikedep333> Will do
15:34:23 <x9c4> #info x9c4 has joined triage
15:34:23 <x9c4> !here
15:34:23 <pulpbot> x9c4: x9c4 has joined triage
15:34:49 <daviddavis> what's the difference between pulp-next and wishlist?
15:34:55 <fao89> so, beyond creating the query, should we discuss something more about it? Or just move to the next topic?
15:35:20 <bmbouter> here's a query https://pulp.plan.io/issues?query_id=168
15:35:35 <bmbouter> which hs the same issues as the pulp-NEXT labels
15:36:05 <bmbouter> daviddavis: it's a wishlist really because pulp4 has no actual planning atm
15:36:42 <fao89> I think dalley  created a wishlist tag
15:36:48 <daviddavis> yea there are two tags
15:37:09 <dalley> right
15:37:22 <ggainey> ah I see
15:37:27 <bmbouter> one of the overall goals is to not have as many tags in general
15:37:46 <bmbouter> we went through eliminating extra data and tags over the course of several weeks
15:38:10 <bmbouter> it's confusing to a broad group what they mean, it dilutes the ones we really need, and it makes it harder to migrate to GHI
15:38:17 * daviddavis puts away his tagging gun
15:38:32 * bmbouter calms down
15:38:35 <bmbouter> :)
15:38:38 <ggainey> hehehe
15:39:14 <daviddavis> I don't have a strong opinion
15:39:22 <ggainey> so since Pulp-NEXT has 4 issues currently, and Wishlist only 1, I think consolidating on Pulp-NEXT and dropping wishlist is a thing to do
15:39:33 <dalley> I'm fine with that
15:39:52 <ggainey> I think a tag for "things that need to go in the next major iteration of pulp" is also useful
15:39:58 <daviddavis> what about bmbouter's proposal to use a query instead of a tag?
15:40:02 <dalley> although I'd almost rather drop pulp-NEXT and put them all in wishlist
15:40:13 <dalley> since there's no formal planning involved
15:40:23 <bmbouter> a wishlist makes more sense to me
15:40:27 <dalley> we can just groom those issues once we do get there
15:40:31 <ggainey> works4me as well
15:40:34 <daviddavis> +1
15:40:43 <bmbouter> we had a "put this stuff in pulp3" and we added to it over time and then we got to pulp3 and threw it out
15:40:57 <bmbouter> but a wishlist is ok it's just a wishlist
15:40:58 <ggainey> heh, aye that is not an uncommon pattern
15:41:01 <ggainey> +1
15:41:18 <bmbouter> so delete pulp-next tag, move it's items to the wishlist tag and keep the wishilist tag?
15:41:21 <dalley> +1
15:41:25 <ggainey> +1
15:41:27 <daviddavis> +1
15:41:31 <bmbouter> dalley: can you do all this?
15:41:40 <dalley> yes
15:41:41 <ttereshc> bmbouter, could you share why it is easier to migrate a query than a tag? or hte idea that queries are not migrated at all?
15:42:08 <ttereshc> I'm +1 to the current proposal
15:42:16 <ttereshc> just want to understand the tag problem
15:42:16 <bmbouter> we will probably migrate both
15:42:35 <bmbouter> the real issue are the first 2 points about usage, migration wise it's probably not harder
15:42:49 <ttereshc> ok, thanks
15:43:53 <fao89> can we move to the next topic?
15:44:01 <ggainey> +1
15:44:13 <ttereshc> sure
15:44:20 <fao89> topic: backport definition and workflow
15:44:49 <bmbouter> ja
15:45:04 <fao89> this also came from last open floor
15:45:15 <ggainey> we really need to have this well-defined/documented
15:45:42 <fao89> +1
15:45:46 <dkliban> is this question arising from the backport request that was not really a backport because it was for a current z stream?
15:45:57 <dkliban> for the issue that went into 3.9.1
15:46:01 <daviddavis> yea
15:46:01 <fao89> yep
15:46:05 <ggainey> dkliban: I think that def made us talk about the issue, aye
15:46:07 <daviddavis> there are backport docs https://docs.pulpproject.org/pulpcore/en/master/nightly/bugs-features.html#for-backport-requests
15:46:14 <daviddavis> not sure if they're insufficient
15:46:53 <daviddavis> I assumed that all cherry picks are backports and follow the same process
15:47:03 <ggainey> daviddavis: are we missing "how many versions back do we support"? (and maybe "do we backport to every version, or only selected ones"?)
15:47:28 <daviddavis> ggainey: that's a good question and I don't have a strong opinion
15:47:39 <ggainey> daviddavis: I know we've kicked it around before
15:47:50 <daviddavis> did we come to a conclusion for the LTS discussion?
15:48:07 <ggainey> daviddavis: I don't believe we did? anybody have a better memory than I do?
15:48:13 <daviddavis> anyway, maybe we can follow up there about supported versions
15:48:13 <ttereshc> I think the pulp-dev thread ended on - we want to release for all versons in between
15:48:21 <ttereshc> but until we have release automation
15:48:25 <ttereshc> we won't do it
15:48:25 <x9c4> that's what i recall too-
15:48:26 <ggainey> (I know we don't like 'long-term support' as a description tho :) )
15:48:27 <bmbouter> yes exactly
15:48:32 <fao89> I think people are discussing it in some email thread
15:48:40 <ggainey> ah! rightright
15:48:42 <bmbouter> basically we can't do anything differently until we have it fully automated
15:48:49 <daviddavis> +1
15:48:51 <ttereshc> and for now we are stuck with unfriendly option, just backport to wherever it was requested
15:48:59 <ggainey> +1, reliable automation is key
15:50:07 <ttereshc> so back to the topic,  do we need any clarifications in the docs?
15:50:36 <ttereshc> and is it clear if the latest z-stream requires backport requests or not?
15:50:38 <fao89> I just read, and it looks good to me
15:50:45 <bmbouter> to me also
15:50:58 <ggainey> yeah, concur
15:51:06 <fao89> next topic?
15:51:12 <ttereshc> ok, so for the latest z stream do we need a backport request or not?
15:51:13 <bmbouter> when we put the new policy into effect we'll update them again then
15:51:21 <bmbouter> I think we do
15:51:21 <ggainey> +1
15:51:46 <ttereshc> ok, I read it differently :) how do we do it in practice?
15:51:53 <bmbouter> backporting is the act of cherrypicking back, and we don't pick all bugs so if someone wants something on even latest z they need to tell us
15:52:05 <dkliban> +1
15:52:14 <ttereshc> I open an issue, it got fixed and it's in master branch
15:52:15 <bmbouter> this aspect is not very clear in the docs I'll admit
15:52:29 <ttereshc> so we do not close the original issue?
15:52:39 <bmbouter> currect and by it being fixed it's really at MODIFIED until release time
15:52:42 <ttereshc> but have a backport request and close it earlier than the original one?
15:52:42 <ggainey> so we're developing on 3.10dev, 3.9 is released, a fix happens in master - it will be in 3.10, if you want it in 3.9.1 , you need a backport request
15:52:53 <fao89> we did, but pointing to the merged issue
15:53:11 <bmbouter> yes it's possible the backport could close earlier
15:53:16 <ggainey> yus
15:53:19 <bmbouter> the fix is being released on that other branch first
15:53:27 <ttereshc> ok, so we did 3.9.1 wrongly then
15:53:40 <fao89> lets rename to cherry-pick request
15:53:41 <ttereshc> I'm +1 to clarify it in docs
15:53:52 <ggainey> ahhh, ok ok
15:54:28 <x9c4> as long as we only cherry-pick from master we are sure it will be in the next major release.
15:54:42 <x9c4> +1 for clarification
15:54:48 <ggainey> +1
15:54:53 <ttereshc> I'll open a PR, since i'm the one who is confused the most
15:54:59 <ggainey> ttereshc++
15:54:59 <pulpbot> ggainey: ttereshc's karma is now 287
15:55:10 <daviddavis> ty
15:55:19 <ttereshc> so basically we do not plan any z releases until someone files a backport request
15:55:32 <daviddavis> that's my understanding
15:55:39 <ggainey> that sounds right/reasonable
15:55:40 <ttereshc> this sounds a bit new to me but reflects the reality
15:55:54 <ttereshc> I think we can move on
15:55:59 <fao89> next topic
15:56:10 <fao89> topic: Permissions to manage repository versions
15:56:23 <fao89> is there a need to have dedicated object-level permissions for repository versions?
15:56:56 <ggainey> I feel like I'm missing context here
15:56:58 <dkliban> i am very conflicted on this
15:57:06 <ttereshc> This question came from a discussion between x9c4  and myself
15:57:19 <ttereshc> basically do we want to give permissions to a specific repo version
15:57:37 <ttereshc> or we can say, if you have certain permission for a repo, you can mange all versions
15:57:43 <ggainey> ahhhh
15:57:59 <ggainey> so the question could be famed as "do repo-versions inherit perms from their repo"?
15:58:12 <x9c4> like a repo_delete_version object permission on the repo-object
15:58:46 <ttereshc> yes, the key here is that a permission applies to ALL versions of a repo
15:58:48 <x9c4> and if we still want to specify permissions on the individual version. (we could have both)
15:58:59 <bmbouter> so a few thoughts on this 1) it's a plugin-by-plugin decision and 2) I advocate to keep it simpler
15:59:03 <ggainey> so, an advantage of per-repo-version perms might be "you can do what you want with this repo - but not with THIS version, it's controlled by me"?
15:59:47 <bmbouter> we could do this, technically it would work
16:00:07 <dkliban> is there a use case for it?
16:00:12 <ggainey> note that I lean (hard) towards "simpler until field use shows we might want complicated"
16:00:31 <bmbouter> and on a long enough timeline we probably will actually and just ship a default policy that maps back to repo-level permissions
16:00:42 <x9c4> IMO the version_delete perm per repo should be enough.
16:00:51 <ttereshc> bmbouter, so to make it plugin-by-plugin decision we'd need to add auto-assignment of permissions in core for repo versions and plugins will choose whether to check them or not
16:01:11 <ttereshc> is it what you imagined?
16:01:35 <bmbouter> it is
16:01:52 <x9c4> bmbouter, we have a problem with shipping new versions of default policies.
16:01:54 <bmbouter> since there is only 1 model not a Detail version of it like most others it would have to be offered by core properly
16:02:27 <bmbouter> x9c4: ok but one problem at a time please
16:02:32 <dkliban> lol
16:02:46 <ttereshc> to finish witht he perms for repo versions
16:02:53 <ttereshc> do we want to add it to the pulpcore now?
16:03:05 <bmbouter> I do not think so without a clear use case for it
16:03:08 <ttereshc> or later whenever someone asks for such granularity
16:03:20 <bmbouter> we are under the gun in many ways we need to be careful about where we spend our effort
16:03:22 <ttereshc> then it will require a data migration
16:03:26 <bmbouter> and also building too much too fast
16:03:26 <ttereshc> later
16:03:28 <ggainey> I'd lean towards wait
16:03:46 <ttereshc> I agree and in favor of waiting
16:03:59 <ttereshc> just pointing out the data migration need later
16:04:04 <ggainey> yus
16:04:05 <bmbouter> ttereshc: now that the accessPolicy comes as part of the signal it wouldn't need a migration
16:04:22 <bmbouter> the permissions are already auto-created by django actually (the 4 generic ones)
16:04:38 <ttereshc> yes, but they are not assigned
16:04:38 <bmbouter> and once the accesspolicy is on the model it will be created post-migrate on the next run
16:04:51 <bmbouter> ah that's true
16:05:09 <bmbouter> then I agree it'll need a migration
16:05:12 <bmbouter> and that's ok to me
16:05:28 <ttereshc> great, we are all in agreement
16:05:35 <x9c4> So they (version permissions) are ready to be used in a not default policy?
16:05:37 <ttereshc> what was thenext problem, x9c4 ?
16:06:04 <bmbouter> x9c4: yes they exist django creates them automatically
16:06:09 <dalley> I lean towards inheriting repository level permissions too
16:06:25 <bmbouter> you'll see something like core.change_repositoryversion in the permissions table today
16:06:36 <ttereshc> +1 to what bmbouter said
16:06:37 <fao89> next topic?
16:06:41 <x9c4> I think we need new version manage perms on the repo,
16:06:54 <ttereshc> x9c4,  we can define it at plugin level
16:07:02 <bmbouter> x9c4: the file prototype had something like this
16:07:08 <bmbouter> so I agree w/ you
16:07:13 <x9c4> +1
16:07:15 <bmbouter> each plugin will need to add this "custom" repo perm
16:07:17 <ttereshc> and check with conditions I just added to pulpcore
16:07:31 <bmbouter> it was called "manage content" or something like that
16:07:39 <bmbouter> but it could have a better/other name
16:07:51 <fao89> oh, I'm getting the messages in the wrong order
16:08:11 <bmbouter> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC#diff-72b7e5101ef16b644073b1d1a1bb3ce40d85f0281001f13a5ab43dec0046a7d9R61
16:08:23 <bmbouter> that was the custom perm in the pulp_file rbac PoC
16:08:43 <bmbouter> but another better name is welcome also
16:08:51 <x9c4> sad it did newer get merged...
16:08:52 <dkliban> that's the perm for creating new repo versions ... right?
16:09:12 <bmbouter> x9c4: it still could it's all there
16:09:41 <bmbouter> dkliban: yes
16:09:49 * bmbouter sings the siren song of RBAC
16:10:32 <x9c4> one thing i (think i) learned in the container rbac story is that permissions can naver be granular enough.
16:10:41 <dkliban> i agree
16:10:46 <ttereshc> yesss
16:10:46 <bmbouter> I also agree
16:11:04 <ttereshc> I hope repo versions are an exception lol
16:11:07 <bmbouter> but we still need to manage our effort with time/priorities
16:11:09 <dkliban> so i think you need a permission for creating new repository versions and another permissions for removing repository versions
16:11:24 * daviddavis loses himself in dance to the RBAC song
16:11:30 <bmbouter> lol
16:11:36 <dkliban> the 'manage content' is the add repository versions
16:11:46 <fao89> next topic?
16:11:51 <x9c4> and sync could be the third.
16:12:04 <dkliban> i think we need a separte meeting for this RBAC discussion
16:12:08 <x9c4> next topic, +1
16:12:09 <bmbouter> we do
16:12:12 <bmbouter> but we also need use cases
16:12:18 <bmbouter> designs need to follow need
16:12:20 <ggainey> +1 to that
16:12:41 <fao89> matrix is giving me messages at random, I'm super confused
16:12:42 <bmbouter> I like all the discussion though, it shows clearly that folks are really thinking about this and I like to see that
16:12:52 <bmbouter> fao89: do you see a black cat twice?
16:13:01 <ttereshc> :D
16:13:07 <mikedep333> fao89: :/
16:13:13 <x9c4> Last thought: use cases should be baked into the access_policy
16:13:29 <ggainey> fao89: I think it's just that we keep agreeing to next-topic, and then "one more thing"-ing :)
16:14:28 <bmbouter> x9c4: yes I agree. the default access policy that is shipped defines the user authZ experience
16:15:48 <ff> I joined here to understand the open floor
16:16:27 <ttereshc> x9c4, dkliban, bmbouter , should I schedule an irc meeting for rbac any time this week?
16:16:29 <bmbouter> welcome!
16:16:39 <mikedep333> ff: Welcome!
16:16:45 <ggainey> Guest57563: people add "general topics that need brainstorming" to the open-floor doc here : https://hackmd.io/SVCMjpwXTfOMqF2OeyyLRw
16:16:51 <x9c4> ttereshc, yes, please.
16:16:54 <bmbouter> ttereshc: only if it's in support of the pulp_container feature push
16:17:17 <bmbouter> what we've discussed here I don't think is needed for pulp_container's feature wrap up
16:17:24 <dkliban> Guest57563: open floor is for discussing topics that are of interest to developers and users of Pulp ... the agenda is https://hackmd.io/@pulp/triage/edit
16:17:27 <bmbouter> but I'd be happy to join so I'm ok if it's scheduled
16:17:55 <Guest57563> I'm fao89 just avoiding matrix
16:18:00 <daviddavis> lol
16:18:01 <ggainey> Guest57563: and we bash out thoughts on them - sometimes enough to get a single response, sometimes just getting to "we need to have a dedicated time for this - who wants to schedule a mtg, who's interested in contributing?"
16:18:06 <ttereshc> x9c4 , maybe if we have an agenda it will help to see what we need to discuss urgently if anything and keep some topics for later, wdyt?
16:18:38 <x9c4> sure works for me.
16:18:57 <ggainey> Guest57563: ha! nice disguise!
16:18:57 <Guest57563> I picked the wrong way to introduce myself here, sorry
16:19:08 <ttereshc> x9c4, would you mind creating it?
16:19:15 <ttereshc> exactly! ggainey
16:19:27 <x9c4> ttereshc, i don't mind.
16:19:29 <bmbouter> x9c4: ttereshc this all sounds good to me
16:19:31 <ttereshc> ty
16:19:46 <ttereshc> dear guest, we can move on
16:19:47 <x9c4> the first one to say yes gets the action item...
16:19:56 <bmbouter> it's true
16:20:00 <bmbouter> or not show up lol
16:20:13 <fao89> timecheck - 10 minutes
16:20:19 <Guest57563> next topic is pretty big - Which pulp_installer FIPS/Vagrant tests, which take **~90 min**, to run for PRs vs cronjobs vs tagged jobs?
16:20:19 <ggainey> oof
16:20:47 <mikedep333> I have a proposed matrix on https://hackmd.io/@pulp/triage/edit
16:20:56 <ttereshc> I guess all of them for cronjobs and tagged jobs?
16:21:18 <Guest57563> mikedep333 I'm +1 to the proposal
16:21:34 <mikedep333> tagged jobs are for releases of pulp, so I think tagged should only have sandbox (installing pulp & pulp_file from PyPI)
16:21:36 <Guest57563> I didn't understand the branch None
16:21:54 <mikedep333> cronjobs could take a while if we do all 15 tests or so, although I can add more and slim down later.
16:22:10 <mikedep333> "branch" is what happens whenever master branch gets updated.
16:22:24 <mikedep333> So it's like "we just merged a PR. Let's test it *again*"
16:22:51 <mikedep333> Although the test will be more recent than when the PR was last updated. Another PR could have been merged since then.
16:23:07 <mikedep333> My main concern is to not hog our GHA runners, which seem to be shared across the entire pulp org.
16:23:07 <ggainey> mikedep333: and that merge would be picked up by the nightly anyway, yes?
16:23:13 <mikedep333> ggainey: exactly
16:23:14 <dkliban> it would be
16:23:36 <bmbouter> I'm not sure failiar with all this, but generally really long things need to run at night
16:23:42 <mikedep333> bmbouter: yeah
16:23:46 <bmbouter> and we want a "small sample" of those long things to run with each PR
16:23:52 <ggainey> so, every PR gets 2 FIPS runs, nightly runs 6, on-tag runs 2
16:24:08 <bmbouter> enough that if it breaks all test environments we'll still know that pre-merge
16:24:10 <bmbouter> if that's possible
16:24:21 <mikedep333> right, although I'm likely to increase the number of nightlies.
16:24:24 <ggainey> so yeah, if the per-PR runs ran a less-than-90-minute subset of FIPS tests?
16:25:02 <mikedep333> I mean, we still test most of the code and scenarios via molecule. We have  9 scenarios molecules, each testing multiple distros, each taking like 13 minutes.
16:25:52 <ggainey> mikedep333: is it possible to not-run a test if what it's based on hasn't changed? like, if pulp hasn't chgd in PyPI since last-run, not useful to rerun that test?
16:25:58 <ggainey> does that make any sense?
16:25:59 <mikedep333> molecule == docker based test environment for ansible / pulp_installer
16:26:52 <mikedep333> ggainey: for the sandbox tests, that does make sense. I could look into that, but it would only really affect the cronjobs, which are less performance sensitive.
16:27:18 <fao89> I think we can go with this proposal and adjust it with the time, if we feel we need to
16:27:24 <ggainey> yeah that's fair
16:27:37 <mikedep333> Great, I'll implement that within my PR.
16:27:41 <x9c4> +1
16:27:44 <dkliban> +1
16:27:49 <ggainey> so, yeah, I'd rather have a generally-viable fips-test-matrix now, and course-correct over the next month or so
16:28:01 <ggainey> mikedep333++
16:28:01 <pulpbot> ggainey: mikedep333's karma is now 139
16:28:09 <ggainey> thanks for the thoughtful approach
16:28:09 <mikedep333> ggainey: exactly, I just don't want ta huge set of omissions or huge hogging of resources at the moment.
16:28:12 <fao89> mikedep333++
16:28:12 <pulpbot> fao89: mikedep333's karma is now 140
16:28:15 <mikedep333> :)
16:28:29 <ggainey> zacly
16:28:46 <x9c4> mikedep333++
16:28:46 <pulpbot> x9c4: mikedep333's karma is now 141
16:28:46 <fao89> next topic?
16:28:52 <ggainey> +1 to next
16:28:55 <ggainey> we have 2 min
16:29:07 <fao89> topic: GHA is warning us that ubuntu-latest will become ubuntu-20.04 sometime soon
16:29:23 <dkliban> i don't see it as a problem
16:29:28 <mikedep333> Yeah, I cannot find a date. The last update was 5 days ago that they are working on it.
16:29:29 <fao89> I think we'll be safe with the change
16:29:36 <mikedep333> fao89: You do?
16:29:54 <mikedep333> It would be ideal to update the plugin-template to ubuntu-20.04 manually. And see if anything breaks.
16:29:59 <fao89> yep, while ago I use ubuntu 20 and it went well
16:30:06 <ggainey> ah ok
16:30:09 <mikedep333> docker does isolate us from most changes though.
16:30:24 <x9c4> fao89, that's foresight!
16:30:34 <ggainey> fao89++
16:30:34 <pulpbot> ggainey: fao89's karma is now 138
16:30:47 <dkliban> we didn't triage any bugs
16:30:51 <x9c4> fao89++
16:30:51 <pulpbot> x9c4: fao89's karma is now 139
16:31:14 <fao89> yep, open floor is taking longer and longer lately
16:31:16 <fao89> #endmeeting