14:30:09 #startmeeting Pulp Triage 2021-06-01 14:30:09 #info daviddavis has joined triage 14:30:09 !start 14:30:09 Meeting started Tue Jun 1 14:30:09 2021 UTC. The chair is daviddavis. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:30:09 Useful Commands: #action #agreed #help #info #idea #link #topic. 14:30:09 The meeting name has been set to 'pulp_triage_2021-06-01' 14:30:09 daviddavis: daviddavis has joined triage 14:30:24 #info ttereshc has joined triage 14:30:24 !here 14:30:25 ttereshc: ttereshc has joined triage 14:30:26 #info ggainey[m] has joined triage 14:30:26 !here 14:30:26 ggainey[m]: ggainey[m] has joined triage 14:32:19 I'd like to have one more person here 14:32:22 #info gerrod has joined triage 14:32:23 !here 14:32:23 gerrod: gerrod has joined triage 14:32:30 Topic: Does it make sense to start doing regular Z releases (for the latest Y stream) and not wait with bugfixes for the next Y release. 14:32:38 #info ppicka has joined triage 14:32:38 !here 14:32:39 ppicka: ppicka has joined triage 14:32:51 I thought we discussed it last time 14:32:56 me too 14:32:59 the issue for this was the installer no? 14:33:17 we did, and iirc the conclusion was "yes, but release-auto needs to be more bulletproof"? 14:33:24 yeah 14:33:35 +1 14:33:37 not sure if the bottleneck is the installer specifically 14:33:38 next topic? 14:33:41 +! 14:33:42 +1 14:33:51 Topic: Have a specific repo to host Github Discussions rather than use pulpcore 14:33:58 Example: https://github.com/github/feedback 14:34:18 there's some discussion about this here too: https://github.com/pulp/pulpcore/discussions/1377 14:34:29 I like the idea - feels less likley to confuse ppl? 14:34:49 #info bmbouter has joined triage 14:34:49 !here 14:34:49 bmbouter: bmbouter has joined triage 14:34:59 but having it have to be a repo is...odd. 14:35:11 yea, I agree 14:35:21 agreed 14:35:59 the problem with using pulpcore is also labels/permissions/etc can't be controlled individually like we could do for a separate repo 14:36:03 re regular Z releases, I'm thinking maybe with 3.14 we could if the automation becomes 1-click and the installer no longer requires z releases. mikedep333 did you have an issue written for the z-releases? 14:36:09 I'm not sold on "just use pulpcore", but I am *definitely* against "every plugin has their own" 14:36:43 @david 14:36:44 bmbouter: can we come back to that topic? I don't want to discuss 2 topics at once 14:36:45 daviddavis: what permissions did we want that we don't have now? 14:36:48 argh, fingers 14:36:55 ok yes let's come back 14:37:15 daviddavis: maybe we should discuss in 1377 yeah? 14:37:18 like if we want to give melanie access to control discussions, she would have to have the commit bit 14:37:25 that works 14:37:39 it didn't cross my mind to use a separate repo before I saw the github using it. Hard to find a better candidate for setting examples to follow. Maybe it feels odd but maybe because we are not used to it. 14:38:06 +1 14:38:15 let's think about it and follow up on https://github.com/pulp/pulpcore/discussions/1377 14:38:28 +1 14:38:39 imo pulpcore only is more confusing than a separate repo 14:38:46 yea 14:38:59 Topic: Libera/matrix bridge is live: https://fosstodon.org/@liberachat/106320476905973945 14:39:00 +1 14:39:17 I don't know what the next steps are here? 14:39:48 daviddavis: have you had any progress RE registering pulp as an official libera project? 14:40:01 from the feedback we've gotten having non-IRC be primary is what I've heard 14:40:04 mcorr: submitted the application I believe 14:40:06 which to me means matrix 14:40:13 or - do we just bite the bullet, and start moving as a community to the matrix-space that mcorr has set up for us? 14:40:14 and libera bridge would allow IRC access 14:40:24 yeah 14:40:48 that was my understanding. I just wasn't sure who was setting up matrix, the bridge, the bot, etc 14:40:52 I've switcted to matrix primary for a week now, and while I'm still fighting w/ it some, it's def doable 14:41:04 yeah I'm unclear on the next steps for the matrix setup 14:41:38 not to overuser the hammer but ... github discussion to coordinate the setup of those things 14:41:46 ? 14:41:47 heh, +1 14:41:50 +1 14:41:58 should we also move the meeting bot to libera to move off of freenode completely? 14:41:58 is there one already? 14:42:05 bmbouter: sometimes, your problem really is just a nail :) 14:42:15 :) 14:42:29 ttereshc: I'd like to. I want to get off freenode ASAP. I keep seeing more reports of channels being taken over by freenode staff. 14:42:38 I agree 14:42:47 is there someone who can take the AI to start the discussion? 14:42:49 we def want out IRC-native presence to be libera ASAP 14:43:33 I can start, I just don't feel familiar enough with the meeting bot internals 14:43:44 me neither but dkliban can probably help 14:43:52 so we'll need official registration of those channels on libera, and someone can go ahead and do that 14:44:00 I've done that 14:44:05 and parallel we can work on matrix setup and then bridge and declare matrix primary 14:44:10 +1 14:44:51 +1 14:44:57 maybe we'll use #pulp-meeting on libera until we have a matrix bot set up and hopefully we can bridge #pulp-meeting 14:45:12 i can help with pulpbot 14:45:16 sweet 14:45:19 dkliban++ 14:45:19 daviddavis: dkliban's karma is now 599 14:45:29 ttereshc++ 14:45:29 daviddavis: ttereshc's karma is now 371 14:45:36 we should file a task for this effor though 14:45:45 bmbouter: a lot of chanserv setup already happened on libera 14:45:51 +1 to task 14:45:57 agreed +1 14:46:00 +1 14:46:22 +1 14:46:39 Topic: no-meeting-Fridays June-August - should we resched the second triage-mtg? Or just have one per week? 14:46:55 I added this 14:47:10 we do not have no-meetings rule yet, is it a suggestion to have it? 14:47:17 if we want to hold to no-meeting-fri for the summer, we're down to just Tues for triage 14:47:41 monday thursday perhaps? 14:47:47 ttereshc: I think it's a fine idea - but if we do, this is one consequence 14:47:52 or just one per week on wednesday 14:48:11 I think we could try just one per week 14:48:19 dalley: my thinking was "is once a week enough?" if it is, Tues is prob fine 14:48:22 it's very uncommon we go the entire time 14:48:33 yeah, I'm cool w/ that, and adjust when/if needed 14:49:07 I'm confused. The topic suggests that we already have no-meetings rule and we are figuring out what to do with the triage. Maybe I'm behind, sorry. DO we have this rule? 14:49:37 not afaik 14:49:53 it's a rule that robin is trying out this summer since other teams are going to try to do so aswell 14:50:04 ttereshc: I think "should we have no-mtg-Fri" prob should have been on the agenda before my bullet :) 14:50:10 emphasis on the "try" 14:50:13 ok, maybe it's a topic to the team meeting if it'snot a rule yet 14:50:30 * for the team meeting 14:50:34 I know last week we talked about it, and were going to discuss in full at the next open-floor - and here we are. But it didn't get added to the agenda 14:50:39 #info fao89 has joined triage 14:50:39 !here 14:50:39 fao89: fao89 has joined triage 14:50:53 yeah, maybe team-mtg is the best place for this 14:51:08 and all of us in the US weren't here yesterday 14:51:12 ok, sorry, I haven't heard about it here 14:52:41 +1 to having trying one triage/openfloor a week 14:52:44 I have the goal of fewer meetings, which summer-goal-of-no-meeting-friday aside, I think would be good 14:52:46 to try at least 14:53:30 +1 14:53:34 looks like there was some discussion on the 25th, from my backscroll-reading 14:54:08 anyway - I'm good w/ trying for no-mtg-Fri, and I'm also good w/ 1-triage-a-week until we figure out we can't 14:54:19 fwiw :) 14:55:02 let's discuss more at monday's team meeting? 14:55:16 or do we want to decide now 14:55:20 sure 14:55:27 whatever works for me 14:55:32 yeah that 14:55:36 I'll add a topic to monday's meeting 14:55:39 I seem to be behind and unaware of the rule 14:55:50 so if it's a triage only question, t's good to decide her 14:55:57 e 14:56:12 if it's to decide on the rule, then +1 to the tesm meeting 14:56:12 Topic: what if recoccuring meeting announcements all went on one thread instead of one per meeting? 14:56:14 https://github.com/pulp/pulpcore/pull/1345/files 14:56:33 um, that PR has nothing to do w/ discussions? 14:56:42 that might be a mispaste 14:56:58 yea, not sure I understand this link 14:57:03 bmbouter: ^ 14:57:14 * bmbouter reads 14:57:24 oh geez wrong link 14:57:27 lol 14:57:29 he probably wanted this one? https://github.com/pulp/pulpcore/discussions/1379 14:57:35 https://github.com/pulp/pulpcore/discussions/1379 14:57:37 yup 14:57:43 one thread works for me 14:57:56 yeah I was thinking: so discussions that get overwhelmed with reocurring meeting minutes lowers information value 14:58:12 so I'd like to try this (and reoccuring meeting posters to try this) 14:58:26 +1 to 1 thread 14:58:32 ahh, I like that idea 14:58:37 +1 14:58:54 +1 14:59:01 do they go up when there is a new comment? 14:59:16 yes 14:59:24 with date as the heading 14:59:28 * bmbouter updates pulpcore to show 14:59:33 pulpcore meeting 14:59:38 see on matrix I could edit that ;) 14:59:46 * daviddavis groans 14:59:51 heh 15:00:14 edit is evil 15:00:20 :) 15:00:38 https://github.com/pulp/pulpcore/discussions/1379#discussion-3391667 15:00:43 if it moves up, then good 15:01:05 yeah it moved up 15:01:23 \o/ 15:01:37 * ggainey[m] cheers 15:01:50 cool, so let's try to keep minutes on a single thread then, yea? 15:01:55 +1 15:02:15 +1 15:02:20 bmbouter, on this thread, there is only one comment from today 15:02:26 what exactly moved up? 15:02:41 ttereshc: the thread in the list of threads at https://github.com/pulp/pulpcore/discussions 15:02:53 the pulpcore meeting minutes thread 15:02:58 oh I also didn't understand the 'move up' yeah great! 15:03:32 oh actually there's no comment 15:03:38 i think ttereshc is wondering how will you update the threads? 15:03:55 like edit the main post or add separate comments 15:03:59 if I add a new comment in a week, will the thread move up? 15:04:00 I'll post a new comment 15:04:17 one for each meeting (apologies to not have two minutes worth of examples) 15:04:27 I bumped this thread and now it's first https://github.com/pulp/pulpcore/discussions/1358 15:04:45 so I think a comment is good so it bumps the thread 15:04:47 if we add a new comment, it will be at the end of the thread which might become inconvenient, so maybe a thread per month or a year? 15:04:58 +1 15:05:07 yeah eventually we'll need new thread 15:05:09 daviddavis, thanks, great that it works 15:05:17 you can sort comments by newest-first 15:05:20 you can filter by newest: https://github.com/pulp/pulpcore/discussions/1358 15:05:22 yea 15:05:24 hehe 15:05:30 cool 15:05:41 yes indeed 15:06:06 ok, let's try single-thread-with-comments then 15:06:09 can we set meeting discussions to auto-sort comments by newest first? 15:06:17 not that I see right now 15:06:19 and by "top" - so upvoted ones go first? not sure what "top" means yet 15:06:30 gerrod: I think that's a per-user decision 15:06:38 ggainey[m]: that's my belief re: top 15:06:58 anyway - there are tools available to us, def worth the experiment I think 15:07:09 we have a bunch of issues so let's move on to the next topic 15:07:12 Topic: Is everything in place for the 3.13 container image release today? (some plugin issues late last week were blocking) 15:07:15 I'll try it and we'll see 15:07:29 looks like we need a release of pulp-ansible 15:07:43 dkliban: am I remembering right you're releasing this? 15:07:54 yes i am 15:07:58 i'll do it today 15:07:59 ggainey[m], on reddit, mods can default change the sorting of comments for a thread, the user can always switch it. Probably just a feature request for github. 15:08:13 is ansible the last blocker? 15:08:20 i believe so 15:08:26 from my test, yes 15:08:31 great! 15:08:37 gerrod: aye, I just don't think it's there currently, is all :) 15:08:42 just ping me when it's ready 15:09:01 dalley++ 15:09:01 daviddavis: dalley's karma is now 378 15:09:04 dkliban++ 15:09:04 daviddavis: dkliban's karma is now 600 15:09:24 should we return to the z-release topic before we triage? fwiw, I don't see mikedep333 around 15:09:26 bmbouter: ^ 15:10:07 I mainly just wanted to say: I think we're close but not ready today to adopt z-stream release weekly 15:10:11 dkliban: before releasing pulp-ansible, I found an issue with pulp-maven release 15:10:29 https://github.com/pulp/pulp_maven/pull/41#issuecomment-851514599 15:10:33 daviddavis: if I should wait for a topic change lmk 15:11:27 bmbouter: no, that's cool and I think we agree on that. I didn't hear any objections. 15:12:01 ok, let's triage then 15:12:08 !next 15:12:09 daviddavis: 8 issues left to triage: 8843, 8839, 8837, 8830, 8782, 8781, 8779, 8744 15:12:09 #topic https://pulp.plan.io/issues/8843 15:12:10 RM 8843 - mdellweg - NEW - Export list fails with 500 when a repository is deleted 15:12:11 https://pulp.plan.io/issues/8843 15:12:26 #info mikedep333 has joined triage 15:12:26 !here 15:12:26 mikedep333: mikedep333 has joined triage 15:12:28 daviddavis: ty 15:12:33 accept and add 15:12:36 +1 15:12:40 +1 15:12:47 #idea Proposed for #8843: accept and add to sprint 15:12:47 !propose other accept and add to sprint 15:12:47 daviddavis: Proposed for #8843: accept and add to sprint 15:13:20 #agreed accept and add to sprint 15:13:20 !accept 15:13:20 daviddavis: Current proposal accepted: accept and add to sprint 15:13:21 #topic https://pulp.plan.io/issues/8839 15:13:21 daviddavis: 7 issues left to triage: 8839, 8837, 8830, 8782, 8781, 8779, 8744 15:13:21 +1 15:13:22 RM 8839 - vk - NEW - file content upload performance needs improvement -- currently about 5x slower than rsync 15:13:23 https://pulp.plan.io/issues/8839 15:13:48 I talked to this user on Thursday and reproduced this 15:14:07 through increasing the chunk size, you can get it down to 43s 15:14:15 half of the time is in the cli and the other half is server side 15:14:46 isn't 43s with increased chunk size / no chunking? 15:14:47 daviddavis: so if you use the API directly, it's faster? 15:14:58 with the defaults it was like 3 minutes IIRC 15:15:02 mmmm 15:15:22 we probably could increase defaults until we reach the default filesystem limit of nginx/apache 15:15:34 ggainey[m]: yea, depending on how you use the api 15:15:35 anyway a user filed this, a user is interested, we should fix 15:15:36 actually, nm, this is triage - this sounds like a task, and one we should accept. Not sure if it goes on the sprint, but probably yes? 15:15:44 I think so 15:15:51 +1 15:15:55 agreed 15:15:58 what is the fix though? 15:16:05 https://github.com/pulp/pulp_installer/blob/master/roles/pulp_webserver/defaults/main.yml#L37 15:16:05 we don't know yet :) 15:16:08 still sprints aren't how work gets done, to me someone saying I'll focus on this is the way 15:16:17 ggainey[m]: I can tell you where all the time is spent 15:16:22 fair enough 15:16:28 add to the task? 15:17:15 most of the info is in the description already 15:17:30 cool 15:17:39 I think this just needs some proposals 15:17:51 I'm just saying, we don't have to have an answer, to triage the issue as "work on this" 15:17:53 cool want to skip and ask for proposals? 15:18:01 that works for me 15:18:05 sure 15:18:09 #idea Proposed for #8839: Skip this issue for this triage session. 15:18:09 !propose skip 15:18:09 daviddavis: Proposed for #8839: Skip this issue for this triage session. 15:18:27 #agreed Skip this issue for this triage session. 15:18:27 !accept 15:18:28 daviddavis: Current proposal accepted: Skip this issue for this triage session. 15:18:28 #topic https://pulp.plan.io/issues/8837 15:18:29 daviddavis: 6 issues left to triage: 8837, 8830, 8782, 8781, 8779, 8744 15:18:30 RM 8837 - knzivid - NEW - Generate openapi python bindings with asyncio enabled 15:18:31 https://pulp.plan.io/issues/8837 15:18:58 huh, interesting 15:19:54 it is possible, we need to change pulp-openapi-generator 15:20:11 that's cool 15:20:23 ok - so, task? 15:20:28 but it may be a breaking change 15:20:40 needs some investigation, but it is doable 15:20:45 ouch 15:21:08 "breaking change", for the whole API at this point, would be really really painful 15:21:50 i don't think it should be breaking 15:22:01 we use 3 different versions of openapi-generator-cli, https://github.com/pulp/pulp-openapi-generator/blob/master/generate.sh#L22 15:22:06 this is just having the requests the bindings do be async no? 15:22:10 I don't have enough firm info to worry about it just yet 15:22:18 yeah same 15:22:31 if async requires a specific release, it may be a breaking change release 15:23:01 let's explore first and then tactically introduce when we know what is needed 15:23:14 +1 15:23:20 it is a matter of verifying if the openapi-generator-cli we use already supports async 15:23:25 so, task and accept? 15:23:52 sorry, task, accept, on-sprint? 15:24:18 I think so 15:24:25 #idea Proposed for #8837: convert to task and add to sprint 15:24:25 !propose other convert to task and add to sprint 15:24:25 daviddavis: Proposed for #8837: convert to task and add to sprint 15:24:41 +1 15:25:17 #agreed convert to task and add to sprint 15:25:17 !accept 15:25:17 daviddavis: Current proposal accepted: convert to task and add to sprint 15:25:18 daviddavis: 5 issues left to triage: 8830, 8782, 8781, 8779, 8744 15:25:19 #topic https://pulp.plan.io/issues/8830 15:25:19 RM 8830 - osapryki - NEW - Memory leak in openapi.json schema generation. 15:25:20 https://pulp.plan.io/issues/8830 15:25:22 sprint has 27 new items and we have a planning meeting tomorrow :/ 15:25:35 yeah sprints are where work goes to get forgotten 15:25:35 I guess we need to figure out how to use them again 15:26:10 triage-wise (4 min left) we need to fix 8830 15:26:17 I haven't looked at the current sprint in months 15:26:19 I think it's pretty clear in terms of the concern 15:26:26 accept and add to sprint? 15:26:33 #idea Proposed for #8830: accept and add to sprint 15:26:33 !propose other accept and add to sprint 15:26:33 daviddavis: Proposed for #8830: accept and add to sprint 15:26:34 yup (RE 8830) 15:26:34 +1 15:26:41 we can't forget this one so I think we need more than sprint 15:26:58 we need someone to take as ASSIGNED 15:27:10 if we can't then ok but I think that's the need 15:27:18 I can take it 15:27:22 fao89++ 15:27:22 daviddavis: fao89's karma is now 176 15:27:26 wooooot 15:27:28 !next 15:27:29 #topic https://pulp.plan.io/issues/8782 15:27:29 daviddavis: 4 issues left to triage: 8782, 8781, 8779, 8744 15:27:30 RM 8782 - alikins - NEW - Intermittent psycopg2.errors.AdminShutdown errors in galaxy_ng dev env and pulp-all-in-one 15:27:31 https://pulp.plan.io/issues/8782 15:27:32 fao89: if you want any discussion about it lmk 15:27:51 I posted my reproduer on this, I think we still need to skip until we understand it 15:28:25 my strategy is equally unworkable to the sprint. I propose skip so it keeps coming back until the triage's accumulate so much work it's intractable (so I don't have a great solution to our sprint problems either really) 15:28:38 dkliban: I wanted to see what you thought about 8782 15:28:41 and we skip 15:29:09 #idea Proposed for #8782: Skip this issue for this triage session. 15:29:09 !propose skip 15:29:09 daviddavis: Proposed for #8782: Skip this issue for this triage session. 15:29:12 #agreed Skip this issue for this triage session. 15:29:12 !accept 15:29:12 daviddavis: Current proposal accepted: Skip this issue for this triage session. 15:29:12 #topic https://pulp.plan.io/issues/8781 15:29:13 daviddavis: 3 issues left to triage: 8781, 8779, 8744 15:29:13 bmbouter, does increasing sleep time help 15:29:14 RM 8781 - newswangerd - NEW - /pulp/api/v3/ throws a 500 15:29:15 https://pulp.plan.io/issues/8781 15:29:29 this is on galaxy side 15:29:32 ttereshc: strangely it doesn't, I tripled it and it didn't.... 15:29:42 ok, tx 15:29:51 yes I think close NOTABUG and ask for pulp-only reproduer 15:30:04 yea I talked to ansible about this 15:30:08 I think WORKSFORME 15:30:14 I have a PR: https://github.com/ansible/galaxy_ng/pull/750 15:30:20 WORKSFORME worksforme 15:30:30 sweet 15:30:37 #idea Proposed for #8781: close as WORKSFORME 15:30:37 !propose other close as WORKSFORME 15:30:38 cool 15:30:39 daviddavis: Proposed for #8781: close as WORKSFORME 15:30:40 +1 15:30:40 #agreed close as WORKSFORME 15:30:40 !accept 15:30:40 daviddavis: Current proposal accepted: close as WORKSFORME 15:30:41 #topic https://pulp.plan.io/issues/8779 15:30:41 daviddavis: 2 issues left to triage: 8779, 8744 15:30:42 RM 8779 - adam.winberg@smhi.se - NEW - Task started on removed worker 15:30:43 https://pulp.plan.io/issues/8779 15:31:02 yup this is our tasking system issue... we don't understand what the issue is here 15:31:08 yupyup 15:31:14 but I'll point out that prior to pulpcore 3.9 we didn't have these issues that I can remember 15:31:24 and we made changes which I think introduced issues 15:31:58 and also ultimately this code will be going away over time, so unless someone has an idea on how to investigate I'm thinking, revert and leave as is until the new system becomes the default 15:32:19 dalley: x9c4 as the tasking system peeps I kind of look to you for feedback on this 15:32:19 are we sure that versions >3.9 aren't just getting more exercise? 15:32:23 we're not 15:32:33 but we had 0 reports of issues except from oleksandr 15:32:37 and it wasn't of this nature 15:32:49 triage-wise I think skip (my not-so-great strategy) 15:32:59 #idea Proposed for #8779: skip 15:32:59 !propose other skip 15:32:59 daviddavis: Proposed for #8779: skip 15:33:00 dalley: x9c4 we should continue this convo in #pulp-dev 15:33:02 yeah 15:33:43 x9c4 doesn't want to :) 15:33:55 lol 15:34:08 #agreed skip 15:34:08 !accept 15:34:08 daviddavis: Current proposal accepted: skip 15:34:09 #topic https://pulp.plan.io/issues/8744 15:34:09 daviddavis: 1 issues left to triage: 8744 15:34:10 RM 8744 - equipe_serveurs@exane.com - NEW - Pulp3 database migration broken with newly release click 8 15:34:11 https://pulp.plan.io/issues/8744 15:34:37 this should be closed 15:34:39 ? 15:34:45 I was going to ask the same 15:34:58 bmbouter: anything else we need to do for this? 15:35:20 +1 for closing 15:35:26 #idea Proposed for #8744: close and see if bmbouter says anything 15:35:26 !propose other close and see if bmbouter says anything 15:35:26 daviddavis: Proposed for #8744: close and see if bmbouter says anything 15:35:45 #agreed close and see if bmbouter says anything 15:35:45 !accept 15:35:45 daviddavis: Current proposal accepted: close and see if bmbouter says anything 15:35:46 daviddavis: No issues to triage. 15:35:46 I don't think so 15:35:48 +! 15:35:51 +1 15:35:53 #endmeeting 15:35:53 !end