15:30:29 <fao89> #startmeeting Pulp Triage 2021-02-19
15:30:29 <fao89> #info fao89 has joined triage
15:30:29 <fao89> !start
15:30:29 <pulpbot> Meeting started Fri Feb 19 15:30:29 2021 UTC.  The chair is fao89. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:30:29 <pulpbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:30:29 <pulpbot> The meeting name has been set to 'pulp_triage_2021-02-19'
15:30:29 <pulpbot> fao89: fao89 has joined triage
15:31:03 <ttereshc> #info ttereshc has joined triage
15:31:03 <ttereshc> !here
15:31:04 <pulpbot> ttereshc: ttereshc has joined triage
15:31:10 <ppicka> #info ppicka has joined triage
15:31:10 <ppicka> !here
15:31:11 <pulpbot> ppicka: ppicka has joined triage
15:31:11 <gerrod> #info gerrod has joined triage
15:31:11 <gerrod> !here
15:31:12 <pulpbot> gerrod: gerrod has joined triage
15:31:12 <daviddavis> #info daviddavis has joined triage
15:31:12 <daviddavis> !here
15:31:13 <pulpbot> daviddavis: daviddavis has joined triage
15:31:52 <ggainey> #info ggainey has joined triage
15:31:52 <ggainey> !here
15:31:52 <pulpbot> ggainey: ggainey has joined triage
15:31:58 <bmbouter> #info bmbouter has joined triage
15:31:58 <bmbouter> !here
15:31:59 <pulpbot> bmbouter: bmbouter has joined triage
15:32:15 <fao89> we don't have any topics for open floor
15:32:24 <fao89> should we start triage?
15:32:31 <ggainey> triiiiiiiiiiiage
15:33:03 <fao89> !next
15:33:04 <fao89> #topic https://pulp.plan.io/issues/8283
15:33:04 <pulpbot> fao89: 3 issues left to triage: 8283, 8282, 8277
15:33:05 <pulpbot> RM 8283 - ggainey - POST - generic-list work broke content guards
15:33:06 <pulpbot> https://pulp.plan.io/issues/8283
15:33:14 <ggainey> accept and add pliz
15:33:17 <ipanova> #info ipanova has joined triage
15:33:17 <ipanova> !here
15:33:17 <pulpbot> ipanova: ipanova has joined triage
15:33:20 <fao89> #idea Proposed for #8283: accept and add to sprint
15:33:20 <fao89> !propose other accept and add to sprint
15:33:20 <pulpbot> fao89: Proposed for #8283: accept and add to sprint
15:33:34 <ttereshc> +!
15:33:35 <ttereshc> +1
15:33:38 <ipanova> +1
15:33:41 <bmbouter> +1
15:33:42 <fao89> #agreed accept and add to sprint
15:33:42 <fao89> !accept
15:33:42 <pulpbot> fao89: Current proposal accepted: accept and add to sprint
15:33:42 <fao89> #topic https://pulp.plan.io/issues/8282
15:33:43 <pulpbot> fao89: 2 issues left to triage: 8282, 8277
15:33:44 <pulpbot> RM 8282 - dalley - NEW - Make "publish_settings" a mandatory parameter on Publication.create()
15:33:45 <pulpbot> https://pulp.plan.io/issues/8282
15:34:30 <ttereshc> it's a task I think
15:34:39 <ggainey> oh - you mean 3.11 and 3.13, I panicked that there was pulp2 changes incoming :)
15:34:40 <ttereshc> and probably it should say 3.11 and 3.13
15:34:42 <ipanova> dalley: it should 3.11 and 3.13?
15:34:45 <ipanova> i have updated that
15:34:48 <ggainey> heh
15:34:52 <fao89> #idea Proposed for #8282: convert to task
15:34:52 <fao89> !propose other convert to task
15:34:52 <pulpbot> fao89: Proposed for #8282: convert to task
15:35:07 <ggainey> ggainey panics, ipanova fixes - who's the better developer? :)
15:35:16 <ggainey> +1 to task
15:35:22 <daviddavis> +1
15:35:29 <dalley> sorry, I'm back
15:35:34 <ppicka> +1
15:35:39 <ipanova> ggainey: :D lol this is a symbiosis
15:35:41 <dalley> this might be a non-issue if we're syncing up
15:35:43 <ggainey> :)
15:35:48 <dalley> the versions*
15:35:54 <x9c4> #info x9c4 has joined triage
15:35:54 <x9c4> !here
15:35:54 <pulpbot> x9c4: x9c4 has joined triage
15:36:10 <dalley> leave it un-triaged for now
15:36:25 <dalley> we may end up closing it
15:36:28 <fao89> skip it then?
15:36:33 <dalley> yes
15:36:37 <fao89> !skip
15:36:38 <fao89> #topic https://pulp.plan.io/issues/8277
15:36:38 <pulpbot> fao89: 1 issues left to triage: 8277
15:36:40 <pulpbot> RM 8277 - wibbit - NEW - Pulp upgrade fails - core.0049_add_file_field_to_uploadchunk
15:36:41 <pulpbot> https://pulp.plan.io/issues/8277
15:36:59 <ggainey> this is the one wibbit hit
15:37:29 <ggainey> I think we need to accept-and-add - right now, if your core_uploadchunks table has entries in it when you go to upgrade, 0049 will fail
15:37:37 <bmbouter> yeah ouch
15:37:40 <bmbouter> we gotta fix dis
15:37:44 <ggainey> yus
15:37:59 <daviddavis> there was a release note about this https://github.com/pulp/pulpcore/blob/master/CHANGES.rst#removals-2
15:38:05 <bmbouter> maybe we can discuss some outside of triage, I don't understand the path to fix
15:38:14 <fao89> #idea Proposed for #8277: accept and add to sprint
15:38:14 <fao89> !propose other accept and add to sprint
15:38:14 <pulpbot> fao89: Proposed for #8277: accept and add to sprint
15:38:39 <ggainey> daviddavis: ah, I'd missed that - but it's not "encouraged" , it's "your upgrade will fail if you don't"
15:38:49 <fao89> #agreed accept and add to sprint
15:38:49 <fao89> !accept
15:38:49 <pulpbot> fao89: Current proposal accepted: accept and add to sprint
15:38:50 <pulpbot> fao89: No issues to triage.
15:39:06 <fao89> I think we can "open floor" this issue
15:39:08 <ipanova> bmbouter:  i agree, the expectations were that users would do what was mentioed in the removal note.
15:39:19 * bmbouter reads the removal
15:39:28 <bmbouter> also we should go through the CIs if we can (if that makes sense)
15:39:53 <ipanova> ggainey: maybe we should clarify our language
15:40:00 <ggainey> at a minimum
15:40:04 <fao89> pulp_ansible is failing
15:40:05 <bmbouter> ipanova: oic, so is that the answer we just say hey rm -rf this? or do we go back and have migration 0049 do that as a data migration?
15:40:35 <fao89> pulp_rpm is failing (seems to be intermittent thing with performance tests)
15:40:38 <ggainey> it's not rm -rf - it's "psql trunc core_uploadchunks;"
15:41:09 <bmbouter> oic this reads to me like a filesystem actoin required
15:41:21 <ggainey> yeah
15:41:32 <bmbouter> if it's a db thing I think migration 0049 should be handling it
15:41:55 <ipanova> bmbouter: i would need to remember the details, i thought this was fs stuff
15:41:57 <ggainey> well, 0049 was released with 3.9. Can you update it?
15:42:06 <ggainey> ipanova: the failure is at migration-time
15:42:33 <bmbouter> I think we can in this specific case because users who weren't affected already ran it and didn't need it. users who are cannot run the migration so our newly shipped version would run just for them
15:42:43 <ggainey> 8277 is, if core_uploadchunks has rows in it, 0049 migration fails
15:42:53 <bmbouter> and I am almost always a never change a shipped migration kind of fellow
15:42:59 <ggainey> bmbouter: that makes sense
15:43:05 <ipanova> ggainey: i read this note as - before upgrade perform an rm -rf
15:43:18 <bmbouter> me2
15:44:04 <bmbouter> so yeah I think the fix is add ^ to the migration 0049, and also clarify this release note
15:44:20 <ggainey> ipanova: the note in the readme absolutely says that , yes - it's just not sufficient
15:45:05 <ipanova> well, we should have specified this step along with the upgrade steps.
15:45:23 <ipanova> now it i too late because the user did not perform it and now has migration failure
15:45:36 <ttereshc> also notes say that "it is encouraged" which is not strong enough if it's a requirement, imo
15:45:42 <ggainey> yup
15:45:57 <bmbouter> I don't think it's too late, the migration fails and 0049 is shows as unrun I expect
15:46:00 <ipanova> ttereshc: yes we should have capslocked a MUST :D
15:46:14 <ttereshc> bmbouter, that would be my expectation as well
15:46:31 <x9c4> But the user can still clean up and run the migration again. So documenting that is a solution.
15:46:31 <ggainey> bmbouter: aye belike - wibbit trunc'd the table and his migration swent successfully afterwards
15:47:00 <bmbouter> oh good
15:47:01 <ipanova> can we just clarify our language?
15:47:07 <bmbouter> I think we should fix for future users then still
15:47:11 <ggainey> concur
15:47:31 <x9c4> So if we truncate the table in the migration, The docs are fine again. right?
15:47:33 <bmbouter> I think the "wibbit got it working" info just tells me we know the scope of the fix is right
15:47:38 <ggainey> like, say, anyone currently on 3.7 upgrading to pulp-latest - would rather it "just worked"
15:47:50 <bmbouter> yeah the docs matter less at least. I would still touch them up though (personally)
15:47:52 <x9c4> Because cleaning your harddrive is "recommended".
15:48:07 <ggainey> x9c4: I don't know what the implications are of leaving old-chunks around on the filesystem - prob just space?
15:48:10 * bmbouter defrags his disk
15:48:26 <ggainey> wow, defragging - haven't thought about that word in forever :)
15:48:29 <bmbouter> yeah at some point we'll clean those up too but in a future far far away
15:48:55 <x9c4> https://www.getdigital.de/Defragged-Zebra.html
15:49:10 <ggainey> niiice
15:49:26 <bmbouter> lolol
15:49:39 <ggainey> anyway - so are we agreed on "fix 0049, revisit docs"?
15:49:55 <x9c4> So i see too options: 1. trunc in the migration; 2. trunc in the Docs.
15:50:13 <ggainey> migration - never require users to open postgres, that's just asking for disaster
15:50:26 <x9c4> +1 here
15:50:36 <ttereshc> I agree, it's bad ux to ask users to ttunc
15:50:40 <bmbouter> if it's one or the other I recommend the migration, it'll Just Work™
15:50:45 <ggainey> yus
15:50:48 <ttereshc> +1
15:51:02 <ipanova> agree
15:51:12 <ttereshc> so it mean we need to insert one migration in between old ones?
15:51:24 <x9c4> And the docs already tell the user to wipe his harddrive clean
15:51:25 <ttereshc> or just fix an existing one
15:51:26 <bmbouter> I think rewrite 0049
15:51:29 <ttereshc> yeah
15:51:31 <bmbouter> well add to it really
15:51:33 <ttereshc> ok
15:51:38 <bmbouter> can we put this onto 3.11?
15:52:04 <ttereshc> bmbouter, when 0049 was added? 3.10? maybe we needa 3.10.z
15:52:10 <ttereshc> to fix the upgrade path
15:52:13 <bmbouter> https://upload.wikimedia.org/wikipedia/en/5/56/311_album_cover.jpg
15:52:15 <x9c4> rewriting. should be fine because if it is recoreded as alredy run, than the user didnt' need that change.
15:52:23 <bmbouter> x9c4: agreed
15:52:26 <ttereshc> +1
15:52:47 <bmbouter> 3.10.z would be right but I don't think we explicitly need it unless someone asks us for it who can't consume 3.11
15:53:05 <bmbouter> 3.10.z is right (no question) I just don't know if we can afford to release timewise
15:53:42 <daviddavis> technically the problem was in 3.9
15:53:46 <ggainey> yes
15:53:59 <daviddavis> so if we really wanted to fix it, we'd have to release a new 3.9 and 3.10
15:54:06 <ttereshc> oh, then 3.9.z and 3.10.z
15:54:08 <ttereshc> yeah
15:54:09 <ggainey> it happened in 3.9. I agree with "fix in 3.11, backport only if we need to"
15:54:14 <daviddavis> +1
15:54:18 <ttereshc> +1
15:54:25 <ipanova> bmbouter: ttereshc it was 3.9
15:54:34 <ipanova> daviddavis: oh you beat me to it
15:54:34 <daviddavis> there's a workaround too: delete all your uploads
15:54:36 <ggainey> (it may be that poor wibbit is the only person who's ever affected , and he already fixed his installation :) )
15:54:46 <daviddavis> not your files but the uploads in the API
15:54:53 <wibbit> ggainey: Yup, I've fixed my stuff
15:54:59 <ggainey> yeah, if you use the API, I *think* it might do the right thing
15:55:04 <ggainey> wibbit++
15:55:04 <pulpbot> ggainey: wibbit's karma is now 11
15:55:08 <dalley> BTW that issue is actually https://pulp.plan.io/issues/7316
15:55:09 <x9c4> And you can delete them via the api, right?
15:55:13 <dalley> files not being deleted
15:55:31 <daviddavis> x9c4: yes
15:55:42 <dalley> maybe not. anyway - might be related
15:55:55 <daviddavis> yea, the issue is that the storage location of uploads changed
15:55:56 <bmbouter> proposal: so add to 3.11 ?
15:55:59 <daviddavis> +1
15:56:12 <ipanova> yeah
15:56:15 <x9c4> +1
15:56:23 <ggainey> +1
15:57:21 <ggainey> yesterday was def a Fun day
15:58:08 <daviddavis> I think our triage lead has abandoned us
15:59:19 <ipanova> heh
15:59:30 <ipanova> this was the last issue to triage i think
16:00:07 <bmbouter> def fun_day(person=ggainey):
16:00:38 <ggainey> heh
16:01:20 <fao89> indeed, I got distracted with postgres, but I'm back
16:01:37 <ggainey> understandable
16:01:37 <fao89> and I believe the open floor/triage has ended
16:01:46 <fao89> #endmeeting
16:01:46 <fao89> !end