14:30:19 <fao89> #startmeeting Pulp Triage 2020-05-29 14:30:19 <fao89> #info fao89 has joined triage 14:30:20 <pulpbot> Meeting started Fri May 29 14:30:19 2020 UTC. The chair is fao89. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:30:20 <pulpbot> Useful Commands: #action #agreed #help #info #idea #link #topic. 14:30:20 <pulpbot> The meeting name has been set to 'pulp_triage_2020-05-29' 14:30:20 <pulpbot> fao89: fao89 has joined triage 14:30:44 <dkliban> #info dkliban has joined triage 14:30:44 <dkliban> !here 14:30:44 <pulpbot> dkliban: dkliban has joined triage 14:30:46 <ppicka> #info ppicka has joined triage 14:30:46 <ppicka> !here 14:30:46 <pulpbot> ppicka: ppicka has joined triage 14:30:53 <fao89> !next 14:30:54 <pulpbot> fao89: 6 issues left to triage: 6851, 6837, 6835, 6834, 6828, 6714 14:30:54 <fao89> #topic https://pulp.plan.io/issues/6851 14:30:55 <pulpbot> RM 6851 - hyu - NEW - Regression in regenerate applicability 14:30:56 <pulpbot> https://pulp.plan.io/issues/6851 14:31:24 <fao89> #idea Proposed for #6851: move to rpm 14:31:24 <fao89> !propose other move to rpm 14:31:24 <pulpbot> fao89: Proposed for #6851: move to rpm 14:31:24 <ttereshc> #info ttereshc has joined triage 14:31:24 <ttereshc> !here 14:31:25 <pulpbot> ttereshc: ttereshc has joined triage 14:31:28 <x9c4> !here 14:31:28 <x9c4> #info x9c4 has joined triage 14:31:29 <pulpbot> x9c4: x9c4 has joined triage 14:31:51 <dkliban> +1 14:31:57 <fao89> #agreed move to rpm 14:31:57 <fao89> !accept 14:31:57 <pulpbot> fao89: Current proposal accepted: move to rpm 14:32:03 <ttereshc> +1 14:32:43 <fao89> !next 14:32:44 <pulpbot> fao89: 5 issues left to triage: 6837, 6835, 6834, 6828, 6714 14:32:44 <fao89> #topic https://pulp.plan.io/issues/6837 14:32:45 <pulpbot> RM 6837 - ekohl - NEW - Various links in installation instructions show a 404 page 14:32:46 <pulpbot> https://pulp.plan.io/issues/6837 14:32:55 <dkliban> #idea Proposed for #6837: accept and add to sprint 14:32:55 <dkliban> !propose other accept and add to sprint 14:32:55 <pulpbot> dkliban: Proposed for #6837: accept and add to sprint 14:32:56 <ggainey> #info ggainey has joined triage 14:32:56 <ggainey> !here 14:32:57 <pulpbot> ggainey: ggainey has joined triage 14:32:57 <ipanova> #info ipanova has joined triage 14:32:57 <ipanova> !here 14:32:58 <pulpbot> ipanova: ipanova has joined triage 14:33:12 <ipanova> +1 14:33:17 <fao89> #agreed accept and add to sprint 14:33:17 <fao89> !accept 14:33:17 <pulpbot> fao89: Current proposal accepted: accept and add to sprint 14:33:18 <pulpbot> fao89: 4 issues left to triage: 6835, 6834, 6828, 6714 14:33:18 <fao89> #topic https://pulp.plan.io/issues/6835 14:33:19 <pulpbot> RM 6835 - dkliban@redhat.com - NEW - .travis/release.py in pulpcore creates the wrong version for the final .dev version commit 14:33:20 <pulpbot> https://pulp.plan.io/issues/6835 14:33:44 <dkliban> i think we need to add more information to the wiki page 14:33:50 <dkliban> but this was mostly user error on my part 14:34:14 <fao89> the problem for me is: where is the right place for documenting it? 14:34:38 <ipanova> i think both wiki and --help helps 14:34:41 <x9c4> "release.py --help" ? 14:34:43 <fao89> I'll add a note to put it on the wiki then 14:35:12 <bmbouter> #info bmbouter has joined triage 14:35:12 <bmbouter> !here 14:35:12 <pulpbot> bmbouter: bmbouter has joined triage 14:35:16 <fao89> #idea Proposed for #6835: accept and add to sprint 14:35:16 <fao89> !propose other accept and add to sprint 14:35:18 <pulpbot> fao89: Proposed for #6835: accept and add to sprint 14:35:20 <dkliban> +1 14:35:22 <x9c4> +1 14:35:22 <fao89> #agreed accept and add to sprint 14:35:22 <fao89> !accept 14:35:23 <pulpbot> fao89: Current proposal accepted: accept and add to sprint 14:35:23 <fao89> #topic https://pulp.plan.io/issues/6834 14:35:24 <pulpbot> fao89: 3 issues left to triage: 6834, 6828, 6714 14:35:25 <pulpbot> RM 6834 - mdellweg - NEW - A pulpcore / plugin pr should be able to point to an experimental version of the pulp-ci base image 14:35:26 <pulpbot> https://pulp.plan.io/issues/6834 14:35:43 <ttereshc> task? 14:35:50 <bmbouter> +1 14:35:54 <dkliban> yeah ... but we should add it to the sprint after that 14:36:15 <dkliban> #idea Proposed for #6834: convert to task 14:36:15 <dkliban> !propose other convert to task 14:36:15 <pulpbot> dkliban: Proposed for #6834: convert to task 14:36:15 <fao89> !propose other convert to a task and add to sprint 14:36:20 <dkliban> haha 14:36:20 <x9c4> +1 14:36:23 <fao89> #idea Proposed for #6834: convert to a task and add to sprint 14:36:23 <fao89> !propose other convert to a task and add to sprint 14:36:23 <pulpbot> fao89: Proposed for #6834: convert to a task and add to sprint 14:36:27 <fao89> #agreed convert to a task and add to sprint 14:36:27 <fao89> !accept 14:36:27 <pulpbot> fao89: Current proposal accepted: convert to a task and add to sprint 14:36:28 <fao89> #topic https://pulp.plan.io/issues/6828 14:36:28 <pulpbot> fao89: 2 issues left to triage: 6828, 6714 14:36:28 <dkliban> +1 14:36:30 <pulpbot> RM 6828 - fao89 - POST - browsable API missing endpoints on /pulp/api/v3/ 14:36:31 <pulpbot> https://pulp.plan.io/issues/6828 14:36:44 <dkliban> #idea Proposed for #6828: accept and add to sprint 14:36:44 <dkliban> !propose other accept and add to sprint 14:36:46 <ttereshc> +1 14:37:00 <fao89> it is good to open floor, due the discussion on the PR 14:37:10 <ipanova> +1 14:37:19 <fao89> #agreed accept and add to sprint 14:37:19 <fao89> !accept 14:37:19 <pulpbot> fao89: Current proposal accepted: accept and add to sprint 14:37:43 <fao89> pulpbot is weird today 14:37:43 <pulpbot> fao89: Error: "is" is not a valid command. 14:37:49 <fao89> !next 14:37:50 <fao89> #topic https://pulp.plan.io/issues/6714 14:37:50 <pulpbot> fao89: 1 issues left to triage: 6714 14:37:51 <pulpbot> RM 6714 - alikins - NEW - drf builtin manage.py 'generateschema' command fails on pulp base viewsets 14:37:52 <pulpbot> https://pulp.plan.io/issues/6714 14:38:37 <dkliban> so think we should convert this to a story or task 14:38:45 <dkliban> the request is to generate openapi v3 schema 14:38:57 <dkliban> we currently only support generating openapi v2 schema 14:39:06 <ttereshc> and to go away from drf_yasg 14:39:08 <ttereshc> ? 14:39:10 <dkliban> yes 14:39:20 <ttereshc> +1 to a story 14:39:23 <dkliban> drf_yasg is not being actively developed anymore 14:39:41 <dkliban> i have noticed that they have not done a release in many months and PRs are not getting merged 14:39:44 <fao89> drf is not mature for it yet 14:40:02 <dkliban> fao89: i agree, but we shuold keep this on our road map 14:40:07 <fao89> I tried to use it, and I had to do many tweaks 14:40:07 <dkliban> and maybe use a different project 14:40:10 <bmbouter> it probably should be an epic 14:40:20 <bmbouter> it will likely be many tasks 14:40:25 <dkliban> bmbouter: +1 14:40:30 <ttereshc> #idea Proposed for #6714: convert to an epic story 14:40:30 <ttereshc> !propose other convert to an epic story 14:40:31 <pulpbot> ttereshc: Proposed for #6714: convert to an epic story 14:40:39 <fao89> #agreed convert to an epic story 14:40:39 <fao89> !accept 14:40:39 <pulpbot> fao89: Current proposal accepted: convert to an epic story 14:40:41 <pulpbot> fao89: No issues to triage. 14:40:53 <fao89> Open floor! 14:40:58 <fao89> https://hackmd.io/SVCMjpwXTfOMqF2OeyyLRw 14:41:07 <fao89> redmine spam! 14:41:36 <bmbouter> aw man I meant to look into this earlier 14:41:36 <fao89> https://pulp.plan.io/issues/5878 14:42:12 <ttereshc> I'm not sure what we can do without the api for removing notes 14:42:27 <ttereshc> it's the majority of spam, imo 14:42:39 <bmbouter> the option discussed last time as a temporary alternative was to require account confirmation 14:42:53 <ttereshc> ah I see 14:43:01 <dkliban> yeah ... i think we should require that 14:43:12 <ipanova> i tend to agree 14:43:25 <bmbouter> https://www.redmine.org/projects/redmine/wiki/Features#User-self-registration-support 14:44:07 <ggainey> account-confirmation is fine, but nowadays is unlikely to have a significant impact 14:44:22 <bmbouter> right now it's set to accoutn activation by email 14:44:23 <ggainey> the SEO-optimizer-folks have scripts to monitor email addresses and auto-confirm 14:44:36 <bmbouter> the issue is that we don't want to auto-confirm 14:44:37 <ggainey> ah, kk 14:44:41 <bmbouter> these are all humans registering 14:44:59 <ggainey> yah 14:45:07 <bmbouter> so the only option is administrators approving accounts 14:45:30 <dkliban> if amins have to verify accounts, we need some kind of comment from the user stating why they are opening the account 14:45:39 <bmbouter> I agree 14:45:39 <ggainey> aye 14:45:52 <bmbouter> and redmine's process won't change so we would have to have email correspondance 14:46:33 <bmbouter> the other option is to not use the API but have the automation drive a firefox browser to remove comments 14:46:37 <bmbouter> not a great option obviously 14:47:14 <dkliban> i've done things like that before with selinium 14:47:29 <dkliban> you can do it without an actual GUI browser 14:47:40 <dkliban> that's an option for selinium 14:47:45 <bmbouter> yup 14:47:50 <bmbouter> I also believe it's diable 14:47:55 <bmbouter> doable 14:48:45 <bmbouter> from a project perspective the cost of developer time (both in cost and opportunity cost) I think makes manual correspondance not really viable 14:48:54 <bmbouter> the spammer cost is extremely low 14:48:58 <bmbouter> our cost extremely high 14:49:33 <ttereshc> you think they will increase the number of registrations? 14:49:59 <bmbouter> even the current rate is several per day 14:50:41 <bmbouter> as in if all we had to handle were say 3-5 users per day it's a non-negligable economic impact on pulp 14:50:53 <bmbouter> and 0 economic impact on spammers 14:51:13 <dkliban> true dat 14:51:22 <ttereshc> I understand that 14:51:24 <bmbouter> ttereshc: do you know if we can lock their accounts ia the redmine api? 14:51:50 <ttereshc> I'm just not sure when we will be able to do the selenium way 14:52:03 <bmbouter> let's say we don't do the selenium way as a thought experiment 14:52:06 <ttereshc> bmbouter, they said yes, I couldn;t find it but I didn't try hard. 14:52:48 <ttereshc> bmbouter, https://www.redmine.org/issues/22434 seems like no 14:53:23 <ttereshc> but you can delete one :/ not that useful but an option 14:53:34 <bmbouter> I wonder if it deletes their content too 14:53:44 <bmbouter> oh I know it doesn't actually 14:53:47 <dkliban> bmbouter: no... marks it as by Anonymous 14:53:50 <bmbouter> it makes their content 'Anonymoous' 14:53:52 <bmbouter> yeah 14:53:53 <ttereshc> I agree 14:54:25 <bmbouter> I think selenium is our best option 14:54:43 <bmbouter> and to raise urgency again w/ plan.io 14:54:44 <dkliban> i agree 14:54:59 <bmbouter> or put a mechanical turk process in place 14:55:03 <dkliban> do we have an algorithm for identifying spam comments? 14:55:24 <dkliban> (btw, i gotta go in 5 mins) 14:55:47 <bmbouter> we could train a bayesian classifier easily, we could also flag comments with non-trusted outbound links (they always contain links) 14:56:18 <dkliban> bmbouter: what do you mean by non-trusted? 14:56:31 <bmbouter> we could have a whitelist over time 14:56:35 <dkliban> gotcha 14:56:52 <dkliban> do these users ever create new issues? 14:56:56 <ttereshc> lmjachky was looking for a neural network task ;) 14:56:56 <bmbouter> they do 14:57:08 <dkliban> ok 14:57:09 <bmbouter> I was thinking the same thing 14:57:18 <ipanova> bmbouter: lmjachky expressed interesed to work on this and do some machine learning 14:57:31 <bmbouter> I'm +1 on him doing it 14:57:32 <dkliban> we should do it! 14:57:34 <ipanova> heh today ttereshc beats me in replies 14:57:35 <ggainey> +1 14:57:36 <dkliban> +1 14:57:45 <dkliban> i am excited about this 14:57:52 <dkliban> am i crazy? 14:57:52 <ipanova> bmbouter: if this can wait until his return we can keep this for him 14:58:04 <bmbouter> I can offer to mentor I have done several neural net supervised and unsupervised AI projects (for fun and profit) 14:58:05 <dkliban> it can def wait 14:58:15 <bmbouter> yes it can wait until he is available 14:58:15 <ipanova> sweet! 14:58:30 <ggainey> (if you've done em for profit, you're way ahead of most of the AI industry ;) ) 14:58:33 <bmbouter> did anyone else want to mentor him on it? 14:58:36 <ipanova> Lubos will be excited as well 14:59:22 <bmbouter> I actually created one last year as well to automatically search github issues for CVEs not yet identified based on language used in the ticket (I evne made a presentation on it w/ a formal analysis) 14:59:32 <fao89> next topic? 14:59:41 <bmbouter> almost, when does he return? 14:59:46 <bmbouter> I'm trying to figure out when it would start 15:00:08 <ipanova> 18 of june 15:00:11 <bmbouter> and also if anyone else wants to mentor so I don't jump into something someone else wanted to 15:00:15 <dkliban> gotta go ... see ya all 15:00:20 <ggainey> seeya! 15:00:24 <bmbouter> lata 15:00:59 <ipanova> bmbouter: sold, i think :D 15:01:02 <bmbouter> lmk if you also want to join, I can offer ot take an AI to setup a kickoff meeting w/ him around the 18th 15:01:27 <ipanova> bmbouter: that would be great 15:01:33 * bmbouter takes the AI 15:01:47 <bmbouter> I'll cc the brno folks since you all have worked closely w/ him too just for awareness 15:01:55 <ttereshc> AI about AI 15:02:03 <bmbouter> ja 15:02:22 <bmbouter> I'm ready to move on ty for the extra time for closure 15:02:52 <ipanova> bmbouter: да 15:03:59 <fao89> next topic: Can we remove the pulp-admin “category” from Redmine? 15:04:00 <ttereshc> +1 to remove pulp-admin category 15:04:02 <bmbouter> спасибо 15:04:16 <ipanova> +1 15:04:30 <bmbouter> I can take care of doing it just after this 15:04:39 <bmbouter> this is my weekly redmine data quality improvement campaign 15:04:46 <x9c4> Whats the use of that category (i any)? 15:05:02 <bmbouter> originally it was for issues filed relating to the pulp2 CLI 15:05:04 <ttereshc> just not needed 15:05:16 <bmbouter> but it never got used meaningfully in any process 15:05:19 <x9c4> +1 15:06:07 <bmbouter> with the consent of this group I will take the action after this 15:06:10 <bmbouter> next topic? 15:06:18 <fao89> topic: schema view html - https://github.com/pulp/pulpcore/pull/724#issuecomment-634275398 15:06:47 <fao89> browsable api could hide endpoints 15:07:00 <fao89> due html comment <! 15:07:05 <bmbouter> yeah so I'm concerned about us modiyfing the regex 15:07:17 <bmbouter> which is what prevents my +1 to merge (although ty for the work anyway fao89) 15:07:28 <bmbouter> https://github.com/pulp/pulpcore/pull/724/files#diff-9bc1ab0842dca699cb7abce66109298dR67 15:07:32 <bmbouter> it does fix the problem 15:07:43 <bmbouter> but I have a question 15:08:05 <bmbouter> why do we have html here at all? 15:08:06 <fao89> is about your question I wanted to talk: Should we be using HTML code in ours then? 15:08:23 <bmbouter> yeah https://github.com/pulp/pulpcore/pull/724#issuecomment-634275398 15:08:31 <fao89> exactly! I bring it to open floor for getting opinions 15:09:12 <bmbouter> fao89: can you link us to an example of where this html is embedeed in the pulp code as an example? 15:09:45 <fao89> it is possible to change the view to raw yaml or json, it won't be beautiful, but will show all endpoint 15:10:16 <bmbouter> that's a solution though, I want to understand why we have this problem and othe rprojects don't 15:10:36 <fao89> it is only possible to see it on the html source code: https://pulp.plan.io/issues/6828#note-3 15:11:45 <bmbouter> but it's coming from somewhere 15:11:51 <fao89> it comes from URLField 15:11:55 <fao89> https://github.com/pulp/pulpcore/pull/724#issuecomment-634267585 15:12:20 <fao89> DRF URLField > django URLValidator 15:13:15 <fao89> every view which uses a serializer with URLField will comment the html 15:13:29 <bmbouter> ok so that validator end up in the schema because it includes the html comment symbol in the schema spec itself 15:13:41 <fao89> yep 15:13:56 <bmbouter> ty for that I did not understand 15:14:25 <fao89> it is a regex for validating domain on the URLField 15:14:53 <bmbouter> yup 15:15:05 <x9c4> https://github.com/pulp/pulp_file/blob/master/pulp_file/app/viewsets.py#L57 <- Is that an example? 15:15:27 <bmbouter> x9c4: this is more like the example I was expecting 15:16:02 <bmbouter> if it's pulp code like ^ I think the resolution would be to remove those things 15:16:34 <x9c4> FileRemoteViewSet has a little bit more 15:16:51 <x9c4> Line 118 15:17:25 <bmbouter> yup 15:17:51 <fao89> it is not the problem 15:17:52 <bmbouter> I was just inspecting the html file you posted and I agree though that the urlfield brings in that character 15:17:59 <bmbouter> fao89: yup I agree 15:18:05 <bmbouter> but we would run into that next 15:18:18 <fao89> I'll try to find an example 15:18:35 <bmbouter> your html file shows it, the text here https://pulp.plan.io/attachments/529414 15:18:43 <bmbouter> grep that for <!- 15:19:37 <x9c4> flake8 also does not like html in the doc strings afaik. (if the corresponding packages are installed.) 15:19:49 <fao89> from what I see only pulp-ansible uses URLField: https://github.com/search?q=org%3Apulp+urlfield&type=Code 15:20:27 <fao89> these guys are the ones who are injecting html comment: https://github.com/pulp/pulp_ansible/blob/8e7d519f8a824a45d1d96e3af06428dae7cb397a/pulp_ansible/app/serializers.py#L272-L284 15:22:01 <bmbouter> fao89: ic you'r saying those lines are actually bringing that regex into the schema 15:22:05 <bmbouter> is that right? 15:22:15 <fao89> yep 15:22:33 <fao89> URLField from DRF is what is doing it 15:23:00 <fao89> the way it validates the url is with this regex which contains <! 15:23:31 <bmbouter> so if you inspect the html output of pulpcore, pulp_file, pulp_rpm, all plugins except pulp_ansible do you get any <!- chars? 15:24:35 <bmbouter> I expected you would? I'm not sure tho 15:25:12 <fao89> yep, only pulp_ansible has <! because it is the only one with URLField 15:25:59 <bmbouter> so that field comes from DRF or django? or does DRF use django? 15:26:15 <bmbouter> oh I see it's importing from DRF 15:26:19 <bmbouter> and that uses django underneath 15:26:31 <fao89> it comes from DRF, but it uses django validator 15:27:00 <bmbouter> is there an upstream DRF and/or django issue about this incompatability when those fields are used w/ openapi schema generation? 15:27:37 <fao89> I don't know, but it is specific for browsable API 15:27:57 <bmbouter> and that's a drf feature yes? 15:28:43 <fao89> I think it is a too specific problem, for seeing it you have to have a browsable API view of a generated openapi schema with view with urlfield 15:29:09 <bmbouter> I think the drf browseable aPI is not compatible with modles that use URLField 15:29:17 <fao89> yep 15:29:24 <bmbouter> and if we had a bug filed against drf like that then pulp could add to its plugin docs that we do not support that 15:29:39 <bmbouter> and then we could work w/ pulp_ansible to s/UrlField/CharField/ 15:30:18 <bmbouter> I don't think we can accept the patch that modifies the validator itself since that's the validator the code is desiring by using URLField to start with 15:30:32 <fao89> and have it on docs, for other plugins avoid to use URLField 15:30:38 <bmbouter> yes exactly 15:30:42 <bmbouter> in the pulpcore plugin writer docs 15:30:58 <bmbouter> fao89: I can help work out the details after open floor if that's helpful 15:31:12 <bmbouter> we're out of time 15:31:21 <fao89> I'll add some notes on the issue, and maybe file other issues 15:31:24 <fao89> yep 15:31:27 <fao89> #endmeeting 15:31:27 <fao89> !end 15:31:34 <fao89> thank you bmbouter 15:31:39 <fao89> #endmeeting 15:31:39 <fao89> !end 15:31:46 <bmbouter> fao89: ty really! great investigation and work 15:32:13 <fao89> pulpbot does not want to end the open floor 15:32:13 <pulpbot> fao89: Error: "does" is not a valid command. 15:32:18 <fao89> #endmeeting 15:32:18 <fao89> !end 15:32:31 <fao89> !friday 15:32:31 <pulpbot> ♪ It's Friday, Friday, gotta get down on Friday ♪ 14:20:06 <fao89> Open floor will start in 10 minutes (no issues for triage) - https://hackmd.io/SVCMjpwXTfOMqF2OeyyLRw 14:30:55 <fao89> !start 14:30:55 <fao89> #info fao89 has joined triage 14:30:55 <pulpbot> fao89: Error: Can't start another meeting, one is in progress. 14:30:56 <pulpbot> fao89: fao89 has joined triage 14:31:09 <fao89> #endmeeting 14:31:09 <fao89> !end