14:30:19 <fao89> #startmeeting Pulp Triage 2020-05-29
14:30:19 <fao89> #info fao89 has joined triage
14:30:20 <pulpbot> Meeting started Fri May 29 14:30:19 2020 UTC.  The chair is fao89. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:30:20 <pulpbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:30:20 <pulpbot> The meeting name has been set to 'pulp_triage_2020-05-29'
14:30:20 <pulpbot> fao89: fao89 has joined triage
14:30:44 <dkliban> #info dkliban has joined triage
14:30:44 <dkliban> !here
14:30:44 <pulpbot> dkliban: dkliban has joined triage
14:30:46 <ppicka> #info ppicka has joined triage
14:30:46 <ppicka> !here
14:30:46 <pulpbot> ppicka: ppicka has joined triage
14:30:53 <fao89> !next
14:30:54 <pulpbot> fao89: 6 issues left to triage: 6851, 6837, 6835, 6834, 6828, 6714
14:30:54 <fao89> #topic https://pulp.plan.io/issues/6851
14:30:55 <pulpbot> RM 6851 - hyu - NEW - Regression in regenerate applicability
14:30:56 <pulpbot> https://pulp.plan.io/issues/6851
14:31:24 <fao89> #idea Proposed for #6851: move to rpm
14:31:24 <fao89> !propose other move to rpm
14:31:24 <pulpbot> fao89: Proposed for #6851: move to rpm
14:31:24 <ttereshc> #info ttereshc has joined triage
14:31:24 <ttereshc> !here
14:31:25 <pulpbot> ttereshc: ttereshc has joined triage
14:31:28 <x9c4> !here
14:31:28 <x9c4> #info x9c4 has joined triage
14:31:29 <pulpbot> x9c4: x9c4 has joined triage
14:31:51 <dkliban> +1
14:31:57 <fao89> #agreed move to rpm
14:31:57 <fao89> !accept
14:31:57 <pulpbot> fao89: Current proposal accepted: move to rpm
14:32:03 <ttereshc> +1
14:32:43 <fao89> !next
14:32:44 <pulpbot> fao89: 5 issues left to triage: 6837, 6835, 6834, 6828, 6714
14:32:44 <fao89> #topic https://pulp.plan.io/issues/6837
14:32:45 <pulpbot> RM 6837 - ekohl - NEW - Various links in installation instructions show a 404 page
14:32:46 <pulpbot> https://pulp.plan.io/issues/6837
14:32:55 <dkliban> #idea Proposed for #6837: accept and add to sprint
14:32:55 <dkliban> !propose other accept and add to sprint
14:32:55 <pulpbot> dkliban: Proposed for #6837: accept and add to sprint
14:32:56 <ggainey> #info ggainey has joined triage
14:32:56 <ggainey> !here
14:32:57 <pulpbot> ggainey: ggainey has joined triage
14:32:57 <ipanova> #info ipanova has joined triage
14:32:57 <ipanova> !here
14:32:58 <pulpbot> ipanova: ipanova has joined triage
14:33:12 <ipanova> +1
14:33:17 <fao89> #agreed accept and add to sprint
14:33:17 <fao89> !accept
14:33:17 <pulpbot> fao89: Current proposal accepted: accept and add to sprint
14:33:18 <pulpbot> fao89: 4 issues left to triage: 6835, 6834, 6828, 6714
14:33:18 <fao89> #topic https://pulp.plan.io/issues/6835
14:33:19 <pulpbot> RM 6835 - dkliban@redhat.com - NEW - .travis/release.py in pulpcore creates the wrong version for the final .dev version commit
14:33:20 <pulpbot> https://pulp.plan.io/issues/6835
14:33:44 <dkliban> i think we need to add more information to the wiki page
14:33:50 <dkliban> but this was mostly user error on my part
14:34:14 <fao89> the problem for me is: where is the right place for documenting it?
14:34:38 <ipanova> i think both wiki and --help helps
14:34:41 <x9c4> "release.py --help" ?
14:34:43 <fao89> I'll add a note to put it on the wiki then
14:35:12 <bmbouter> #info bmbouter has joined triage
14:35:12 <bmbouter> !here
14:35:12 <pulpbot> bmbouter: bmbouter has joined triage
14:35:16 <fao89> #idea Proposed for #6835: accept and add to sprint
14:35:16 <fao89> !propose other accept and add to sprint
14:35:18 <pulpbot> fao89: Proposed for #6835: accept and add to sprint
14:35:20 <dkliban> +1
14:35:22 <x9c4> +1
14:35:22 <fao89> #agreed accept and add to sprint
14:35:22 <fao89> !accept
14:35:23 <pulpbot> fao89: Current proposal accepted: accept and add to sprint
14:35:23 <fao89> #topic https://pulp.plan.io/issues/6834
14:35:24 <pulpbot> fao89: 3 issues left to triage: 6834, 6828, 6714
14:35:25 <pulpbot> RM 6834 - mdellweg - NEW - A pulpcore / plugin pr should be able to point to an experimental version of the pulp-ci base image
14:35:26 <pulpbot> https://pulp.plan.io/issues/6834
14:35:43 <ttereshc> task?
14:35:50 <bmbouter> +1
14:35:54 <dkliban> yeah ... but we should add it to the sprint after that
14:36:15 <dkliban> #idea Proposed for #6834: convert to task
14:36:15 <dkliban> !propose other convert to task
14:36:15 <pulpbot> dkliban: Proposed for #6834: convert to task
14:36:15 <fao89> !propose other convert to a task and add to sprint
14:36:20 <dkliban> haha
14:36:20 <x9c4> +1
14:36:23 <fao89> #idea Proposed for #6834: convert to a task and add to sprint
14:36:23 <fao89> !propose other convert to a task and add to sprint
14:36:23 <pulpbot> fao89: Proposed for #6834: convert to a task and add to sprint
14:36:27 <fao89> #agreed convert to a task and add to sprint
14:36:27 <fao89> !accept
14:36:27 <pulpbot> fao89: Current proposal accepted: convert to a task and add to sprint
14:36:28 <fao89> #topic https://pulp.plan.io/issues/6828
14:36:28 <pulpbot> fao89: 2 issues left to triage: 6828, 6714
14:36:28 <dkliban> +1
14:36:30 <pulpbot> RM 6828 - fao89 - POST - browsable API missing endpoints on /pulp/api/v3/
14:36:31 <pulpbot> https://pulp.plan.io/issues/6828
14:36:44 <dkliban> #idea Proposed for #6828: accept and add to sprint
14:36:44 <dkliban> !propose other accept and add to sprint
14:36:46 <ttereshc> +1
14:37:00 <fao89> it is good to open floor, due the discussion on the PR
14:37:10 <ipanova> +1
14:37:19 <fao89> #agreed accept and add to sprint
14:37:19 <fao89> !accept
14:37:19 <pulpbot> fao89: Current proposal accepted: accept and add to sprint
14:37:43 <fao89> pulpbot is weird today
14:37:43 <pulpbot> fao89: Error: "is" is not a valid command.
14:37:49 <fao89> !next
14:37:50 <fao89> #topic https://pulp.plan.io/issues/6714
14:37:50 <pulpbot> fao89: 1 issues left to triage: 6714
14:37:51 <pulpbot> RM 6714 - alikins - NEW - drf builtin manage.py 'generateschema' command fails on pulp base viewsets
14:37:52 <pulpbot> https://pulp.plan.io/issues/6714
14:38:37 <dkliban> so think we should convert this to a story or task
14:38:45 <dkliban> the request is to generate openapi v3 schema
14:38:57 <dkliban> we currently only support generating openapi v2 schema
14:39:06 <ttereshc> and to go away from drf_yasg
14:39:08 <ttereshc> ?
14:39:10 <dkliban> yes
14:39:20 <ttereshc> +1 to a story
14:39:23 <dkliban> drf_yasg is not being actively developed anymore
14:39:41 <dkliban> i have noticed that they have not done a release in many months and PRs are not getting merged
14:39:44 <fao89> drf is not mature for it yet
14:40:02 <dkliban> fao89: i agree, but we shuold keep this on our road map
14:40:07 <fao89> I tried to use it, and I had to do many tweaks
14:40:07 <dkliban> and maybe use a different project
14:40:10 <bmbouter> it probably should be an epic
14:40:20 <bmbouter> it will likely be many tasks
14:40:25 <dkliban> bmbouter: +1
14:40:30 <ttereshc> #idea Proposed for #6714: convert to an epic story
14:40:30 <ttereshc> !propose other convert to an epic story
14:40:31 <pulpbot> ttereshc: Proposed for #6714: convert to an epic story
14:40:39 <fao89> #agreed convert to an epic story
14:40:39 <fao89> !accept
14:40:39 <pulpbot> fao89: Current proposal accepted: convert to an epic story
14:40:41 <pulpbot> fao89: No issues to triage.
14:40:53 <fao89> Open floor!
14:40:58 <fao89> https://hackmd.io/SVCMjpwXTfOMqF2OeyyLRw
14:41:07 <fao89> redmine spam!
14:41:36 <bmbouter> aw man I meant to look into this earlier
14:41:36 <fao89> https://pulp.plan.io/issues/5878
14:42:12 <ttereshc> I'm not sure what we can do without the api for removing notes
14:42:27 <ttereshc> it's the majority of spam, imo
14:42:39 <bmbouter> the option discussed last time as a temporary alternative was to require account confirmation
14:42:53 <ttereshc> ah I see
14:43:01 <dkliban> yeah ... i think we should require that
14:43:12 <ipanova> i tend to agree
14:43:25 <bmbouter> https://www.redmine.org/projects/redmine/wiki/Features#User-self-registration-support
14:44:07 <ggainey> account-confirmation is fine, but nowadays is unlikely to have a significant impact
14:44:22 <bmbouter> right now it's set to accoutn activation by email
14:44:23 <ggainey> the SEO-optimizer-folks have scripts to monitor email addresses and auto-confirm
14:44:36 <bmbouter> the issue is that we don't want to auto-confirm
14:44:37 <ggainey> ah, kk
14:44:41 <bmbouter> these are all humans registering
14:44:59 <ggainey> yah
14:45:07 <bmbouter> so the only option is administrators approving accounts
14:45:30 <dkliban> if amins have to verify accounts, we need some kind of comment from the user stating why they are opening the account
14:45:39 <bmbouter> I agree
14:45:39 <ggainey> aye
14:45:52 <bmbouter> and redmine's process won't change so we would have to have email correspondance
14:46:33 <bmbouter> the other option is to not use the API but have the automation drive a firefox browser to remove comments
14:46:37 <bmbouter> not a great option obviously
14:47:14 <dkliban> i've done things like that before with selinium
14:47:29 <dkliban> you can do it without an actual GUI browser
14:47:40 <dkliban> that's an option for selinium
14:47:45 <bmbouter> yup
14:47:50 <bmbouter> I also believe it's diable
14:47:55 <bmbouter> doable
14:48:45 <bmbouter> from a project perspective the cost of developer time (both in cost and opportunity cost) I think makes manual correspondance not really viable
14:48:54 <bmbouter> the spammer cost is extremely low
14:48:58 <bmbouter> our cost extremely high
14:49:33 <ttereshc> you think they will increase the number of registrations?
14:49:59 <bmbouter> even the current rate is several per day
14:50:41 <bmbouter> as in if all we had to handle were say 3-5 users per day it's a non-negligable economic impact on pulp
14:50:53 <bmbouter> and 0 economic impact on spammers
14:51:13 <dkliban> true dat
14:51:22 <ttereshc> I understand that
14:51:24 <bmbouter> ttereshc: do you know if we can lock their accounts ia the redmine api?
14:51:50 <ttereshc> I'm just not sure when we will be able to do the selenium way
14:52:03 <bmbouter> let's say we don't do the selenium way as a thought experiment
14:52:06 <ttereshc> bmbouter, they said yes, I couldn;t find it but I didn't try hard.
14:52:48 <ttereshc> bmbouter, https://www.redmine.org/issues/22434  seems like no
14:53:23 <ttereshc> but you can delete one :/ not that useful but an option
14:53:34 <bmbouter> I wonder if it deletes their content too
14:53:44 <bmbouter> oh I know it doesn't actually
14:53:47 <dkliban> bmbouter: no... marks it as by Anonymous
14:53:50 <bmbouter> it makes their content 'Anonymoous'
14:53:52 <bmbouter> yeah
14:53:53 <ttereshc> I agree
14:54:25 <bmbouter> I think selenium is our best option
14:54:43 <bmbouter> and to raise urgency again w/ plan.io
14:54:44 <dkliban> i agree
14:54:59 <bmbouter> or put a mechanical turk process in place
14:55:03 <dkliban> do we have an algorithm for identifying spam comments?
14:55:24 <dkliban> (btw, i gotta go in 5 mins)
14:55:47 <bmbouter> we could train a bayesian classifier easily, we could also flag comments with non-trusted outbound links (they always contain links)
14:56:18 <dkliban> bmbouter: what do you mean by non-trusted?
14:56:31 <bmbouter> we could have a whitelist over time
14:56:35 <dkliban> gotcha
14:56:52 <dkliban> do these users ever create new issues?
14:56:56 <ttereshc> lmjachky was looking for a neural network task ;)
14:56:56 <bmbouter> they do
14:57:08 <dkliban> ok
14:57:09 <bmbouter> I was thinking the same thing
14:57:18 <ipanova> bmbouter: lmjachky expressed interesed to work on this and do some machine learning
14:57:31 <bmbouter> I'm +1 on him doing it
14:57:32 <dkliban> we should do it!
14:57:34 <ipanova> heh today ttereshc beats me in replies
14:57:35 <ggainey> +1
14:57:36 <dkliban> +1
14:57:45 <dkliban> i am excited about this
14:57:52 <dkliban> am i crazy?
14:57:52 <ipanova> bmbouter: if this can wait until his return we can keep this for him
14:58:04 <bmbouter> I can offer to mentor I have done several neural net supervised and unsupervised AI projects (for fun and profit)
14:58:05 <dkliban> it can def wait
14:58:15 <bmbouter> yes it can wait until he is available
14:58:15 <ipanova> sweet!
14:58:30 <ggainey> (if you've done em for profit, you're way ahead of most of the AI industry ;) )
14:58:33 <bmbouter> did anyone else want to mentor him on it?
14:58:36 <ipanova> Lubos will be excited as well
14:59:22 <bmbouter> I actually created one last year as well to automatically search github issues for CVEs not yet identified based on language used in the ticket (I evne made a presentation on it w/ a formal analysis)
14:59:32 <fao89> next topic?
14:59:41 <bmbouter> almost, when does he return?
14:59:46 <bmbouter> I'm trying to figure out when it would start
15:00:08 <ipanova> 18 of june
15:00:11 <bmbouter> and also if anyone else wants to mentor so I don't jump into something someone else wanted to
15:00:15 <dkliban> gotta go ... see ya all
15:00:20 <ggainey> seeya!
15:00:24 <bmbouter> lata
15:00:59 <ipanova> bmbouter: sold, i think :D
15:01:02 <bmbouter> lmk if you also want to join, I can offer ot take an AI to setup a kickoff meeting w/ him around the 18th
15:01:27 <ipanova> bmbouter: that would be great
15:01:33 * bmbouter takes the AI
15:01:47 <bmbouter> I'll cc the brno folks since you all have worked closely w/ him too just for awareness
15:01:55 <ttereshc> AI about AI
15:02:03 <bmbouter> ja
15:02:22 <bmbouter> I'm ready to move on ty for the extra time for closure
15:02:52 <ipanova> bmbouter: да
15:03:59 <fao89> next topic: Can we remove the pulp-admin “category” from Redmine?
15:04:00 <ttereshc> +1 to remove pulp-admin category
15:04:02 <bmbouter> спасибо
15:04:16 <ipanova> +1
15:04:30 <bmbouter> I can take care of doing it just after this
15:04:39 <bmbouter> this is my weekly redmine data quality improvement campaign
15:04:46 <x9c4> Whats the use of that category (i any)?
15:05:02 <bmbouter> originally it was for issues filed relating to the pulp2 CLI
15:05:04 <ttereshc> just not needed
15:05:16 <bmbouter> but it never got used meaningfully in any process
15:05:19 <x9c4> +1
15:06:07 <bmbouter> with the consent of this group I will take the action after this
15:06:10 <bmbouter> next topic?
15:06:18 <fao89> topic: schema view html - https://github.com/pulp/pulpcore/pull/724#issuecomment-634275398
15:06:47 <fao89> browsable api could hide endpoints
15:07:00 <fao89> due html comment <!
15:07:05 <bmbouter> yeah so I'm concerned about us modiyfing the regex
15:07:17 <bmbouter> which is what prevents my +1 to merge (although ty for the work anyway fao89)
15:07:28 <bmbouter> https://github.com/pulp/pulpcore/pull/724/files#diff-9bc1ab0842dca699cb7abce66109298dR67
15:07:32 <bmbouter> it does fix the problem
15:07:43 <bmbouter> but I have a question
15:08:05 <bmbouter> why do we have html here at all?
15:08:06 <fao89> is about your question I wanted to talk: Should we be using HTML code in ours then?
15:08:23 <bmbouter> yeah https://github.com/pulp/pulpcore/pull/724#issuecomment-634275398
15:08:31 <fao89> exactly! I bring it to open floor for getting opinions
15:09:12 <bmbouter> fao89: can you link us to an example of where this html is embedeed in the pulp code as an example?
15:09:45 <fao89> it is possible to change the view to raw yaml or json, it won't be beautiful, but will show all endpoint
15:10:16 <bmbouter> that's a solution though, I want to understand why we have this problem and othe rprojects don't
15:10:36 <fao89> it is only possible to see it on the html source code: https://pulp.plan.io/issues/6828#note-3
15:11:45 <bmbouter> but it's coming from somewhere
15:11:51 <fao89> it comes from URLField
15:11:55 <fao89> https://github.com/pulp/pulpcore/pull/724#issuecomment-634267585
15:12:20 <fao89> DRF URLField > django URLValidator
15:13:15 <fao89> every view which uses a serializer with URLField will comment the html
15:13:29 <bmbouter> ok so that validator end up in the schema because it includes the html comment symbol in the schema spec itself
15:13:41 <fao89> yep
15:13:56 <bmbouter> ty for that I did not understand
15:14:25 <fao89> it is a regex for validating domain on the URLField
15:14:53 <bmbouter> yup
15:15:05 <x9c4> https://github.com/pulp/pulp_file/blob/master/pulp_file/app/viewsets.py#L57 <- Is that an example?
15:15:27 <bmbouter> x9c4: this is more like the example I was expecting
15:16:02 <bmbouter> if it's pulp code like ^ I think the resolution would be to remove those things
15:16:34 <x9c4> FileRemoteViewSet has a little bit more
15:16:51 <x9c4> Line 118
15:17:25 <bmbouter> yup
15:17:51 <fao89> it is not the problem
15:17:52 <bmbouter> I was just inspecting the html file you posted and I agree though that the urlfield brings in that character
15:17:59 <bmbouter> fao89: yup I agree
15:18:05 <bmbouter> but we would run into that next
15:18:18 <fao89> I'll try to find an example
15:18:35 <bmbouter> your html file shows it, the text here https://pulp.plan.io/attachments/529414
15:18:43 <bmbouter> grep that for <!-
15:19:37 <x9c4> flake8 also does not like html in the doc strings afaik. (if the corresponding packages are installed.)
15:19:49 <fao89> from what I see only pulp-ansible uses URLField: https://github.com/search?q=org%3Apulp+urlfield&type=Code
15:20:27 <fao89> these guys are the ones who are injecting html comment: https://github.com/pulp/pulp_ansible/blob/8e7d519f8a824a45d1d96e3af06428dae7cb397a/pulp_ansible/app/serializers.py#L272-L284
15:22:01 <bmbouter> fao89: ic you'r saying those lines are actually bringing that regex into the schema
15:22:05 <bmbouter> is that right?
15:22:15 <fao89> yep
15:22:33 <fao89> URLField from DRF is what is doing it
15:23:00 <fao89> the way it validates the url is with this regex which contains <!
15:23:31 <bmbouter> so if you inspect the html output of pulpcore, pulp_file, pulp_rpm, all plugins except pulp_ansible do you get any <!- chars?
15:24:35 <bmbouter> I expected you would? I'm not sure tho
15:25:12 <fao89> yep, only pulp_ansible has  <! because it is the only one with URLField
15:25:59 <bmbouter> so that field comes from DRF or django? or does DRF use django?
15:26:15 <bmbouter> oh I see it's importing from DRF
15:26:19 <bmbouter> and that uses django underneath
15:26:31 <fao89> it comes from DRF, but it uses django validator
15:27:00 <bmbouter> is there an upstream DRF and/or django issue about this incompatability when those fields are used w/ openapi schema generation?
15:27:37 <fao89> I don't know, but it is specific for browsable API
15:27:57 <bmbouter> and that's a drf feature yes?
15:28:43 <fao89> I think it is a too specific problem, for seeing it you have to have a browsable API view of a generated openapi schema with view with urlfield
15:29:09 <bmbouter> I think the drf browseable aPI is not compatible with modles that use URLField
15:29:17 <fao89> yep
15:29:24 <bmbouter> and if we had a bug filed against drf like that then pulp could add to its plugin docs that we do not support that
15:29:39 <bmbouter> and then we could work w/ pulp_ansible to s/UrlField/CharField/
15:30:18 <bmbouter> I don't think we can accept the patch that modifies the validator itself since that's the validator the code is desiring by using URLField to start with
15:30:32 <fao89> and have it on docs, for other plugins avoid to use URLField
15:30:38 <bmbouter> yes exactly
15:30:42 <bmbouter> in the pulpcore plugin writer docs
15:30:58 <bmbouter> fao89: I can help work out the details after open floor if that's helpful
15:31:12 <bmbouter> we're out of time
15:31:21 <fao89> I'll add some notes on the issue, and maybe file other issues
15:31:24 <fao89> yep
15:31:27 <fao89> #endmeeting
15:31:27 <fao89> !end
15:31:34 <fao89> thank you bmbouter
15:31:39 <fao89> #endmeeting
15:31:39 <fao89> !end
15:31:46 <bmbouter> fao89: ty really! great investigation and work
15:32:13 <fao89> pulpbot does not want to end the open floor
15:32:13 <pulpbot> fao89: Error: "does" is not a valid command.
15:32:18 <fao89> #endmeeting
15:32:18 <fao89> !end
15:32:31 <fao89> !friday
15:32:31 <pulpbot> ♪ It's Friday, Friday, gotta get down on Friday ♪
14:20:06 <fao89> Open floor will start in 10 minutes (no issues for triage) - https://hackmd.io/SVCMjpwXTfOMqF2OeyyLRw
14:30:55 <fao89> !start
14:30:55 <fao89> #info fao89 has joined triage
14:30:55 <pulpbot> fao89: Error: Can't start another meeting, one is in progress.
14:30:56 <pulpbot> fao89: fao89 has joined triage
14:31:09 <fao89> #endmeeting
14:31:09 <fao89> !end