Add a widget to pip's docs to collect user feedback

[nlhkabu]

Following on from #8783

Collecting user feedback about pip's docs by email is a great first step, but it would be better if we could:

• collect this data in a way that is more user friendly
• send the data to a database, spreadsheet, etc. for easier analysis

For this, we should consider installing a feedback widget.

Preliminary mockups from @jdittrich:

Possible solutions:

1. Collaborate with read the docs (cc @ericholscher) to build a widget integrated into Sphinx
2. Build something custom for pip's docs
3. Find a third party widget that will fulfil our requirements

[jdittrich]

Here is a proof-of-concept if you will (in case you want to see how this could work), although a bit old-ish: https://github.com/jdittrich/visualBugs. If you click on "send feedback" in google’s webapps you get a similar function (though I do not know how they do it)

[humitos]

Hi! I'm Manuel and I'm working at Read the Docs. Eric shared this issue the other day with us and I wanted to chime in and leave a comment here to start tracking it and see where this idea ends.

Spoiler: I don't have a good solution.

• I know Read the Docs used to have a commenting system many years ago (I wasn't at the company) but for some reason that I don't know it was removed. I read some issues/emails where we started pointing people looking for this, to something like sphinxcontrib-disqus which seems kind of deprecated now.
  • allows you to make a public comment on each of the pages of the documentation
  • guest and authenticated comments (depends on how it's configured by the disqus account)
  • I personally used it in some sites and I haven't experienced SPAM
  • documentation with lot of readers should probably require moderation
• Months ago I knew https://web.hypothes.is/ (live example on Read the Docs' documentation) and I thought it was a good idea to explore for documentation (as personal private notes but also as a way to share a note/feedback publicly) and I built sphinx-hypothesis that helps you to easily add it in your documentation.
  • only authenticated users with a hypothesis account
  • allows comments and highlights at a word level: you can select a word, a sentence or a paragraph and leave a comment on that selection
  • public and private comments
  • each comment can be a thread with replies

None of them are probably a good fit (public comments is not the same as feedback), but they give a good context to start asking some question about how the widget you have in mind could work. Some questions that came to my mind so far,

1. Guest comments should be allowed? If yes, how are we going to deal with SPAM? If no, does it make sense to be logged in with your Read the Docs account to comment?
2. What is the extra value added by a widget that allows you to send feedback in a text-field over a link on each page (maybe a sticky-floating "Feedback" button) to mailto:[email protected]?subject=...&body=...?
3. If a user sends feedback using the widget, how do we contact the user back?
4. What type of feedback do we want to collect? Typos in the documentation? Sections that are not clear? Troubleshooting?
5. Who are going to read this feedback? Is this going to be a support channel?
6. Supposing that there is a backend already implemented in Read the Docs that allows the Javascript widget to send a POST request to RTD servers:
   • how this feedback should be presented to maintainers of the Read the Docs' project?
   • Is it enough to just show a list of each feedback reported where it can be opened and deleted?
   • how do you think Read the Docs could provide extra value with this data?

As you can see, I have more questions than answers and I'm not seeing a clear path forward yet. I personally wouldn't start building something from scratch without been tested the email link (you proposed in #8783) first to understand how people use it and grow from there or maybe trying to highly configure/adapt the hypothesis JS client (which is nice) to behave in the way we want (in case it's possible).

Days ago, @jd41 opened an issue on Read the Docs about something similar readthedocs/readthedocs.org#7401 that may be interesting to take a look as well.

I'm interested in keep thinking about this and see where we go. Let me know if I can be useful in any particular way.

[jd41]

I briefly wrote about Chatwoot on that issue, see readthedocs/readthedocs.org#7401 (comment)

[jdittrich]

Thanks for the questions and ideas. Some additional information from my side:

I would see the commenting/feedback/… mainly as way to get to know where users struggle to understand something. While I think it can be nice that people see each others comments and even comment on them, this would be a different topic for me.

Guest comments should be allowed? If yes, how are we going to deal with SPAM? If no, does it make sense to be logged in with your Read the Docs account to comment?

…as comments would not be public should be no spam problem. As for the accounts: Feedback is often most valuable from who have a harder time to deal with problems, so I think no account should be needed.

What is the extra value added by a widget that allows you to send feedback in a text-field over a link on each page (maybe a sticky-floating "Feedback" button) to mailto:[email protected]?subject=...&body=...?

• Using a widget, we could guide the users better through the infos we would
• They would be directly on the page and there is a lower barrier to writing
• They could post without giving us their mail address
• potentially, we could get screenshots and/or highlighted text sections

If a user sends feedback using the widget, how do we contact the user back?

This will not always be needed, but people could optionally provide their mail address.

What type of feedback do we want to collect? Typos in the documentation? Sections that are not clear? Troubleshooting?

What people struggle with – so "Sections that are not clear". It is nice if they report typos, but would not be my main concern.

Who are going to read this feedback? Is this going to be a support channel?

In the case of the project we are currently involved in the people who want to improve the communication would read the and use the feedback. Thus, there would be no support channel. If people provide a way to reach them, they might get an helpful answer, but it is mainly means to gather feedback, not a replacement for a forum, mailing-list or hotline.

Is it enough to just show a list of each feedback reported where it can be opened and deleted?

Yes. If there is additional data like screenshots or highlighted text, it should be there as well, but in any case a line in a table or a list entry should suffice.

how do you think Read the Docs could provide extra value with this data?

It could help authors to improve the documentation, particularly including newcomers to their projects. Most feedback channels like mailinglist, forum, mail to author have higher barriers and thus feedback tends to come from people who are more experienced and thus might not be very helpful to e. g. provide feedback on the tutorial or the "getting started" section.

[humitos]

Message out of the thread, probably, but today I found this https://mopinion.com/ while taking a look at this site that uses it https://docs.insolar.io/en/latest/ and I thought it could be something you may be interested to explore a little.

[pradyunsg]

I have a plan for tackling this in the future: we can use a Google form with pre-filled responses.

It will require the user to make two clicks to submit an empty response, but this is fairly feasible; especially if we include it as a 👍🏽 / 👎🏽 response; where clicking either takes you to the relevant feedback page!

[ericholscher]

@pradyunsg I'll be curious to hear how this works, as it would be great to recommend to folks in the RTD/Sphinx docs.

[dgarcia360]

Hi @nlhkabu!

I found this issue while searching for "feedback widget" + "Sphinx" on Google.

Find a third party widget that will fulfil our requirements

I'm building PushFeedback, a feedback widget that integrates well with Sphinx.

We have a free plan for open-source project in exchange of keeping the widget branding. If you're interested in exploring integrating PushFeedback, please sign up for an account and let me know to remove the limitations for your project.

Additionally, I'd be more than happy to assist with a pull request to integrate the Sphinx extension.

Looking forward to hearing from you!