Creation portal - Review flow UX issue

[kiranharidas187]

@krgauraw @pallakartheekreddy @Krishnaj20 Currently we are facing the following UX issues

1. Currently when the Reviewer Request changes a modal opens. The Reviewer will have to enter all the changes in one shot and then click on the Submit Review button. There is no option for the user to save the comment so that he can go through other content and update the review comment. As a real user, it's very difficult to note down all the questions in a paper and enter them in modal in one shot.
2. As a content creator, I have a button called View Comments where I can see all the changes requested by the Reviewer. But when I click on the same button on the question editor page the modal is coming as blank.

Suggested solution respectively

1. If there are APIs for saving and updating the comment, we can introduce a Save button on the Request change modal, so that the comments can be saved to the backend, and onload of the modal we can get the latest comment from the backend. Please let us know if these APIs are already available. If not available please let us know about the effort to implement it.
2. Can we show the Reviewer comment t fo the question details page also?

Please let us know your feedback on this

cc- @aks30

[Krishnaj20]

@kiranharidas187 - One of the large scale adopters of Sunbird Knowlg & inQuiry in the Indian Edtech domain has curated hundreds of thousands of assets with the existing workflow without any issues. So, it will be useful, if you can provide more details about this use case and share specifics about any experiments that were done to understand the need. Also, this will help you take a judicious call whether this capability is a 'must have' or a 'nice to have' one, and accordingly prioritize it.
In the proposed approach, request you to think about workflows for multiple scenarios. For ex., if multiple reviewers are reviewing the asset and have added their comments, and the asset is rejected by a specific reviewer then whose comments should be shown to the creator? If you can tag the PRD to this thread, I can go thru these scenarios and add my comments.
I'd strongly recommend to finalize the functional workflows first before starting the tech design and implementation discussions.

@pallakartheekreddy @krgauraw @alok-os

[pramodkvarma]

In general, commenting within the context is always easier for reviewers than one generic place all comments are written down. Difficult for reviewer to remember comments, make references to context of that comment, etc. Difference between Word/GoogleDoc in place comments (easy) vs sending comments via email after reading a PDF (painful).

[Prateek-slokam]

@Krishnaj20
The use case is mostly about bringing ease

1. To the reviewer for putting up their comments for the changes that they want to suggest.
   The observation and survey form are different than the till date used question sets as each question is trying to capture the opinions, requirements or present availability for the user and thus, each question and the entire form has a subjectivity with a derived context of the location, purpose and persona for which it is designed. The reviewer needs to follow onto this context while reviewing and further specifically need to give comments according to the matter of each question, question set (like sections or criteria) within the entire question set of observation and survey. To do this in a single go would require the reviewers to lay down their list of comments separately first and mark them in platform with the matter to which it points.
   Eg. We have seen this reviewing behaviour of contextual comments to the questions and question sets in the observation and survey in works in the current design of template based creation. The comments if are specific to question or say a particular score or hint/tip in the question, would not make sense without the comment pointing to that matter.
2. To the creator to easily and clearly understand the suggestions that have been recommended with the context of substance the change have been recommended in.
   As within the above example the creator too needs to have an idea of the particular point of reference of a suggestion. As each question has a different information to capture with no 'Book designed format' for the questions and question sets, it becomes tricky for creator otherwise to understand via a single comment all the changes that are to be made.
3. In saving of the reviewer's work and navigation by content creator through the form
   The length of the observation forms are huge, especially like in the case of an observation with rubric where a general form might end up having about 80-100 questions. In such a case to go through the entire forms in a single go would be difficult and so for reviewer to be able to put up comments at each individual page of creator and rather than in a single go would make a huge difference in the experience and efficiency. With the same idea the creator would also feel lost with a single comment box for so many questions.

The ultimate Plan for a review flow, is in line with this above mentioned use case and I think to what @pramodkvarma sir is also suggesting. You can look up the Jira ticket - [28057]. The UI is also attached in the same.
This is the PRD for your reference.

However, we have upon discussions with the engineering team also realised that this is a larger effort which will require longer time and so as the not delay the release because of this single flow we are looking at it into two phases now. The intermediatory solution (or phase 1) is what Kiran mentioned in the discussion where we are looking to at least make the reviewer part easier by providing them option of continuously being able to add and save comments and the creator having the access to the comment dialog box at all places. In phase 2 (which would be after the first release) we are targeting for achieving the design mentioned in the PRD to bring ease and contextuality to the entire review flow.

[Krishnaj20]

@Prateek-slokam - thanks for providing the context. This is useful. I'd recommend you to change the scope of the story as per the phase1 implementation scope. Phase 2 can be added to the backlog. Apologies, if this is already taken care of.
@kiranharidas187 - Since this is a new feature, I'm assuming the tech design was already run with @vrayulu @maheshkumargangula @pallakartheekreddy ? If not, I'd recommend you to share the tech design document for feedback/clarifications.

[Prateek-slokam]

@Krishnaj20 Sorry for the delay in response, The stories for the first phase review flow enhancements are put up - https://project-sunbird.atlassian.net/browse/SB-29586 and https://project-sunbird.atlassian.net/browse/SB-29587. @kiranharidas187 Can you please take it forward as Krishna is suggesting.

[Krishnaj20]

@Prateek-slokam @aks30 - In the last meeting b/w Ekstep & SL teams, @maheshkumargangula has volunteered to help SL team come up with the initial design and the implementation design for this contribution. I hope you folks have made progress on that front.
In the spirit of Open source contributions, I request you to follow the below process:
a) Once, you with the help @maheshkumargangula, have the initial design ready, pls share it in this thread for the review of @pallakartheekreddy @vrayulu & the community. We'd recommend you to proceed with the implementation after this review.
b) Are the PRD & JIRA stories shared in this thread ready for review? Pls let me know - I'll review and add my comments. Also I'd request you to include wireframes for all the scenarios. The existing design in the story has only one screen.

Pls do reach out to us if you have any queries.

@alok-os @madhucr

[kiranharidas187]

MOM of call held on 12th May

• As the suggestion made above, we will be implementing a save button in the Request change modal. Story ref - https://project-sunbird.atlassian.net/browse/SB-29587
• Complete comments flow will be implemented in the next phase
• Mahesh will talk to DC and let us know if this approach is fine by 13th May EOD

cc- @Krishnaj20 @pallakartheekreddy @aks30 @maheshkumargangula

[maheshkumargangula]

@kiranharidas187 @aks30 - Let us have a PRD explaining the capability that we are planning to enable here and have a high-level design to support the requirement. After that, we will be able to decide on whether the shortcut implementation whatever is proposed by you in our previous call is valid or not.

Also, along with the shortcuts, we should plan and define the required changes to implement the final version of this feature and any changes and data migration involved to deploy the final version of this feature.
Please define these and schedule a call with DC and other required people for review.
Cc: @Krishnaj20 @pallakartheekreddy @vrayulu

[aks30]

@maheshkumargangula During the call on 28th April, we in the interest of time agreed on the Phase 1 approach and decided to subsequently come up with the design and implementation as per the PRD @Prateek-slokam has shared as that entails a whole new feature development from scratch and would require much time to be spent on coming up with the design of the same. Can we for now go ahead without that being a blocker?

cc - @alok-os, @vijiurs, @Krishnaj20

[vijiurs]

@maheshkumargangula , The Phase 1 enhancement enables a reviewer to keep saving the review feedback in draft mode as he/she reviews the question set. This feature would be required in the final solution too. Given this, I think we should go ahead with the Phase 1 design and implementation and not wait for the design of the final solution.
@aks30 , Please share the design and implementation details with @maheshkumargangula and team.
cc- @alok-os, @Krishnaj20 , @madhucr.

[kiranharidas187]

@maheshkumargangula @pallakartheekreddy We have created a confluence doc for the Phase 1 approach. Please let us know if this approach is fine.

cc- @vijiurs

[pramodkvarma]

Basically you should create a topic dynamically for every unique content item with IDs that are derived. If discussion APIs do not allow creation of topics with custom ID/name (I think it does). Btw topic name also need not be human readable since these are internal topics that only shows up in embedded form within content. So either ID or name can be constructed in a unique fashion using content ID.

If none of the above works, I suppose you will have to store topic ID as part of content metadata (not all discussion IDs, just parent topic ID).

@vrayulu can you please help?

[vrayulu]

@joffinjoy @aks30 - here is an option for using discussion forums to manage review comments on assets (content, collection, question, questionSet, etc):

• create a discussions category "Reviewer Comments"
• create one forum per asset dynamically when the asset is created (or submitted for review for the first time). You can provide the object type (e.g.: QuestionSet) and object identifier (e.g.: question set identifier) as input to the createForum API. And there is an API fetch the forum created for a specific object.
• within the forum, create multiple topics - one per one version of the object (@joffinjoy - each time an object is published, the version number of the object is updated in KP. we need to use this version number to differentiate the review comments given in each cycle of review-publish). And use the tags capability to add the version number as a tag on the topic.
• within each topic, there could be multiple posts for comments by the reviewer and replies by the creator.

Hope this helps... however, please validate this design for multiple scenarios and see if it is able to fulfil all of them.

[pramodkvarma]

Yes. Thanks @vrayulu

[joffinjoy]

@vrayulu

create one forum per asset dynamically when the asset is created (or submitted for review for the first time)

We are confused about the concept of forums here. I mean, from the documentation it seems like the exact hierarchy is Category -> Sub-Category -> Topics -> Posts. This was confirmed by @VenkatKadiveti in the discussion post I opened recently. Question #1 in https://github.com/orgs/Sunbird-Lern/discussions/27. Are you using forums and Sub-Categories interchangeably?

Going through Forum APIs listed at: https://lern.sunbird.org/learn/product-and-developer-guide/discussion-forum/api-documentation/forum-apis isn't helping me clear this confusion for myself. Category (or sub-category) feels synonymous to forums.

You can provide the object type (e.g.: QuestionSet) and object identifier (e.g.: question set identifier) as input to the createForum API.

Does input here mean the tags or do you want it to be part of the forum id like @pramodkvarma suggested for posts (Using questionSet Id as the forum name)? My query #3 in the linked post was regarding fetching posts/topics by their tags. Venkat shared the curl that will do that. Does the same apply to forums as well? Because we couldn't find an API to create a forum and pass in object ids in the 'API listing page'. Can you please clarify this?

within the forum, create multiple topics - one per one version of the object

Okey! Understood exactly what versioning you meant. But in that case, a second version happens when a questionSet that is already published undergoes a modification, leading to the creation of an image node(at this point, the original questionset is untouched). And this image node is the one that is getting reviewed. But since we are not going to show previous comments from earlier review-publish cycle, is this necessary?

But instead if we are considering versioning between creator and reviewer, Like creator sends a questionset for review and reviewer reviews it and sends it back, this becomes version one. Then the creator modifies the QS according to the review comments and sends it again for review, this time it becomes version 2 of reviewer comments. This isn't any way related to publish process at all. Basically each creation-review or review-modify-review becomes a version of its own. Wouldn't this be a better approach as far as versioning is concerned?

cc: @VenkatKadiveti

[vrayulu]

@aks30 @joffinjoy - if you need more help in doing this design, lets get into a call and you will be able to understand it better. Please write to me on my email and we can identify a time.

P.S.: the summary of the call should be added back to this thread post the call.

[aks30]

We (@vrayulu , @Prateek-slokam , @vijiurs , @kiranharidas187 ) have had a meeting and confirmed taking this requirement in 2 phases.

Phase 1 - Allow the reviewer to save comments as a draft before he clicks on the final publish/reject.
Phase 2 - Bring in discussion forum to enable Google Docs kind of comment for questionSet and collections.

We have put out the design document as per our discussion for Phase 1 here - https://project-sunbird.atlassian.net/l/cp/my7WTrV7

@vrayulu request you to check and share if any comments or changes on the same.

[kameshbhr]

I have a few questions on the design that I have added in the confluence page.

[aks30]

Have added responses for the same

[kameshbhr]

I have further added my comments. Would like @vrayulu to go through it. If required we can plan a quick call to close on it.