Last modified: 2012-11-28 14:31:20 UTC
Currently AFT5 uses 2*2 different ways to ask for improvement. It is fairly likely that a respondent doesn't notice both questions in either path, as one of them is in very large font, and another is in small gray text. For the "Yes, I found what I'm looking for" path, the big headline reads "Great. Would you like to add a comment?" The smaller gray text in the textarea says: "How could this article be improved?" For the "No, I didn't find what I'm looking for" path, the big headline reads "Sorry about that. Any suggestions for improvement?" The smaller gray text in the textarea reads: "What were you looking for?" So respondents are answering four different questions: 1) "Would you like to add a comment?" 2) "How could this article be improved?" 3) "Any suggestions for improvement?" 4) "What were you looking for?" Each of these surely has merits and drawbacks, but irrespective of that, the fact that we have four different questions, and no way for feedback reviewers to infer which question the feedback-giver answered, is seriously problematic. To give an example: A feedback-giver writes "pulse valve" in the feedback form for the article "Air suction valve". If this response is in answer to question 4), it might indicate that the reader was looking for information about pulse valves and didn't find it. If it's in response to question 3), it might indicate that the reader has specific issues with that part of the article. These four different questions may also make respondents seem less literate or willing to provide clear feedback than they actually are. As a respondent I might think I gave a perfectly clear answer ("I was looking for X, so I wrote X"), but the feedback-reviewer has no way to parse the response. Likewise, the fact that the "Yes" path has such a prominent encouragement of "comments" (as opposed to suggestions for improvement) may bias the responses in favor of opinions and useless feedback. In short, we should IMO ask a single question irrespective of the yes/no response. For example: "Did you find what you were looking for? [Yes] [No]" -> Yes: "Great. Do you have any suggestions that could make the article better?" Gray text: "Write your suggestion here, e.g.: 'The article would benefit from an illustration.'" No: "Sorry about that. Do you have any suggestions that could make the article better?" Gray text: "Write your suggestion here, e.g.: 'The article would benefit from an illustration.'" Asking a single consistent question with consistent instructions would help feedback-receivers parse the feedback, could improve feedback quality, and would make testing of alternatives easier.
Thanks, Erik, you make a very good point, which is well thought-out and much appreciated. I agree with your overall proposal, though I recommend that we shorten your proposed language, to make it easier to read and to translate. Here's my recommended copy: "Did you find what you were looking for? [Yes] [No]" -> Yes: "Great. Any suggestion for improvement?" Gray text: "Write your suggestion here (e.g.: 'This article needs a picture).'" No: "Sorry about that. Any suggestion for improvement?" Gray text: "Write your suggestion here (e.g.: 'This article needs a picture).'"
The shorter version works for me as well, thanks! We may still have some issues with the context of the question, which is a yes/no question, not being apparent to a person looking at the feedback stream ("Why does this feedback only say 'no'?"), but at least with a single consistent question and hint, it should be easier to figure out for feedback-reviewers what's going on.
Pushed to Gerrit (https://gerrit.wikimedia.org/r/#/c/22544/) & prototype (where you probably won't see the update because I can't clear the messages cache)
(In reply to comment #3) > Pushed to Gerrit (https://gerrit.wikimedia.org/r/#/c/22544/) & prototype (where Status Merged
