Hey Buddy

My buddy is Jonathan Nation, whose blog can be viewed here: http://cs303-jn.blogspot.com/

To recap a bit, Jonathan's idea is to integrate quizzes into Wikipedia pages to test whether this increases knowledge retention. He's going to test whether quizzes work best at the beginning, end, or interspersed throughout Wikipedia content; also, does giving the user negative/positive feedback affect their scores? I think the premise is excellent; I remember reading a thread on Reddit (which Jonathan also references) where a lot of people suggested that Wikipedia quizzes would make the site more fun and increase their knowledge retention. I think there are several challenges in this experiment, however. Quiz design is not a trivial task; I think a lot of pilot testing will be needed to refine these down into fair assessments of ability.

Also, I think it's going to be necessary to have a large sample size, due to a few issues. First, there are several treatments; this project is a lot more ambitious than mine in terms of all the different hypotheses and treatments it is trying to test. This will multiply the number of subjects needed. Also, there is large potential for bias to work its way into this study; if one particular user already happens to know a lot about a particular subject, then clearly they will do disproportionately well on the quiz. Also, things like attentiveness, time of day, general level of interest of the article, and other circumstances could greatly affect peoples' ability to answer quizzes. With a large sample size, subject-specific biases can be diminished. Another possible factor is that if volatile Wikipedia pages are chosen, then edits may happen between various treatments and thus make it more difficult to ensure consistency. This is probably easily mitigated by either choosing more stable Wikipedia articles (based on edit history), or by serving users a static version of the page that is pulled at a certain date and held constant.

All in all I'm looking forward to this experiment and the product itself. A Wikipedia quiz browser extension or website that users can edit themselves and share (like Wikipedia itself) would be a great companion tool to Wikipedia and something I would use.