“There is, of course, nothing scandalous about divisions of opinion among scientists. This is a condition for scientific progress” (Grove, 1989, p. 133).
Although it is rare in the history of JUX, on a few occasions we have received a commentary on a published article. When we believe a commentary will be of interest to our readers, we might invite the author to prepare a publishable version, and, if so, we offer the authors of the original paper the opportunity to respond.
In our May 2023 issue, we published a paper entitled “Talking About Thinking Aloud: Perspectives from Interactive Think-Aloud Practitioners.”
On June 29, 2023, we received an email from Rolf Molich expressing some concerns about the paper and requesting a response from the Editors and reviewers regarding the acceptance of the paper. This request caught us a bit by surprise. After some deliberation, we responded on July 26, 2023, with an invitation to prepare a letter to the editors for publication. But we explained that our double-blind review policy prevented us from sharing information about reviewers and the contents of their reviews (a policy used by many scientific journals). After some discussion via email about our review policy, this invitation was accepted.
We then informed the authors of the original paper, Liam O’Brien and Stephanie Wilson, about the commentary and offered them an opportunity to respond to it, which they accepted on November 7, 2023.
We are pleased to offer this commentary (in the form of a letter to the editors) and the authors’ response in this issue of JUX. This “conversation” illustrates some of the differences between quantitative and qualitative research with regard to research goals and definitions of validity and generalizability.
Our Response to Comments on JUX Review and Publication Practices
Most of the commentary is focused on the details of the paper, “Talking About Thinking Aloud,” but some comments were directed at the JUX review and publication practices. We respond to those here.
Generalization Value
In his section on Generalization Value, Mr. Molich quoted from our former “Call for Papers” page on our website: “Authors are invited to submit manuscripts addressing various aspects of quantitative and qualitative usability studies that have a strong generalization value to other practitioners working with any human-interactive product.” Note that since the letter was written, as part of a previously planned revision of the website, we reorganized our submission information and renamed the page, “Submit an Article.” Readers should be aware that the quote is accurate and, even though it is no longer present on our website, the generalizability of quantitative and qualitative research is an important criterion for our reviewers and our final publication decisions.
JUX’s Review Process
Mr. Molich has expressed concerns not only about the paper but also about our “review process, error culture, and the quality of published papers” and encourages us (the Editors) and reviewers to constantly strive to improve our skills and the quality of JUX.
We agree that it is always worthy to strive to improve skills and quality, but we do not agree that the process by which this paper was published indicates serious problems with our review process or the quality of our published papers. We use a double-blind review process, with reviews sent to three members of our editorial reviewers. We only allow people to join this group of reviewers if they have a track record of publication and reviewing. These three reviewers were highly seasoned UX professionals, all of whom had at some point in their careers founded well-known UX consultancies and had received honors from UX organizations.
If there are concerns among our authors, reviewers, or readers regarding our review and publication policies, please feel free to contact us via email at jus@uxpa.org so we can discuss the issue(s) or, if you prefer, contact the UXPA Director of Publications (currently andrew.schall@uxpa.com).
References
Grove, J. W. (1989). In defense of science: Science, technology, and politics in modern society. University of Toronto Press.
