Great post. This is a really interesting question when you begin to break a usability study down and compare it to the three pillars of scientific procedure: hypothesis, experimentation, and proof.

When you consider how a usability study usually begins, with some sort of hunch that an aspect of the design isn’t quite performing up to par, that mimics a hypothesis by definition pretty well. Copernicus proposed the hypothesis that the Earth and planets revolve around the sun after he made certain celestial observations that didn’t quite support the notion that the Earth stood at the center of the universe. That was his “hunch” or hypothesis.

Then you move on to the second pillar, “Experiment”, which through good usability testing practices certainly can quality in my opinion as a real experiment.

Finally, I do agree with you on the Conclusions part. With usability testing recommendations you do go a bit beyond the formal conclusions, building upon what your conclusions gave you, and using your expert opinion to decide actionable next steps the client can take.

Thanks for writing about these questions. They are ignored all too often, yet they are so critical to a robust research praxis, so thank you.

I would argue, respectfully, that your approach cannot be ethnomethodological or even interpretivist in either case.

You would be using “subjective” measures such as facial expressions, for example, but you would not be engaging in the interpretive practice so central to qualitative research. Your research design would initially have to be far less structured, for example (e.g., not your laptop, but THEIR computer, sitting somewhere in their house or office). The participant would tell you about the topic, because you would guide them, but they may or may not find it relevant to show you the computer. They certainly would not complete (or attempt to complete) a pre-determined flow — that is engineered entirely by your research desire to uncover its effectiveness.

In this sense, I would argue usability studies are necessarily positivist but they may be less “scientific” than say, lab-based experiments. But your guiding of the “user,” your pre-determined desire to find out a specific insight about the flow itself, and your bringing of the laptop, e.g., “an instrument” makes this far more a mirror of positivist research design.

Interpretive research, would argue, necessarily puts more power in the hands of the “participants” and therefore really can’t get to the answers you really need to get to in usability studies.

Thanks again for writing such important work!