Abstract
User satisfaction and usefulness are measured using usability studies that involve real customers. Given the nature of software development and delivery, having to conduct usability studies can become a costly expense in the overall budget. A major part of this expense is the participant costs. Under this condition, it is desirable to reduce the number of participants without sacrificing the quality of the experiment. If a company could use a smaller participant pool and get the same results as the entire pool; this would result in significant savings. Given a participant pool of size N, is there a subset of N that would yield the same results as the entire population? This research addresses this question using a data-mining clustering tool called Applications Quest.
Practitioner’s Take Away
Conducting user studies can be very expensive when selecting several experiment participants. The approach of collecting participant demographics and other information as input into clustering is a mechanism for selecting ideal participants. As a guideline for practitioners:
- Identify your target user base.
- Develop a survey instrument to collect information about them that you deem relevant to the evaluation task. This may include demographic, system experience, etc.
- Process the completed surveys in a clustering tool using a K-Means approach where you can specify the number of clusters. Ideally, you could use Applications Quest because it will cluster the applicants and make recommendations on which applicants should be used in your experiments. If you don’t have Applications Quest, there are alternative methods, which are described next.
- If you don’t have Applications Quest, you can cluster the surveys and then hand pick individuals from each cluster. This could be challenging if the clusters are large, or if you have a large number of clusters.
