Positive Developments but Challenges Still Ahead: A Survey Study on UX Professionals’ Work Practices

Abstract

This paper describes and gives an overview of User Experience (UX) professionals’ work practices—their environment, practices, tools, and challenges. First, we reviewed 32 empirical studies about usability and UX work to identify key issues in usability and UX work practices. For the identified key issues, we collected data from 422 UX professionals surveyed in five different countries using a comprehensive questionnaire with 62 questions. Our results show that UX professionals individually know about usability and UX concepts, methods, and tools. They typically employ between one and five Human Computer Interaction (HCI) theories on average and use one to three different techniques and tools. On the organizational level, UX is involved from early to late stages and is generally well known within all levels of the organization. On the country and community level, UX professionals generally do not report themselves as belonging to a professional community, despite the fact that the survey was administered via channels of the respective communities in the survey countries. Overall, this survey shows that UX professionals have considerable work experience and strong UX expertise self-confidence. This may be considered as indicating a positive development of the UX profession.

Keywords

user experience, usability, UX professional, UX work environment, UX practices, UX tools, UX challenges

Introduction

All over the world, organizations increasingly look to usability and UX in their design and development processes. Large and small companies, government agencies, NGOs, and other organizations are going digital and require the skills and knowledge provided by UX professionals. To meet this need, UX communities are maturing in many countries and emerging in other countries. It is, however, a common concern that there are still many challenges in the UX professionals’ work—even if the topic has been studied for decades, UX professionals are still facing challenges in their work (Ardito et al., 2014; Boivie et al., 2006; Iivari, 2006; Wale-Kolade & Nielsen, 2016).

Existing research paints a problematic picture of organizational integration of usability and UX. Across countries, social, organizational, and cultural issues with usability and UX persist (Katre et al., 2010). For example, there are challenges in integrating user perspective into agile development (Bruun et al., 2018; Cajander et al., 2013; Larusdottir et al., 2017; Wale-Kolade, 2015), management does not appreciate usability (Cajander et al., 2006; Rajanen & Iivari, 2007; Wale-Kolade & Nielsen, 2016), and in general nobody appears to take responsibility for organizational integration of UX and usability (Ardito et al., 2014; Cajander, 2010; Wale-Kolade & Nielsen, 2016). Hence, it is still contested and controversial how well UX professionals’ expertise and work practices play together with organizational practices and the software development life cycle (Ardito et al., 2011; Bruun et al., 2018; Dillon et al., 1993; Hussein, Mahmud, Tap, & Osman, 2012; Kou & Gray 2018; Marsden & Holtzblatt, 2018; Vukelja et al., 2007). This is surprising as the HCI research and practitioner communities have devoted a lot of time and effort to improving the position of UX professionals in organizations. This study aims to explore whether any positive developments can be identified.

In this study, we review findings from previous empirical studies on UX work and practices. We use the term usability when we refer to specific usability definitions and activities, such as the ISO usability concept or usability test; in all other instances, we use the term UX to cover both usability and UX. Based on the review, we identify a set of key issues for the UX professional work and develop a set of research questions that we examine with data collected from a survey conducted in early 2016 with 422 respondents from five countries. We report findings on (a) UX professionals’ knowledge and understanding of UX; (b) UX professionals’ UX activities, methods, and tools; (c) the integration of UX work into development life cycle; and (d) the active use of resources and involvement in local UX communities. We discuss the key factors identified and argue that many positive developments can be identified as regards the UX professionals’ work in organizations. The professionals seem to face less challenges than what used to be the case. However, we also point out that many challenges remain across decades, and the HCI community should be prepared to remedy the challenges still. For that purpose, both research community and practitioners benefit a lot from this type of rich insight into the work practices and challenges of UX professionals.

Related Work

In this section, we review related empirical studies on UX professionals’ work. We identified 32 empirical studies conducted during 1985–2014 that reported on the background of UX professionals; their practices in terms of processes, methods, and tools; their challenges faced in the organization; as well as professionals’ relationships within national or international communities. We categorize the literature findings into three key issues that can be further divided in more specific issues or sub-categories: (a) UX professionals’ knowledge and practices, (b) organizational factors that include evaluation practices and work challenges, and (c) country and UX community related issues.

UX Professionals’ Knowledge and Practices

Relevant literature showed that related to UX professionals, it is important to understand their basic knowledge and understanding of UX as well as their work practices. A summary of studies in these two sub-categories are presented in the following sections.

Basic Knowledge and Understanding of Usability and UX

Research on usability and UX has focused on understanding these concepts and has contributed to both producing a variety of definitions as well as a debate on what usability and UX represent (Hertzum, 2010; Law et al., 2014; Tractinsky, 2018). However, for practitioners, the ISO standards’ definitions represent the main instruments for communicating and operationalizing these concepts in practice in order to achieve a minimum level of quality in use (Dzida, 1996; Bevan et al., 2015; Marghescu, 2009). International Standardization Organization (ISO) 9241 Part 11 standard (1998) defined usability as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.” A decade later, UX was defined in ISO 9241-210 (2010) as “a person’s perceptions and responses that result from the use or anticipated use of a product, system or service.” Other definitions of UX such as the ones that define UX as a holistic experience (McCarthy & Wright, 2004) are popular and may thus draw interest among practitioners.

The empirical studies reviewed showed quite divergent findings, and these were observed in various cultural contexts and years. Through survey studies, Clemmensen (2003, 2005) found that the knowledge of the Danish usability community was focused on general usability and HCI concepts, which were used as means for communication and cooperation. On the other hand, Jääskeläinen and Heikkinen (2010) pointed to the differences in understanding UX between UX professionals and end users, which was amplified due to the personal characteristics. Lizano et al. (2013) showed that there is a consensus on the understanding of usability among software developers who are interested in usability evaluation. In contrast, Law et al. (2009) studied the understanding of UX among UX professionals and found that UX, which at the time was a fairly new concept, was mostly understood as a context-dependent and subjective concept as part of user-centered design (UCD), and there was no consensus on a clear description of the term.

The Practices of UX Professionals

A second sub-category of studies in the relevant literature focused on the work practices of UX professionals. One of the early survey studies (Gould & Lewis, 1985) revealed that system designers believed that the use of the system design principles—namely early focus on users, empirical measurement, and iterative design—could support usability. Gunther et al. (2001) studied the popular methods and activities in UCD process, which were usability testing, prototyping, heuristic evaluation, and customer interviews. However, Vredenburg et al. (2002) reported that effectiveness of the UCD process was rarely measured, despite that the measures were known by the professionals. A survey conducted in France’s UX community by Roche et al. (2014) showed that the preference and use of methods changed according to the professional’s expertise, academic background, and the sector in which they worked. Besides, only popular methods such as usability testing were frequently employed among the professionals whereas specific methods such as card sorting were known as anecdotes. These studies indicate that UX professionals are familiar with the concept of usability testing and major HCI concepts. However, the rest of the tools and methods are adopted in a limited manner.

Organizational Integration, Challenges, and Needs

There are a number of studies that address the integration of UX practices in the software/system development process (SDLC-Software Development Life Cycle). Some of the studies were realized before the 2000s and mostly included software developers. This category also includes studies that focus on user contact and involvement in the system development process. A sub-category of studies focuses on usability evaluation in particular.

Work Challenges of UX Professionals

Early studies dating back to the end of the 1980s reported a prominent problem that prevented the integration of usability methods in software/systems development organizations. This problem was the lack of understanding, direct interaction, and support between designers and other parties contributing to the development process, such as marketing (Borgholm & Madsen, 1999; Grudin & Poltrock, 1989; Poltrock & Grudin, 1994; Rauch & Wilson, 1995). Some following studies also indicated a lack of support by upper management and a resulting difficulty in convincing top management about the potential value of usability (Bekker & Vermeeren, 1996; Borgholm & Madsen, 1999; Clegg et al., 1997). However, lack of top management support was also viewed as a problem in more recent studies (e.g., Ardito et al., 2011; Gulliksen et al., 2004).

Moreover, Bekker and Vermeeren (1996) showed that the potential barriers that hindered the employment of usability methods in the system development process were insufficient resources (such as time, money, and equipment), lack of information about the user, and lack of information about the application domain and complexity of the application. Similarly, other factors hindering the integration of usability and UX practices were identified as late contact and collaboration in the process between software engineers and HCI practitioners, which resulted in unfixed usability problems (Jerome & Kazman, 2005), unyielding developer mindset, increased resource demands, and lack of customer participation (Ardito et al., 2011; Bak et al., 2008). Studies emphasized also the growing need for user involvement in the process (e.g., Clegg et al., 1997; Poltrock & Grudin, 1994).

Using empirical data from two case studies of examining usability practices in the public administration in Italy, Catarci et al. (2002) grouped the identified obstacles to the adoption of usability in organization in two categories: (a) organizational/contractual deficiencies such as the limited involvement of users and lack of interest toward the preference of projects that value usability and (b) methodological and cultural deficiencies such as the customer unawareness of usability and designers ignoring the role of users in the process. Moreover, in another case study, Iivari (2006) showed that the integration of usability practices in software development organizations differed due to varying archetypal “cultures of usability work.” Thus, organizational culture and organizational knowledge about usability and UX were seen as barriers or determinants to the adoption of good usability and UX practices. Rosenbaum et al. (2000), then again, showed that the size of the organization did not affect how the organizations perceived the impact of usability in the development process. Moreover, Mao et al. (2005) found that UCD expenditure often exceeded 10% of the overall project budget; however, the adoption of UCD methodology by the organizations varied. Although a growing interest toward the inclusion of usability has been reported during the years, usability professionals still do not play an important role in the post-deployment phase (Boivie et al., 2006; Chilana et al., 2011; Ji & Yun, 2006).

In summary, the reviewed studies show that the work challenges that prevent the adoption of usability and UX in the software/system development process due to organizational factors are lack of information about the user due to the limited involvement of users in the process; insufficient resources such as time, money, and equipment; lack of communication and collaboration among the responsible parties in the process such as software engineers and UX professionals; and lack of support by the upper management and other departments such as marketing.

User Contact and Evaluation

One of the earliest large-scale studies conducted throughout nine European countries in this sub-category showed that only a limited number of the organizations had user research teams and facilities to conduct usability tests and this led the designers to do their own evaluations (Dillon et al., 1993). A survey study in Switzerland realized that in the late 2000s, even though organizations preferred to have more contact with end users, they still did not prefer to rely on user insights in the decision process (Vukelja et al., 2007). A following survey study in Norway indicated that even though some organizations claimed the need for software development and usability methods to be integrated, usability requirements analysis was still more valued than usability testing (Bygstad et al., 2008). More recent studies have showed that usability and user involvement still have low priority in the projects due to the following reasons: Developer mindset undervalues the contribution of real user in the process, and most of the organizations do not invest in funding usability tests due to lack of time (Hussein et al., 2014; Inal & Guner, 2016).

Presence of a UX Community in the Country of Work

Although the number of the studies is limited, cross-cultural studies have from early on revealed the importance of the presence of an established community of practice among UX professionals. A study conducted between six organizations in Denmark and the USA showed that the more experienced a community was with usability practices, as in the case of Denmark that has its roots in the Scandinavian system development perspective, the more that community valued the contribution of the involvement of real users in relevant design and development processes (Borgholm & Madsen, 1999). A study on the “institutionalization” of HCI in China and India supported the notion that the presence of a community of practice through the establishment of a local organization could contribute to wide spread usability practice in professional sectors (Smith et al., 2007). It has also been reported that in order to support the rapidly growing UCD practice in China, practitioners should be supported with training (Zhou et al., 2008).

Despite all these empirical inquiries on usability and UX professionals’ work and practices, there is a need for an overall understanding of the situation. During the past decades, the world around us has dramatically changed—the fast-paced digitalization has led to an increased importance given to usability and UX as product, system, or service attributes. Simultaneously, during the same decades, HCI research and practitioner communities have worked hard to legitimize their work and to make it effective in development. It is time to see whether any positive developments can be identified. In the following, an empirical study to indicate UX professionals’ work practice is presented.

Methods

The following sections discuss the research design and questions, which countries were selected and the reasons for selecting those countries, information about the study participants, and information about the questionnaire and data collection process.

Research Design and Questions

The aim of this study was to describe and give an overview of UX professionals’ work practice, work environment, tools, and challenges. Based on the literature review presented in the previous section, we developed our research questions listed as follows:

• RQ1: What is the knowledge and understanding of UX professionals about usability and UX?
• RQ2: What are the activities, methods, and tools involved in UX professionals’ work?
• RQ3: How well is the work of UX professionals organizationally integrated?
• RQ4: What challenges and needs do UX professionals encounter in their work?
• RQ5: What kinds of associations and country specific issues are involved in UX professionals work?

Selection of Countries

In order to answer our research questions, a cross-national survey was implemented. Data were collected from UX professionals working in Denmark, Finland, France, Malaysia, and Turkey as these together represent geographic and cultural diversity. We relied on convenience and purposive sampling, executing the study in countries of the researchers showing initial interest in this study and inviting UX professionals in these countries to participate in the survey. We intentionally included cultural diversity into the sample and tried to locate countries representing variety in terms of geographical position such as North-European, Central-European, South-East-European, and Asian. The selection includes countries with an extensive background in HCI (Finland, Denmark) and in ergonomics (France), as well as countries with a relatively recently established UX community (Turkey, Malaysia). Finland and Denmark represent Nordic countries in which the influence of the Scandinavian tradition of participatory design is assumed to be visible in terms of UX professionals engagement with users, whereas Malaysia and Turkey are seen as representatives of countries with emerging HCI communities that so far have not been studied from this perspective and that are assumed to bring more inclusiveness and variety into the dataset.

Participants

A total of 422 UX professionals participated in the study. The distribution by country was as follows: 123 (29.1%) professionals were from Malaysia, 90 (21.3%) from Turkey, 86 (20.4%) from Finland, 64 (15.2%) from France, 49 (11.6%) from Denmark, and 10 (2.4%) from other countries. Of the participants, 213 (50.5%) were male and 188 (44.5%) were female, and the remaining 21 (5%) did not state their gender. The mean age of the participants was 35.2 years (SD = 8.3).

Questionnaire and Data Collection

The questionnaire started with an information sheet explaining the goals of the study and instructions to the participants. The core part of the survey contained questions that aimed to gather information related to the following seven categories: organization and work environment, usability and UX understanding, UX activities and tools utilized in work, integration of UX work, UX communities, UX activities in the country of work, and demographics. In the closing section of the survey, the UX professionals were invited to a follow-up study.

The questionnaire was implemented using an online survey tool over a period of eight weeks between January and March 2016. The questionnaire was distributed through local UX associations, communities, mailing lists, and personal networks of researchers. Reminder emails were sent two and four weeks after the initial emails. There were 62 questions in the questionnaire. It was expected that participants allocate approximately 30 minutes to complete the survey (see Appendix).

Data were collected by using different survey tools in each country. The questionnaire was translated into local languages by researchers. The questionnaires were back-translated to check and ensure the accuracy of the translation. Participants were given the option to choose between their local language and English. At the end of the data collection process, data from each country were merged and cleaned. The data were coded with respect to variables names, values, and level of measurement. Duplicate data were removed and the final data set consisted of 422 valid respondents.

In the current study, we analyzed all data that were collected through the questionnaire to provide an overview of UX professionals work practice and to answer the above-mentioned research questions. In a previous study, we reported data from 19 questions focused on background information and UX professionals’ understanding of usability and UX concepts (Rajanen et al., 2017). The data analysis in this study is based on descriptive statistical analysis and frequency distribution graphs performed using MS Excel and SPSS 20.0.

Results

In the following sections, we report the findings from the questionnaire. We begin with the demographics of the participants, followed by our findings related to usability and UX understanding, knowledge, activities, methods, tools, communities, organizational integration, challenges, and needs.

Demographics

The general profile of the survey participants was that of a professional working in small teams within large organizations that had no or not very visible software or quality maturity certificates and used agile/lean development methodologies. Participants also needed to have more than a decade of work experience Our participants tended to be familiar with local language, and they lived and worked in the same country. They had acquired a technically oriented university degree and some formal HCI education. They had considerable work experience in the UX field. They stayed updated with the new knowledge in the field and were self-identified as UX professionals (subject matter experts). However, along with these prominent characteristics, variation in the practitioners’ profiles existed. In the following sections, we provide the demographic details on the individuals and their organizations.

Language, Country of Work, Education, Education in HCI, Work Experience, Job Title, Follow Development in The Field

The majority of the respondents (82.7%, n = 349) reported that they spoke their native language and were employed in their native country (90.3%, n = 381). Concerning the educational levels of the participants, 21 (5%) had a basic or diploma, 118 (28%) a Bachelor’s degree, 213 (50.5%) a Master’s degree, and 49 (11.6%) had a PhD. About one third had last graduated from areas related to computer science and information systems (32.2%, n = 136); followed by media and communication (13.3%, n = 56); psychology (6.9%, n = 29); arts (5.5%, n = 23); business and management (5.2%, n = 22); and electronic, automation, communication engineering and electronics (4.3%, n = 18). However, quite a large proportion of the respondents mentioned other areas such as educational sciences, economics, architecture, and medical and health sciences (27.3%). Table 1. describes the results related to the education level and graduation field of the respondents.

Table 1. Education level and graduation field of UX professionals

Education level and field of study		n	%
Education level	Basic or diploma	21	5.0
	Bachelor’s degree	118	28.0
	Master’s degree	213	50.5
	PhD	49	11.6
	Missing	21	5.0
Graduation field	Computer science and information systems	136	32.2
	Media and communication	56	13.3
	Psychology	29	6.9
	Arts	23	5.5
	Business and management	22	5.2
	Electronic and automation	18	4.3
	Other	115	27.3
	Missing	23	5.5

 

Most of the participants had received formal education related to HCI. Only 3.8% (n = 16) of the participants had not received any formal education related to HCI. Also, 42.4% (n = 179) had completed one type of formal HCI education, 13.5% (n = 57) had completed two types, 6.9% (n = 29) three types, and 3.8% (n = 16) four types (such as completing HCI courses, completing a thesis or dissertation in the field, participating in projects, and obtaining certifications). Regarding vocational training, the majority (75.4%, n = 318) reported that they had not received any vocational training. Only 19.4% (n = 82) had received vocational training in HCI.

The participants had considerable work experience, and they worked at all levels in organizations, though not always with usability or UX in their job title. The average length of the participants’ total work experience was 11.2 years (median = 10; SD = 7.9) with experience in UX area being 6.7 years (median = 5; SD = 5.6). The average time in their current position was 4.3 years (median = 3; SD = 4.6). The participants included both beginners and experienced participants, with a considerable number of the participants (n = 189) who had less than 5 years’ work experience in UX field (Figure 1).

Figure 1. Years of UX work experience (frequency distribution).

Concerning their positions in their workplace, 106 participants (25.1%) worked in low-level/middle management positions, 66 (15.6%) in top management, 34 (8.1%) at entry level positions, and 16 (3.8%) as experts including academic specialists. The job titles of half of the participants (45.5%, n = 192) included usability or UX-specific titles such as usability/UX designer, expert, strategist, manager, researcher, analyst, engineer or consultant, while the other half (47.2%, n = 199) reported non-UX related titles.

The participants followed recent developments in the fields of UX and were confident about their own expertise in UX. The majority (84.1%, n = 355) reported that they kept up with novel applications and practices in the UX field. On a 5-point scale from novice to expert, they rated their level of knowledge concerning UX to be 3.41 (SD = 1.13).

Organization Type, Location, and Branch

Most participants were employed in in-house UX teams in local, private sector organizations. The participants’ occupational status was employed by a company (83.4%, n = 352) or as an entrepreneur (7.1%, n = 30) or freelancer (3.8%, n = 16). Concerning the type of organization, more than half of the participants (61.4%, n = 259) worked in in-house UX teams, and 20% (n = 83) worked as UX consultants, including one-person companies. The remaining 12.6% (n = 53) stated that they worked in companies operating in various areas, such as research and development, education, and academia (not as a professional but as an educator). Many participants worked in national (local) organizations (56.2%, n = 237), but also many worked for international organizations (43.8%, n = 185). The majority of the participants (66.1%, n = 279) worked in private companies, 68 (16.1%) were employed in the public sector, and the remaining were employed in other types of organizations such as academic institutions and public-private partnerships. The principal work areas were information and communication (45.3%, n = 191); education (22.5%, n = 95); professional, scientific, and technical activities (18.5%, n = 78); financial and insurance activities (10.9%, n = 46); manufacturing (8.1%, n = 34); and administrative and support service activities (7.8%, n = 33).

Figure 2. Organization size.

As stated previously, the participants worked in small teams within large organizations that had no or very little software or quality maturity certificates and used agile/lean development methodologies. The participants also had more than a decade of experience in UX work. More than half of the participants (60%, n = 253) worked in large-scale organizations with more than 250 employees, followed by small-scale (less than 50 employees; 21.6%, n = 91), then medium-scale (50–250 employees; 16.6%, n = 70) organizations (Figure 2). In addition, 116 (27.5%) participants reported that their organization had a software maturity or quality certificate, while 193 (45.7%) had no idea about the certification status of their organization, and 113 (26.8%) reported that their organization did not have such certification.

Figure 3. Software/system/product development methodologies.

Regarding the software/system/product development methodology (Figure 3) that had been in use in the participants’ organizations for the last two years, Agile/Lean was mentioned by the majority of the participants (67.8%, n = 286), followed by rapid prototyping (41.9%, n = 177) and waterfall (39.6%, n = 167). According to the responses of the participants, the average number of UX professionals employed in all organizations was 16.5, the average number of years of organizational experience in UX activities was 8.3, and the average number of people on the UX team in the most recent project was 4.2.

Usability and UX Understanding and Knowledge

To answer the first research question, in the survey we asked participants to provide their understanding of the concepts of usability and UX as well as to report on the theories, frameworks, and methods they use in their work. Thus, the participants were given two definitions of usability and two definitions of UX and asked to rate the definitions according to their importance. The definitions of usability were chosen so that one reflected the ISO 9241-11 perspective focusing on the quality in use of a product, service, or system, while the other definition reflected usability in a broader sense—that of the fit between the product and the organization adopting it, which is referred to in the literature as organizational usability (Elliott & Kling, 1996; Sørensen & Al-Taitoon, 2008). Moreover, the definitions of UX were chosen so that one reflected the system-oriented user experience (Kujala et al., 2011), and the other definition reflected a human-oriented view (McCarthy & Wright, 2004).

Regarding usability, the first definition, “Usability describes how a product can support its users to be effective, efficient and satisfied in its use,” was chosen as the most important by the majority of the participants (55.8%, n = 231) and somewhat more important by 91 participants (22%). Only four participants (1%) thought that the most important definition of usability was “the match between the product and the organization adopting it,” and 12 participants (2.9%) found this second definition was somewhat more important.

Concerning the definitions of UX, 16.9% of the participants (n = 70) considered that the definition, “UX is the perceived attractiveness, ease of use, utility, and degree of usage of the product,” was the most important, and 10.6% of the participants (n = 44) thought it was somewhat more important. Similarly, 22.2% of the participants (n = 92) chose the definition “UX is the combined experience of the composition of elements, sensory qualities, related emotions, and the context” as the most important, followed by 18.6% of the participants (n = 77), who considered it to be somewhat more important. Contrary to the case in usability definitions, a significant number of participants (31.6%, n = 131) reported that both UX definitions were equally important. For a detailed analysis on how people rated the definitions of usability and UX, see Rajanen et al. (2017).

On average, the participants utilized in their work about four to five different theories, frameworks, and methods of UX, such as activity theory, mental models, user-centered design, cognitive dimensions, inclusive design, phenomenology, and information processing theory. The average number of theories, frameworks, and methods with which the participants were familiar was 4.87 ranging from 0 to 17 (Figure 4). As shown in Figure 4, most of the participants employed a quite narrow theoretical background on UX in their work in that most of the participants (62.3%, n = 263) reported that they used in their work between one and five theories, frameworks, or methods of UX. However, among the theories, frameworks, and methods mentioned by the respondents, some were broad in scope such as user experience (n = 346), usability (n = 337), and user-centered design (n = 302). More specific approaches were also reported such as mental models (n = 153), information processing theory (n = 107), cognitive dimensions (n = 83), activity theory (n = 69), sustainable design (n = 67), cognitive work analysis (n = 67), design activism (n = 63), and ethnomethodology (n = 62).

Figure 4. Number of theories, frameworks, and methods with which the participants were familiar.

UX Activities, Methods, and Tools

To answer the second research question, we surveyed the type of UX work respondents were engaged in. We addressed the following issues: user involvement, usability testing, usability activities, and tools for collecting user feedback, remote usability testing, and prototyping. Generally, the participants carried out 5–10 different types of UX studies during a year, using a few different IT tools. When asked what UX activities (such as persona, wireframing, prototyping, card sorting, benchmarking, mock-up, user research, eye-tracking, and psychophysiological studies) they performed within the last year, the average number of activities was 8.32 with the minimum number being 0 and the maximum being 22. A significant number of the participants had performed very few UX activities within the last year, while more than one third of the participants (34.6%, n = 146) reported that they undertook more than 10 UX activities. Top most common activities conducted across the sample were related to prototyping, namely, wireframing (n = 237), sketching (n = 218), mock-up (n = 212), and digital prototyping (n = 205).

Figure 5. Top 10 most common UX activities conducted.

Requirements gathering (n = 203) and benchmarking (n = 196) were the next most common activities. Furthermore, paper prototyping was reported by 194 participants followed by the satisfaction surveys (n = 186), and other surveys or online research (n = 180). Usability testing was conducted by several respondents as well, namely, at the customer location (172) or in a lab (158). User research using focus groups, field studies, and observations were conducted by 169 respondents. Other relatively common activities were persona, ideation, heuristic or expert review, analyzing metrics, card sorting, and competitive studies. Eye-tracking and psychophysiological research were conducted less commonly; only 62 and 19 respondents, respectively, reported these approaches. Other less common activities were remote usability testing and living labs (Figure 5).

Usability and UX Testing

The participants usually recruited and met end users in face-to-face meetings during repeated usability testing sessions with six test users or more, or in follow-up sessions. Most participants (67.5%, n = 285) reported that they had face-to-face contact with end users in their recent projects and 66.1% (n = 279) generally performed usability tests. Concerning the number of usability tests typically conducted before deployment or implementation, 134 participants (31.8%) stated that they had performed three or more rounds of usability tests in their recent projects. Sixty-nine participants (16.4%) had performed two rounds, 49 (11.6%) one round of usability tests, and 7.8% (n = 33) had not performed any usability tests in their recent project (Figure 6).

Figure 6. Number of usability tests conducted by the participants in recent projects before deployment/implementation.

The participants were also asked about the number of users that had participated in the usability testing process in their recent projects: 41.7% (n = 176) had recruited 6 to 50 test users, 15.4% (n = 65) 1 to 5 users, and 4% (n = 17) more than 50 users. The participants who reported to have performed usability tests in their recent projects were also asked to explain their process of user selection for testing, and 34.4% of the participants (n = 145) preferred recruiting users themselves, while 13.3% (n = 56) preferred utilizing professional recruitment agencies. Finally, most participants (74.2%, n = 313) did follow-up with the development teams after completing the UX activities.

Remote Usability Testing

Interestingly, the majority of the participants (72.7%, n = 307) did not perform remote usability testing. Only 71 (16.8%), 24 (5.7%), and 6 (1.4%) participants reported that they used one, two, and three remote usability testing tools, respectively.

Prototyping

The participants used a small number of different low- and high-fidelity prototyping tools. Designing prototypes is one of the job responsibilities of UX professionals, and thus having experience in using low- and high-fidelity prototyping tools is essential for these professionals. The average number of low-fidelity prototyping tools (such as Balsamiq mockups, Axure, paper prototyping, and InDesign) used by the participants was found to be 1.9 with the minimum number being 0 and the maximum number being 7. Ninety-one (21.6%, n = 91) participants reported that they did not use any low-fidelity prototyping tools. A significant number of participants (59.7%, n = 252) reported that they used between one and three low-fidelity prototyping tools in their projects.

Figure 7. Number of tools used for prototyping.

Similarly, many participants (26.3%, n = 111) reported to have no experience in using high-fidelity prototypes such as Axure, InVision, HTML+CSS, Java, and Pixate. The average number of high-fidelity prototyping tools used by the participants was 1.83 ranging from 0 to 9. Similar to the responses regarding experience in using low-fidelity prototyping tools, a significant number of participants (55.9%, n = 236) reported that they used between one and three high-fidelity prototyping tools in their projects (Figure 7).

Use of Tools for Collecting User Feedback

The participants were expected to use various tools such as email, social media, survey tools, Loop11, Tech Smith Morae, Optimizely (A/B testing), and Silverback to receive quick user feedback during the development process. However, the average number of tools used was 1.78 ranging from 0 to 7. Noteworthy, 78 participants did not use any tools for getting feedback from users, and 110 (26.1%), 108 (25.6%), and 68 (16.1%) reported that they used one, two, or three tools, respectively.

Organizational Integration, Challenges, and Needs

To answer the third and fourth research questions regarding the integration of UX work in the organization and the associated challenges and needs, we surveyed the general perception of how well UX is integrated in projects, the stages in the development life cycle where UX is employed, and what is the level of familiarity with UX of various functions in the organization. Moreover, we asked what the main challenges and improvement areas were from the perspective of the respondents.

Knowledge of UX Within Organization

The participants were asked about the extent to which they thought UX was known or familiar to people in their organizations. They reported that UX was best known to designers and managers, and least known to marketing and customer support. UX professionals and designers were rated as the most knowledgeable groups in the organizations, whereas customer support teams and people from marketing departments were considered to have the least knowledge. A large number of participants stated that project managers and top managers in their organization also had sufficient knowledge and familiarity regarding UX (Figure 8).

Figure 8. Familiarity of different employees with UX according to the participants.

System Development Phase Involvement

The participants were involved in their recent projects from the start to the end, with UX activities integrated. The participation level and time of UX professionals in a project are very critical to get more benefits from their expertise. According to the responses, most of the participants were an important part of projects with 206 participants (48.8%) being involved in all phases of software/system/product development processes from the start to the end. The remaining 41.2% (n = 174) stated that they only participated in the early stages of the development process in their projects. The integration of UX activities into the development process can also give information about the maturity level of an organization regarding UX activities and attitudes toward the field. The numbers of participants who reported that UX activities were fully, mostly, or moderately integrated into the development process of their recent projects were 81 (19.2%), 113 (26.8%), and 122 (28.9%), respectively. Of the remaining participants, 32 (7.6%) and 55 (13%) reported that UX activities were not included or mostly not included in their projects, respectively (Figure 9).

Figure 9. Integration of UX activities into the development process.

UX Professionals’ Challenges and Needs

The participants faced several challenges in organizational integration of UX such as lack of resources, knowledge, good communication, qualified UX people, and support from top management. The problems and challenges faced by participants were reported to be insufficient resources such as time, money, and equipment (48.3%, n = 204); lack of understanding/knowledge about UX in their organization, team, or project (39.3%, n = 166); and low priority of UX issues in the organization (34.4%, n = 145). Other major problems included communication problems with developers (29.6%, n = 125); lack of qualified UX professionals in the organization, team, or project (29.1%, n = 123); and lack of organizational/management support (26.5%, n = 112). Only 22 (5.2%) of the participants stated that they did not experience any problems working as a UX professional.

Figure 10. Problems and challenges faced by the participants.

The participants wanted more top management support, money, better internal cooperation, better organization-wide tools, more UX training, and more communication with developers. The participants were asked about the changes they thought were necessary to improve the effectiveness of their work. The most significant responses given by the participants were related to increased support from upper management (40%, n = 156); increased budget for user research in the organization, team, or project (35.8%, n = 151); and higher level of internal cooperation (33.2%, n = 140). These responses were followed by better tools and methods used by the organization, team, or project (32.2%, n = 136); providing education/training for UX professionals (30.3%, n = 128); and improved communication with developers (29.9%, n = 126). Only 11 participants (2.6%) stated that they were satisfied with the current situation.

Local UX Communities

Finally, to answer the fifth research question, we asked the participants to provide information on their participation in professional communities and meetings, and their knowledge about UX regulations and introduction in their country of work.

Information About UX Communities

The participants were in general not members of any professional community (though this varied significantly across the five countries), and thus they did not attend professional meetings. Most of the participants (63%, n = 266) reported that they were not members of any national UX community. Only 32.5% (n = 137) had a membership in a national community. Some of the communities they were associated with includes UXPA Turkey, UXPA Istanbul, UX Denmark, SIGCHI Finland, IxDA Helsinki, KäytettävyysOSY/Sytyke, SIGCHI Finland, MINDS’ Arts and Creativity Exhibition, ErgoHM, Flupa UX Paris, UX Malaysia, Käyttäjän ystävät ry, France-Luxembourg User Experience Professionnals Association (Flupa), DADA ry, itsmf, Cognitilist, Association d’anciens élèves de l’ENSC (Bordeaux), Réseau des Jeunes Chercheurs en Ergonomie, Ergoihm, Agile UX Amsterdam, SIGCHI.dk, Infininit, DUXFOR, Malaysian SIGCHI – 3, ego ihm, ergolist, arpège, UX Linkedin, Facebook, Google+, ErgoIHM ErgoListe, Le Groupe Associatif du Master Ergonomie et Sociologie de la ville de Nice, Tout le monde UX, Human Factors and Ergonomics of Malaysia, Aalto Alumni, Palvelumuotoilijat ry, Design for All Finland ry. Similarly, the majority of the participants (80.8%, n = 341) reported that they were not affiliated with any international UX community. Very few participants (14.7%, n = 62) had an international membership regarding UX. Many of the participants (71.1%, n = 300) stated that they had not had any chance to attend UX meetings such as conferences, workshops, or training programs over the last 12 months. Of the remaining participants, 58 (13.7%) had attended one, 22 (5.2%) two, and 11 (2.6%) three UX meetings.

Information About UX Activities in the Country of Work

The participants were asked which year the words usability and UX started to be used in their countries. The participants reported that usability and UX became common terms in their country around 2001 and 2008, respectively, but this varied a lot across the five countries. The responses varied ranging from 1930 to 2016 for usability, and from 1950 to 2016 for UX. For example, the majority of the participants from Turkey, Malaysia, and France stated that usability started to be used between 2000 and 2016, whereas the participants from Denmark and Finland mostly stated that usability started to be used before the 2000s. The participants were generally not familiar with the government regulations, requirements, or laws regarding UX; most participants (74.9%, n = 316) reported not having information about them. In addition, most participants (69.4%, n = 293) reported that they did not have any UX work experience outside their country, while 25.8% (n = 109) had such work experience abroad.

Discussion

In this section, the findings are discussed and their implications for HCI research and practice are addressed.

Need for a Coherent Body of Knowledge on UX Practice

Our results show that the field still lacks a common approach to study the UX profession and practice. The existing studies address different questions or ask different questions when addressing the same overall questions. For these reasons, comparing our survey of UX professionals to the previous surveys is not easy. Also, other reasons contribute to this difficulty beyond the obvious reason that every author wants to make their own contributions and be unlike the others. Firstly, demographic questions such as gender, age, organization size, and business sector show variation in the studies done with respect to the target population and the sampling method employed. For instance, similar to the studies of Clemmensen (2003; 2005) and Bak et al. (2008), our study collected data from UX professionals in a variety of countries purposively (purposive sampling, Etikan et al., 2016), while Chilana et al. (2011), Jääskeläinen and Heikkinen (2010), and Rauch and Wilson (1995) used convenience sampling. Another example is that we sampled from both small and large companies; however, the majority of the compared studies do not report organization size, while one study (Vukelja et al., 2007) gathered data from mostly large companies and another one (Grudin & Poltrock, 1989) from only large companies. Secondly, it is hard to compare different studies’ approaches to capturing UX professionals’ perception of UX because there is only one study (Dillon et al., 1993) that asked the respondents about the definition of the usability. Some of the compared studies (Ardito et al., 2011; Bak et al., 2008) focused instead on the definition of usability evaluation to figure out how the respondents describe what they understand by this concept. Thirdly, it is not possible to say completely “agree or disagree” for some questions across studies because there may be both similarities and differences at the same time. For instance, Bak et al. (2008), Ardito et al. (2011), Gulliksen et al. (2004), Vukelja et al. (2007), and Grudin and Poltrock (1989) identified several obstacles to usability work, but only some of the previous results were found similar to our results or similar across various studies. Besides, previous studies focused on different kinds of obstacles than we do, for example, we addressed obstacles from the perspective of a UX professional, while in other studies obstacles to usability evaluation were addressed. Fourthly, even if some questions seem similar or aim to measure similar issues, question format, options presented to the respondents, and their answers are different; this prevented us from saying that participants can say they agree or disagree. For instance, some studies (e.g., Mao et al. 2005) asked a question including both UCD theories/methods and interaction design activities, while we asked them in different questions (without merging) which provided more options to our respondents. Finally, another problem is about the respondents’ profile. Very few previous studies (Chilana et al., 2011; Clemmensen, 2003, 2005; Jääskeläinen & Heikkinen, 2010) collected data from usability or UX professionals, while most of the compared studies recruited software developers, project managers, user-centered design practitioners, user interface designers, and so on.

However, comparing surveys of UX professionals should be valuable and necessary for developing a coherent body of knowledge about UX professionals and their practice. In Table 2, we show how our study and the studies that we have reviewed overlap and, where possible, how our study agrees or disagrees with the previous studies. We limit our comparison to show (a) how past survey studies centered around specific questions and (b) how we covered these past survey studies. Hence, Table 2 does not include the four case studies and the two interview studies in our review.

Table 2. The scope and agreement of 26 questionnaire studies of UX professionals, compared to questions from our study.

Questions		Gulliksen et al. (2004)	Ji and Yun (2006)	Dillon et al. (1993)	Roche et al. (2014)	Chilana et al. (2011)	Hussein et al. (2013)	Clemmensen (2005)	Zhou et al. (2008)	Bygstad et al. (2008)	Hussein et al. (2012)	Law et al. (2009)	Vredenburg et al. (2002)	Clemmensen (2003)	Vukelja et al. (2007)	Grudin and Poltrock (1989)	Gunther et al. (2001)	Ardito et al. (2011)	Bak et al. (2008)	Bekker and Vermeeren (1996)	Jerome and Kazman (2005)	Mao et al. (2005)	Rosenbaum et al. (2000)	Jääskeläinen and Heikkinen (2010)	Lizano et al. (2013)	Rauch and Wilson (1995)	Gould and Lewis (1985)	Total
Current job title																												23
Place of work																												19
Challenges in being UX professional		X	O	O		X	O				O					X	X	O	O	O	X		O			X		14
Business sector																												13
Occupational status																												11
Work experience																												10
Level of UX knowledge																												10
Interaction design activities																												9
Organization’s size																												8
Graduation field																												8
Age																												8
Usability testing																												7
Gender																												7
Theories, frameworks, methods used																												7
Education																												7
Organization’s type																												7
Development methodology																												7
Development process involved				X		O	O								X						X						O	6
HCI specific formal education																												6
UX integration																												5
Size of UX team																												5
Face-to-face contact with end users																												4
User recruitment method																												4
Nationality																												3
Recent project																												2
Follow-up usability process																												2
UX definition																												2
Organization’s current position																												2
Type of organization																												1
Usability definition																												1
Number of users involved in usability tests																												1
International UX experience																												1
In-service training in HCI																												1
How to keep up evolving UX																												1
Total		15	14	13	12	12	12	11	11	11	9	9	9	8	8	8	8	7	6	6	6	6	6	5	4	4	2	222

Note: “X” marks agreement between our results and the result from a previous survey. “O” marks disagreement. A gray colored cell marks that the question was covered by the previous study. Items in the first column are sorted by number of occurrences in previous questionnaires, and the 26 questionnaire studies are sorted by number of questions addressed.

From Table 2 we learn that all or most studies asked about job title, place of work, and business sector. The purpose of these questions is to identify respondents that fall under the category of professionals investigated in the study. The questions also indicate that the participants are not drawn from a homogenous sample such as members of a UX professional association (Etikan et al., 2016). Furthermore, challenges in being a UX professional is a question of interest in most studies. The motivation of these kind of studies is often to help the professional field to move forward, and hence the authors want to identify barriers and challenges. We found that our results tended to agree with eight out of the 14 previous studies that asked this question.

In contrast to the few common questions across previous studies shown in the first rows of Table 2, the lower rows of the table show us that most of the questions about UX professionals’ work practices were asked by less than a third of the 26 previous studies. This scattered picture of questions asked in previous studies is clear in the lower part of Table 2 and testifies a real need for discipline level discussions about which are the key dimensions of being a UX professional.

Looking forward, in our study we asked a number of questions that were not asked in previous studies, but which we believe are important. They include obvious questions such as the number of UX professionals and of usability tests conducted, which tools are used by UX professionals, and how to improve UX and usability evaluation. However, we also asked types of questions not asked before: These questions related to national, cultural, and country differences, such as language skills and what the country has to offer a UX professional, including questions about national and international UX community membership, history data about when the terms usability and UX were first used in the country, and questions about government regulations related to UX and usability issues.

Main Results Compared with Previous Survey Studies

Our findings indicate a positive situation with many UX professionals having a considerable work experience and formal education in HCI, as well as a toolbox equipped with relevant knowledge and practices including various HCI theories, methods, techniques, and tools. In earlier studies (e.g., Clemmensen, 2005; Hussein et al., 2019), UX professionals were young, highly educated across all faculties (including humanities and social sciences), international, and had less than five years of work experience with little or no formal education in HCI. Furthermore, most of them were employed as consultants, and fewer were employed in in-house UX groups in large organizations. These organizations hiring UX professionals were private companies with strong software methodology approaches but with little experience in integrating usability and UX. However, it turned out that the UX professionals in our survey typically worked in small teams within large organizations, which had no or not very visible software or quality maturity certificates and used agile/lean development methodologies. Also, the participants had more than a decade of work experience and on average more than five years of UX work experience. The UX professionals typically were local people, speaking local languages, with technically oriented university degrees and some formal HCI education, had considerable work experience, and were followers of new knowledge in the field, and clearly self-identified as usability and UX professional experts.

Our findings on UX professional’s knowledge and understanding about usability and UX, however, is somewhat similar to that reported by several of the previous studies (e.g., Clemmensen, 2003). These similarities include that UX professionals are interested in, but know few HCI theories, and use a small number (out of the very many possible) of UX activities, methods, and tools in their work. Also, in our study there seemed to be a consensus on understanding of usability (Lizano et al., 2013), while lack of it as regards to understanding of UX (Law et al., 2009). Our findings on UX professionals’ activities, methods, and tools also seem to align quite well with the existing literature showing that usability testing and prototyping are common (Gunther et al., 2001; Roche et al., 2014).

As regards to the organizational integration of UX work, we found in contrast to the previous studies (e.g., Dillon et al., 1993; Hussein et al. 2012, 2013, 2014; Vukelja et al., 2007) that UX professionals report early and have consistent involvement in system development and a high priority in the organization. Previous studies have found that developer mindset undervalued the contributions of real users in the process, and most of the organizations did not invest in funding usability tests due to lack of time. Even the ones that had contact with users did not prefer to rely on user insights in the decision process.

Furthermore, also in contrast to existing studies (e.g., Boivie et al., 2006; Jerome & Kazman, 2005; Ji & Yun, 2006), most of the UX professionals in our survey reported that their UX work was not so challenged by a set of persistent problems in the organization such as those identified by Bekker and Vermeeren (1996) and Bak et al. (2008), for example, lack of knowledge about users (Chilana et al., 2011) or usability (Catarci et al., 2002); insufficient time, money, and equipment; poor communication with developers (Poltrock & Grudin, 1994); lack of support from management and marketing (Ardito et al., 2011; Borgholm & Madsen, 1999; Gulliksen et al., 2004); and lack of time spent on user research (Christensen, 2018).

In addition, in contrast to findings from previous studies (Clemmensen, 2005; Smith et al., 2007), it turned out that most of the UX professionals responding to our survey were not members of any national or international UX community. The expectation was that the presence of a community of practice through the establishment of a local organization can contribute to the diffusion of usability practice in industry, as suggested by Smith et al. (2007). Moreover, similar to findings in a previous study in Denmark (Borgholm & Madsen, 1999), we found that the more experienced with usability practices a community is, the more that community values the contribution of the involvement of real users in relevant design and development processes. However, the positive finding was that UX has matured in the countries of this study in the sense that both concepts of usability and UX according to our respondents were used in their country.

Research Implications—Positive Developments but Challenges Still Ahead

The surveyed UX professionals were surprisingly positive about their organizational integration. It seems that the world may be truly changing in this respect compared to the previous studies. In addition, we see it as very positive that our respondents were well educated and experienced UX professionals who also occupied managerial positions in their organizations. There seemed to be big groups of UX professionals in organizations with several professionals working on the same projects, their involvement ranged from the very early phases until the end, and the people in their organizations being quite aware and knowledgeable about usability and UX. Hence, there seems to be a clear difference to the earlier studies that report that usability and UX professionals complain about the perceived low priority of usability and UX issues in their organizations (de Lima Salgado et al., 2016; Gulliksen et al., 2004) and the lack of understanding about usability and UX in the organizations (Hussein et al., 2012; Ji & Yun, 2006).

However, almost all respondents identified challenges and targets for improvement in their work. Unfortunately, they emphasized the same old issues that have been emphasized for decades. The UX professionals also in this study wanted their organizations to give more priority to and emphasis on UX activities and to have better understanding and knowledge about UX. They also complained about insufficient resources such as time, money, and equipment and the lack of communication with developers in their organizations. However, unlike the previous studies (Ardito et al., 2011; Boivie et al., 2006; Gulliksen et al., 2004) reporting that UX professionals wanted more support from top management, UX professionals in this study were so positive about organizational support that only one third of the UX professionals reported the lack of organizational or management support as one of the main challenges that they encounter in their work. On the other hand, when UX professionals in this study were asked about the changes they thought were necessary to improve the effectiveness of their work, more support from upper management was the most popular answer. Here we see opportunities for further work. One interesting issue is that many UX professionals seem to be in managerial positions nowadays. We hope they can make a difference here: UX professionals do not exist only in power-weak entry positions, but they may be influential, senior people who should be able to make a difference in their organizations. We expect to see positive developments in this respect in the future.

Furthermore, there seems to be a potential for post-graduate HCI training programs and/or more UX community building. On one hand, a significant number of UX professionals in this study reported that they did not receive not only enough formal education but also any vocational education about UX. On the other hand, most of the UX professionals had limited theoretical background about UX activities as many were only aware of a very few and popular theories, frameworks, or methods. Getto and Beecher (2016) pointed out that “UX education within academia is a difficult endeavor to undertake, but a necessary one” (p. 162). Usability professionals who receive education/training on HCI have more awareness and knowledge about usability theories (Clemmensen, 2003). Therefore, it seems that in order to have more and in-depth knowledge about UX, there should be more education or training opportunities for UX professionals to improve their professional skills. Given the situation described above, there seems to be a persistent need for local and international UX communities to develop career models for UX professionals that ensure international outlook and a high level of knowledge and use of HCI theories, methods, and tools.

In summary, the main contribution of this study is the assessment of the development of the UX profession through a comprehensive survey instrument built up on the previous survey studies and a comparison of the findings with results of the previous survey studies in a comprehensive way.

Limitations

Three limitations should be noted in interpreting the results of this study. First, this study is about UX professionals work practices in five countries, of which no one country is an English-only speaking country, and the study results may in subtle ways be specific to these five countries beyond the already discussed sampling issues and rationales. For example, UX professionals work practices in Malaysia may be shaped by the general working culture in Malaysia, which may be different from countries where UX began, such as the USA. Then again, we maintain that there is a need in the global UX community to acknowledge the great cultural and geographical variety involved. In this study, we managed to include emerging yet already strong UX communities located in different parts of the world. We do acknowledge the need for investigating UX professionals work practices even within a larger range of countries, and in particular how UX work practices are integrated into local cultural settings. Second, our survey was longer than previous surveys of UX professionals work practices (see Table 2), and this may have led to potentially more bias and lower completion rates than for previous surveys. However, our rationale for this was that we wanted to compare across the previous surveys and hence include the questions that the previous surveys used. Furthermore, we argue that for the UX professionals as a respondent group, long questionnaires are less likely to lead to low response rates because these respondents (a) are familiar with the practices and challenges of survey research and should therefore be more tolerant toward lengthy questionnaires and (b) should be interested in the survey topic as it is about their own profession and practice and development of them. This latter point was also emphasized to the respondents in the motivating texts we used in the invitations to participate: We emphasized the importance of this data collection endeavor for the profession (and implicitly the respondents’ career), and we used several reminders. These means may well have matched or even exceeded the boost in the response rate that would have been obtained by shortening the survey. However, we acknowledge that additional data and analysis are needed to provide solid arguments for scientific representativeness of the UX professionals’ responses. Third, the data that we present in this paper is from 2016, and UX professionals’ work practices may have changed since then. Hence our results do not entirely reflect the current situation: Likely this is the case for the use of tools, as new tools appear frequently, and for learning on the job, as new ways of learning have recently received popularity (e.g., MOOCs). However, many issues relating to the UX profession and work practice do not change very fast. Actually, many issues relating to the UX work practices, integration into organizations, maturity of usability in a country, and more, have remained for decades and are still highly relevant. Nevertheless, we acknowledge that continuous studies on the UX professionals’ work practices and in particular updates on their tool use and online learning-on-the-job are of value. We look forward to other studies coming after this one.

Conclusion

In this paper, we have integrated fragmented findings from various survey studies of UX professionals’ work and developed and conducted a comprehensive survey to assess UX work environment and practices. Our findings showed that UX professionals have considerable work experience and strong UX expertise self-confidence. Individually they know about usability and UX, and they typically employ about five HCI theories, frameworks, or methods in their work and use one to three different tools and techniques in their tasks such as prototyping and collecting user feedback. On the organizational level, UX is involved from the early to late stages and is well known within all levels of the organization. On the country level, UX communities seem to include few of the UX professionals as members, while usability and UX have become common terms in the countries of this survey.

Future interview studies with UX professionals are warranted to gain an in-depth understanding of their work practices, challenges, and organizational realities. In this study, data were analyzed descriptively to understand UX professionals’ work practices. Future research should pay more attention to the cross-cultural and cross-country differences and similarities of factors faced by the countries to determine the effects of these factors on the integration of UX into organizational projects. Furthermore, future research should contribute to an HCI-wide common theoretical framework on the relation between organizational IT development and the UX profession.

Tips for Practitioners

Based on this study, we make the following recommendations to usability and user experience practitioners:

• The usability and UX field evolve with the development of new concepts, methods, and tools that become available to you as professional. Participate in training programs and community events in order to keep up with the development of the field and adopt the best practices as illustrated in this study.
• There may still be challenges ahead to integrate usability and UX work in various projects in your organization. Ensure you are aware of these challenges in your project and organization and adopt a proactive attitude to overcome possible problems. For example, ensure you communicate with all project stakeholders (developers, upper management, customers, users) to avoid misunderstandings and to clarify the benefits of usability and UX work for the project and organization.
• If in your organization or project, usability and UX work is not optimally integrated or supported, you can use this paper to provide upper management a broader perspective of how other companies and projects integrate usability and UX work.
• Participating in survey studies on usability and UX work practices is one way you as a professional can contribute to improving the state of the art in this field.
• In order to expand the UX culture in your country, taking an active part in building and developing a UX community is important.
• Try adopting a diverse theoretical ground by referring to different theories, frameworks, and methods in the HCI literature.
• As UX managers, provide extra resources in terms of time, money, and equipment for UX professionals in your organization.
• Support communication with different teams such as developers. The coherent relationship between project members who may have a different mindset and background will play an important role in the success of the project.

Acknowledgments

We thank Amélie Roche from Ecole Nationale Superieure de Cognitique (ENSC), Bordeaux Polytechnical Institute, France, for collecting the data from the French participants.

References

* Refers to studies included in the 26 survey studies presented in Table 2.

Ardito, C., Buono, P., Caivano, D., Costabile, M. F., & Lanzilotti, R. (2014). Investigating and promoting UX practice in industry: An experimental study. International Journal of Human-Computer Studies, 72(6), 542–551.

* Ardito, C., Buono, P., Caivano, D., Costabile, M. F., Lanzilotti, R., Bruun, A., & Stage, J. (2011, July 7–9). Usability evaluation: A survey of software development organizations. Proceedings of the 23rd International Conference on Software Engineering & Knowledge Engineering (pp. 282–287). SEKE’2011, Miami Beach, FL, USA.

* Bak, J. O., Nguyen, K., Risgaard, P., & Stage, J. (2008, October). Obstacles to usability evaluation in practice: a survey of software development organizations. 5th Nordic Conference on Human-Computer Interaction: Building Bridges (pp. 23–32). NordiCHI08, Lund, Sweden. https://doi.org/10.1145/1463160.1463164

* Bekker, M. M. & Vermeeren, A. P. O. S. (1996). An analysis of user interface design projects: information sources and constraints in design. Interacting with Computers, 8(1), 112–116.

Bevan, N., Carter, J., & Harker, S. (2015, August 2–7). ISO 9241-11 revised: What have we learnt about usability since 1998? In M. Kurosu (Ed.), Human-computer interaction: 17th international conference (Vol. 9169, pp. 143–151). HCI 2015: Los Angles, CA.

Boivie, I., Gulliksen, J., & Göransson, B. (2006). The lonesome cowboy: A study of the usability designer role in systems development. Interacting with computers, 18(4), 601–634.

Borgholm, T. & Madsen, K. H. (1999). Cooperative usability practices. Communications of the ACM, 42(5), 91–97.

Bruun, A., Larusdottir, M. K., Nielsen, L., Nielsen, P. A., & Persson, J. S. (2018, September). The role of UX professionals in agile development: A case study from industry. In Proceedings of the 10th Nordic Conference on Human-Computer Interaction (pp. 352–363). NordiChi ’18, Oslo, Norway.

* Bygstad, B., Ghinea, G., & Brevik, E. (2008). Software development methods and usability: Perspectives from a survey in the software industry in Norway. Interacting with Computers, 20(3), 375–385.

Cajander, Å. (2010). Usability–Who cares? The introduction of user-centred systems design in organisations [Doctoral Thesis, Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Human-Computer Interaction]. http://uu.diva-portal.org/smash/record.jsf?pid=diva2%3A310201&dswid=8140

Cajander, Å., Gulliksen, J., & Boivie, I. (2006, October 14–18). Management perspectives on usability in a public authority: A case study. In Proceedings of the 4th Nordic Conference on Human-Computer Interaction: Changing Roles (pp. 38–47). NordiCHI ’06, Oslo, Norway.

Cajander, Å., Larusdottir, M., & Gulliksen, J. (2013). Existing but not explicit-the user perspective in scrum projects in practice. In P. Kotzé, G. Marsden, G. Lindgaard, J. Wesson, & M. Winckler (Eds.), INTERACT 2013, Part III. LNCS (Vol. 8119, pp. 762–779). Springer, Heidelberg.

Catarci, T., Matarazzo, G., & Raiss, G. (2002). Driving usability into the public administration: The Italian experience. International Journal of Human-Computer Studies, 57(2), 121–138.

* Chilana, P. K., Ko, A. J., Wobbrock, J. O., Grossman, T., & Fitzmaurice, G. (2011). Post-deployment usability: A survey of current practices. In Proceedings of the International Conference on Human Factors in Computing Systems (pp. 2243–2246). CHI 2011, Vancouver, BC, Canada.

Christensen, S. Y. (2018) Designing user experience design: Determining the role of UX in software organizations. [Master thesis, Radford University].

Clegg, C., Axtell, C., Damodaran, L., Farbey, B., Hull, R., Lloyd-Jones, R., Nicholls, J., Sell, R. & Tomlinson, C. (1997). Information technology: A study of performance and the role of human and organizational factors. Ergonomics, 40(9), 851–871.

* Clemmensen, T. (2003, September 1­–5). Usability professionals’ Personal interest in basic HCI theory. In Conference on Human-Computer Interaction (pp. 939–646). INTERACT’03, Zurich, Switzerland.

* Clemmensen, T. (2005). Community knowledge in an emerging online professional community: The case of Sigchi.dk. Knowledge and Process Management, 12(1), 43–52.

de Lima Salgado, A., Amaral, L. A., Freire, A. P., & Fortes, R. P. M. (2016, September). Usability and UX practices in small enterprises: Lessons from a survey of the Brazilian context. In Proceedings of the 34th ACM International Conference on the Design of Communication (Article No. 18, pp. 1–9). SIGDOC ’16, Silver Springs, MD, USA. https://doi.org/10.1145/2987592.2987616

* Dillon, A., Sweeney, M., & Maguire, M. (1993). A survey of usability engineering within the European IT industry. In J. Alty, S. Guest, & D. Diaper (Eds.), HCI’93. People and Computers VII. Cambridge University Press.

Dzida, W. (1996). International usability standards. ACM Computing Surveys (CSUR), 28(1), 173–175.

Elliott, N., & Kling, R. (1996, January 3–6). Organizational usability of digital libraries in the courts. In Proceedings of the 29th Hawaii International Conference on System Sciences (Vol. 5, pp. 62–71). HICSS-29, Wailea, HI, USA.

Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and purposive sampling. American Journal of Theoretical and Applied Statistics, 5(1), 1–4.

Getto, G., & Beecher, F. (2016). Toward a model of UX education: Training UX designers within the academy. IEEE Transactions on Professional Communication, 59(2), 153–164.

* Gould, J. D. & Lewis, C. (1985). Designing for usability: Key principles and what designers think. Communications of the ACM, 28(3), 300–311.

* Grudin, J. & Poltrock, S. E. (1989). User interface design in large corporations: Coordination and communication across disciplines. In Proceedings of SIGCHI Conference on Human Factors in Computing Systems: Wings for the Mind (pp. 197–203).

* Gulliksen, J., Boivie, I., Persson, J., Hektor, A., & Herulf, L. (2004, October). Making a difference: A survey of the usability profession in Sweden. In Proceedings of the 3rd Nordic Conference on Human-Computer Interaction (pp. 207–215). NordiCHI04, Tampere, Finland.

* Gunther, R., Janis, J., & Butler, S. (2001). The UCD decision matrix: How, when, and where to sell user-centered design into the development cycle. OVO Studios, Usability Labs and Services. Retrieved January 20, 2017, from http://www.ovostudios.com/upa2001/.

Hertzum, M. (2010). Images of usability. International Journal of Human-Computer Interaction, 26(6), 567–600.

Hussein, I., Mahmud, M., & Md Tap, A. O. (2014, September 2­–5). A survey of user experience practice: A point of meet between academic and industry. In Proceedings of the 3rd International Conference on User Science and Engineering. i-USEr, Shah Alam, Malaysia.

* Hussein, I., Mahmud, M., Md Tap, A. O., & Laura, J. (2013). Does user-centered design (UCD) matter? Perspectives of Malaysian IT organizations. International Journal of Interactive Digital Media, 1(1), 71–78.

* Hussein, I., Mahmud, M., Md Tap, A. O., & Osman, A. (2012, July 9–12). User Experience Design (UXD): A survey of user interface development practices in Malaysia. In Proceedings of the Southeast Asian Network of Ergonomics Societies Conference. SEANES, Langkawi, Malaysia.

Hussein, I., Hussain, A., Mkpojiogu, E. O. C., & Mahmud, M. (2019). A UX community of practice: Design goals, practice motivations and values. International Journal of Advanced Science and Technology, 28(10), 21–29.

Iivari, N. (2006). ‘Representing the User’ in software development-a cultural analysis of usability work in the product development context. Interacting with Computers, 18(4), 635–664.

Inal, Y. & Guner, H. (2016). Understanding software developers’ awareness and knowledge about user experience and usability. Pamukkale University Journal of Engineering Sciences, 22(5), 384–389.

International Standard Organization. (1998). Ergonomic requirements for office work with visual display terminals (VDTs) – Part 11: Guidance on usability. (ISO Standard No. 9241-11:1998). https://www.iso.org/standard/16883.html

International Standard Organization. (2010). Ergonomics of human-system interaction – Part 210: Human-centred design for interactive systems. (ISO Standard No. 9241-210:2010). https://www.iso.org/standard/52075.html

* Jääskeläinen, A. & Heikkinen, K. (2010, October 17). Divergence of user eXperience: Professionals vs. end users. In Proceedings of the First International Workshop on the Interplay between User Experience and Software Development (pp. 8–12). I-UxSED 2010, Reykjavik, Iceland.

* Jerome, B., & Kazman, R. (2005). Surveying the solitudes: An investigation into the relationships between human computer interaction and software engineering in practice. In A. Seffah, J. Gulliksen, M. C. Desmarais (Eds.) Human-Centered Software Engineering — Integrating Usability in the Software Development Lifecycle (Human-Computer Interaction Series, Vol 8., pp. 59–70). Springer, Dordrecht

* Ji, Y. G., & Yun, M. H. (2006). Enhancing the minority discipline in the IT industry: A survey of usability and User-Centered design practice. International Journal of Human-Computer Interaction, 20(2), 117–134.

Katre, D., Orngreen, R., Yammiyavar, P., & Clemmensen, T. (2010, October 7-8). Human work interaction design: Usability in social, cultural and organizational contexts. In the Proceedings of the Second IFIP WG 13.6 Conference (Revised Selected Papers, Vol. 316, Springer). HWID 2009, Pune, India.

Kou, Y., & Gray, C. M. (2018, January). Towards professionalization in an online community of emerging occupation: Discourses among UX practitioners. In Proceedings of the 2018 ACM Conference on Supporting Groupwork (pp. 322 –334). GROUP ’18, Sanibel Island, FL, USA.

Kujala, S., Roto, V., Väänänen-Vainio-Mattila, K., Karapanos, E., & Sinnelä, A. (2011). UX curve: A method for evaluating long-term user experience. Interacting with Computers, 23(5), 473–483.

Larusdottir, M., Gulliksen, J., & Cajander, Å. (2017). A license to kill–Improving UCSD in Agile development. Journal of Systems and Software, 123, 214–222.

* Law, E. L.-C., Roto, V., Hassenzahl, M., Vermeeren, A. P., & Kort, J. (2009, April 4–9). Understanding, scoping and defining user experience: A survey approach. In Proceedings of the 27th International Conference on Human Factors in Computing Systems (pp. 719–728). CHI 2009, Boston, MA, USA.

Law, E. L.-C., van Schaik, P., & Roto, V. (2014). Attitudes towards user experience (UX) measurement. International Journal of Human-Computer Studies, 72(6), 526–541.

* Lizano, F., Sandoval, M. M., Bruun, A., & Stage, J. (2013). Usability evaluation in a digitally emerging country: A survey study. In P. Kotzé, G. Marsden, G. Lindgaard, J. Wesson, M. Winckler (Eds.) Human-Computer Interaction – INTERACT 2013. INTERACT 2013. Lecture Notes in Computer Science (Vol. 8120, pp. 298–305). Springer, Berlin, Heidelberg.

* Mao, Y. J., Vredenburg, K., Smith, P. W., & Carey, T. (2005). The state of user-centered design practice. Communications of the ACM, 48(3), 105–109.

Marghescu, D. (2009). Usability evaluation of information systems: A review of five international standards. In W. Wojtkowski, G. Wojtkowski, M. Lang, K. Conboy, C. Barry (Eds.) Information Systems Development (pp. 131–142). Springer, Boston.

Marsden, N., & Holtzblatt, K. (2018, April). How Do HCI professionals perceive their work experience?: Insights from the comparison with other job roles in IT. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (Paper No: LBW522). CHI ’18, Montreal, QC, Canada.

McCarthy, J., & Wright, P. (2004). Technology as experience. Interactions, 11(5), 42–43.

Poltrock, S. E. & Grudin, J. (1994). Organizational obstacles to interface design and development: two participant-observer studies. ACM Transactions on Computer-Human Interaction, 1(1), 52–80.

Rajanen, D., Clemmensen, T., Iivari, N., Inal, Y., Rızvanoğlu, K., Sivaji, A., & Roche, A. (2017, September 25­–29). UX professionals’ definitions of usability and UX–A comparison between Turkey, Finland, Denmark, France and Malaysia. In Proceedings of the IFIP Conference on Human-Computer Interaction (pp. 218–239). INTERACT 2017, Mumbai, India.

Rajanen, M., & Iivari, N. (2007, September 10–14). Usability cost-benefit analysis: How usability became a curse word? In Proceedings of the 11th IFIP Conference on Human-Computer Interaction (pp. 511–524). INTERACT 2007, Rio de Janeiro, Brazil.

* Rauch, T. & Wilson, T. (1995). UPA and CHI surveys on usability processes. ACM SIGCHI Bulletin, 27(3), 23–25.

* Roche, A., Lespinet-Najib, V., & André, J. M. (2014). Use of usability evaluation methods in France: The reality in professional practices. In the Proceedings of the 3rd International Conference on User Science and Engineering. i-USEr, Shah Alam, Malaysia.

* Rosenbaum, S., Rohn, J. A., & Humburg, J. (2000). A toolkit for strategic usability: Results from workshops, panels, and surveys. In the Proceedings of the Conference on Human Factors in Computing Systems (pp. 337–344). CHI ’00, The Hague, The Netherlands.

Smith, A., Joshi, A., Liu, Z., Bannon, L., Gulliksen, J., & Li, C. (2007). Institutionalizing HCI in Asia. In C. Baranauskas, P. Palanque, J. Abascal, S. D. J. Barbosa (Eds.) Human-Computer Interaction. INTERACT 2007. Lecture Notes in Computer Science (Vol. 4663.) Springer, Berlin, Heidelberg.

Sørensen, C., & Al-Taitoon, A. (2008). Organisational usability of mobile computing—volatility and control in mobile foreign exchange trading. International Journal of Human-Computer Studies, 66(12), 916–929.

Tractinsky, N. (2018). The usability construct: A dead end? Human–Computer Interaction, 33(2), 131–177.

* Vredenburg, K., Mao, J. Y., Smith, P. W., & Carey, T. (2002, April). A survey of user-centered design practice. In the Proceedings of the SIGCHI Conference on Human factors in Computing Systems (pp. 471–478). CHI ’02, Minneapolis, Minnesota, USA.

* Vukelja, L., Müller, L., & Opwis, K. (2007). Are engineers condemned to design? A survey on software engineering and UI design in Switzerland. In C. Baranauskas, P. Palanque, J. Abascal, S. D. J. Barbosa (Eds.) Human-Computer Interaction – INTERACT 2007 Lecture Notes in Computer Science (Vol. 4663). Springer, Berlin, Heidelberg.

Wale-Kolade, A. Y. (2015). Integrating usability work into a large inter-organisational agile development project: Tactics developed by usability designers. Journal of Systems and Software, 100, 54–66.

Wale-Kolade, A., & Nielsen, P. A. (2016). Apathy towards the Integration of Usability Work: A Case of System Justification. Interacting with Computers, 28(4), 437–450.

* Zhou, R., Huang, S., Qin, X, & Huang, J. (2008). A survey of user-centered design practice in China. In Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (pp. 1885–1889). 2008 IEEE, Singapore.

Appendix: Usability / UX Professionals’ Practice, Knowledge, and Tools

An appendix including the survey questions can be found in the PDF of this article.

Published in: in Volume 15, Issue 4,