The challenges and opportunities of user experience in AR based applications

Augmented Reality (AR) and Virtual Reality (VR) are getting their momentum to become a major mass media. AR, as a supporting technology, has been around for many years. However, popular smart devices, such as smartphones, have so far failed to provide a friendly experience for AR applications. In recent years, applications such as Pokémon and Snapchat have changed the developers’ and people’s perceptions of AR applications. Nevertheless, there are still many challenges associated with AR from the user experience (UX) design perspective.

Transformation from 2D into 3D requires a mindset change

The major challenge that we have to overcome is the transformation from 2D into the 3D design. Despite living in the 3D world, the designers learned to communicate with the 3-dimensional world with the use of 2D design instruments. The designers used pens and papers to come up with an application prototype for the 3D world in the 2D environment.

The transformation is not a very straightforward task, as it requires a change in the designer’s mindset, e.g., the shift from the 2D world to the 3D world. In 2D design, we deal with only x and y-axis and the designers use objects such as text, icons, and animations for interacting with the system. Unlike in 2D, in 3D solutions, the designer has to deal with an additional z-axis which represents the depth of the objects. The transformation from 2D design to 3D design is actually associated with the following challenges:

1. How to combine the real and virtual images? This issue actually deals with human perception. For example, how can we create an illusion that matches the real and virtual image, such as depth and shadows? In simple words: “How could we fool the user’s senses?”
2. How could the interaction happen in a real-time environment? This issue deals with physical components. For example, how must a virtual object be presented in a real environment? Do we create objects based on audio or visual commands?
3. How are the objects presented and constructed in 3D? This issue deals with technology. For example, which technology supports and constructs the best illusions for the user in different contexts, e.g., on the move or in a car?
4. How to engage the user with the application? This issue deals with psychology. For example, how can we emotionally attach the user to the application for a sustainable usage?

Context of use is a key factor in AR design

The overall objectives of the 2D and 3D design, from the user experience perspective, are almost the same: the application must be easy to user, easily memorable, and leave a lasting experience. The users of the potential AR based applications have a big role. Therefore, identifying the users and investigating their needs are crucial elements in creating a robust application.

The major difference, however, in an AR oriented application is that the user interacts with the application in a real-time environment. Therefore, knowing the context of the use for the application is crucial, for example, whether the application is used while driving a car or while sitting in a coffee shop. Hence, the context of use affects significantly the design solutions.

Unlike in 2D, where the live environment has not greatly affected the application performance and user experience, in AR the knowledge and experience are actually constructed in the context where the user interacts with the application. Furthermore, the 3D volumetric that interacts with light and shadow and the animated media, such as images and videos, are the two main objects in AR. The Animated Media is the same in 3D as it is in traditional 2D media.

Nowadays, smartphones have become the main window into the AR world. This is, however, associated with some design considerations, such as, how should the users interact with the application; and how long do they have to interact with it, e.g., for drivers, voice commands would be preferable and safer than touch buttons.

How to create a positive experience?

Retaining the user’s continuous positive experience is yet another challenge that the designer must overcome. The AR is often constructed in a live situation where any obstructions such as lights, movements, and changes in the camera’s focus can result in a negative experience for the user. Hence, the designer must pay special attention to the context and environment where the live AR objects are constructed.

In addition, creating a positive experience mandates that the designer has to think about the hardware that the potential user is going to utilize, such as, sensors, displays, input/output devices, etc. For example, the interaction with smartphones is different from the interaction with Microsoft HoloLens. Finally, design theory helps significantly in coming up with an application that emotionally attaches the users and helps to create a lasting experience for them.

The design principles enable us to deliver the intended message efficiently and effectively to users. AR enables us to associate emotions with the application in a live context. We do not apply AR merely for aesthetic purposes or as a gimmick. Through AR technology, we pursue to make the application more delightful, personalized, and alive for our user. Hence, AR is a means to improve the emotional connection of the mobile application with users.

There is still room for development

User experience design for AR is still immature and needs further investigations. Further experiments are needed to come up with a proper AR UX guideline. How does AR make the application interface and interaction more comprehensive for users? For example, furniture manufacturers use AR to help users to assemble a product.

In Haaga-Helia we have an ongoing project (Adaptive Study Management Platform: Freedom through Personalization) to orient our freshmen to their new university even before they officially start their studies. Using our AR solution, the students in the Degree Programme in Business Information Technology (BITE) learn about Haaga-Helia’s common applications (Moodle, Peppi, MyNet) and the Pasila campus premises and key people. Finally, AR gamification is used to introduce the BITE curriculum to the new students.

Special thanks goes to Riitta Blomster for proofreading the text.

 

The writer Amir Dirin is a lecturer at Haaga-Helia University of Applied Sciences.