https://www.mdu.se/

Visual perception plays an important role for recognizing possible hazards. In the context of heavy machinery, relevant visual information can be obtained from the machine's surrounding and from the human-machine interface that exists inside the cabin. In this paper, we propose a method that classifies the occurring collisions by combining the data collected by the eye tracker and the automatic logging mechanism in the mixed reality simulation. Thirteen participants were asked to complete a test scenario in the mixed reality simulation, while wearing an eye tracker. The results demonstrate that we could classify the occurring collisions based on two visual perception conditions: (1) whether the colliding objects were visible from the participants' field of view and (2) whether the participants have seen the information presented on the human-machine interface before the collisions occurred. This approach enabled us to interpret the occurring collisions differently, compared to the traditional approach that uses the total number of collisions as the representation of participants' performance.

This paper investigates the possible visualization using transparent displays, which could be placed on the excavator’s windshield. This way, the information could be presented closer to operators’ line of sight, without fully obstructing their view. Therefore, excavator operators could acquire the supportive information provided by the machine without diverting their attention from operational areas. To ensure that there is a match between the supportive information and operators’ contextual needs, we conducted four different activities as parts of our design process. Firstly, we looked at four relevant safety guidelines to determine which information is essential to perform safe operations. Secondly, we reviewed all commercially available technologies to discover their suitability in the excavator context. Thirdly, we conducted a design workshop to generate ideas on how the essential information should look like and behave based on the performed operation and the chosen available technology. Fourthly, we interviewed seven excavator operators to test their understanding and obtain their feedback on the proposed visualization concepts. The results indicated that four out of six visualization concepts that we proposed could be understood easily by the operators and we also revised them to better suit the operators’ way of thinking. All the operators also positively perceived this approach, since all of them included at least three visualization concepts to be presented on the windshield.

The work presented in this dissertation is in the field of interaction design through visual interaction with information systems in heavy industrial vehicles within agriculture, forestry, and construction. The dissertation investigates how interaction technologies and the application of interaction design methods can be used to provide an efficient interaction and a good user experience for the operator interacting with the heavy vehicle information system. Having a proficient user interaction with these systems is becoming central due to the increasing levels of information- and computation-rich functionality. The possibilities given by new interaction technologies are pushing the frontiers of how and where interaction between humans and computers occurs. This development comes with both opportunities and risks, making the information exchange with the operator a decisive factor for a successful and safe vehicle operation.

This thesis covers the theoretical, methodological, technological, and empirical aspects of design research through interaction design in the heavy vehicle’s domain. A practice-based procedure, through sketching, is used to approach and bring design theories and design perspectives together into a model covering design research, process, and elements of design. The application area is approached by observations of operators, focusing on their attention to vehicle displays. This work presents empirical results with evaluation of the application of pop-up user study methods, and the use of online based user studies within the heavy vehicle domain. The limited attention to digital information providers, observed in the operator studies, led the work into the study and exploration of designs that could visualize information closer to the area in front of the vehicle. The aim was to improve the operator’s situation awareness, by showing warning and navigation information using transparent displays. The realization of these design prototypes contributed to the development of a novel mixed reality simulation environment. The resulting evaluation of the designs indicates an improved information detection, at a lower perceived effort level, using transparent display interfaces.

We have investigated the visualization of safety information for mobile crane operations utilizing transparent displays, where the information can be presented closer to operators' line of sight with minimum obstruction on their view. The intention of the design is to help operators in acquiring supportive information provided by the machine, without requiring them to divert their attention far from operational areas. We started the design process by reviewing mobile crane safety guidelines to determine which information that operators need to know in order to perform safe operations. Using the findings from the safety guidelines review, we then conducted a design workshop to generate design ideas and visualisation concepts, as well as to delineate their appearances and behaviour based on the capability of transparent displays. We transformed the results of the workshop to a low-fidelity paper prototype, and then interviewed six mobile crane operators to obtain their feedback on the proposed concepts. The results of the study indicate that, as information will be presented closer to operators' line of sight, we need to be selective on what kind of information and how much information that should be presented to operators. However, all the operators appreciated having information presented closer to their line of sight, as an approach that has the potential to improve safety in their operations.

There are more types of vehicles than the automobile. Many are used for purposes other than transporting passengers or goods. They are often dedicated to enable the user in performing specific manual tasks, in parallel to driving. Such heavy vehicles range from construction vehicles, such as excavators and articulated haulers, to agriculture vehicles, such as tractors and harvesters. They also include speciality vehicles such as lifts and cranes. Recent advances in information technology radically increases their productivity and safety. Moreover, heavy vehicles are increasingly sensor and software-driven, as well as connected and integrated with information systems. This development creates new interaction challenges and research areas. The aim of this workshop is to gather practitioners, researchers, and professionals who wish to explore the potential opportunities, identify research challenges, and innovate in the domain of heavy vehicles.

The paper presents a complementary method, called online video study, to conducting field studies in remote locations, by using available videos on YouTube. There are two driving factors for the online video study. Firstly, there are some occasions where conducting field studies are difficult, for example, due to the remoteness of the location where the research subject is located. Secondly, there is a growing interest among researchers to use available data on the internet as their research data source. To give a context, the study specifically investigates how operators of forest harvesters work in their natural settings. The online video study was started by collecting suitable videos on YouTube using certain criteria. We found 26 videos that meet our criteria, which also provide diverse samples of forest harvesters, operators, and working situations. We used five prior field studies, which investigated forest harvesters-related issues, to evaluate the feasibility of our approach. The results of the online video study method are promising, since we are able to find answers for research questions that we have predefined. The paper does not only contribute to the understanding of how operators of forest harvesters work in natural settings, but also the feasibility of conducting the online video study, which can be utilized when the research subject is located in remote locations.

Operating heavy vehicles, for instance an excavator, requires a high level of attention to the operation done using the vehicle and awareness of the surroundings. Digital transformation in heavy vehicles aims to improve productivity and user experience, but it can also increase the operators mental load because of a higher demand of attention to instrumentation and controls, subsequently leading to reduced situation awareness. One way to mitigate this, is to display information within the operators’ field of view, which enhances information detectability through quick glances, using mixed reality interfaces. This work explores two types of mixed reality visualizations and compares them to a traditional display setup in a simulated excavator environment. We have utilized eye-tracking glasses to study users’ attention to the task, surrounding awareness, and interfaces, followed by a NASA-RTLX questionnaire to evaluate the users’ reported mental workload. The results indicate benefits for the mixed reality approaches, with lower workload ratings together with an improved rate in detection of presented information.

The machinery used in industrial applications, such as in agriculture, construction, and forestry, are increasingly equipped with digital tools that aim to aid the operator in task completion, improved productivity, and enhanced safety. In addition, as machines are increasingly connected, there are even more opportunities to integrate external information sources. This situation provides a challenge in mediating the information to the operator. One approach that could be used to address this challenge is the use of augmented reality. This enables the system-generated information to be combined with the user’s perception of the environment. It has the potential to enhance the operators’ awareness of the machine, the surroundings, and the operation that needs to be performed. In this paper, we review the current literature to present the state of the art, discuss the possible benefits, and the use of augmented reality in heavy machinery.

The machinery used in industrial applications, such as in agriculture, construction, and forestry, are increasingly equipped with digital tools that aim to aid the operator in task completion, improved productivity, and enhanced safety. In addition, as machines are increasingly connected, there are even more opportunities to integrate external information sources. This situation provides a challenge in mediating the information to the operator. One approach that could be used to address this challenge is the use of augmented reality. This enables the system-generated information to be combined with the user's perception of the environment. It has the potential to enhance the operators' awareness of the machine, the surroundings, and the operation that needs to be performed. In this paper, we review the current literature to present the state of the art, discuss the possible benefits, and the use of augmented reality in heavy machinery.

Designing and testing new information and safety features for industrial vehicles do not need to involve the realization of high-fidelity and expensive simulators. We propose a low-cost mixed reality environment which allows for rapid development and rearrangement of a virtual and physical setup of a simulator for industrial vehicles. Our mixed reality simulator allows for safe testing of controls, information, and safety features to support drivers of industrial vehicles. In this paper, we test the implications of showing extra digital information to excavator drivers through a virtual environment, an external head-up display as well as a head-down display. Through user tests we have seen first indications that projected information through our mixed reality system and content on a head-up display is perceived as more helpful and intuitive than using head-down displays, when controlling our industrial vehicle simulator. Moreover, we have seen that the fear of overseeing an obstacle or other important information is lower when using a head-up display, in comparison to other tested visualization options. © Springer International Publishing AG, part of Springer Nature 2018.