Multimodal Human–Machine Interfaces for Coordinated Control of Heterogeneous Autonomous Systems
This PhD project explores a new paradigm for the control and supervision of heterogeneous autonomous systems, focusing on multimodal human–machine interfaces (HMI) for coordinated operation of ground, aerial, and maritime (naval) drones. As autonomous systems are increasingly deployed in defence, security, policing, and emergency-response operations, the ability for human operators to intuitively and safely command multiple autonomous platforms becomes a critical challenge.
While autonomy has advanced significantly, current systems often rely on complex, fragmented interfaces that require operators to manage each platform separately through graphical interfaces, manual controllers, or domain-specific command languages. This leads to high cognitive workload, reduced situation awareness, and increased risk of error—particularly in time-critical or safety-critical missions.
The project aims to develop natural, unified, and robust interaction mechanisms that allow operators to command and supervise heterogeneous autonomous systems using speech, gestures, body movement, gaze, and wearable interfaces. By focusing on intent-based control and multimodal interaction, the research seeks to simplify human–machine communication while preserving safety, reliability, and operator authority.
The project explicitly targets dual-use applications, addressing both defence and security contexts as well as civilian use cases such as policing support, search-and-rescue, and disaster response.
The project will address the following key areas:
- Design of multimodal human–machine interfaces for autonomous systems
- Voice-based and gesture-based command and control
- Intent-based task specification for heterogeneous robotic platforms
- One-to-many and supervisory control paradigms
- Human factors, cognitive workload, and situation awareness
- Safety mechanisms and error mitigation in human–autonomy interaction
Previous research and study goals
Recent developments in autonomous systems increasingly emphasize distributed and cooperative operation, where multiple platforms collaborate across different domains (land, air, sea). Despite advances in autonomy, human operators remain responsible for mission intent, exception handling, and ethical oversight.
Traditional control interfaces do not scale well to such scenarios, as they require detailed, low-level interaction with individual platforms. This project investigates how multimodal interaction techniques can enable operators to express high-level intent rather than low-level commands, allowing autonomous systems to handle execution while humans maintain supervision and control.
The research goal is to design scalable, intuitive, and safety-aware interaction frameworks that reduce operator workload, improve trust in autonomous systems, and support effective coordination across heterogeneous platforms.
Responsibilities and (foreseen) tasks:
- Research and design multimodal human–machine interaction concepts for autonomous systems
- Develop voice- and gesture-based control mechanisms for heterogeneous robots
- Implement intent-based command representations and control abstractions
- Integrate multimodal interfaces with robotic software frameworks (e.g., ROS-based systems)
- Design and conduct user studies to evaluate usability, workload, and performance
- Perform simulation-based and experimental validation campaigns
- Publish results in peer-reviewed journals and conferences
Applicants should fulfil the following requirements:
- A degree in engineering sciences (preferably in robotics, computer science, AI, or related fields)
- A clear interest in autonomous systems and human–machine interaction
- Excellent English communication skills
- Strong computer science background
- Programming skills in C++ and Python
- Strong writing and analytical skills
- Ability to work independently and as part of an international research team
The following experience is beneficial:
- Experience with ROS and robotic middleware
- Experience with human–machine interfaces or interaction design
- Experience with autonomous robots (UGV, UAV, USV)
- Knowledge of sensors, perception, and control systems
- Familiarity with experimental evaluation and user studies
Supervisors:
Main supervisor: Tenured Associate Professor Vladimir Kuts: Department of Mechanical and Industrial Engineering
Co-Supervisor: Tenured Associate Professor Raivo Sell: Department of Mechanical and Industrial Engineering
Co-Supervisor: Major Janar Pekarev: Deputy Commander of the Force Transformation Command for Innovation
Tallinn University of Technology (TalTech) is an international scientific community with approximately 9,000 students and 2,000 employees; it is one of the largest universities in Estonia, the leading EU country in digitalisation. The university's strengths are broad multidisciplinary study/research interests, a modern research environment, and strong collaboration with international educational and research institutions. TalTech is aiming to be an organisation leading the way to a sustainable digital future.
The Department of Mechanical and Industrial engineering focuses on the engineering side of self driving vehicles, developing new coatings and additive manufacturing developments. The curriculums on Bachelor, Masters and Doctor level have hundreds of graduates each year. We also provide engineering services for industry and our partners, starting with modelling and finishing with production optimization.
For information about the admission process, please visit the PhD Admission homepage
Applications can be submitted from 01.06.2026 to 30.06.2026
