A User Interface Prototyping Tool for Exploring Supervisory Control of Automation During Event-Paced Scenarios

Abstract

Usability testing is a critical step in the process of agile iterative development. Static and lower-fidelity user-interface mock-ups can suffice during early phases of design or for user-paced environments, but higher-fidelity test beds are required to evaluate event-paced environments especially those exploring the supervisory control of automation. Increased fidelity is desirable as early as possible during the design process to explore the supervisory control design paradigm and generate more numerous and higher-quality user feedback. Complex AI-based systems are more difficult to test during early development phases. Static and low-fidelity prototypes typically cannot adequately represent the complex automation behaviors making usability more difficult to measure. A high fidelity user-interface prototyping approach was employed to simulate the capabilities of pending algorithms that are under development by multiple laboratories for a future intelligent air-defense system. The prototyping tool supported usability data collection by simulating the proposed automation and an initial supervisory control user-interface design within dynamic, group, air defense scenarios. This approach yielded high-quality user feedback early in the system design process even while being subjected to the constraints imposed by remote testing requirements due to COVID-19.