A tower controller utters “wizz air nine kilo lima palanga tower good day cleared touch and go runway three four wind …” and inputs the same clearance content for the aircraft callsign into the flight strip system. Communicating the same information twice (verbally and electronically) is not very efficient and can be supported by an assistant based speech recognition (ABSR) system.
While the HAAWAII project focusses on En-Route and TMA traffic, the same algorithms and machine learning approaches can also be used for the ATC tower environment. To be more specific, it is used in the multiple remote tower environment of DLR’s exercise in SESAR2020’s “PJ.05-W2-97-ASR” solution.
The figure below shows DLR’s hardware setup and user interface of the controller working position. It consists of three rows of monitors for the outside view of three remote airports called Vilnius, Kaunas, and Palanga (red zoomed box at the top). Below are three radar screens plus a ground radar for the largest airport. The prototypic electronic flight strip system on the desk visualizes different columns to maintain flights and ground vehicles for different procedural (flight) phases in specific bays.
Coming back to the above-mentioned utterance: The ABSR system will immediately transform the utterance by the speech to text module (transcription) and output all recognized words in the outside view: “wizz air nine kilo lima palanga tower good day cleared touch and go runway three four wind two eight zero degrees six knots”. Meanwhile, the command extraction algorithms will transform the text to ATC concepts (annotation), i.e., the callsign, command types, values, units, etc. are extracted due to the European wide agreed ontology: “WZZ9KL GREETING, WZZ9KL STATION PALANGA_TOWER, WZZ9KL CLEARED TOUCH_GO 34, WZZ9KL INFORMATION WINDDIRECTION 280, WZZ9KL INFORMATION WINDSPEED 6 kt”.
The extracted ATC concepts are used to display the content of the speech recognition and understanding process below the transcription in the outside view and for the main purpose of flight strip maintenance in a simple form on the flight strip display (red zoomed box on lower right). The aircraft callsign will be shown in inverted colors directly after uttering. The icon for a “touch-and-go” as well as the runway name “34” will be highlighted in dark green to give the controller 10 seconds for checking and potentially correcting the ABSR output. The automatic maintenance of the electronic flight strips relieves the controller from the former manual task. Some details on procedures and trials are explained in this video: link to video.
The validation trials with five controllers from ON (Lithuania) and five controllers from ACG (Austria) took place from February 14 to March 3, 2022 at DLR TowerLab in Braunschweig, Germany. Results on command recognition rates, human factors questionnaires, and many more will be published after the trials. First results will also be presented at an online open day a few weeks after the trials.
Video: YouTube
Links with more information: SESAR , DLR, LinkedIn
