C2Land

In the current project phase, an optical localization system for eVTOLs is being developed. The information-rich visual environment is captured by several cameras in the visible and near-infrared range. From this, various features of the vertipad are detected using computer vision and artificial intelligence methods. Typical landing field features such as lines, circles and letters are used and investigated, as well as QR code like ArUco-markers. Geometric relationships are then exploited to perform accurate self-localization.

While satellite-based navigation systems generally achieve sufficient accuracy to perform precise landings, the validity of the position solution cannot be verified without additional complex ground infrastructure. Thus, with current (SBAS) procedures aircraft can only be guided to a minimum altitude of about 60 meters (200 ft). With the independent optical solution, the integrity of the entire localization solution can be checked autonomously and continuously. In this way, the confidence in the overall solution can be increased to the point where it can be used until touchdown. Thereby, the integrity requirements are determined dynamically in a top-down approach and in a technology-independent holistic concept. The so-called Total Capability Approach (TCA) considers the available error budget across all systems and enables better utilization of the same throughout the individual systems while maintaining the same level of security.

In a further step, the Institute of Flight System Dynamics at the Technical University of Munich takes the dynamic properties of the complex and novel aircraft into account when generating approach trajectories. Using a specialized flight controller, control commands can be sent to the aircraft to perform autonomous landings. To ensure safety between traffic participants during landing, f.u.n.k.e Avionics GmbH is developing a transponder-based anti-collision system and provides avionics components for the exchange, recording and telemetric transmission of system data.

In collaboration with a german air taxi company, the C2Land system will be installed in an eVTOL as an experimental vehicle and will be further developed and validated in several test campaigns. At the end of the project duration, a fully automated landing of this aircraft without additional ground infrastructure shall be demonstrated.

Until now in regular operations, fully automatic landings for fixed-wing aircraft have also only been carried out in poor visibility conditions and with the use of costly ground infrastructure. In order to exploit the potential of satellite-based inertial navigation (GNSS/INS) for landing independent of further infrastructure, an optical positioning solution for on-board autonomous independent integrity monitoring was developed.

Using cameras operating in the visible and infrared range and specially developed image processing algorithms, the runway is optically detected and the relative position of the aircraft is determined. With the help of the precise and reliable navigation solution, a specialized control system can guide the approach and safely touch down on the centerline. Together with the Technical University of Munich and with the aircraft manufacturer Diamond Aircraft Industries GmbH (DAI), it was possible to demonstrate a fully automatic landing with optical-assisted GNSS/INS navigation for the first time.

The optical localization and guidance system is now being expanded to include other flight phases, such as taxi, takeoff and cruise, and the range of applications is being extended to include night operations, among other things. New developments in the field of computer vision and neural networks are constantly being investigated and the system optimized.

The Institute of Flight Guidance has extensive experience in the development and application of image processing algorithms in the aviation domain. Both, feature-based methods and machine learning methods are used in multi-stage and intelligently coupled algorithms. Algorithms are preferably investigated and used which, according to the current standard, make an aviation certification of the system possible.

For runway detection, the image is first converted into a gray-scale image with a defined depth. Special nonlinear methods ensure minimal loss of information, in particular in the case of infrared images. The image is then binarized, for example using the Otsu method. Edges or contours are extracted using Canny Edge Detection, among other methods. Straight lines are filtered. Finally, special filter criteria identify the left and right runway boundaries, the threshold and centerline. These elements can be identified much more robustly at long distances than corner points or other features. By exploiting the relationships of projective geometry, the own position in relation to the runway can be determined. This approach is supported by feature trackers and optical flow algorithms that can compensate for a momentary loss of runway visibility. Additionally, SLAM techniques are also being investigated to further enhance situational awareness. Filtering and fusion algorithms produce a joint robust position solution.

The existing data recording system and algorithms are also ideal for collecting training and validation data for machine learning. Deep neural networks (DNNs) have already been successfully trained for runway detection.

For air taxis the landing site is not yet fully defined, but will be similar to those of helicopters. In principle, the existing procedures can be used for recognition with minor modifications. Markers can be filtered by width and color. Since corner points are easily recognizable, a position can be calculated by setting up the Perspective-n-Point (PnP) problem and a bundle adjustment. If QR code like ArUco markers are also used, confidence in correct detection can be further increased by reading the binary code. Circular landing sites can be localized by geometric circle-ellipse relations.

Development is supported by suitable libraries and frameworks such as OpenCV and TensorFlow. We are constantly investigating new methods and appreciate participation in the context of student assignments and student assistantships.