Universidade NOVA de Lisboa Background
The large diffusion of lightweight drones, fostered by the lowering of the prices, has unlocked their employment in several professional activities, such as law enforcement, environmental protection, monitoring of critical installations, etc. Within these contexts, the capacity to retrieve the position and the metric of a target in real-time is of the utmost importance. This capability is not available unless the imaging system is fitted with other devices such as a laser pointer/range finder, which is very uncommon for lightweight drones.
Laser pointer/range finders are instead regular components for imaging systems installed into larger aerial platforms, such as helicopters or military-grade drones. They can measure the distance to the target, which can be used to pinpoint its location and estimate its metrics. However, the laser pointer/range finder retrieves only a punctual measure (located in the image center). Thus, only the targets located in the image center can be accurately assessed. The estimation in other locations of the image is usually performed assuming a flat terrain, which is a simplification that may introduce a relevant error.
In some cases, even the distance to the target measured by the laser may not be accurate enough to meet the requirements of the surveillance. For example, estimating the body stature of the people in a scene, which is one of the most distinctive biometric traits for person reconnaissance, requires achieving a centimetric accuracy.
Technology Overview
The technology works with a single image acquired by a monocular optical camera, whose acquisition parameters must be known. The camera may work in either visible or infrared spectrum. Besides straight vertical measures, the method can perform measurements in any orientation, including horizontally or along vertical planes (see ).
Firstly, the method requires correcting the distortion of the image due to lens curvature. However, the method is structured in a way that only a few pixels need to be corrected, not the entire image. If the length of one or more ground features appearing in the image can be retrieved (e.g., length of a wall), this information can be used to reduce the number of required parameters in the calculation (i.e., focal length, pixel pitch, camera roll and pitch, number of pixels spanning each target), thus allowing to reach a centimetric accuracy. The calculation can be performed independently from the position of the target in the image, and multiple targets can be analysed concurrently. Existing maps or a Geographic Information System (GIS) can be used to retrieve the length of ground features, whose process can be automatized via Artificial Intelligence algorithms.
In those cases where no ground features are available, a digital model of the terrain could be used instead. However, in this case, both intrinsic and extrinsic camera parameters are required, which may introduce degradation of the accuracy.
Finally, one of the outcomes of calculation is the camera-to-target distance, which can be used to retrieve the position (coordinates) of the target(s).
Stage of Development
Technology Readiness Level (TRL) 4.
The method has been extensively tested in controlled environments for validation and accuracy analysis. It confirmed that the accuracy may be below +/-2cm using a ground feature as reference. Ground Sample Distance (spatial resolution) and distance to the target influence the accuracy, although the highest impact is given by the number of pixels spanning the ground feature (the higher the number, the better).
When ground features are not available, the overall accuracy is closely related to the quality of the positioning system (e.g., GPS). When the terrain is very uneven, the resolution of the digital terrain model may play an important role as well.
Benefits
The method does not require special equipment to work or special procedures, even a regular-market drone fitted with an optical camera can be used. No need for special procedures to collect the data or dedicated training as well. This is one the first technologies capable to retrieve the position and the metrics of a target in real-time with simple aerial cameras.
Advanced imaging systems fitted with lasers or other devices can equally benefitting of the method, because it allows to operate in every location of the image with the same level of accuracy. In fact, the method is capable to retrieve the distance to the target(s) in every point of the image (not just the centre) to pinpoint its location and estimates its metrics. This approach would greatly increase the accuracy, especially for operations conducted in areas where the terrain is not flat.
The method can be used to generate “measuring tool” to manually perform measurements during the flight. Alternatively, the metrics of all the targets of interest (e.g., height of the people) can be retrieved automatically displayed on screen. This can be achieved by employing machine learning algorithms for the automatic reconnaissance of relevant targets. A tool capable to retrieve such information in real-time would greatly enhance the situational awareness during the flight for efficient decision-making.
The very limited time needed for the correction of the distortion allows to obtain quick results (real-time) even when multiple targets are considered. Moving targets can be analysed because only one image (or video frame) is required for the calculation.
The accuracy is highly dependent on parameters such as GPS error, but it can drastically improve up to centimetric level by considering reference ground features, which are very common in locations such as urban areas. This technology is one of the few in the market – probably the only one- capable to reach this level of quality. It is therefore deemed the only solution employable when accurate assessments with aerial camera are needed such as reconnaissance of people via biometric traits like body stature.
Applications
The capacity to retrieve the position and the metric of a target in real-time is of the utmost importance for any kind of surveillance activity, such as law enforcement and environmental protection, although it may be also required in any kind of flight activity, even leisure.
Opportunity
Available for exclusive or non-exclusive licensing.