Clough’s paper addresses advancements in unmanned aerial vehicles (UAV) autonomy in terms of autonomous control levels. The paper specifically addresses the issue of how to quantitatively analyze or gather metrics on UAV autonomy. This is an interesting issue because how do you gather measurable data based upon a concept such as autonomy?
The first step in defining the metrics for autonomy is to define what autonomy is. There have been papers describing the different levels of autonomy, as in Parasuraman’s “A Model for Types and Levels of Human Interaction with Automation”. The key is to map these different levels of autonomy to some measurable criteria. The interesting piece about this article is that the author and his team only found two examples of where autonomy was mapped to some metric diagram.
The paper then brings up an interesting point about autonomy in general. If you are replacing a human, why not measure like one. This makes perfect sense in that we measure human activity and progress all the time. It would only make sense to apply the same domain of metrics to the automation system space since this system is essentially replacing the human operators. The insight is that if we are designing algorithms to replace pilot decision functions. If the machines replace humans, why not look to the human effectiveness community for metrics? The choice was to modify the OODA (observe, orient, decide and act) loop. The ACL metrics have been used successfully at the AFRL in developing plans and programs in autonomous UAV controls research.
The article itself does not seem to take itself seriously. There are numerous examples of sarcastic or humorous comments throughout the article. This is interesting because this is a published white paper that is addressing issues from a tactical military perspective at an Air Force Research Laboratory. One would think that this military laboratory would only release documents filtered through as professional as to not give a negative impression on the research activities.
In the end, the point was to have a method available to measure autonomy in a system. This is important since there will have to be a trust involved with the human components. If the humans can see actual measurable progress with autonomous systems, than they will be more apt to trust the system and allow more decisions to be delegated to it, freeing up the human operator to perform other functions.
Reference:
- Clough, Bruce T. “Metrics, Schmetrics! How The Heck Do You Determine A UAV’s Autonomy Anyway?” <http://www.dtic.mil/dtic/tr/fulltext/u2/a515926.pdf>
