Researchers create autonomous drones that chat like people and grab with hands
Capable of 'thinking, entity control and environmental awareness'
Chinese researchers have claimed a breakthrough in autonomous robotics by giving a swarm of drones the ability to chat intelligently in human language as they collaborate over complex tasks.
They foresee their chatting drones being used in security, disaster recovery and logistics, where they could apply an apparently unique ability to use "group-chat" conversation among themselves and their human overseers, and to navigate real-world, open environments.
Language is essential to human thought, communication and hence, civilisation, researchers from Northwestern Polytechnical University in Shaanxi, China, posited in a reprint of their study last week.
If machines used language likewise, "to interact, hear, think, express what they see", might they become "truly intelligent"?, they speculated.
Their response to this question, based on the Chinese large language machine learning model Scholar·Puyu, effectively gave the drones "brains", they claimed.
Using this as the basis for a "group chat control framework", the researchers, from the Polytechnic's Institute of Opto-electronics and Intelligence, said they created a means for multi-machine and human-machine dialogue, breaking interaction barriers between human and machine.
Understanding the physical world through travel
Large language models (such as GPT4) have raised hope for general artificial intelligence, they said. But "reading thousands of books is not as good as travelling thousands of miles. Large models need to truly 'walk' into the physical world in order to truly understand complex tasks and solve practical problems."
Led by Professor Li Xuelong, the team had set out to recreate the profound autonomy intrinsic to human cognition. They created a model of human cognition consisting of three interacting dimensions: "thinking, entity control and environmental awareness". Scholar·Puyu, built by four Chinese and Hong Kong Universities with Shangtang Technology, is reportedly based on a three-dimensional evaluation framework of "ability-task-indicator".
Those three attributes of human intelligence - dialogue, environmental perception and autonomous control - were the defining capabilities of autonomous drone clusters, the researchers said. Combined with the "group chat-style" control framework, it gave drones the capability for intelligent interaction. It gave them the power of "active perception" as well, including the ability to navigate autonomously in open environments, and the means to handle complex tasks.
Hands capable of grasping
Given drones that understood human dialogue, people could better instruct them to do complex tasks. The system created a "group chat" between users, and drones that incorporated sounds, images and drone status into natural language as well.
The researchers gave the drones the power of "active environmental perception" by fusing data from multiple sensors, and programming them with algorithms for visual positioning, for avoiding obstacles and for low-altitude search.
They gave the drones "hands, growing out and capable of grasping", turning them into flying robots. They also developed other such "effectors" not described in the public statement.
Given their collaborative control mechanism and environmental perception, the researchers claim they could make clusters of heterogeneous drones, the difference of each presumably defined by some specialised ability, that could then together search for a target, position it in space, and grab it.
Creating composite intelligent agents, and giving them the ability to handle complex tasks, was the focus of the LLM research.
"The large-model autonomous drone cluster is a successful attempt to apply the three-dimensional interaction model of biological intelligence - 'thinking, entity control and environmental awareness' - to autonomous agents, said the statement.