A group of researchers from Universitat Politècnica de Catalunya in Barcelona and the telecom company Huawei have retooled an artificial intelligence technique used for chess and self-driving cars to make OTNs run more efficiently.
OTNs require rules for how to divvy up the high amounts of traffic they manage and writing the rules for making those split-second decisions becomes very complex. If the network gives more space than needed for a voice call, for example, the unused space might have been better put to use ensuring that an end user streaming a video doesn’t get “still buffering” messages.
The researchers’ new approach to this problem combines two machine learning techniques: The first, called reinforcement learning, creates a virtual “agent” that learns through trial and error the particulars of a system to optimize how resources are managed. The second, called deep learning, adds an extra layer of sophistication to the reinforcement-based approach by using so-called neural networks, which are computer learning systems inspired by the human brain, to draw more abstract conclusions from each round of trial and error.
“Deep reinforcement learning has been successfully applied to many fields,” said one of the researchers, Albert Cabellos-Aparicio. “However, its application to computer networks is very recent. We hope that our paper helps kickstart deep-reinforcement learning in networking and that other researchers propose different and even better approaches.”
So far, the most advanced deep reinforcement learning algorithms have been able to optimize some resource allocation in OTNs, but they become stuck when they run into novel scenarios. The researchers worked to overcome this by varying the manner in which data are presented to the agent.
After learning the OTNs through 5,000 rounds of simulations, the deep reinforcement learning agent directed traffic with 30 percent greater efficiency than the current state-of-the-art algorithm.
One thing that surprised Cabellos-Aparicio and his team was how easily the new approach was able to learn about the networks after starting out with a blank slate.
“This means that without prior knowledge, a deep reinforcement learning agent can learn how to optimize a network autonomously,” Cabellos-Aparicio said. “This results in optimization strategies that outperform expert algorithms.”
With the enormous scale some optical transport networks already have, Cabellos-Aparicio said, even small advances in efficiency can reap large returns in reduced latency and operational costs.
Next, the group plans to apply their deep reinforcement strategies in combination with graph networks, an emerging field within artificial intelligence with the potential to transform scientific and industrial fields, such as computer networks, chemistry and logistics.
