As telecom operators wrestle with stagnating revenues and soaring infrastructure costs, the rapid rise of AI is exposing the limits of the rigid, hardware-centric networks built for 3G, 4G and 5G. Shekar Ayyar, Chairman and CEO of US-based networking firm Arrcus, argues that carriers, having been “too late” to capitalise on the cloud wave, now face a far more consequential transition.
This time, he warns, they must adopt software-defined and disaggregated networks or risk losing the ability to monetise AI altogether.
In an exclusive interview with TechObserver.in’s Mohd Ujaley, Ayyar says the centre of gravity in AI infrastructure is shifting rapidly from compute to networking as power limits and distributed inferencing push workloads toward the edge. He sees India playing a defining role in this evolution, both as a rapidly expanding market for next-generation data centres and telecom networks and as a deep talent base where Arrcus plans to double its engineering footprint in the next 12 to 15 months.
“Telecom operators missed the cloud wave,” Ayyar says. “The AI wave can be a saving grace if they make their infrastructure flexible and monetisable.”
Edited Excerpts:
What makes this moment particularly ripe for software-defined and disaggregated networking in the telecom sector?
In the telecom space people have historically built very large capital investment infrastructures for every generation of communications like 3G, 4G, 5G etc, and increasingly they are finding out that two things are happening. One, the infrastructure is so rigid that it is not allowing them to create new services that are monetisable and therefore recoup their return on investment.
The second is that the cost of this infrastructure is also getting more and more exorbitant, and so if all that they can do is go from 3G to 4G, 4G to 5G and 5G to 6G, and keep charging consumers for data and voice, the consumers are not going to keep paying more and more money for that in order for the telecoms providers to recoup this. So as a result, the best thing for them to do is think about creative models where their infrastructure is going to allow them to create and deploy new services that they can monetise.
Now they could have done this when the cloud wave came into being as an example, but honestly the telecom operators were too little too late in the cloud world and they essentially failed. Now there is an AI wave that is coming upon us, and if these operators can figure out how to take their infrastructures and make them basically flexible to allow them to monetise the AI wave, then that could be a good saving grace for them as they go forward.
So this is one reason why I feel like a software defined disaggregated network infrastructure and architecture, particularly for the communications and telecom world, is not just important, it is essential, because if you do not do this you will essentially die.
This is the telecom view of the world. If you take a broader lens, the broader lens basically says that the world is moving from what used to be cloud computing to now AI computing. So everything is now becoming about AI whereas previously everything was about the cloud. That means that large models are being trained and then these large models are going to be used by agents, by edges, by inference points to go and draw conclusions which are going to make all our lives better. That is the promise of AI.
Under that promise of AI, if you look at what the infrastructure requirement is, if you build a large data centre with rigid cages that have purpose built equipment that cannot be changed, then you are building an infrastructure that may be okay for some form of training an AI algorithm, but the moment you want to go from training to inferencing, or the moment you start getting constraints on things like power, you are now hard coded or straight jacketed into an environment that you cannot change.
So the demands of AI are increasingly going to start making inferencing as well as distributed computing a necessary part of that architecture, and that is another big reason why a software-defined disaggregated network paradigm is going to be important and critical. Like I said it is not just important, it is an essential part of what people need to start building their networks around.
So these are two big reasons. One is telecom carriers that are spending huge amounts of capital investment and then scratching their heads on how to monetise that, and on the other side the AI world moving from training large scale data centres to now inferencing and starting to face constraints like power.
What would you advise CIOs of large legacy enterprises who want to embrace AI but are held back by monolithic infrastructure?
Two things I would say. One is infrastructure transformations do not need to be disruptive in the sense of operational disruption. They can be disruptive in the sense of technology disruption but you can always be brownfield. You can always create introductions and then take layers of your technology and upgrade them rather than upgrading everything all at once. So that could well be the answer to some of these corporations that are steeped in a legacy environment and looking to get the benefits of agility from new technology.
Frankly any customer where Arrcus technology gets introduced is often a brownfield environment. People go in and say look, I am still going to go and buy whatever a gazillion dollars of routers and switches from companies like Cisco, Juniper and Arista, but here is side by side an environment where I am going to start introducing Arrcus, and as our confidence in you starts building up we are going to give you more and more real estate to play in so that you become a bigger part of this infrastructure. This is exactly how we operate today.
So companies like you mentioned, whether they are State Bank of India or LIC or educational institutions in India, they could all adopt this approach. In fact, we cannot yet publish the names, but some of the largest financial institutions in India are starting to move over to Arrcus in a similar vein.
Now the other side of it is, I actually have a lot more optimism that even a country like India where there are such legacy infrastructures can in fact leapfrog when the requirement is high. We have seen this before. I grew up in an era when getting a landline took 13 years. Today India has 1.2 billion mobile connections. That’s the power of reinventing infrastructure at scale.
Companies need to reinvent themselves. Maybe they keep their old stuff and then they start something new over here. This becomes the next generation architecture. It is completely software defined. It is built for agility and can not only look to make incremental improvements but look to leading the world and being an example of next generation architecture and technology. I have my optimism that at least some subset of companies in India will opt for that approach.
Many organisations assume that buying GPUs is synonymous with building AI capability. What critical infrastructure priorities are they missing?
A lot of people are inferring that AI means GPUs and that by ordering a bunch of GPUs they have AI and they have basically done their job, which is far from the truth. So on the infrastructure side I would say the focus and the energy is quickly going to shift from compute to network, because once you have capacity constraints, you can either have overcapacity of GPUs because you ordered too much and you do not know how to use them, or you can have power constraints that tell you that you cannot put 300 racks in this place, you can only put 24, and so you have to combine two or three powered areas to get to one single consolidated architecture.
Or you can have inferencing that is getting done at the cell sites and cell towers or in autonomous vehicles and cars, and therefore by design these are smaller modular compute capacities that need to be networked together. So I am saying do not assume that buying a GPU is your job done. Be smarter than that and ask the question of what you need more than the compute capacity, and in fact how you can address the requirements of constrained compute capacity to get what you want out of AI by building a smarter network.
One other example here is that we are working very closely with Nvidia and Broadcom on their SmartNICs, and you can use SmartNICs to offload things from processors and therefore release more capacity in your core compute stack, and that allows you to get more from less. These are all techniques in networking that people need to get adapted so that they can use their infrastructure more smartly rather than just buying and stacking up more and more GPUs.
Now on another dimension, infrastructure is not the only part of the problem. You need to know how to use the infrastructure, how to create the right agents, how to make these agents a part of your fabric of whatever it is that you are running as a business, how to train people to take advantage of AI and not be afraid of it, and how to create the right level of operational visibility and security around these parameters to enjoy the benefits of AI without being burdened by the ills of bad use of AI. So the operational complexity of using AI and adapting to it and making sure that you are getting the benefits is equally as important as the infrastructure itself and building the right kind of platform.
These are two parts of the problem that people are not actually getting their heads around as much as they need to, because to your earlier point they are basically saying I am just going to buy GPUs and or rent capacity on GPUs and that is AI for me. But I would strongly encourage people to look at this more holistically so that they can start spending their time, energy and money in these other areas to make AI successful.
How is AI transforming the role of the edge, and what innovations do you expect in edge networking?
Previous iterations of the edge were more focused on things like IoT and edge computing. I think what AI does is, I like to describe this as one fundamental requirement, which is that you have to know how to use it for proper inferencing as opposed to just training your model. That is point number one, and therefore use of AI in inferencing necessarily means that those inference points are going to be distributed all over the globe, and you need to know how to network those points at the edges. You need to know what gets done natively at the edge. You need to know how edges communicate with each other and how an edge communicates with a mothership where most of the training is being done.
These are the reasons why an edge based networking fabric or network infrastructure is going to become more important as people recognise that AI is not just about training large models but about using that trained model to draw conclusions at all of these edge points.
The second reason why edges are going to become very important in the context of AI is constraints on the compute environment. Whether that is the cost of AI chips or the cost of power to power up these AI chips, you are not going to have the luxury of building large giant data centres with racks of GPUs consuming unlimited amounts of power. That is not going to be a reality because everybody is going to have a limitation on power.
Everybody is going to have to figure out how to take the most power from what you can get in different points in the network and then collectively bring that to your advantage to build a large trained model. That by design means that you are going to have your compute capacity located at different points wherever you can find power, and then you have to network these things together to pretend to be a larger compute capacity pool that can give you the results you want for training.
So, the requirements of inferencing and the power constraints on training are both going to make the edge a very important and required part of the fabric for AI, and therefore by design network infrastructure that brings these edges into being is going to become very important for AI success.
You recently expanded in India. What are your strategic priorities for the Indian market?
Customer opportunity in India is growing through the roof right now, because if you look at the three areas that we focus on as customers, one is data centre and network operators, the second is telecom companies and the third is larger enterprises. All these three groups in India are growing to a point where they need their infrastructures refreshed.
Data centre capacity, I mean every large data centre company in the world is looking at India as an opportunity for building data centres. Telecom environments, I mean it is one of the largest markets alongside China for telecom services to be delivered to mobile phones. The third one is enterprises. Notwithstanding your comment about enterprises in India having large legacy infrastructures, I am actually quite bullish about the opportunity for some of them at least to reinvent themselves.
Therefore, I think these three groups of companies are very well present and represented in India, and we see this as a huge market opportunity for us and for anyone else that is bringing agile infrastructure to the market.
As we look ahead, I always think that India can be both a producer as well as a consumer. I think the last big production of technology from India was the IT wave which then led to a lot of outsourcing, but largely that has been a process where there is speculation that it needs to get reinvented to adapt itself to the AI wave.
If you look at all the requirements of AI, there is still a tonne of things that need to be done to train models and to make sure they are properly being used, and I think India could be a huge hotbed of sourcing this sort of technology from India and making it available to the world. When you do that, underneath all this you need the right kind of adaptable infrastructure, and that is what excites us a lot.
How is Arrcus planning to scale its workforce and footprint in India?
Our India operation even today is basically half our global operation. We are largely India and US focused and it is fifty-fifty. As we go ahead, I see the India operation expanding substantially, and my goal is that in the next 12 to 15 months we should double our capacity in India.
We have one big centre in Bengaluru and we are now starting to explore other places to start our next centres in India. From a revenue and bookings goal standpoint, I would like our India operation to contribute at least 10 percent of our global bookings. So we have very big aspirations for both the technology growth as well as our go to market in India.
Our company is a very technology rich company, so anytime we do more expansion you can expect that 70 to 80 percent of that expansion capacity will be core engineering and technical disciplines, and the remaining 20 percent will be a combination of sales and go-to-market onboarding.
In Bengaluru and other centres around Bengaluru we see a lot of networking talent that can be very helpful in our growth. However, our bar is very high, so we are looking to bring on board very selectively people with good experience. They can be fresh college graduates to people with 15 to 20 years of experience. We will take the entire gamut, but they all need to be jumping in saying this is an excellent opportunity to join a pre IPO company at the peak of its growth, and we believe that this is a great time to be doing that in India.
You previously led cloud and telecom strategy at VMware. Which parts of that experience shape what you are building at Arrcus?
To me the liberation of the application workloads from the underlying hardware, which is what VMware created with the hypervisor layer, was a breathtaking moment. It led to cloud infrastructures and containerised models and everything you hear about in compute technology today. It had its roots in that one fundamental aspect of separating out the application workloads from the underlying hardware.
In the network we are still 30 to 40 years behind. We have not had that VMware moment in the network. So to me, if I can take that same learning from what the compute industry went through and bring that to the benefit of the networking industry and show them that network functions and workloads can be liberated from the underlying hardware infrastructure by creating this layer of network operating system, which we do at Arrcus, that would be phenomenal. That is really my taking of learnings from there and bringing it into this new world.
