Edge AI: Why the Future of AI Compute is at the Edge

AI has been the talk of the tech town over the last two years, and we’ve already seen some real utility value of AI-powered tools like ChatGPT, Midjourney, and more.

However, while the spotlight has been on generative AI and large language models (LLMs), these technologies are impractical for industries requiring real-time and local decision-making, such as manufacturing.

Edge AI seeks to address this challenge by processing massive datasets on compact computing platforms, using smaller, more efficient machine learning models to deliver real-time AI insights.

As experts note, AI computing processes should happen where it creates the most value for the enterprise. And for many industrial enterprises, this is close to where the data is generated – right at the edge.

Like never before, there’s a strong drive to innovate at the edge, evidenced in NTT Data’s 2023 Edge Advantage Report which surveyed 600 enterprises across multiple industries and showed “about 70% of enterprises are now using edge solutions to solve business challenges.”

But what really is edge AI and how is it going to change the future of AI compute?

Edge AI: Improving Operational Efficiency

Edge AI describes the processing of data when and where it’s generated (that is, at the edge of a network), rather than relying solely on remote cloud servers. Proponents of this approach state that by moving computing power to the edge, companies can access insights that help improve their operational efficiency.

Related:Microsoft, BlackRock Launch $30B AI Data Center Investment Fund

Kaihan Krippendorff, founder of Outthinker Networks and co-author of Proximity, believes “AI compute isn’t just the future, but it’s already here,” and adds that “manufacturing enterprises, in particular, are moving AI systems out of the cloud and onto the edge, enabling faster data processing and reducing latency and cloud-server costs.”

“Moving AI systems out of the cloud and onto the edge, particularly in manufacturing environments, results in faster processing of data from Internet of Things (IoT) devices and sensors, connected by local networks within factories. This reduces latency, cuts down on cloud-server costs, and opens up new possibilities for localized innovation.”

Edge AI is transforming manufacturing, bringing real-time data processing and decision-making directly to the factory floor (Image: Alamy)

But it’s not just Krippendorff who thinks edge AI is already here. Several other industry experts, like Paul Bloudoff, senior director of edge services at NTT, also share the sentiment.

“We are seeing several enterprises pushing for these solutions,” says Bloudoff. “The reason is that Edge AI brings actionable intelligence and real-time decision-making possibilities into operational technology environments, such as the factory floor.”

Related:How LLMs on the Edge Could Help Solve the AI Data Center Problem

Now, the edge AI market is rapidly expanding, and some key players driving this shift include NTT Data, Siemens, IBM, and Microsoft, with each bringing its unique approach to the market.

NTT, for example, recently launched its ultralight edge AI platform, which is a fully managed edge AI solution that “breaks down IT-OT silos, enabling advanced AI use-cases for industrial and manufacturing.”

Siemens, on the other hand, has developed an industrial edge platform that allows manufacturers to deploy AI applications directly on the factory floor. IBM’s edge application manager focuses on delivering AI solutions for edge devices in industries like healthcare, telecommunications, and automotive.

Real-World Applications of Edge AI

Edge AI is already being deployed across numerous industries, with early adopters reporting significant improvements in operational efficiency, cost reduction, and innovation capabilities. One of the most notable areas where edge AI is making an impact is in manufacturing.

With edge AI, manufacturers can now process data locally, enabling real-time monitoring, predictive maintenance, and intelligent decision-making at the factory level.

Related:Private Cloud vs. Public Cloud: Which Option is Cheaper?

For instance, IoT sensors on production machines can generate streams of data that are immediately analyzed by edge AI systems to predict potential equipment failures. This allows manufacturers to perform maintenance before a breakdown occurs, reducing downtime and increasing overall productivity. 

One company that has already successfully implemented edge AI in its manufacturing processes is the global home appliance and electronics maker, Haier. Leveraging edge AI at its local factories, Haier said it has optimized production processes and gained the ability to customize products for local markets more effectively.

This strategy of combining cloud processing with edge-based applications has become so successful that Haier now offers its integrated AI systems to other manufacturers under COSMOPlat, the company Haier established in 2017.

But edge AI’s impact isn’t limited to manufacturing. Smart cities are also using edge AI to enhance urban infrastructure and services, optimize traffic management, monitor public safety, and even track environmental conditions like air quality and weather patterns in real-time.

Hurdles on the Way to the Edge

Despite the growing interest in edge AI, its adoption hasn’t been without challenges. According to the 2023 Edge Advantage Report, “nearly 40% of those planning edge deployments expressed concerns about the capabilities of their current infrastructure to support the technology.”

Bloudoff acknowledges these concerns, noting that many enterprises face significant obstacles in integrating edge solutions with their existing IT and OT infrastructure.

Read more of the latest next-gen data center news

A key challenge is ensuring data from IoT devices, sensors, and machinery can be collected, processed, and analyzed seamlessly at the edge. This requires robust hardware and software solutions capable of handling the volume and complexity of data generated in real-time environments.

However, solutions like NTT’s ultralight edge AI platform have been designed to help simplify the deployment and management of AI applications at the edge. The platform’s auto-discovery feature scans the entire IT and OT environment to inventory assets, identify vulnerabilities, and streamline data collection.

“By automating the discovery and collection of diverse IoT and OT devices into a single data plane for real-time decision-making, we’re removing barriers to edge adoption,” says Bloudoff.

AI on the Edge and Sustainability

Beyond operational efficiency and innovation, edge AI also presents a unique opportunity for enterprises to meet their sustainability goals. Since edge AI processes data locally, it reduces the need for energy-intensive cloud-based computing, cutting down on the carbon footprint associated with long-distance data transmission and cloud storage.

As Bloudoff notes, enterprises that deploy AI applications at the edge are not only enhancing productivity but are also taking steps toward more sustainable operations. “By reducing energy consumption and network congestion, Edge AI is helping companies minimize their environmental impact while driving digital transformation,” he adds.

As sustainability becomes an increasingly important consideration for businesses worldwide, edge AI’s ability to deliver both operational and environmental benefits will likely be a driving force behind its continued adoption.