Edge AI expert Jack Ferrari shares his insights on why this tech is a good fit for manufacturing operations.
For manufacturing facilities, an IoT strategy for equipment data collection and analysis is an essential step toward digital transformation, providing the data required to generate data-driven insights, predictive maintenance and other benefits. However, connecting machines and equipment to the internet raises challenges for IT teams, including security concerns, data storage, bandwidth and computing power.
To tackle these challenges, many teams consider whether to process data at the edge, or in the cloud. Historically, while the edge has benefits such as processing speed, low bandwidth and security, cloud solutions offer unmatched computing power. To use complex computing solutions such as AI models, you may think cloud is the only option. However, vendors like MathWorks are proving that AI at the edge can provide the best of both worlds.
Engineering.com recently spoke with Jack Ferrari, Edge AI Product Manager at MathWorks, to learn more about Edge AI and how manufacturers use it.
Engineering.com (Eng.com): What are the benefits of edge devices compared to ‘dumb’ sensors that just send data straight on to a pc or to the cloud?
Jack Ferrari (JF): Running AI models locally on edge devices instead of the cloud brings several benefits. First, the inference time (or, response time) of the model can be greatly reduced, as data is no longer required to be shuffled back and forth over the Internet. Secondly and for the same reason, edge AI enhances data privacy (all data fed to/from the model stays on the device) and makes applications more reliable (less prone to network outages). Finally and thirdly, edge AI can lower costs by reducing/eliminating cloud hosting and storage fees.
Eng.com: What trends and technologies drive edge AI’s adoption across industries?
JF: The large and rapidly growing number of IoT devices across industries (expected to reach 40 billion by 2030) are generating massive amounts of data at the edge, driving the need for local processing to handle data efficiently, reduce latency and lower cloud costs. Advancements in hardware, like AI accelerators and software, like new model compression techniques, are working in tandem to enable the adoption of edge AI.
Eng.com: Do you think industry-wide adoption of edge technology is driven by devices becoming more cost effective, energy efficient and/or powerful, or are edge trends driven by other, more strategic factors?
JF: A combination of the two is influencing edge AI’s adoption: On the technology side, new hardware platforms are being designed with AI workloads in mind. Recent advancements in microcontrollers (MCUs), digital signal processors (DSPs) and AI accelerators (like Neural Processing Units (NPUs) are enabling the deployment of models that were previously impossible to consider running at the edge. Besides simply having greater horsepower, these new chips are being optimized to execute AI workflows with greater energy efficiency. At the same time, the ecosystem of software tools used to compress AI models and program them on edge devices is becoming more robust and user-friendly, making the technology more accessible. Strategically, edge AI is enabling companies to differentiate their products in new ways. For example, by adding real-time processing and decision-making capabilities, enhancing device security by handling all data processing locally and enabling the personalization of AI models through techniques like on-device learning.
Eng.com: In many industries, security and IP concerns hold back adoption of AI tools. Is this seen in manufacturing?
JF: Security and IP concerns can indeed impact AI adoption in manufacturing. However, processing sensitive data at the edge (close to where it originates), rather than transmitting it to the cloud, can reduce exposure to potential breaches, offering a way to address these concerns.
Eng.com: What benefits can engineers expect when using edge AI?
JF: There are four primary benefits to using edge AI:
- Lower latency: AI models can deliver predictions and classifications more quickly, which is crucial for engineers working on time-sensitive applications. This rapid response can enhance user experience and enable real-time decision- making, particularly in scenarios where milliseconds matter, such as autonomous vehicles or live data monitoring.
- Lower costs: Reducing data transmission and storage fees, along with improved energy efficiency, leads to significant cost savings. For engineers, this means more budget can be allocated to other critical projects or resources and they can ensure their systems have higher uptime and availability, even during network outages, thus maintaining service continuity.
- Enhanced privacy: By processing incoming data on-device instead of transmitting it to the cloud, engineers can ensure higher levels of data privacy and security. This is particularly beneficial in industries where sensitive information is handled, as it reduces the risk of data breaches and ensures compliance with privacy regulations, making it easier to protect user data.
- Improved reliability: As edge AI does not rely on continuous cloud connectivity, it can continue to function during network outages. This ensures that critical operations, like monitoring and control systems in manufacturing, remain active even if the cloud connection is lost.
Eng.com: What are common challenges associated with edge AI?
JF: While it’s becoming easier to implement AI models on the edge, organizations should be mindful of several challenges that accompany the technology:
- Resource constraints: Edge devices typically have limited processing power, memory and storage. Complex AI models may run slowly or not at all. To mitigate this, proper care should be taken in selecting model architectures that are well-suited for the edge device they will eventually be deployed to. Additionally, models can be further optimized for edge deployment with compression techniques like projection, pruning and quantization.
- Model deployment: Translating AI models from the high-level languages where they are defined and trained (like Python or MATLAB) to low-level languages that can be compiled to run on edge devices (like C or C++) can be challenging. MathWorks tools facilitate this process by automating the conversion, ensuring efficient deployment on a diverse range of hardware. For example, Airbus used GPU Coder to deploy deep learning models, trained in MATLAB for defect detection, onto embedded GPUs. GPU Coder automatically translated their MATLAB code into the corresponding CUDA code, which could be compiled and run on their embedded system.
- Model maintenance: After deploying AI models to the edge, organizations should have a plan for keeping them updated over time. This can take several forms:
- Over-the-air (OTA) updates, where new model files and weights are sent to edge devices over a network connection.
- On-device training (or, incremental learning), where models are updated and refined directly on the device using local data, allowing for personalization without the need to communicate with the cloud.
Eng.com: Are there edge AI use cases that are applicable across multiple industries?
JF: Beyond classic examples like image classification, object detection and semantic segmentation, one interesting application of edge AI MathWorks is seeing used across industries are virtual sensors (or, software sensors). AI-based virtual sensors can be used to infer sensor data that might be difficult or expensive to measure directly, by analysing data from other sensors in real-time. One great example is for estimating the state of charge of a battery. While difficult to measure directly, it can be inferred from the values of other, more easily attainable values, like current, voltage and operating temperature. By using AI models trained on historical battery performance data, the virtual sensor can predict the state of charge more accurately and adapt to changes in battery health and usage patterns, providing real-time insights without the need for additional hardware. Virtual sensors are applicable to multiple industries, including automotive, aerospace, manufacturing and healthcare. As another example, Poclain Hydraulics used MATLAB to design and deploy a neural network-based virtual sensor for monitoring the temperature of motors used in power machinery.
Eng.com: Do you think that the trend toward AI and cloud-based IoT systems make custom-built systems into dinosaurs, in other words would a manufacturer be ‘crazy’ to consider building a solution in-house?
JF: While AI and cloud-based IoT systems offer scalable and cost-effective solutions, the decision to build a system in-house depends on a manufacturer’s specific needs and capabilities. Companies with specialized requirements or strong internal expertise may benefit from custom solutions, while others might prefer the speed and lower upfront costs of cloud-based platforms. Ultimately, the choice hinges on factors like customization, security and time to market.
Eng.com: As the complexity increases of the devices we use to monitor and maintain our equipment, is there a growing need to monitor and maintain the edge and IT devices as well? How do we do that?
JF: Yes, as the complexity and number of AI-enabled edge devices increases, so does the need for monitoring and maintenance. Over time, the input data to AI models can drift or differ significantly from the data they were originally trained on, negatively impacting model accuracy and performance. Organizations should anticipate this and consider approaches to continuously update their models, whether through OTA updates or incremental learning.
