Transforma logo

AIoT Chipsets: enhancing the connected devices ecosystem

NOV 22, 2024 | Paras Sharma
 
region: ALL vertical: ALL Artificial IntelligenceInternet of ThingsEdge Computing

The emergence of advanced artificial intelligence (AI) applications is acting as a catalyst for the development of advanced and sophisticated AI chips. In this blog, we will provide a brief overview of the AIoT chipsets market (as discussed in detail in our ‘AI Chipsets: cloud AI drives scale, AIoT drives innovation’ report), with a focus on edge AI chipsets and a brief introduction to a range of emerging edge AI chipset vendors.

While superior computational performance has traditionally been the primary focus for chip makers, the diverse requirements of AIoT applications demand a balance among computational power, memory, latency, and energy efficiency to meet domain-specific needs. Meanwhile, the rise of smaller, application-specific language models, which require less computing power than large language models, is driving the development of specialised edge AI and AIoT chips.

Cloud AI and edge AI chips

The processing of large datasets is often undertaken using AI processing in the cloud, often demanding high computational power, while on-device or processing at the edge requires real-time processing of data and demands low-power consumption and high-performance chips, resulting in increased complexities and intricacies for the development of specialised chips.

With the growing number of IoT devices and the integration of AI into these devices, the demand for AIoT applications will grow and so, the development of low-cost, energy-efficient and high-performance chips will accelerate. From autonomous vehicles to factory robots, edge devices must often deliver instantaneous responses.

The evolution of AI on SoCs (Systems-on-Chips) can efficiently support a plethora of real-time compute-intensive applications. This requirement for specialised AI chips will support a proliferation of companies developing AI SoCs for different use cases.

A balancing act of power, performance, and area

AIoT chipmakers are under immense pressure to strike a balance between power, performance, and area, while designing an AI chip integrated with billions of transistors etched together. The performance requirements of chips in different edge or AIoT contexts can vary widely; for instance, autonomous vehicles demand higher performance and can accommodate higher power consumption for image processing while a smart camera processor may require a lower cost chipset that is more power-efficient.

Environmental factors are also shaping the Edge AI and AIoT SoC market, with research focusing on achieving high computing performance with low latency and minimal energy consumption. For instance, Fraunhofer IPMS in partnership with Robert Bosch GmbH, the Indian Institute of Technology in Kanpur and the Technical University of Munich, developed a chip design optimised for AIoT applications, that uses ferroelectric field effect transistors (FeFET) that can store data without power supply and can deliver up to 885 TOPS/W (Trillion or Tera Operations per Second per Watt).

The proliferating vendor landscape for Edge AI, or AIoT, chips

With the increasing adoption of IoT, many use cases demand real-time data processing near the source to enable instantaneous responses. This is especially important as connected devices are often deployed in remote locations with limited connectivity, making cloud-based processing challenging. Moreover, some IoT applications, like autonomous vehicles, are extremely sensitive to latency, where even minimal delays can result in critical failures. Thus, a plethora of companies are venturing into the Edge AI chips market landscape. Discussed below are some of the key Edge AI, or AIoT, chip vendors.

  • Hailo Technologies. Established in 2017, Hailo is a provider of specialised AI chips for running AI workloads on edge and on AIoT devices. The company boasts a rich portfolio of edge AI processors for a variety of AI use cases such as object detection, running LLMs on edge devices, and semantic segmentation. The company offers the Hailo-15 family of AI vision processors for video and image processing, Hailo-8 AI accelerators for edge devices to run deep learning applications, and generative AI accelerators to run LLMs.
  • Mythic. Headquartered in Texas (US), Mythic is a provider of edge AI and AIoT-focussed power-efficient chips using a single-chip analogue compute-in-memory architecture without requiring the need for DRAM. The company offers the M1076 Analog Matrix Processor which is capable of delivering up to 25 TOPS for managing edge AI applications. The processor consists of a 4-lane PCIe 2.1 interface with up to 2GB/s of bandwidth and can support up to 80 million weights for AI workloads.
  • Nvidia. Nvidia offers the Jetson platform to support robotics and embedded edge AI applications, supported by the JetPack SDK for software development. Meanwhile, the Drive AGX platform is intended to support automated driving functions and in-cabin experience. The Clara for Medical Devices platform supports processing of streaming data in real-time.
  • Perceive. Acquired by Amazon in August 2024, Perceive is a provider of edge AI and AIoT processing chips, with a focus on edge processing of LLMs. The company offers low-power edge processing chips which are designed for a variety of applications including connected cameras and appliances. Ergo2, the company’s latest generation of edge AI chip, has a footprint of 7mm x 7mm and does not require external DRAM. The company claims that the chip can run inferences on 30 fps video feeds with a power consumption of just 17mW.
  • SiMa.AI. SiMa.ai is a provider of embedded machine learning (ML) systems-on-chip (SoCs), allowing customers to perform entire applications on a single chip. The company targets AIoT use cases which place a priority on low power consumption, specifically in the range of 5W-25W. According to SiMa, its chips can work as a single-edge platform for AI applications spanning across computer vision and multimodal generative AI.
  • Syntiant. Headquartered in California (US), Syntiant is a provider of low-power processors for edge AI and AIoT applications, and as of March 2022, the company had shipped more than 20 million edge AI chips. The company focuses on acoustic event detection and video processing used in industries such as in security to offer real-time data processing with minimal latency. The company’s latest chip, called NDP250, is optimised for speech recognition.

Conclusion

The edge AI and AIoT markets are becoming more fragmented to match the diverse needs of AIoT applications, with defined use-cases and a priority placed on optimising cost, power consumption and processing power for specific use cases resulting in significant levels of innovation and opportunities for new market entrants.

Watch the replay of our AIoT webinar

If AIoT is a topic of interest to you, check out the replay of our 21st November 2024 webinar, 'AIoT: the impact and opportunities of combining AI and IoT' where our analysts assess the increasing overlap between AI and IoT, the opportunities associated with AIoT and the new challenges it presents, including sharing our forecasts of the emerging AIoT market.

AIoT webinar image.jpg

NOV 21, 2024 Previous Post
Enriching AutoTech: Nine domains of change
All Blog Posts