“Machine learning applications are increasingly common, and many associate them with high memory requirements, cloud computing servers, or highly parallel GPU architectures. Many programs do require expensive computational operations, which makes them incompatible with edge processing. However, companies also know that not all deep learning systems require this capability.
“
At the SENSOR + TEST 2022 conference held last month in Nuremberg, Germany, ST (STMicroelectronics) detailed the ISM330IS, the industry’s first sensor with an Intelligent Sensor Processing Unit (ISPU). ST announced the technology in early 2022, and the ISPU is a C-enabled programmable embedded digital signal processor (DSP) capable of running machine learning and deep learning algorithms. As such, it is the next stage of edge AI, or the “age of online life” as ST advocates. The ISM330IS includes a floating-point unit for unit-precision calculations, a first among motion sensors.
From an idea to a new generation of sensors, what challenges has ST overcome?
It’s critical to look back at the major innovations of the past four years to understand how we got here. It started when ST published a paper examining the feasibility of machine learning cores inside inertial sensors. Before this study, sensors only collected data. Therefore, all calculations have to be done on the microcontroller. The reasoning behind this system is relatively simple, inertial sensors are small, low-power devices. Adding a powerful processor not only violates these constraints, it also creates significant integration and manufacturing challenges. ST combines DSP and accelerometer with gyroscope without compromising processing power, memory storage or sensing accuracy.
What did the first sensor with a machine learning core accomplish?
The 2018 ST paper was groundbreaking because it addressed these challenges, and was later dubbed the LSM6DSOX product, the first-ever inertial sensor capable of accommodating eight decision trees in parallel to run machine learning algorithms. Therefore, applications can be run locally with very low power consumption. New applications are starting to appear, especially after integrating devices into SensorTile.box. For example, a cry detector could be created for a baby left in a car. Likewise, two UCL projects have used this sensor to develop stand/sit detection and a digital stethoscope.
What other sensors have a machine learning core?
LSM6DSOX is also the start of a new developer community. ST provides a machine learning core repository on GitHub and makes the Unico GUI software tool more accessible to help programmers who wish to take advantage of the machine learning core in the LSM6DSOX. In addition, ST has released more powerful sensors. More demanding applications such as virtual reality headsets can be targeted. ST also introduced the LSM6DSV16X with enhanced machine learning cores and a better performance-per-watt ratio for systems with tighter power constraints. As such, ST’s sensors with a machine learning core contribute in part to the next era of automation, and the ISM330IS opens an important new chapter in this saga.
From new processing cores to new applications
What is ISPU made of?
The ISPU of the ISM330IS provides 8 KB RAM for data and 32 KB RAM for applications. It also features a 32-bit RISC Harvard architecture running at 10 MHz, a four-stage pipeline, a floating-point unit, and a set of 16-bit length instructions optimized for neural network processing. The processor can raise an interrupt in as little as four clock cycles (Arm Cortex cores typically do it in 15 clock cycles), and it can also handle 16-bit multiplications in one cycle. It communicates with the host MCU using SPI or I2C. The developer simply loads the C code into the ISPU’s volatile memory when the host processor starts up.
FPUs enable applications to run inference at the edge with greater flexibility. Once the condition is met, the program throws an interrupt to the microcontroller. Again, this architecture enables the ISPU to increase performance compared to previous generation devices while still operating at the microwatt level. So it’s a major leap forward and a much more efficient system than the previous decision trees at the heart of machine learning. Furthermore, despite its powerful computing power, the ISM330IS still fits into the market standard 3mm x 2.5mm x 0.83mm LGA package. As a result, designers can adopt new components without significantly changing their PCBs.
How does the ISM330IS stand out?
Machine learning applications are increasingly common, and many associate them with high memory requirements, cloud computing servers, or highly parallel GPU architectures. Many programs do require expensive computational operations, which makes them incompatible with edge processing. However, companies also know that not all deep learning systems require this capability. Therefore, running inference on mobile devices (e.g. smartphones) or local industrial installations is gaining attention. Image recognition, anomaly detection, and predictive maintenance all require reliable AI performance within a compact power envelope. Likewise, cameras used in wireless home security systems use artificial intelligence to recognize faces or pets, while ISPUs can provide intelligence for always-on displays on mobile systems.
The ISM330IS is a new solution to this challenge as it requires only 0.59 mA in high performance mode. In contrast, the ISM330DHCX requires 1.5 mA. The latter has a more powerful gyroscope, which, among other things, does partially explain the difference. However, the numbers also show the optimization of the new device and the efficiency of the processing cores. In fact, low-power microcontrollers rarely have an FPU because they usually require a lot of power. However, the ISM330IS still manages to keep its power consumption low enough for battery powered systems.
The Links: 2MBI75L-060 FP15R12NT3
