“Easy TinyML - Practical TinyML examples

with Arduino”
Massimo Banzi – Arduino

Italian Area Group – June 1, 2021

 tinyML Italian Committee

Alessandro Grande Danilo Pietro PAU Massimo Banzi Pietro Montino

Director of Technology, System Research and Arduino co-founder Machine learning
Edge Impulse Applications Technical Engineer, VoiceMed
Director, IEEE and ST
Fellow
 tinyML Talks Sponsors

tinyML Strategic Partner

tinyML Strategic Partner

tinyML Strategic Partner

tinyML Strategic Partner

tinyML Strategic Partner

Additional Sponsorships available – contact Olga@tinyML.org for info

 collaboration with
Focus on:
(i) developing new use cases/apps for tinyML vision; and (ii) promoting tinyML tech & companies in the developer
community

Open now

Submissions accepted until August 15th, 2021

Winners announced on September 1, 2021 ($6k value)
Sponsorships available: sponsorships@tinyML.org
https://www.hackster.io/contests/tinyml-vision
Successful tinyML Summit 2021:
www.youtube.com/tinyML with 150+ videos

tinyML Summit-2022, January 24-26, Silicon Valley, CA

 June 7-10, 2021 (virtual, but LIVE)

Sponsorships are being accepted: sponsorships@tinyML.org

 Italian tinyML meetup objectives

• Cre a t e a h o m e fo r e ve ryt h in g t in yML in It a ly

• Co n n e c t in d ivid u a ls in t e re s t e d in t in yML
• Ed u c a t e o n e a n o t h e r o n t h e la t e s t a d va n c e m e n t s
• In s p ire a n e w g e n e ra t io n o f t in yML p ro fe s s io n a ls

• St ro n g c o n n e c t io n w it h it a lia n a c a d e m ia a n d in d u s t ry
• Ta lk s w ill b e (m o s t ly) in it a lia n
• Lo c a t io n w ill ro t a t e in d iffe re n t it a lia n a n d s w is s c it ie s
 The next big opportunity in Tech

“Th e g ro w t h o f Tin yML is d ire c t ly t ie d t o t h e e xp lo s ive g ro w t h o f Io T d e vic e s . Glo b a l Io T d e vic e s will

grow from 6.6 billion in 2020 to 23.72 billion by 2026 . Ea c h n e w d e vic e a n d c o n n e c t io n re p re s e n t s a n
o p p o rt u n it y fo r AI a n d ML. As a re s u lt , t h e TinyML market will grow from 15.2 million shipments in
2020 to 2.5 billion in 2030. ”

ABI Re s e a rc h
 TinyML

● Ru n n in g n e u ra l n e t w o rk a t a n e n e rg y c o s t o f le s s t h a n 1m W
● A d e vic e ru n n in g o n a c o in b a t t e ry t h a t h a s a life t im e o f 1 ye a r
● 32-b it m ic ro c o n t ro lle rs c o s t in g ~ 1$
● Ha vin g a c lo c k s p e e d lo w e r t h a n 1 GHz
● Ho ld in g le s s t h a n 1 MB o f RAM
Massimo Banzi

Massimo Banzi is the co-founder of the Arduino project. He is an

Interaction Designer, Educator and Open Source Hardware advocate.
Massimo started the first FabLab in Italy which led to the creation of
Officine Arduino, a FabLab/Makerspace based in Torino. Massimo has
taught workshops and has been a guest speaker at institutions all over the
world. Massimo is also the author of “Getting Started with Arduino”
published by O’Reilly. He is a regular contributor to the Italian edition of
Wired Magazine and Che Futuro, an online magazine about innovation. He
currently teaches Interaction Design at SUPSI Lugano in the south of
Switzerland and is a visiting professor at CIID in Copenhagen.
TinyML with Arduino

Massimo Banzi
01.06.2021
Introduction
In this talk we walk through some examples of TinyML on Arduino devices
to show how simple it is to build working examples of ML with minimal
coding using three different platforms compatible with Arduino.

2
Arduino
Enabling anyone
to innovate by
making complex
technologies
simple to use.

3
29M+ Users/Year on arduino.cc
900K+ Arduino Forum Users
17M+ IDE downloads/year
1.2M+ Users of Arduino Create Cloud
4000+ Libraries Available for Arduino

923K 302K 408K

4
Arduino Today
Easy for
Beginners,
Fast for
professionals

5
 Arduino Hardware
> RIGHT CLICK ON IMAGE
> REPLACE IMAGE
> UPLOAD FROM COMPUTER /
DRIVE /
WEB URL

(PLEASE REMEMBER TO KEEP THE LEFT-SIDE WHITE GRID ON TOP)

6
Arduino Nano
BLE Sense
● Nano form factor
● nRF52840 Arm Cortex M4f @
64MHz
● 1MByte Flash
● 256Kb Ram
● Bluetooth 5
● Sensors: IMU, Microphone,
Gesture, Light, Proximity,
Pressure, Temperature , Humidity

7
7
Tiny Machine
Learning Kit
• VGA Camera and Nano BLE Sense
• Used for the EdX course on TinyML

8
8
https://sites.google.com/g.harvard.edu/tinyml/home 10
https://www.edx.org/professional-certificate/harvardx-tiny-machine-learning 11
Arduino Edge Control / Smart Agriculture 12
Portenta H7
● Portenta form factor
● STM32H747
● Dual core
○ Arm Cortex M7@480MHz
○ Arm Cortex M4@240MHz
● up to 64 MByte SDRAM
● up to 128 MByte QSPI Flash
● Common Criteria Crypto
● WiFi b/g/n 65Mbps + BT 5.1
● 100Mbit Ethernet
● CAN
● USB-C with DisplayPort output

13
13
Portenta Vision
Shield
● Portenta form factor
● Low power cameras
○ Himax HM-01B0
○ Himax HM-0360
● Dual microphone (beamforming)
● Ethernet
● Lora
● SD Card

14
14
15
 TinyML on Arduino
Tensorflow Lite Micro

Title 16
16
17
Arduino Portenta Machine Control 18
Arduino IDE 2.0 Beta 19
19
OpenMV IDE (Supports Portenta H7, Nano 33 BLE support coming soon) 20
20
Examples
Arduino Tiny ML

21
 TinyML on Arduino
Tensorflow Lite Micro

TensorFlow Lite for Microcontrollers / Gesture Recognition 22

22
23
 Gesture Recognition

We’re going to use the code provided here:

https://github.com/arduino/ArduinoTensorFlowLiteTutorials/tree/master/GestureToEmoji
24
24
Load IMU_Capture.ino and collect datasets 25
25
TensorFlow Lite for Microcontrollers / Gesture Recognition 26
26
TensorFlow Lite for Microcontrollers / Gesture Recognition 27
27
TensorFlow Lite for Microcontrollers / Gesture Recognition 28
28
Gather “punch.csv” and “flex.csv” 29
29
30
30
 Machine Learning
• Go to this colab:
• https://colab.research.google.com/github/arduino/ArduinoTe
nsorFlowLiteTutorials/blob/master/GestureToEmoji/arduino_t
inyml_workshop.ipynb
• Drag punch.csv and flex.csv files from your computer to the
tab to upload them into colab.
• Follow the instructions on the Notebook and you’ll end up with
“model.h” file you can import into arduino

TensorFlow Lite for Microcontrollers / Gesture Recognition 31

31
 Classifying IMU Data
• Open ArduinoSketches/IMU_Classifier/IMU_Classifier.ino in the
Arduino IDE.
• Switch to the model.h tab
• Replace the contents of model.h with the version you
downloaded from Colab
• Upload the sketch: Sketch -> Upload
• Open the Serial Monitor: Tools -> Serial Monitor
• Press the button, and perform a gesture
• The confidence of each gesture will be printed to the Serial
Monitor (0 -> low confidence, 1 -> high confidence)

TensorFlow Lite for Microcontrollers / Gesture Recognition 32

32
TensorFlow Lite for Microcontrollers / Gesture Recognition 33
33
TensorFlow Lite for Microcontrollers / Gesture Recognition 34
34
OpenMV / Tensorflow / Portenta Vision Shield / Image Recognition 35
35
REQUIRED HARDWARE AND SOFTWARE
• Portenta H7 board
• Portenta Vision Shield - LoRa or Portenta Vision Shield -
Ethernet
• USB-C cable (either USB-A to USB-C or USB-C to USB-C)
• An Edge Impulse account for training the ML model
• Fruits (or other objects) to create the classification model
🍏🍌🍐

36
36
37
37
38
38
39
39
40
40
41
41
42
42
43
43
44
44
45
45
46
46
47
47
48
48
Examples
Arduino Tiny ML

49
STM / STM32 Cube AI (Portenta Family) 50
50
Edge Impulse / https://docs.edgeimpulse.com/docs/responding-to-your-voice 51
51
Qeexo / https://github.com/qeexo/Qeexo_AutoML_Arduino_Nano33BLE_pub 52
52
SensiML 53
53
https://experiments.withgoogle.com/collection/tfliteformicrocontrollers 54
54
That’s a wrap,
Thank you!

Massimo Banzi
askmassimo@arduino.cc
Please take the 5 question poll
and continue the conversation
@ tinyML.org/forums
 tinyML Talks Sponsors

tinyML Strategic Partner

tinyML Strategic Partner

tinyML Strategic Partner

tinyML Strategic Partner

tinyML Strategic Partner

Additional Sponsorships available – contact Olga@tinyML.org for info

Arm: The Software and Hardware Foundation for tinyML
1 1
Connect to Application
high-level
frameworks
2
Profiling and Optimized models for embedded
2
debugging AI Ecosystem
Supported by tooling such as Partners
end-to-end tooling Arm Keil MDK 3 Runtime
(e.g. TensorFlow Lite Micro)

3 Optimized low-level NN libraries

Connect to (i.e. CMSIS-NN)
Runtime

RTOS such as Mbed OS

Stay Connected
Arm Cortex-M CPUs and microNPUs
@ArmSoftwareDevelopers

@ArmSoftwareDev
Resources: developer.arm.com/solutions/machine-learning-on-arm
1313 © 2020 Arm Limited (or its affiliates)
 WE USE AI TO MAKE OTHER AI FASTER, SMALLER
AND MORE POWER EFFICIENT

Automatically compress SOTA models like MobileNet to <200KB with

little to no drop in accuracy for inference on resource-limited MCUs

Reduce model optimization trial & error from weeks to days using
Deeplite's design space exploration

Deploy more models to your device without sacrificing performance or

battery life with our easy-to-use software

BECOME BETA USER bit.ly/testdeeplite

| Confidential Presentation ©2020 Deeplite, All Rights Reserved PAGE 14
TinyML for all developers
Dataset

Acquire valuable Enrich data and train

training data securely ML algorithms

Edge Device Impulse

Real sensors in real time
Open source SDK

Embedded and edge Test impulse with

compute deployment real-time device
options data flows

Test
Copyright © EdgeImpulse Inc. www.edgeimpulse.com
 Maxim Integrated: Enabling Edge Intelligence
Advanced AI Acceleration IC Low Power Cortex M4 Micros Sensors and Signal Conditioning

The new MAX78000 implements AI inferences at Large (3MB flash + 1MB SRAM) and small (256KB Health sensors measure PPG and ECG signals
low energy levels, enabling complex audio and flash + 96KB SRAM, 1.6mm x 1.6mm) Cortex M4 critical to understanding vital signs. Signal chain
video inferencing to run on small batteries. Now microcontrollers enable algorithms and neural products enable measuring even the most
the edge can see and hear like never before. networks to run at wearable power levels. sensitive signals.
www.maximintegrated.com/MAX78000 www.maximintegrated.com/microcontroller www.maximintegrated.com/sensor
s s
 Qeexo AutoML
Automated Machine Learning Platform that builds tinyML solutions for the Edge using sensor data

Key Features End-to-End Machine Learning Platform

▪ Supports 17 ML methods:
▪ Multi-class algorithms: GBM, XGBoost, Random
Forest, Logistic Regression, Gaussian Naive Bayes,
Decision Tree, Polynomial SVM, RBF SVM, SVM, CNN,
RNN, CRNN, ANN
▪ Single-class algorithms: Local Outlier Factor, One
Class SVM, One Class Random Forest, Isolation Forest
For more information, visit: www.qeexo.com
▪ Labels, records, validates, and visualizes time-series
sensor data Target Markets/Applications
▪ On-device inference optimized for low latency, low power ▪ Industrial Predictive ▪ Automotive
consumption, and small memory footprint applications Maintenance
▪ Mobile
▪ Supports Arm® Cortex™- M0 to M4 class MCUs ▪ Smart Home
▪ IoT
▪ Wearables
 Advancing AI Perception
Object detection, speech IoT/IIoT
res earch to make
recognition, contextual fus ion

efficient AI ubiquitous Reas oning

Edge cloud

Efficient
Scene unders tanding, language
Automotive
Power efficiency Pers onalization unders tanding, behavior prediction
learning
Model des ign, Continuous learning, Robus t learning
compres s ion, quantization, contextual, always -on, through minimal data,
algorithms, efficient privacy-pres erved, uns upervis ed learning,
hardware, s oftware tool dis tributed learning on-device learning
Action
Reinforcement learning Cloud
A platform to s cale AI for decis ion making

acros s the indus try Mobile

Qualcomm AI Res earch is an initiative of Qualcomm Technologies , Inc.

Build Smart IoT Sensor
Devices From Data
SensiML pioneered TinyML software
tools that auto generate AI code for the
intelligent edge.

• End-to-end AI workflow
• Multi-user auto-labeling of time-series data
• Code transparency and customization at each
step in the pipeline

We enable the creation of production-

grade smart sensor devices. sensiml.com
SynSense builds sensing and inference hardware for ultra-
low-power (sub-mW) embedded, mobile and edge devices.
We design systems for real-time always-on smart sensing,
for audio, vision, IMUs, bio-signals and more.

https://SynSense.ai
 Copyright Notice
This multimedia file is copyright © 2021 by tinyML Foundation. All
rights reserved. It may not be duplicated or distributed in any form
without prior written approval.

The content of this presentation is the work and opinion of the

author(s) and is reproduced here as presented at a tinyML Talk.

tinyML is a registered trademark of the tinyML Foundation.

www.tinyml.org
 Copyright Notice
This presentation in this publication was presented as a tinyML® Talks webcast. The content reflects the
opinion of the author(s) and their respective companies. The inclusion of presentations in this
publication does not constitute an endorsement by tinyML Foundation or the sponsors.

There is no copyright protection claimed by this publication. However, each presentation is the work of
the authors and their respective companies and may contain copyrighted material. As such, it is strongly
encouraged that any use reflect proper acknowledgement to the appropriate source. Any questions
regarding the use of any materials presented should be directed to the author(s) or their companies.

tinyML is a registered trademark of the tinyML Foundation.

www.tinyML.org