The goal of this tutorial is to show you a way to easily add AI to a project , without any knowledge in this field, using the software  Nano AI Studio and its Arduino compatible libraries!

This tutorial guides you through building a cardboard touchpad that relies on vibration analysis and an Embedded AI algorithm running on an Arduino UNO R4 . The UNO emulates a USB keyboard device.

Vibration data from the cardboard is captured using a basic accelerometer connected via the Qwiic connector. Within the UNO microcontroller, vibrations are classified using a NanoEdge AI library.

Based on the detected class, the touchpad triggers either a "PageUp" or "PageDown" keystroke.

Hardware:

1. Arduino UNO R4 Wifi

1. LIS3DH Triple-Axis Accelerometer (any accelerometer should work)

Software:

1. Arduino IDE (for Windows user, you may want to use the 1.8.19 desktop version)

1. NanoEdge AI Studio

NanoEdge AI Studio:

NanoEdge is a free machine learning software developed by STMicroelectronics which allow to easily create and integrate AI libraries to any cortex M microcontroller. Essentially, select a project type, import data locally, run a benchmark to find the best model automatically, test the model if you want and get an AI library.

In Nanoedge AI Studio, four kinds of projects are available, each serving a different purpose:

1. Anomaly detection (AD): to detect a nominal behavior and an abnormal one. Can be retrained directly on board.
2. 1 class classification (1c): Create a model to detect both nominal and abnormal behavior but with only nominal data. (In case you cannot collect abnormal examples)
3. N class classification (Nc): Create a model to classify data into multiple classes that you define
4. Extrapolation (Ex): Regression in short. To predict a value instead of a class from the input data (a speed or temperature for example).

Steps:

Open Arduino IDE and create a new project:

1. Copy data logger source code available below.
2. Click on Sketch > Include Library > Adafruit_LiS3DH to install the library.

Be careful : If your board is emulating a keyboard, you need to double press the reset button to be able to flash it.

Open the serial in Arduino IDE to check that the accelerometer data is correctly coming to your computer. (don't forget to close the serial after that)

Now we will automatically create an AI model able to recognize classes of gestures using vibration patterns.

Open NanoEdge AI Studio and create a new " N-Class Classification " project.

In project settings:

1. Set target to "UNO R4 Wifi"
2. Sensor to 3-axes accelerometer

In the Signal step , collect one dataset per class ("Nothing","Swipe","Multitap" or any gesture that you want) using serial port.

Collect one kind of gesture per dataset!

Here is how to proceed:

1. Make sure the board is connected to the pc with the data logger code flashed on it
2. Click ADD SIGNAL > FROM SERIAL
3. Make sure to select the right come port
4. Click START/STOP to collect data (100 buffers per signal should be enough)
5. Once finished click CONTINUE and then IMPORT

If everything is correct, you should see a new dataset added with plots and information.

Create the AI model:

Once you have all the classses that you want to recognize, go to the Benchmark step.

1. Click RUN NEW BENCHMARK.
2. Select all your datasets and click START.

NanoEdge AI Studio will take your data and look for a model that is able to classify them. (it also applies pretreatment on its own to your data)

You get the accuracy of the model and its RAM and Flash requirements.

You should reach around 99% pretty fast if you collected good data. You can stop the benchmark when it happens.

Create the AI model:

Once you have all the classses that you want to recognize, go to the Benchmark step.

1. Click RUN NEW BENCHMARK.
2. Select all your datasets and click START.

NanoEdge AI Studio will take your data and look for a model that is able to classify them. (it also applies pretreatment on its own to your data)

You get the accuracy of the model and its RAM and Flash requirements.

You should reach around 99% pretty fast if you collected good data. You can stop the benchmark when it happens.

Create the demo:

1. Open a new project in Arduino IDE
2. Get the main code below and paste it in your project
3. Click on Sketch > Include Library > Adafruit_LIS3DH
4. Add the Nanoedge AI Library (select the previously extracted zip):
   1. Compile the code.
   2. Flash the code.

   It is finished, you can play with it.

    

   Through this example, this tutorial shows how to add AI to an Arduino project using NanoEdge AI Studio. It is now up to you to think about other use case and do them on your own!/* If you want to use NEAI functions please, include NEAI library
   * in your Arduino libraries then, uncomment NEAI parts in the following code
   */
   
   /* Libraries part */
   #include <Wire.h>
   #include <Adafruit_LIS3DH.h>
   #include <Adafruit_Sensor.h>
   #include <Keyboard.h>
   #include <NanoEdgeAI.h>
   #include "knowledge.h"
   float input_user_buffer[DATA_INPUT_USER * AXIS_NUMBER]; // Buffer of input values
   float output_class_buffer[CLASS_NUMBER]; // Buffer of class probabilities
   uint16_t id_class = 0;
   
   /* Macros definitions */
   #define SERIAL_BAUD_RATE 115200
   
   /* Default address is 0x18 but, if SDO is powered at 3v3,
   * address is set to 0x19, so you need to change it
   * depending on your current hardware configuration.
   */
   #define SENSOR_I2C_ADDR 0x18
   
   /* Sensor data rates.
   * You can choose from:
   * LIS3DH_DATARATE_1_HZ
   * LIS3DH_DATARATE_10_HZ
   * LIS3DH_DATARATE_25_HZ
   * LIS3DH_DATARATE_50_HZ
   * LIS3DH_DATARATE_100_HZ
   * LIS3DH_DATARATE_200_HZ
   * LIS3DH_DATARATE_400_HZ
   * LIS3DH_DATARATE_LOWPOWER_1K6HZ
   * LIS3DH_DATARATE_LOWPOWER_5KHZ
   */
   #define SENSOR_DATA_RATE LIS3DH_DATARATE_400_HZ
   
   /* Sensor ranges.
   * You can choose from:
   * LIS3DH_RANGE_16_G
   * LIS3DH_RANGE_8_G
   * LIS3DH_RANGE_4_G
   * LIS3DH_RANGE_2_G
   */
   #define SENSOR_RANGE LIS3DH_RANGE_2_G
   
   /* NanoEdgeAI defines part
   * NEAI_MODE = 1: NanoEdgeAI functions = AI Mode.
   * NEAI_MODE = 0: Datalogging mode.
   */
   #define NEAI_MODE 1
   #define SENSOR_SAMPLES 256
   #define AXIS 3
   
   Adafruit_LIS3DH lis = Adafruit_LIS3DH();
   
   /* Global variables definitions */
   static uint16_t neai_ptr = 0;
   static float neai_buffer[SENSOR_SAMPLES * AXIS] = {0.0};
   
   
   /* Initialization function: In this function,
   * code runs only once at boot / reset.
   */
   void setup() {
   /* Init serial at baud rate 115200 */
   Serial.begin(SERIAL_BAUD_RATE);
   
   /* Init I2C connection between board & sensor */
   if (!lis.begin(SENSOR_I2C_ADDR)) {
   Serial.println("Can't initialize I2C comm with LIS3DH sensor...\n");
   while(1);
   }
   Serial.println("OK");
   /* Init LIS3DH with desired settings: odr & range */
   lis.setRange(SENSOR_RANGE);
   lis.setDataRate(SENSOR_DATA_RATE);
   
   /* Initialize NanoEdgeAI AI */
   enum neai_state error_code = neai_classification_init(knowledge);
   if (error_code != NEAI_OK) {
   Serial.println("Error starting NanoEdge AI lib");
   /* This happens if the knowledge does not correspond to the library or if the library works into a not supported board. */
   }
   
   Keyboard.begin();
   delay(1000);
   }
   
   /* Main function: Code run indefinitely */
   void loop() {
   /* Get data in the neai buffer */
   while (neai_ptr < SENSOR_SAMPLES) {
   /* Check if new data if available */
   if (lis.haveNewData()) {
   /* If new data is available we read it ! */
   lis.read();
   /* Fill neai buffer with new accel data */
   neai_buffer[AXIS * neai_ptr] = (float) lis.x;
   neai_buffer[(AXIS * neai_ptr) + 1] = (float) lis.y;
   neai_buffer[(AXIS * neai_ptr) + 2] = (float) lis.z;
   /* Increment neai pointer */
   neai_ptr++;
   }
   }
   /* Reset pointer */
   neai_ptr = 0;
   
   /* Depending on NEAI_MODE value, run NanoEdge AI functions
   * or print accelerometer data to the serial (datalogging)
   */
   
   if (NEAI_MODE) {
   neai_classification(neai_buffer, output_class_buffer, &id_class);
   
   if (id_class == 1) {
   Keyboard.write(KEY_PAGE_DOWN);
   delay(100);
   } else if (id_class == 2) {
   Keyboard.write(KEY_PAGE_UP);
   delay(100);
   }
   } else {
   /* Print the whole buffer to the serial */
   for (uint16_t i = 0; i < AXIS * SENSOR_SAMPLES; i++) {
   Serial.print((String)neai_buffer[i] + " ");
   }
   Serial.print("\n");
   }
   
   /* Clean neai buffer */
   memset(neai_buffer, 0.0, AXIS * SENSOR_SAMPLES * sizeof(float));
   }

/* If you want to use NEAI functions please, include NEAI library
* in your Arduino libraries then, uncomment NEAI parts in the following code
*/

/* Libraries part */
#include <Wire.h>
#include <Adafruit_LIS3DH.h>
#include <Adafruit_Sensor.h>

/* Macros definitions */
#define SERIAL_BAUD_RATE 115200

/* Default address is 0x18 but, if SDO is powered at 3v3,
* address is set to 0x19, so you need to change it
* depending on your current hardware configuration.
*/
#define SENSOR_I2C_ADDR 0x18

/* Sensor data rates.
* You can choose from:
* LIS3DH_DATARATE_1_HZ
* LIS3DH_DATARATE_10_HZ
* LIS3DH_DATARATE_25_HZ
* LIS3DH_DATARATE_50_HZ
* LIS3DH_DATARATE_100_HZ
* LIS3DH_DATARATE_200_HZ
* LIS3DH_DATARATE_400_HZ
* LIS3DH_DATARATE_LOWPOWER_1K6HZ
* LIS3DH_DATARATE_LOWPOWER_5KHZ
*/
#define SENSOR_DATA_RATE LIS3DH_DATARATE_400_HZ

/* Sensor ranges.
* You can choose from:
* LIS3DH_RANGE_16_G
* LIS3DH_RANGE_8_G
* LIS3DH_RANGE_4_G
* LIS3DH_RANGE_2_G
*/
#define SENSOR_RANGE LIS3DH_RANGE_2_G

/* NanoEdgeAI defines part
* NEAI_MODE = 1: NanoEdgeAI functions = AI Mode.
* NEAI_MODE = 0: Datalogging mode.
*/
#define NEAI_MODE 1
#define SENSOR_SAMPLES 256
#define AXIS 3

Adafruit_LIS3DH lis = Adafruit_LIS3DH();

/* Global variables definitions */
static uint16_t neai_ptr = 0;
static float neai_buffer[SENSOR_SAMPLES * AXIS] = {0.0};


/* Initialization function: In this function,
* code runs only once at boot / reset.
*/
void setup() {
/* Init serial at baud rate 115200 */
Serial.begin(SERIAL_BAUD_RATE);

/* Init I2C connection between board & sensor */
if(!lis.begin(SENSOR_I2C_ADDR)) {
Serial.println("Can't initialize I2C comm with LIS3DH sensor...\n");
while(1);
}
Serial.println("OK");
/* Init LIS3DH with desired settings: odr & range */
lis.setRange(SENSOR_RANGE);
lis.setDataRate(SENSOR_DATA_RATE);

delay(1000);
}

/* Main function: Code run indefinitely */
void loop() {
/* Get data in the neai buffer */
while(neai_ptr < SENSOR_SAMPLES) {
/* Check if new data if available */
if(lis.haveNewData()) {
/* If new data is available we read it ! */
lis.read();
/* Fill neai buffer with new accel data */
neai_buffer[AXIS * neai_ptr] = (float) lis.x;
neai_buffer[(AXIS * neai_ptr) + 1] = (float) lis.y;
neai_buffer[(AXIS * neai_ptr) + 2] = (float) lis.z;
/* Increment neai pointer */
neai_ptr++;
}
}
/* Reset pointer */
neai_ptr = 0;


/* Print the whole buffer to the serial */
for(uint16_t i = 0; i < AXIS * SENSOR_SAMPLES; i++) {
Serial.print((String)neai_buffer[i] + " ");
}
Serial.print("\n");


/* Clean neai buffer */
memset(neai_buffer, 0.0, AXIS * SENSOR_SAMPLES * sizeof(float));
}