
This SDK has been patched by Embedded Artists for the iMXRT1064 Developer's Kit.
The SDK was released on 2021-12-01 and is based on NXP's 2.10.0 SDK (SDK_2_10_0_MIMXRT1064xxxxA.zip).

This is what has been patched:
* Set CPU speed according to Commercial/Industrial CPU
* Correction of the VDD_SOC_IN voltage.
* LWIP projects - added reading of the MAC address from the onboard I2C EEPROM
* Wi-Fi and Bluetooth projects
* Added an I2C driver for the gpio expander (PCA6416) and code to use it
* Added an I2C driver for the PWM gpio expander (PCA9530) and code to use it
* Modified pin muxing
* SEMC projects - changed algorithm for memory test and now test entire 32MB instead of only 4KB
* Examples using a disaplay have been updated to use PCA6416/PCA9530 for
  RST/PWR/BL signals
* BOARD_USER_BUTTON has been redirected to SW5/WAKEUP button on the uCOM Carrier Board
* USER_LED has been changed to the blue RGB LED using PCA6416
* Adjusted the USB interface number (it is different for host and device examples)
* Added support for Embedded Artists 2DS M.2 Module (EAR00386) in the NXP Wi-Fi examples
* Added support for Embedded Artists 1ZM M.2 Module (EAR00364) in the NXP Wi-Fi examples
* Added support for Embedded Artists 1XK M.2 Module (EAR00385) in the NXP Wi-Fi examples
* Changed reset pin for SD card examples

This has been added:
* LWIP projects - option to use 100/10Mbps Ethernet-PHY Adapter
* AWS projects - option to use 100/10Mbps Ethernet-PHY Adapter
* AzureRTOS projects - option to use 100/10Mbps Ethernet-PHY Adapter
* I2C probe example
* Wi-Fi (serial) examples for the CMWC1ZZABR-107-EVB (a.k.a ABR Module)

This has been removed:
* All projects for the expansion board AGM01

Important things to note:
* Read section "8 - Known Issues" in docs/MCUXpresso SDK Release Notes for EVK-MIMXRT1064.pdf
  to see known issues with the current version of the SDK.
* For Iperf examples, set compiler optimization to -O3 or similar to improve performance.
* If the hardware seems unresponsive and the debugger cannot connect/flash/erase the current program
  then the most likely cause is the running program preventing the access. To stop the currently
  running program and regain control:
  1) Press and hold down the ISP_ENABLE button (SW1)
  2) Press and hold down the RESET button (SW3)
  3) Let go of the RESET button
  4) Wait an extra second or two
  5) Release the ISP_ENABLE button
  6) The hardware is now in a mode where programming/erasing it should work



Connectors:
* J29 (micro USB) is the default UART and unless specified otherwise it is setup for 115200 8/N/1


Everything below this line is the original content of the readme file.
=======================================================================



Overview
========
Keyword spotting example based on Keyword spotting for Microcontrollers [1].

Input data preprocessing

Raw audio data is pre-processed first - a spectrogram is calculated: A 40 ms
window slides over a one-second audio sample with a 20 ms stride. For each
window, audio frequency strengths are computed using FFT and turned into
a set of Mel-Frequency Cepstral Coefficients (MFCC). Only first 10 coefficients
are taken into account. The window slides over a sample 49 times, hence
a matrix with 49 rows and 10 columns is created. The matrix is called a spectrogram.

In the example, static audio samples ("off", "right") are evaluated first
regardless microphone is connected or not. Secondly, audio is processed directly
from microphone.

Classification

The spectrogram is fed into a neural network. The neural network is a depthwise
separable convolutional neural network based on MobileNet described in [2].
The model produces a probability vector for the following classes:
"Silence", "Unknown", "yes", "no", "up", "down", "left", "right", "on", "off",
"stop" and "go".

Quantization

The NN model is quantized to run faster on MCUs and it takes in a quantized
input and produces a quantized output. An input spectrogram needs to be scaled
from range [-247, 30] to range [0, 255] and round to integers. Values lower
than zero are set to zero and values exceeding 255 are set to 255. An output
of the softmax function is a vector with components in the interval (0, 255)
and the components will add up to 255).   

HOW TO USE THE APPLICATION:
Say different keyword so that microphone can catch them. Voice recorded from
the microphone can be heared using headphones connected to the audio jack.
Note semihosting implementation causes slower or discontinuous audio experience. 
Select UART in 'Project Options' during project import for using external debug
console via UART (virtual COM port).

[1] https://github.com/ARM-software/ML-KWS-for-MCU
[2] https://arxiv.org/abs/1704.04861

Files:
  main.cpp - example main function
  ds_cnn_s.tflite - pre-trained TensorFlow Lite model converted from DS_CNN_S.pb
    (source: https://github.com/ARM-software/ML-KWS-for-MCU/blob/master/Pretrained_models/DS_CNN/DS_CNN_S.pb)
    (for details on how to quantize and convert a model see the eIQ TensorFlow Lite
    User's Guide, which can be downloaded with the MCUXpresso SDK package)
  off.wav - waveform audio file of the word to recognize
    (source: Speech Commands Dataset available at
    https://storage.cloud.google.com/download.tensorflow.org/data/speech_commands_v0.02.tar.gz,  
    file speech_commands_test_set_v0.02/off/0ba018fc_nohash_2.wav)
  right.wav - waveform audio file of the word to recognize
    (source: Speech Commands Dataset available at
    https://storage.cloud.google.com/download.tensorflow.org/data/speech_commands_v0.02.tar.gz,
    file speech_commands_test_set_v0.02/right/0a2b400e_nohash_1.wav)
  audio_data.h - waveform audio files converted into a C language array of audio signal
    values ("off", "right") audio signal values using Python with the Scipy package:
    from scipy.io import wavfile
    rate, data = wavfile.read('yes.wav')
    with open('wav_data.h', 'w') as fout:
      print('#define WAVE_DATA {', file=fout)
      data.tofile(fout, ',', '%d')
      print('}\n', file=fout)
  timer.c - timer source code
  audio/* - audio capture and pre-processing code
  audio/mfcc.cpp - MFCC feature extraction matching the TensorFlow MFCC operation
  audio/kws_mfcc.cpp - ausio buffer handling for MFCC feature extraction
  model/get_top_n.cpp - top results retrieval
  model/model_data.h - model data from the ds_cnn_s.tflite file converted to
    a C language array using the xxd tool (distributed with the Vim editor
    at www.vim.org)
  model/model.cpp - model initialization and inference code
  model/model_ds_cnn_ops.cpp - model operations registration
  model/output_postproc.cpp - model output processing


Toolchain supported
===================
- IAR embedded Workbench  9.10.2
- Keil MDK  5.34
- MCUXpresso  11.4.0
- GCC ARM Embedded  10.2.1

Hardware requirements
=====================
- Mini/micro USB cable
- EVK-MIMXRT1064 board
- Personal computer

Board settings
==============
Disconnect camera device from the J35 connector if connected (possible signal interference).

Prepare the Demo
================
1. Connect a USB cable between the host PC and the OpenSDA USB port on the target board. 
2. Open a serial terminal with the following settings:
   - 115200 baud rate
   - 8 data bits
   - No parity
   - One stop bit
   - No flow control
3. Download the program to the target board.
4. Either press the reset button on your board or launch the debugger in your IDE to begin running the demo.

Running the demo
================
The log below shows the output of the demo in the terminal window:

Keyword spotting example using a TensorFlow Lite model.
Detection threshold: 25

Static data processing:
Expected category: off
----------------------------------------
     Inference time: 32 ms
     Detected:        off (100%)
----------------------------------------

Expected category: right
----------------------------------------
     Inference time: 32 ms
     Detected:      right (98%)
----------------------------------------


Microphone data processing:
----------------------------------------
     Inference time: 32 ms
     Detected: No label detected (0%)
----------------------------------------

----------------------------------------
     Inference time: 32 ms
     Detected:         up (85%)
----------------------------------------

----------------------------------------
     Inference time: 32 ms
     Detected:       left (97%)
----------------------------------------
