
This SDK has been patched by Embedded Artists for the iMXRT1176 Developer's Kit.
The SDK was released on 2022-03-02 and is based on NXP's 2.11.0 SDK (SDK_2_11_0_MIMXRT1176xxxxx.zip).

This is what has been patched:
* LWIP projects - added reading of the MAC address from I2C EEPROM either on the 100Mbit adapter
  or on the uCOM board
* Added an I2C driver for the gpio expander (PCA6416) and code to use it
* Added an I2C driver for the PWM gpio expander (PCA9530) and code to use it
* SDRAM size has been corrected to 32Mb (including linker files, MPU and DCD)
* 1G Ethernet PHY has been changed from RTL8211F to AR8031DS
* SEMC projects - changed to correct settings for the SDRAM
* SEMC projects - changed algorithm for memory test and now test entire 32MB instead of only 4KB
* Examples using eLCDIF/LCDIFv2 have been updated to use PCA6416/PCA9530 for
  RST/PWR/BL signals
* BOARD_USER_BUTTON has been redirected to SW5/WAKEUP button on the uCOM Carrier Board
* USER_LED has been changed to the blue RGB LED using PCA6416
* Touch: I2C bus and GPIOs have been changed for RST/INT
* Camera pins
* Adjusted the USB interface number (it is different for host and device examples)
* Changed the Wi-Fi examples to use the Embedded Artists 1XK M.2 Module (EAR00385) as default
* Corrected the ethernet PHY addresses
* Changed CORE clock depending on speed grading of MCU (798MHz for Industrial, 996MHz for Commercial)
* Changed SEMC clock to be within maximum speed for SDRAM (now 148.5MHz, was 198MHz)
* Many of the projects have been updated to use a more complete pin_mux.c file where all
  necessary pins have been initialized. The SDK examples used to only configure the pins
  that they use (and often not every pin) and most of the time the configuration was only
  for MUX:ing and not the PAD settings (pull up/down/none, drive strength and slew).
* Converted the AzureRTOS examples to use the BOARD_NETWORK_USE_100M_ENET_PORT (same as all
  other networking examples) instead of using the old EXAMPLE_USE_1G_ENET_PORT

This has been added:
* HDMI support to most GUI examples. HDMI at 1024x768@60 is the default resolution but
  that can be changed per project in display_support.h/elcdif_support.h/lcdifv2_support.h.
* Added ADT example for TensorFlow Lite
* I2C probe example
* EDID reader example
* Wi-Fi (serial) examples for the CMWC1ZZABR-107-EVB (a.k.a ABR Module)

This has been removed:
* All projects for the expansion board AGM01

Important things to note:
* Read section "8 - Known Issues" in docs/MCUXpresso SDK Release Notes for MIMXRT1170-EVK.pdf
  to see known issues with the current version of the SDK.
* For Iperf examples, set compiler optimization to -O3 or similar to improve performance.
* If the hardware seems unresponsive and the debugger cannot connect/flash/erase the current program
  then the most likely cause is the running program preventing the access. To stop the currently
  running program and regain control:
  1) Press and hold down the ISP_ENABLE button (SW1)
  2) Press and hold down the RESET button (SW3)
  3) Let go of the RESET button
  4) Wait an extra second or two
  5) Release the ISP_ENABLE button
  6) The hardware is now in a mode where programming/erasing it should work


Connectors:
* J29 (micro USB) is the default UART for the CM7 core and unless specified otherwise it is setup for 115200 8/N/1
* J30 (micro USB) is the default UART for the CM4 core and unless specified otherwise it is setup for 115200 8/N/1
* For 1Gbit Ethernet examples, use connector J25 on uCOM Carrier Board
* For 100Mbit Ethernet examples, use ethernet adapter connected between J12 on uCOM Carrier Board
  and J37 on the adapter. These four connections are also needed:
    1) uCOM Carrier Board, JP38:1 -> adapter JP37:1
    2) uCOM Carrier Board, JP38:2 -> adapter JP37:2
    3) uCOM Carrier Board, JP27:1 -> adapter JP39:2
    4) uCOM Carrier Board, JP27:2 -> adapter JP39:1
* The two CSI examples can use either an OV5640 camera in connector J23 or a camera in connector J24
* The EIQ examples that use a camera expects the camera in connector J24 (J23 might work for some
  of the examples but runs much slower)
* Some GUI examples are configured for the RK055AHD091 display which should be in connector "C" on
  the uCOM Carrier Board. Ignore the readme text about connecting extra 5V power.
* The default for GUI examples is to use an HDMI adapter in connector "C" on
  the uCOM Carrier Board. Ignore the readme text about connecting extra 5V power.


!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
THIS PROJECT IS NOT DIRECTLY COMPATIBLE WITH THE HARDWARE AND WILL NOT WORK.

There is no audio codec mounted on the uCOM Carrier Board
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!



Everything below this line is the original content of the readme file.
=======================================================================



Overview
========
Keyword spotting example based on Keyword spotting for Microcontrollers [1].

Input data preprocessing

Raw audio data is pre-processed first - a spectrogram is calculated: A 40 ms
window slides over a one-second audio sample with a 20 ms stride. For each
window, audio frequency strengths are computed using FFT and turned into
a set of Mel-Frequency Cepstral Coefficients (MFCC). Only first 10 coefficients
are taken into account. The window slides over a sample 49 times, hence
a matrix with 49 rows and 10 columns is created. The matrix is called a spectrogram.

In the example, static audio samples ("off", "right") are evaluated first
regardless microphone is connected or not. Secondly, audio is processed directly
from microphone.

Classification

The spectrogram is fed into a neural network. The neural network is a depthwise
separable convolutional neural network based on MobileNet described in [2].
The model produces a probability vector for the following classes:
"Silence", "Unknown", "yes", "no", "up", "down", "left", "right", "on", "off",
"stop" and "go".

Quantization

The NN model is quantized to run faster on MCUs and it takes in a quantized
input and produces a quantized output. An input spectrogram needs to be scaled
from range [-247, 30] to range [0, 255] and round to integers. Values lower
than zero are set to zero and values exceeding 255 are set to 255. An output
of the softmax function is a vector with components in the interval (0, 255)
and the components will add up to 255).   

HOW TO USE THE APPLICATION:
Say different keyword so that microphone can catch them. Voice recorded from
the microphone can be heared using headphones connected to the audio jack.
Note semihosting implementation causes slower or discontinuous audio experience. 
Select UART in 'Project Options' during project import for using external debug
console via UART (virtual COM port).

[1] https://github.com/ARM-software/ML-KWS-for-MCU
[2] https://arxiv.org/abs/1704.04861

Files:
  main.cpp - example main function
  ds_cnn_s.tflite - pre-trained TensorFlow Lite model converted from DS_CNN_S.pb
    (source: https://github.com/ARM-software/ML-KWS-for-MCU/blob/master/Pretrained_models/DS_CNN/DS_CNN_S.pb)
    (for details on how to quantize and convert a model see the eIQ TensorFlow Lite
    User's Guide, which can be downloaded with the MCUXpresso SDK package)
  off.wav - waveform audio file of the word to recognize
    (source: Speech Commands Dataset available at
    https://storage.cloud.google.com/download.tensorflow.org/data/speech_commands_v0.02.tar.gz,  
    file speech_commands_test_set_v0.02/off/0ba018fc_nohash_2.wav)
  right.wav - waveform audio file of the word to recognize
    (source: Speech Commands Dataset available at
    https://storage.cloud.google.com/download.tensorflow.org/data/speech_commands_v0.02.tar.gz,
    file speech_commands_test_set_v0.02/right/0a2b400e_nohash_1.wav)
  audio_data.h - waveform audio files converted into a C language array of audio signal
    values ("off", "right") audio signal values using Python with the Scipy package:
    from scipy.io import wavfile
    rate, data = wavfile.read('yes.wav')
    with open('wav_data.h', 'w') as fout:
      print('#define WAVE_DATA {', file=fout)
      data.tofile(fout, ',', '%d')
      print('}\n', file=fout)
  train.py - model training script based on https://www.tensorflow.org/tutorials/audio/simple_audio
  timer.c - timer source code
  audio/* - audio capture and pre-processing code
  audio/mfcc.cpp - MFCC feature extraction matching the TensorFlow MFCC operation
  audio/kws_mfcc.cpp - ausio buffer handling for MFCC feature extraction
  model/get_top_n.cpp - top results retrieval
  model/model_data.h - model data from the ds_cnn_s.tflite file converted to
    a C language array using the xxd tool (distributed with the Vim editor
    at www.vim.org)
  model/model.cpp - model initialization and inference code
  model/model_ds_cnn_ops.cpp - model operations registration
  model/output_postproc.cpp - model output processing


Toolchain supported
===================
- IAR embedded Workbench  9.10.2
- Keil MDK  5.34
- MCUXpresso  11.5.0
- GCC ARM Embedded  10.2.1

Hardware requirements
=====================
- Mini/micro USB cable
- EVK-MIMXRT1170 board
- Personal computer

Board settings
==============
No special settings are required.

Prepare the Demo
================
1. Connect a USB cable between the host PC and the OpenSDA USB port on the target board. 
2. Open a serial terminal with the following settings:
   - 115200 baud rate
   - 8 data bits
   - No parity
   - One stop bit
   - No flow control
3. Download the program to the target board.
4. Either press the reset button on your board or launch the debugger in your IDE to begin running the demo.

Running the demo
================
The log below shows the output of the demo in the terminal window:

Keyword spotting example using a TensorFlow Lite model.
Detection threshold: 25

Static data processing:
Expected category: off
----------------------------------------
     Inference time: 32 ms
     Detected:        off (100%)
----------------------------------------

Expected category: right
----------------------------------------
     Inference time: 32 ms
     Detected:      right (98%)
----------------------------------------


Microphone data processing:
----------------------------------------
     Inference time: 32 ms
     Detected: No label detected (0%)
----------------------------------------

----------------------------------------
     Inference time: 32 ms
     Detected:         up (85%)
----------------------------------------

----------------------------------------
     Inference time: 32 ms
     Detected:       left (97%)
----------------------------------------
