You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-[Generate and build the project](#generate-and-build-the-project)
13
+
-[Application output](#application-output)
14
+
-[Trademarks](#trademarks)
15
+
-[Licenses](#licenses)
10
16
11
-
A collection of ML examples using the CMSIS pack from [ml-embedded-eval-kit](https://review.mlplatform.org/plugins/gitiles/ml/ethos-u/ml-embedded-evaluation-kit/+/refs/heads/main). Currently, a couple of examples are supported:
17
+
## Introduction
12
18
13
-
- Object detection
14
-
- Keyword spotting
19
+
This repository has a collection of Machine Learning (ML) examples using the CMSIS-Pack from
- Object detection - detects objects in the input image.
24
+
- Keyword spotting - detects specific keywords in the input audio stream.
17
25
18
-
- Arm® and Cortex® are registered trademarks of Arm® Limited (or its subsidiaries) in the US and/or elsewhere.
19
-
- Arm® and Ethos™ are registered trademarks or trademarks of Arm® Limited (or its subsidiaries) in the US and/or
20
-
elsewhere.
21
-
- Arm® and Corstone™ are registered trademarks or trademarks of Arm® Limited (or its subsidiaries) in the US and/or
22
-
elsewhere.
23
-
- Arm®, Keil® and µVision® are registered trademarks of Arm Limited (or its subsidiaries) in the US and/or elsewhere.
24
-
- TensorFlow™, the TensorFlow logo, and any related marks are trademarks of Google Inc.
26
+
Use this import button to open the solution in Keil Studio Cloud: [](https://studio.keil.arm.com/?import=https://github.com/Arm-Examples/mlek-corstone-300-examples.git)
25
27
26
-
## Licenses
28
+
## Overview
27
29
28
-
The application samples are provided under the Apache 2.0 license, see [License](./LICENSE).
30
+
The examples presented in this repository showcase how to build and deploy end-to-end Machine
31
+
Learning applications using existing code from various CMSIS-packs. These examples are built
32
+
using Google's [TensorFlow Lite Micro framework](https://www.tensorflow.org/lite/microcontrollers)
33
+
and Arm's [ML Embedded Evaluation Kit](https://review.mlplatform.org/plugins/gitiles/ml/ethos-u/ml-embedded-evaluation-kit/+/refs/heads/main/Readme.md)
34
+
API's. The embedded evaluation kit API pack has ready-to-use machine learning API's for several
35
+
use cases covering typical `voice`, `vibration` and `vision` applications.
29
36
30
-
Application input data sample files (audio or image files) and the neural network model files have
31
-
been converted into C/C++ type arrays and are distributed under Apache 2.0 license. The models have
32
-
been processed by the [Vela compiler](https://pypi.org/project/ethos-u-vela/) and then converted
33
-
into C/C++ arrays to be baked into the example applications.
37
+
Although the target platform here is Arm® Corstone™-300, the examples can be easily ported to new
38
+
targets, potentially using physical (or virtual) peripherals for live data being fed into the
39
+
neural network model. Corstone™-300 platform gives us the choice of executing the ML workload on
40
+
Arm® Cortex®-M55 CPU or running it more efficiently on Arm® Ethos™-U55 NPU. The examples are set
41
+
up to use the NPU by default with unsupported operators falling back on the CPU.
This example uses a neural network model that specialises in detecting human faces in images.
46
+
The input size for these images is 192x192 (monochrome) and the smallest face that can be
47
+
detected is of size 20x20. The output of the application will be co-ordinates for rectangular
48
+
bounding boxes for each detection.
49
+
50
+
### Keyword spotting
39
51
40
-
## Prerequisites
52
+
This example can detect up to twelve keywords in the input audio stream. The audio file used
53
+
contains the keyword "down" being spoken.
54
+
55
+
More details about the input for this example can be found [here](https://review.mlplatform.org/plugins/gitiles/ml/ethos-u/ml-embedded-evaluation-kit/+/refs/heads/main/docs/use_cases/kws.md#preprocessing-and-feature-extraction).
56
+
57
+
## Building the examples
58
+
59
+
We recommend using [Keil Studio Cloud](https://studio.keil.arm.com/?import=https://github.com/Arm-Examples/mlek-corstone-300-examples.git) for building these examples.
60
+
This is by far the easiest way to get started. However, it is possible to build these projects locally too and the following sections cover the requirements for such a
61
+
set up.
62
+
63
+
### Prerequisites
41
64
42
65
For developing on a local host machine, we recommend a Linux based system as we test
43
66
the flow of the instructions in that environment, but a Windows based machine should
44
67
also work.
45
-
### Tools
46
68
47
-
The following tools are required if building on a local machine (and not using the project via Keil Studio Cloud):
69
+
#### Tools
70
+
71
+
The following tools are required if building on a local machine (and not using the project via
72
+
Keil Studio Cloud):
48
73
49
74
-[CMSIS-Toolbox 1.0.0](https://github.com/Open-CMSIS-Pack/devtools/releases) or higher.
50
-
- Arm Compiler 6.18 (part of Keil MDK or Arm Development Studio, evaluation version sufficient for compilation).
75
+
- Arm Compiler 6.18 (part of Keil MDK or Arm Development Studio, evaluation version sufficient
76
+
for compilation).
51
77
- Arm Virtual Hardware for Corstone-300 v11.18.1 or a local installation of Fixed Virtual Platform
52
78
53
79
> **NOTE**: For Linux, we recommend using the script installer as this sets up the
@@ -69,14 +95,16 @@ The following tools are required if building on a local machine (and not using t
1. Use the `csolution` command to create `.cprj` project files:
96
124
```
@@ -119,22 +147,69 @@ The `csolution.yml` file lists the software packs used in all projects. These ca
119
147
> 1. During the build process required packs may be downloaded.
120
148
> 2. The generator specified depends on CMake and the host platform OS.
121
149
122
-
## Execute Project
150
+
###Execute project
123
151
124
-
The project is configured for execution on Arm Virtual Hardware which removes the requirement for a physical hardware board.
152
+
The project is configured for execution on Arm Virtual Hardware which removes the requirement for
153
+
a physical hardware board.
125
154
126
155
- When using a Fixed Virtual Platform installed locally:
127
156
```
128
157
> <path_to_installed_FVP> -a ./out/kws/AVH-U55/Debug/kws.Debug+AVH-U55.axf -C ethosu.num_macs=256
129
158
```
130
-
> **NOTE**: The FVP defaults to running 128 MAC configuration for Arm Ethos-U55 NPU.
159
+
> **NOTE**: The FVP defaults to running 128 MAC configuration for Arm® Ethos™-U55 NPU.
131
160
> However, our default neural network model for the NPU is for 256 MAC configuration.
132
161
133
-
-[Keil Studio Cloud](https://studio.keil.arm.com/) integrates also the Arm Virtual Hardware VHT_Corstone_SSE-300_Ethos-U55 model. The steps to use the example are:
134
-
- Start [Keil Studio Cloud](https://studio.keil.arm.com/) and login to the system using your account.
135
-
- Drag and drop this directory (cmsis-pack-examples) into the project pane.
136
-
- Select the *Active Project* from the drop down - it should show all projects referenced in the `csolution` file.
162
+
-[Keil Studio Cloud](https://studio.keil.arm.com/) integrates also the Arm Virtual Hardware
163
+
VHT_Corstone_SSE-300_Ethos-U55 model. The steps to use the example are:
164
+
- Start [Keil Studio Cloud](https://studio.keil.arm.com/) and login to the system using your
165
+
account.
166
+
- Drag and drop this directory (cmsis-pack-examples) into the project pane, or use the import
167
+
button provided above, under [ML-examples](#ml-examples).
168
+
- Select the *Active Project* from the drop-down - it should show all projects referenced in
169
+
the `csolution` file.
137
170
- Select *Target hardware* from the drop-down: **SSE-300 MPS3**
138
-
- Click **Run project** which executes the project build step and then starts running on Arm Virtual Hardware.
171
+
- Click **Run project** which executes the project build step and then starts running on Arm
172
+
Virtual Hardware.
139
173
140
174
> **Note:** Arm Virtual Hardware models are also available on AWS Marketplace.
175
+
176
+
### Application output
177
+
178
+
Once the project can be built successfully, the execution on target hardware will show output of
179
+
the application in `Output` window in Keil Studio Cloud. Currently, this includes the following:
180
+
- Arm® Ethos™-U55 NPU version information
181
+
- Information about model's memory allocation
182
+
- Running inference on specified input
183
+
- Output of inference
184
+
- Simulation information such as simulated time, user time, system time, etc
185
+
186
+
The output is different for the two example applications:
187
+
- object detection application will detect two objects on the sample input image and will
188
+
present the detected bounding boxes for objects in the image.
189
+
- keyword spotting application will detect a keyword in the sample audio file and will present
190
+
the highest confidence score and the associated keyword label. Threshold, applied for a label
191
+
to be deemed valid, is also shown
192
+
193
+
## Trademarks
194
+
195
+
- Arm® and Cortex® are registered trademarks of Arm® Limited (or its subsidiaries) in the US and/or elsewhere.
196
+
- Arm® and Ethos™ are registered trademarks or trademarks of Arm® Limited (or its subsidiaries) in the US and/or
197
+
elsewhere.
198
+
- Arm® and Corstone™ are registered trademarks or trademarks of Arm® Limited (or its subsidiaries) in the US and/or
199
+
elsewhere.
200
+
- Arm®, Keil® and µVision® are registered trademarks of Arm Limited (or its subsidiaries) in the US and/or elsewhere.
201
+
- TensorFlow™, the TensorFlow logo, and any related marks are trademarks of Google Inc.
202
+
203
+
## Licenses
204
+
205
+
The application samples are provided under the Apache 2.0 license, see [License](./LICENSE).
206
+
207
+
Application input data sample files (audio or image files) and the neural network model files have
208
+
been converted into C/C++ type arrays and are distributed under Apache 2.0 license. The models have
209
+
been processed by the [Vela compiler](https://pypi.org/project/ethos-u-vela/) and then converted
210
+
into C/C++ arrays to be baked into the example applications.
0 commit comments