Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@
MTX_WEBRTCICESERVERS2_0_USERNAME=<username> # WebRTC credentials e.g. intel1234
MTX_WEBRTCICESERVERS2_0_PASSWORD=<password>
```
> **Note:** For GPU-based pipelines, set `privileged_access_required: true` in the `helm/values_<SAMPLE_APP>.yaml` file to enable access to host hardware devices.
> **Note:** For GPU/NPU based pipelines, set `privileged_access_required: true` in the `helm/values_<SAMPLE_APP>.yaml` file to enable access to host hardware devices.

4. Install pre-requisites for all instances

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
```sh
cp helm/values_pallet-defect-detection.yaml helm/values.yaml
```
> **Note:** For GPU-based pipelines, set `privileged_access_required: true` in the `helm/values.yaml` file to enable access to host hardware devices.
> **Note:** For GPU/NPU based pipelines, set `privileged_access_required: true` in the `helm/values.yaml` file to enable access to host hardware devices.

3. Optional: Pull the helm chart and replace the existing helm folder with it

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ This section collects guides for Pallet Defect Detection sample application.
- [Manage pipelines](./how-to-guides/manage-pipelines.md)
- [Run multiple AI pipelines](./how-to-guides/run-multiple-ai-pipelines.md)
- [Use GPU For Inference](./how-to-guides/use-gpu-for-inference.md)
- [Use NPU For Inference](./how-to-guides/use-npu-for-inference.md)
- [Use Your AI Model and Video](./how-to-guides/use-your-ai-model-and-video.md)
- [Change the Input Video Source](./how-to-guides/change-input-video-source.md)
- [Scale Video Resolution](./how-to-guides/scale-video-resolution.md)
Expand All @@ -27,6 +28,7 @@ This section collects guides for Pallet Defect Detection sample application.
./how-to-guides/manage-pipelines
./how-to-guides/run-multiple-ai-pipelines
./how-to-guides/use-gpu-for-inference
./how-to-guides/use-npu-for-inference
./how-to-guides/use-your-ai-model-and-video
./how-to-guides/change-input-video-source
./how-to-guides/scale-video-resolution
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,15 +95,15 @@ If not available, you can simulate this by downloading the sample model from edg

Note the instance-id of the pipeline launched.

8. Verify the pipeline is running. You can View the WebRTC streaming on `http://<HOST_IP>/mediamtx/<peer-str-id>` by replacing `<peer-str-id>` with the value used in the original cURL command to start the pipeline.
8. Verify the pipeline is running. You can View the WebRTC streaming on `https://<HOST_IP>/mediamtx/<peer-str-id>` by replacing `<peer-str-id>` with the value used in the original cURL command to start the pipeline.

![WebRTC streaming](../_assets/webrtc-streaming.png)

### Downloading model with Model Download

At this point, user would like to restart the pipeline with a newer model. The new model can be a retrained version of the existing model or a different model altogether. We use the [Model Download](https://github.com/open-edge-platform/edge-ai-libraries/blob/main/microservices/model-download/docs/user-guide/index.md) microservice to help download the model. It supports downloading public models as well as Geti models from a running Geti server. To learn more about the microservice, see how to [get started with it](https://github.com/open-edge-platform/edge-ai-libraries/blob/main/microservices/model-download/docs/user-guide/get-started.md).

For our demonstration, we will assume the pallet defect detection model has been retrained and is available for downloaded from a Geti server using the Model Download service. Also, the downloaded location is accessible by the dlstreamer pipeline server. In our example, it is `/tmp/tmp-models`. The `/tmp`dir is already accessible by the sample application. If not, please add it to the `volumes` section of `dlstreamer-pipeline-server service in docker-compose file.
For our demonstration, we will assume the pallet defect detection model has been retrained and is available for downloaded from a Geti server using the Model Download service. Also, the downloaded location is accessible by the dlstreamer pipeline server. In our example, it is `/tmp/models`. The `/tmp`dir is already accessible by the sample application. If not, please add it to the `volumes` section of `dlstreamer-pipeline-server service in docker-compose file.

9. Stop the running pipeline by using the pipeline instance "id".

Expand Down Expand Up @@ -143,7 +143,7 @@ If not available, you can simulate this by downloading the sample model from edg
./sample_start.sh -p pallet_defect_detection_mlops
```

11. View the WebRTC streaming on `http://<HOST_IP>/mediamtx/<peer-str-id>` by replacing `<peer-str-id>` with the value used in the original cURL command to start the pipeline.
11. View the WebRTC streaming on `https://<HOST_IP>/mediamtx/<peer-str-id>` by replacing `<peer-str-id>` with the value used in the original cURL command to start the pipeline.

## Additional resources

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ RUN chmod +x /home/pipeline-server/src-gst-gencamsrc/autogen.sh
RUN chmod +x /home/pipeline-server/install_gencamsrc_gstreamer_plugin.sh
RUN /home/pipeline-server/install_gencamsrc_gstreamer_plugin.sh

# For Ubuntu24 intel/dlstreamer-pipeline-server:3.1.0-ubuntu24 base image
# For Ubuntu24 intel/dlstreamer-pipeline-server:2026.0.0-ubuntu24-rc2 base image
RUN apt-get update && apt-get install -y libwxgtk-webview3.2-dev

# For Ubuntu 22 with intel/dlstreamer-pipeline-server:3.1.0-ubuntu22, uncomment the line below and comment the above line
Expand All @@ -67,7 +67,7 @@ USER intelmicroserviceuser
Run the following command to build the image:

```bash
docker build -t intel/dlstreamer-pipeline-server:3.1.0-ubuntu24-gencamsrc-balluff -f BalluffDockerfile .
docker build -t intel/dlstreamer-pipeline-server:2026.0.0-ubuntu24-rc2-gencamsrc-balluff -f BalluffDockerfile .
```

This command builds your Docker image using the steps defined above.
Expand All @@ -76,7 +76,7 @@ This command builds your Docker image using the steps defined above.

After the build completes, update .env and start the container:

> update .env DLSTREAMER_PIPELINE_SERVER_IMAGE=intel/dlstreamer-pipeline-server:3.1.0-ubuntu24-gencamsrc-balluff
> update .env DLSTREAMER_PIPELINE_SERVER_IMAGE=intel/dlstreamer-pipeline-server:2026.0.0-ubuntu24-rc2-gencamsrc-balluff

```bash
docker compose up -d
Expand Down Expand Up @@ -111,7 +111,7 @@ cp .env_pallet-defect-detection .env
Update the `.env` file with the newly created image as below and modify any other required variables.

```bash
DLSTREAMER_PIPELINE_SERVER_IMAGE=intel/dlstreamer-pipeline-server:3.1.0-ubuntu24-gencamsrc-balluff
DLSTREAMER_PIPELINE_SERVER_IMAGE=intel/dlstreamer-pipeline-server:2026.0.0-ubuntu24-rc2-gencamsrc-balluff
```


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ USER intelmicroserviceuser
Run the following command to build the image:

```bash
docker build -t intel/dlstreamer-pipeline-server:3.1.0-ubuntu24-gencamsrc-basler -f BaslerDockerfile .
docker build -t intel/dlstreamer-pipeline-server:2026.0.0-ubuntu24-rc2-gencamsrc-basler -f BaslerDockerfile .
```

This command builds your Docker image using the steps defined above.
Expand All @@ -62,7 +62,7 @@ This command builds your Docker image using the steps defined above.

After the build completes, update .env and start the container:

> update .env DLSTREAMER_PIPELINE_SERVER_IMAGE=intel/dlstreamer-pipeline-server:3.1.0-ubuntu24-gencamsrc-basler
> update .env DLSTREAMER_PIPELINE_SERVER_IMAGE=intel/dlstreamer-pipeline-server:2026.0.0-ubuntu24-rc2-gencamsrc-basler

```bash
docker compose up -d
Expand Down Expand Up @@ -97,7 +97,7 @@ cp .env_pallet-defect-detection .env
Update the `.env` file with the newly created image as below and modify any other required variables.

```bash
DLSTREAMER_PIPELINE_SERVER_IMAGE=intel/dlstreamer-pipeline-server:3.1.0-ubuntu24-gencamsrc-basler
DLSTREAMER_PIPELINE_SERVER_IMAGE=intel/dlstreamer-pipeline-server:2026.0.0-ubuntu24-rc2-gencamsrc-basler
```

### Step 3: Run the Setup Script
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# How to use NPU for inference

## Pre-requisites

To take full advantage of hardware acceleration, pipelines can be designed so that different stages—such as decoding and inference—are executed on the most suitable hardware devices.

Low-power accelerators like a Neural Processing Unit (NPU) can offload neural network computation from the CPU or GPU, enabling more efficient resource utilization and improved overall system performance.

DLStreamer and the DLStreamer Pipeline Server support inference on NPU devices, allowing applications built on these frameworks to leverage NPU acceleration for improved efficiency and performance.

Before running inference on an NPU, ensure that:
- The host system includes a supported NPU device
- The required NPU drivers are installed and properly configured

For detailed setup instructions, refer to the [documentation](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/dev_guide/advanced_install/advanced_install_guide_prerequisites.html#optional-prerequisite-2-install-intel-npu-drivers).

For containerized application, following additional changes are required.

### Provide NPU access to the container

This can be done by making the following changes to the docker compose file.

```yaml
services:
dlstreamer-pipeline-server:
group_add:
# render group ID for ubuntu 22.04 host OS
- "110"
# render group ID for ubuntu 24.04 host OS
- "992"
devices:
# you can add specific devices in case you don't want to provide access to all like below.
- "/dev:/dev"
```
The changes above adds the container user to the `render` group and provides access to the NPU devices.

### Hardware specific encoder/decoders

Unlike the changes done for the container above, the following requires a modification to the media pipeline itself.

Gstreamer has a variety of hardware specific encoders and decoders elements such as Intel specific VA-API elements that you can benefit from by adding them into your media pipeline. Examples of such elements are `vah264dec`, `vah264enc`, `vajpegdec`, `vajpegdec`, etc.

Additionally, one can also enforce zero-copy of buffers using GStreamer caps (capabilities) to the pipeline by adding `video/x-raw(memory: VAMemory)` for Intel NPUs.

Read DL Streamer [docs](https://dlstreamer.github.io/dev_guide/gpu_device_selection.html) for more details.

### NPU specific element properties

DL Streamer inference elements also provides property such as `device=NPU` and `pre-process-backend=va` which should be used in pipelines with NPU memory. It performs mapping to the system memory and uses VA pre-processor. Read DL Streamer [docs](https://dlstreamer.github.io/dev_guide/model_preparation.html#model-pre-and-post-processing) for more.

## Tutorial on how to use NPU specific pipelines

> Note - This sample application already provides a default `docker-compose.yml` file that includes the necessary NPU access to the containers.

The pipeline `pallet_defect_detection_npu` in `pipeline-server-config.json` contains NPU specific elements and uses NPU backend for inferencing. Follow the steps below to run the pipeline.

### Steps

1. Ensure that the sample application is up and running. If not, follow the steps [here](../get-started.md#set-up-the-application) to setup the application and then bring the services up

>If you're running multiple instances of app, start the services using `./run.sh up` instead.

```sh
docker compose up -d
```
2. Start the pipeline.
```sh
./sample_start.sh -p pallet_defect_detection_npu
```

This will start the pipeline. The inference stream can be viewed on WebRTC, in a browser, at the following url:

>If you're running multiple instances of app, ensure to provide `NGINX_HTTPS_PORT` number in the url for the app instance i.e. replace <HOST_IP> with <HOST_IP>:<NGINX_HTTPS_PORT>

```bash
https://<HOST_IP>/mediamtx/pdd/
```

## Deploying with Helm

### Intel GPU K8S Extension

If you're deploying a NPU based pipeline (example: with VA elements like `vapostproc`, `vah264dec` etc., and/or with `device=NPU` in `gvadetect` in `dlstreamer_pipeline_server_config.json`) with Intel GPU k8s Extension, ensure to set the below details in the file `helm/values.yaml` appropriately in order to utilize the underlying NPU.

```sh
gpu:
enabled: true
type: "gpu.intel.com/i915"
count: 1
```

### Without Intel GPU K8S Extension

If you're deploying a NPU based pipeline (example: with VA elements like `vapostproc`, `vah264dec` etc., and/or with `device=NPU` in `gvadetect` in `dlstreamer_pipeline_server_config.json`) without Intel GPU k8s Extension, ensure to set the below details in the file `helm/values.yaml` appropriately in order to utilize the underlying NPU.

```sh
privileged_access_required: true
```
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@

### Updates in v2.6.0

- Consumed latest DL Streamer Pipeline Server 2026.0.0 image. Ubuntu24 variant of the image is default now.
- NGINX,COTURN and MINIO ports made configurable as environment variables
- Added support for simultaneous deployment of multiple applications in the same host via docker and helm
- Model retrained with Geti v2.13.1
- Remove Model Registry service and its references
- MLOps is now demonstrated with Model Download microservice instead of Model Registry
- MLOps is now demonstrated with Model Download microservice instead of Model Registry
- Removed Model Registry service and its references
- NGINX,COTURN and MINIO ports made configurable as environment variables
- Consumed latest DL Streamer Pipeline Server 2026.0.0 image. Ubuntu24 variant of the image is default now.
- Model retrained with Geti v2.13.1
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@
MTX_WEBRTCICESERVERS2_0_USERNAME=<username> # WebRTC credentials e.g. intel1234
MTX_WEBRTCICESERVERS2_0_PASSWORD=<password>
```
> **Note:** For GPU-based pipelines, set `privileged_access_required: true` in the `helm/values_<SAMPLE_APP>.yaml` file to enable access to host hardware devices.
> **Note:** For GPU/NPU based pipelines, set `privileged_access_required: true` in the `helm/values_<SAMPLE_APP>.yaml` file to enable access to host hardware devices.

4. Install pre-requisites for all instances

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@
```sh
cp helm/values_pcb-anomaly-detection.yaml helm/values.yaml
```
> **Note:** For GPU-based pipelines, set `privileged_access_required: true` in the `helm/values.yaml` file to enable access to host hardware devices.

> **Note:** For GPU/NPU based pipelines, set `privileged_access_required: true` in the `helm/values.yaml` file to enable access to host hardware devices.
3. Optional: Pull the helm chart and replace the existing helm folder with it

- Note: The helm chart should be downloaded when you are not using the helm chart provided in `edge-ai-suites/manufacturing-ai-suite/industrial-edge-insights-vision/helm`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,15 +95,15 @@ If not available, you can simulate this by downloading the sample model from edg
```
Note the instance-id of the pipeline launched.

8. Verify the pipeline is running. You can View the WebRTC streaming on `http://<HOST_IP>/mediamtx/<peer-str-id>` by replacing `<peer-str-id>` with the value used in the original cURL command to start the pipeline.
8. Verify the pipeline is running. You can View the WebRTC streaming on `https://<HOST_IP>/mediamtx/<peer-str-id>` by replacing `<peer-str-id>` with the value used in the original cURL command to start the pipeline.

![WebRTC streaming](../_assets/webrtc-streaming.png)

### Downloading model with Model Download

At this point, user would like to restart the pipeline with a newer model. The new model can bea retrained version of the existing model or a different model altogether. We use [Model Download](https://github.com/open-edge-platform/edge-ai-libraries/blob/main/microservices/model-download/docs/user-guide/index.md) microservice to help download the model. It supports downloading public models as well as geti models from a running Geti server. To learn more about it, see [here](https://github.com/open-edge-platform/edge-ai-libraries/blob/main/microservices/model-download/docs/user-guide/get-started.md).

For our demonstration, we will assume the pcb anomaly model has been retrained and is available for downloaded from a Geti server using the Model Download service. Also, the downloaded location is accessible by the dlstreamer pipeline server. In our example, it is `/tmp/tmp-models`. The `/tmp`dir is already accessible by the sample application. If not, please add it to the `volumes` section of `dlstreamer-pipeline-server service in docker-compose file.
For our demonstration, we will assume the pcb anomaly model has been retrained and is available for downloaded from a Geti server using the Model Download service. Also, the downloaded location is accessible by the dlstreamer pipeline server. In our example, it is `/tmp/models`. The `/tmp`dir is already accessible by the sample application. If not, please add it to the `volumes` section of `dlstreamer-pipeline-server service in docker-compose file.

9. Stop the running pipeline by using the pipeline instance "id".

Expand Down Expand Up @@ -142,7 +142,7 @@ If not available, you can simulate this by downloading the sample model from edg
./sample_start.sh -p pcb_anomaly_detection_mlops
```

11. View the WebRTC streaming on `http://<HOST_IP>/mediamtx/<peer-str-id>` by replacing `<peer-str-id>` with the value used in the original cURL command to start the pipeline.
11. View the WebRTC streaming on `https://<HOST_IP>/mediamtx/<peer-str-id>` by replacing `<peer-str-id>` with the value used in the original cURL command to start the pipeline.

## Additional resources

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@

### Updates in v1.2.0

- Consumed latest DL Streamer Pipeline Server 2026.0.0 image. Ubuntu24 variant of the image is default now.
- NGINX,COTURN and MINIO ports made configurable as environment variables
- Added support for simultaneous deployment of multiple applications in the same host via docker and helm
- Model retrained with Geti v2.13.1
- Remove Model Registry service and its references
- MLOps is now demonstrated with Model Download microservice instead of Model Registry
- MLOps is now demonstrated with Model Download microservice instead of Model Registry
- Removed Model Registry service and its references
- NGINX,COTURN and MINIO ports made configurable as environment variables
- Consumed latest DL Streamer Pipeline Server 2026.0.0 image. Ubuntu24 variant of the image is default now.
- Model retrained with Geti v2.13.1
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@
MTX_WEBRTCICESERVERS2_0_USERNAME=<username> # WebRTC credentials e.g. intel1234
MTX_WEBRTCICESERVERS2_0_PASSWORD=<password>
```
> **Note:** For GPU-based pipelines, set `privileged_access_required: true` in the `helm/values_<SAMPLE_APP>.yaml` file to enable access to host hardware devices.
> **Note:** For GPU/NPU based pipelines, set `privileged_access_required: true` in the `helm/values_<SAMPLE_APP>.yaml` file to enable access to host hardware devices.

4. Install pre-requisites for all instances

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
```sh
cp helm/values_weld-porosity.yaml helm/values.yaml
```
> **Note:** For GPU-based pipelines, set `privileged_access_required: true` in the `helm/values.yaml` file to enable access to host hardware devices.
> **Note:** For GPU/NPU based pipelines, set `privileged_access_required: true` in the `helm/values.yaml` file to enable access to host hardware devices.

3. Optional: Pull the helm chart and replace the existing helm folder with it
- Note: The helm chart should be downloaded when you are not using the helm chart provided in `edge-ai-suites/manufacturing-ai-suite/industrial-edge-insights-vision/helm`
Expand Down
Loading
Loading