You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
:warning: It is recommended to use virtual environments !
43
+
### 1.1 Install with uv (recommended)
53
44
54
45
```bash
55
-
$ pyenv install 3.7.3
56
-
$ pyenv virtualenv 3.7.3 vad-venv
57
-
$ pyenv activate vad-venv
46
+
# Install uv if you don't have it
47
+
curl -LsSf https://astral.sh/uv/install.sh | sh
48
+
49
+
# Install all dependencies
50
+
uv sync
58
51
```
59
52
53
+
### 1.2 Development setup
54
+
60
55
```bash
61
-
$ pip install -r requirements.txt
62
-
$ pip install -e .
63
-
```
56
+
# Install with dev dependencies (ruff, pytest, etc.)
57
+
uv sync
64
58
65
-
## 1.2 Virtual environment installation
59
+
# Run linters
60
+
make lint
66
61
67
-
## 1.3 Docker installation
62
+
# Run tests
63
+
make test
68
64
69
-
You can pull the latest image from DockerHub and run Python commands inside the container:
70
-
```bash
71
-
$ docker pull filippogrz/tf-vad:latest
72
-
$ docker run --rm --gpus all -v /var/run/docker.sock:/var/run/docker.sock -it --entrypoint /bin/bash -e TF_FORCE_GPU_ALLOW_GROWTH=true filippogrz/tf-vad
65
+
# Auto-format code
66
+
make format
73
67
```
74
68
75
-
If you want to build the docker image and run the container from scratch, run the following commands.
69
+
### 1.3 Docker installation
76
70
77
-
Build the docker image:
71
+
Build and run the CPU Docker image:
78
72
```bash
79
-
$ make build
73
+
make build
74
+
make local-nobuild
80
75
```
81
-
(This might take a while.)
82
76
83
-
Run the docker image:
77
+
For GPU support (requires NVIDIA Docker runtime):
84
78
```bash
85
-
$ make local-nobuild
79
+
make build-gpu
86
80
```
87
81
88
82
## 2. Introduction
89
83
90
84
### 2.1 Goal
91
85
92
-
The purpose of this project is to design and implement
93
-
a real-time Voice Activity Detection algorithm based on Deep Learning.
86
+
The purpose of this project is to design and implement a real-time Voice Activity Detection algorithm based on Deep Learning.
94
87
95
-
The designed solution is based on MFCC feature extraction and
96
-
a 1D-Resnet model that classifies whether a audio signal is
97
-
speech or noise.
88
+
The designed solution is based on a simple pipeline with MFCC feature extraction and a small 1D-ResNet model (PyTorch) that classifies whether an audio signal is speech or noise.
98
89
99
90
### 2.2 Results
100
91
@@ -107,89 +98,149 @@ Raw and post-processed inference results on a test audio signal are shown below.
107
98
![alt text](pics/inference_raw.png"Raw VAD inference")
108
99
![alt text](pics/inference_smooth.png"VAD inference with post-processing")
109
100
101
+
### 2.3 Model & features
102
+
103
+
Each audio window of `SEQ_LEN = 1024` samples (16 kHz) is converted into a
104
+
**16 × 65** feature tensor stacking:
105
+
106
+
* 5 MFCC coefficients,
107
+
* 5 MFCC deltas (1st order),
108
+
* 5 MFCC delta-deltas (2nd order),
109
+
* 1 RMS energy.
110
+
111
+
These features feed a configurable 1D-ResNet (`vad.model.Resnet1D`):
112
+
stacked residual blocks (3 × `Conv1d → BatchNorm1d` with a 1×1 shortcut) →
113
+
global average pooling → a fully connected head producing a single speech logit.
114
+
The architecture is fully described by the `ModelConfig` dataclass, so the same
115
+
configuration must be used at training, export, and inference time.
116
+
110
117
## 3. Project structure
111
118
112
-
The project `voice_activity_detection/` has the following structure:
113
-
*`vad/data_processing/`: raw data labeling, processing,
114
-
recording & visualization
115
-
*`vad/training/`: data, input pipeline, model
116
-
& training / evaluation / prediction
117
-
*`vad/inference/`: exporting trained model & inference
119
+
The core code lives flat inside `vad/`:
120
+
*`vad/model.py`: the `Resnet1D` model architecture and its `ModelConfig` dataclass
121
+
*`vad/data.py`: feature extraction, dataset building & the PyTorch DataLoader
122
+
*`vad/train.py`: training loop & model export (state dict + TorchScript)
Please download the LibriSpeech ASR corpus dataset from https://openslr.org/12/,
122
-
and extract all files to: `/path/to/LibriSpeech/`.
133
+
Please download the LibriSpeech ASR corpus dataset from https://openslr.org/12/,
134
+
and extract all files to: `/path/to/LibriSpeech/`.
123
135
124
-
The dataset contains approximately 1000 hours of 16kHz read English speech
136
+
The dataset contains approximately 1000 hours of 16kHz read English speech
125
137
from audiobooks, and is well suited for Voice Activity Detection.
126
138
127
-
I automatically annotated the `test-clean` set of the dataset with a
139
+
I automatically annotated the `test-clean` set of the dataset with a
128
140
pretrained VAD model.
129
141
130
-
Please feel free to use the `labels/` folder and the pre-trained VAD model (only for inference) from this
142
+
Please feel free to use the `labels/` folder and the pre-trained VAD model (only for inference) from this
131
143
[ link ](https://drive.google.com/open?id=1ZPQ6wnMhHeE7XP5dqpAEmBAryFzESlin).
132
144
145
+
**Important note:** As this is only a toy project, it is designed to split the `test-clean` sub-dataset intro train / val / test for quick iteration, but can be extended to a full large-scale dataset.
0 commit comments