This project seeks to develop an ”Intelligent Transport Assist” system that enables visually impaired commuters to independently identify and access buses in real-time.
The system detects and reads Malayalam bus name boards from live video input, and provides clear audible announcements using a text-to-speech engine.
- Bus Nameboard Detection using YOLOv8
- OCR (Optical Character Recognition) for Malayalam using Tesseract
- Text Categorization into primary (main place) and secondary (route details)
- Fuzzy Matching to handle OCR errors
- Audio Announcements using VITS TTS model in Malayalam
- Real-time processing on Jetson Orin Nano hardware
- Python
- YOLOv8 (Ultralytics)
- Tesseract OCR (with
mal.traineddata) - VITS TTS (Text-to-Speech)
- OpenCV, NumPy, Pillow, Sounddevice
- Jetson Orin Nano + JetPack SDK
- FuzzyWuzzy for string matching
- Video Input from camera
- YOLOv8 detects the bus nameboard
- Image Preprocessing (grayscale, denoise, threshold)
- Text Extraction using Tesseract OCR (Malayalam)
- Categorization into main destination and sub-routes
- Text Matching with route database using fuzzy matching
- Speech Output generated using VITS TTS
- Repeat detection avoided using frequency-based filtering
- Achieved Precision: 82.6% and Recall: 92.1% during validation
- Real-time inference and audio announcement tested on live video
- Handles most common OCR errors using fuzzy match + filtering
- YOLOv8 Weights: Click here
- Location Directory (text files): Click here
- Malayalam OCR Model: Click here
- Test Video (Zoomed): Click here
- Test Video (Non-Zoomed): Click here
- Improve robustness under low-light conditions using IR cameras or image enhancement
- Add support for scrolling LED bus nameboards
- Train a deep learning-based Malayalam OCR for higher accuracy
- Add multilingual support (Hindi, Tamil, etc.)
- Deploy as a mobile application or cloud-integrated service
- Use GPS + live database to update route files dynamically