Licence Plate Detection

Mike Tango Bravo · Updated 2024-10-10

First we need to perform object detection to find the license plates on moving vehicles. For this we can use ultralytics Yolo.

Then, we need to track the object (license plate), using sort.

Finally, after detecting and tracking the license plate, we need to read the license plate. For this we can use the python library EasyOCR:

In the end, we want to create a stream of data that basically has two items:

A timestamp
The license plate number

Training Yolo object detection

We need a dataset of license plate images to train Yolo. We could take a bunch of pictures of license plates, or we could use a publicly available image dataset:

Open Images Dataset v7

So, we can download many images of license plates. To do this, we need a script:

If we are using our own images of license plates, we would need to annotate the dataset.

CVAT.ai

Create a new user account (openvino)
Start a new project (matricula_tester)
Add a label (alpr)
Create a new task

To set up your Raspberry Pi with YOLO (You Only Look Once) and the Pi IR-wide camera to capture and read vehicle license plate numbers, follow these steps:

Install Python dependencies:

You'll need OpenCV, NumPy, and other Python packages for image processing.

sudo apt-get install python3-opencv python3-pip pip3 install numpy

Install YOLOv5

You can use YOLOv5, which is easier to set up on the Raspberry Pi.

git clone https://github.com/ultralytics/yolov5 cd yolov5 pip3 install -r requirements.txt

Installing Hailo

https://www.youtube.com/watch?v=_aCyR8XJcws

Set pcie to gen2/gen3(gen3 is faster than gen2):

Add following text to /boot/firmware/config.txt

#Enable the PCIe external connector

dtparam=pciex1

#Force Gen 3.0 speeds

dtparam=pciex1_gen=3

NOTE

If you want to use gen2, please comment dtparam=pciex1_gen=3

Install hailo-all and reboot:

Open the terminal on the Raspberry Pi 5 and enter the following command to install the Hailo software

sudo apt install hailo-all
sudo reboot

Check Software and Hardware:

Open terminal on the Raspberry Pi5, and input command as follows to check if hailo-all have been installed.

mtb@OpenFJ:~ $ hailortcli fw-control identify
Executing on device: 0000:01:00.0
Identifying board
Control Protocol Version: 2
Firmware Version: 4.18.0 (release,app,extended context switch buffer)
Logger Version: 0
Board Name: Hailo-8
Device Architecture: HAILO8L
Serial Number: HLDDLBB241600246
Part Number: HM21LB1C2LAE
Product Name: HAILO-8L AI ACC M.2 B+M KEY MODULE EXT TMP

Open terminal on the Raspberry Pi5, and input command as follows to check if hailo-8L have been connected.

mtb@OpenFJ:~ $ lspci | grep Hailo
0000:01:00.0 Co-processor: Hailo Technologies Ltd. Hailo-8 AI Processor (rev 01)

Validate installation of TAPPAS

mtb@OpenFJ:~ $ gst-inspect-1.0 hailotools
Plugin Details:
  Name                     hailotools
  Description              hailo tools plugin
  Filename                 /lib/aarch64-linux-gnu/gstreamer-1.0/libgsthailotools.so
  Version                  3.29.1
  License                  unknown
  Source module            gst-hailo-tools
  Binary package           gst-hailo-tools
  Origin URL               https://hailo.ai/

  hailoaggregator: hailoaggregator - Cascading
  hailocounter: hailocounter - postprocessing element
  hailocropper: hailocropper
  hailoexportfile: hailoexportfile - export element
  hailoexportzmq: hailoexportzmq - export element
  hailofilter: hailofilter - postprocessing element
  hailogallery: Hailo gallery element
  hailograytonv12: hailograytonv12 - postprocessing element
  hailoimportzmq: hailoimportzmq - import element
  hailomuxer: Muxer pipeline merging
  hailonv12togray: hailonv12togray - postprocessing element
  hailonvalve: HailoNValve element
  hailooverlay: hailooverlay - overlay element
  hailoroundrobin: Input Round Robin element
  hailostreamrouter: Hailo Stream Router
  hailotileaggregator: hailotileaggregator
  hailotilecropper: hailotilecropper - Tiling
  hailotracker: Hailo object tracking element

  18 features:
  +-- 18 elements

mtb@OpenFJ:~ $ gst-inspect-1.0 hailo
Plugin Details:
  Name                     hailo
  Description              hailo gstreamer plugin
  Filename                 /lib/aarch64-linux-gnu/gstreamer-1.0/libgsthailo.so
  Version                  1.0
  License                  unknown
  Source module            hailo
  Binary package           GStreamer
  Origin URL               http://gstreamer.net/

  hailodevicestats: hailodevicestats element
  hailonet: hailonet element
  synchailonet: sync hailonet element

  3 features:
  +-- 3 elements

Set Up the Pi IR-wide Camera

Enable the camera on the Raspberry Pi: Run sudo raspi-config, go to Interfacing Options, and enable the camera. Then reboot the Pi.
Test the camera: You can test whether the camera works using raspistill or raspivid.
Install picamera2: (Optional if you are using picamera library)
```
bash
```
Copy code

sudo apt install python3-libcamera python3-kms++ python3-picamera2

3. Prepare YOLO for License Plate Detection

Download a pretrained model: YOLOv5 has several pretrained models. Use one of them (like yolov5s.pt) for testing.
Copy code

python3 detect.py --weights yolov5s.pt --source 0 # 0 for Pi camera
Train or fine-tune for License Plate Detection: To accurately detect license plates, you might want to fine-tune YOLO on a dataset of license plates or use a pretrained model for license plate detection.

You can download pre-trained models from sources like OpenALPR or create a custom dataset with labeled images of license plates.

4. Connect YOLO with License Plate Recognition

You can integrate YOLO with a license plate recognition software like OpenALPR:

Install OpenALPR:
```
bash
```
Copy code

sudo apt-get install openalpr openalpr-daemon openalpr-utils
Run OpenALPR: Once you have the image captured using the Pi camera, run OpenALPR on the image to extract the license plate:
```
bash
```
Copy code

alpr -c us your_image.jpg

5. Create the Pipeline

Capture images using the Pi IR-wide camera.
Use YOLO to detect the license plates in the image.
Pass the detected license plate region to OpenALPR for text recognition.

Example Python Script:

python

Copy code

import cv2 from picamera2 import Picamera2 import torch # Load YOLO model model = torch.hub.load('ultralytics/yolov5', 'custom', path='best.pt') # Initialize the Pi camera picam2 = Picamera2() picam2.start() # Capture frame frame = picam2.capture_array() # Perform YOLO detection results = model(frame) # Process results to find license plates for result in results.xyxy[0]: x1, y1, x2, y2, conf, cls = result # Crop the license plate region license_plate = frame[int(y1):int(y2), int(x1):int(x2)] # Save or process the license plate with OpenALPR cv2.imwrite('license_plate.jpg', license_plate) os.system("alpr -c us license_plate.jpg")

This script captures a frame, runs YOLO to detect license plates, and uses OpenALPR to recognize the license plate number. You can fine-tune it based on your needs.

Let me know if you need help with any specific part of this setup!