Artificial Intelligence in Autonomous Vehicle

Decision-Making

AI decides how the vehicle will behave in various situations, such as merging onto a highway, stopping at a crosswalk, or handling intersections.(Behavioural Planning)

Some autonomous systems use reinforcement learning (a trial-and-error learning approach) in simulations to improve decision-making and prepare for complex real-world situations.(Reinforcement Planning)

 Control Systems

Ai algorithm translates their decisions  physically to the braking and acceleration systems.

 AI enables vehicles to respond immediately to changes, like sudden stops from cars ahead or unexpected obstacles, ensuring safe operation.

 Communication and Coordination

• Vehicle-to-Everything (V2X) Communication: AI processes V2X communications, allowing the vehicle to exchange data with other vehicles and infrastructure to anticipate traffic light changes, avoid road hazards, and enhance overall safety.
• Platooning: AI enables autonomous vehicles to travel closely together in a formation, reducing drag, improving fuel efficiency, and allowing efficient coordination in traffic.

Continuous Learning and Improvement

• Data Collection and Analysis: Autonomous vehicles continuously gather data, which is then analyzed to improve AI models over time, making the vehicle safer and more efficient with each update.
• Simulation Testing: AVs undergo extensive AI-powered simulations to prepare for complex driving scenarios, testing millions of situations that the vehicle may encounter in real life.

Passenger Interaction and Personalization

• Natural Language Processing (NLP): AI-driven NLP allows passengers to interact with the vehicle via voice commands, making it easier to control destinations, get updates, and access various vehicle functions.
• User Comfort: AI personalizes the in-car experience by adjusting climate, seat positions, and entertainment settings based on passenger preferences.

Safety and Redundancy

• Fault Detection and Self-Diagnostics: AI monitors vehicle health and can detect malfunctions, triggering alerts or performing safe shutdowns as needed.
• Redundancy Systems: Critical systems are often duplicated, ensuring the vehicle can maintain control if one system fails.

Ethical Decision-Making

• Ethics Algorithms: AI can be programmed with ethical considerations for unavoidable scenarios, although these decision-making frameworks are still evolving and remain a topic of debate.

Step-by-Step Guide to Implement YOLOv5 for Object Detection

Install the required packages:

pip install torch torchvision

pip install opencv-python

pip install -r requirements.txt

Download YOLOv5 Pre-trained Weights: We’ll use a pre-trained YOLOv5 model on the COCO dataset, which contains classes like cars, trucks, and people.

python detect.py –weights yolov5s.pt –source 0 

Create the Program to Detect Objects in a Video Stream:

Here’s the Python code to run object detection on a video file or live webcam feed.

import torch

import cv2

from yolov5.models.experimental import attempt_load

from yolov5.utils.torch_utils import select_device

from yolov5.utils.general import non_max_suppression, scale_coords

from yolov5.utils.plots import plot_one_box

# Load YOLOv5 model

model = attempt_load(‘yolov5s.pt’, map_location=’cpu’)  # or ‘cuda’ for GPU

model.eval()

device = select_device(‘cpu’)  # Choose ‘cpu’ or ‘cuda’ based on availability

# Open video capture (0 for webcam, or use a video file path)

cap = cv2.VideoCapture(0)

while cap.isOpened():

    ret, frame = cap.read()

    if not ret:

        break

    # Prepare frame for YOLOv5

    img = torch.from_numpy(frame).to(device)

    img = img.permute(2, 0, 1).float() / 255.0

    img = img.unsqueeze(0)

    # Perform inference

    with torch.no_grad():

        pred = model(img)[0]

    pred = non_max_suppression(pred, 0.25, 0.45, classes=None, agnostic=False)

    # Process detections

    for i, det in enumerate(pred): 

        if len(det):

            # Rescale boxes from img_size to original frame size

            det[:, :4] = scale_coords(img.shape[2:], det[:, :4], frame.shape).round()

            for *xyxy, conf, cls in det:

                label = f'{model.names[int(cls)]} {conf:.2f}’

                plot_one_box(xyxy, frame, label=label, color=(0, 255, 0), line_thickness=2)

    # Display the frame with detections

    cv2.imshow(‘YOLOv5 Object Detection’, frame)

    # Press ‘q’ to exit

    if cv2.waitKey(1) & 0xFF == ord(‘q’):

        break

cap.release()

cv2.destroyAllWindows()

Explanation of the program

-pip install torch torchvisionpip install opencv-pythonpip install -r requirements.txt 

Download YOLOv5 Pre-trained Weights

You’ll need pre-trained weights to perform object detection effectively. The pre-trained weights yolov5s.pt are available from the YOLOv5 repository, and these weights are trained on the COCO dataset, which contains classes such as cars, trucks, and people.

You can test the weights using:

 python detect.py –weights yolov5s.pt –source 0

Dectect video objects

The following code runs YOLOv5 on a video or live webcam feed to detect and display bounding boxes around objects.

Explanation of Code

1. Import Libraries:

1. • torch for PyTorch (used by YOLOv5).
   • cv2 for handling video capture and image display.
   • YOLOv5-specific utilities for loading the model, performing non-max suppression (NMS), and plotting bounding boxes.

2. Load YOLOv5 Model:

1. • attempt_load loads the YOLOv5 model weights (yolov5s.pt), which is used to detect objects.
   • The model is loaded on the device specified by select_device, which can be ‘cpu’ or ‘cuda’ (for GPU).

3. Set Up Video Capture:

1. • cv2.VideoCapture(0) initializes video capture from the webcam. You can also specify a file path for a video file instead of 0.

4. Loop Through Frames:

1. • Capture each frame and check if the frame was successfully read.
   • Preprocess the frame for YOLOv5 by converting it to a tensor and adjusting its format to fit the model requirements.
   • Normalize pixel values to the range [0, 1] and add an extra dimension.

5. Perform Object Detection:

1. • With torch.no_grad(), disable gradient calculation for efficiency.
   • Run inference on the prepared frame (model(img)[0]).
   • Apply Non-Max Suppression (NMS) with non_max_suppression to eliminate duplicate detections.

6. Process Detections:

1. • Rescale detected boxes to the original frame size using scale_coords.
   • For each detection, draw a bounding box around the detected object with its label and confidence score.

7. Display Frame:

1. • Display the processed frame using cv2.imshow.
   • Exit the loop by pressing ‘q’.

8. Cleanup:

1. • Release video capture and close the display window.