Section 7.4 Chapter 7 Interactive Mastery Suite (Lab 7 Preparation)

Before embarking on the physical or simulated implementation in LAB 7: End-to-End Autonomous Driving, use these interactive exercises to test your mastery of the data pipelines, regression evaluations, and bias mitigation strategies introduced in Chapter 7.

Subsection 7.4.1 Section 7.4 Interactive Exercises

Subsubsection 7.4.1.1 Exercise 7.4.1: ActiveCode Challenge — Complete the End-to-End Data Augmentation Pipeline

In Section 7.3, we learned that horizontal image flipping paired with steering angle inversion (\(\omega_{\text{new}} = -\omega\)) is critical for eliminating left/right dataset bias (e.g., escaping the "Right-Turn Trap").

Task: Complete the augment_driving_dataset() function below.

Iterate through the recorded dataset of (image_matrix, linear_velocity_v, angular_velocity_w).
For every sample where the robot is turning right (\(\omega > 0.1\)), generate a synthetic left-turn sample by:
- Horizontally flipping the 2D image matrix (reversing each row).
- Keeping linear velocity \(v\) unchanged.
- Inverting the angular velocity (\(\omega_{\text{new}} = -\omega\)).
🔗
🔗
Append both the original and augmented samples to the returned dataset.

def flip_image_horizontally(image_grid):
    """Flips a 2D matrix horizontally by reversing each row."""
    return [row[::-1] for row in image_grid]

def augment_driving_dataset(dataset):
    augmented_dataset = []

for image, v, w in dataset:
        # 1. Keep the original sample
        augmented_dataset.append((image, v, w))

# 2. Check if we need to augment (e.g., if there is a non-zero turn rate)
        if abs(w) > 0.05:
            # TODO: Generate flipped image and inverted angular velocity
            flipped_image = flip_image_horizontally(image)
            inverted_w = -w  # Reverse turn direction!

# Append synthetic sample
            augmented_dataset.append((flipped_image, v, inverted_w))

return augmented_dataset

# Simulated Training Set (3x3 mock image grids, linear velocity v, angular velocity w)
# Sample 1: Sharp Right Turn (w = 0.8)
sample_1_image = [
    [0, 0, 1],
    [0, 0, 1],
    [0, 0, 1]
]

# Sample 2: Driving Straight (w = 0.0)
sample_2_image = [
    [0, 1, 0],
    [0, 1, 0],
    [0, 1, 0]
]

raw_dataset = [
    (sample_1_image, 0.5, 0.8),  # (Image, v, w) -> Right turn
    (sample_2_image, 0.5, 0.0)   # (Image, v, w) -> Straight
]

# Run Augmentation
balanced_dataset = augment_driving_dataset(raw_dataset)

print(f"Original Dataset Size: {len(raw_dataset)} samples")
print(f"Augmented Dataset Size: {len(balanced_dataset)} samples\n")

print("--- Augmented Dataset Breakdown ---")
for i, (img, v, w) in enumerate(balanced_dataset, 1):
    turn_type = "RIGHT TURN" if w > 0 else ("LEFT TURN" if w < 0 else "STRAIGHT")
    print(f"Sample {i}: v = {v:.1f} m/s | w = {w:.1f} rad/s -> [{turn_type}]")
    print(f"Image Row 0: {img[0]}")

Subsubsection 7.4.1.2 Exercise 7.4.2: Parsons Problem — End-to-End Behavioral Cloning Execution Pipeline

Reorder the steps below to form the correct sequence for collecting data, mitigating dataset bias, training an end-to-end regression model, and deploying it on physical or simulated hardware for Lab 7.

Checkpoint 7.4.1.

Arrange the steps in the order they must occur, from initial teleoperated data collection through final autonomous deployment.

Teleoperate the robot along field tracks while recording synchronized camera frames and (v, w) joystick commands.
---
Perform data inspection to identify imbalances (e.g., over-representation of right turns).
---
Apply horizontal image flipping and steering inversion to balance left/right turn distributions.
---
Preprocess image pixels and train a regression model to minimize Mean Absolute Error (MAE) against human control labels.
---
Evaluate trained model on test data to ensure MAE < 0.1 before hardware deployment.
---
Deploy trained model weights to the robot to drive autonomously in real time from live camera inputs.

Subsubsection 7.4.1.3 Exercise 7.4.3: Conceptual Check — Troubleshooting Behavioral Cloning Failures

Scenario Matching Check

Match each robot failure scenario observed during testing to its underlying architectural or dataset root cause.

Checkpoint 7.4.2.

Observed Failure Behavior	Root Cause
1. "Wall Trapping": The robot drives straight down a hallway fine, but as soon as it drifts slightly off-center toward a wall, it fails to recover and crashes straight into it.	A. Environmental/Lighting Bias: The training dataset was recorded under different illumination conditions than the test arena.
2. "One-Way Bias": When encountering a symmetrical fork in the path, the robot always steers right, even when an obstacle blocks the right path.	B. Lack of Recovery Demonstrations: The human driver only recorded smooth, centered driving data and never demonstrated how to steer away from walls.
3. "Sunlight Hesitation": The robot navigates correctly indoors under fluorescent lighting, but stops randomly or turns wildly when driven near outdoor windows.	C. Imbalanced Turn Distribution: The training set suffered from the "Right-Turn Trap" (e.g., recorded on a clockwise track).

Answer.

1 → B (If a human only recorded perfect centered driving, the robot never learned how to recover from near-wall positions.)
2 → C (Always favoring one steering direction indicates an imbalanced training set.)
3 → A (Failing under new glare or sunlight conditions stems from environmental/lighting bias.)

Reading Questions 7.4.2 Reading Questions

Check your understanding

1. Exercise 7.4.4: Regression Metrics for End-to-End Systems Conceptual Check.

When evaluating an end-to-end behavioral cloning model in Python before deploying it to physical hardware in Lab 7, why do we evaluate performance using regression metrics like Mean Absolute Error (MAE) rather than classification accuracy (e.g., 95% correct)?

End-to-End models output continuous velocity numbers \((v, \omega)\text{,}\) making classification accuracy mathematically undefined without arbitrary threshold bins.
Correct! Motor commands like linear velocity \(v\) in m/s and turn rate \(\omega\) in rad/s are continuous variables. Because there are infinite possible numerical velocity values, we measure performance using regression distance metrics like MAE or MSE rather than discrete class accuracy.
Regression metrics run faster on microcontrollers than classification metrics.
Incorrect. The choice between regression and classification metrics is about the nature of the output (continuous vs. discrete), not computational speed.
MAE automatically corrects for physical wheel friction during runtime.
Incorrect. MAE is an offline evaluation metric; it measures prediction error and does not adjust anything during runtime.
Classification accuracy can only be used on black-and-white images, whereas regression works on RGB images.
Incorrect. Both classification and regression models can take RGB images as input; the distinction is in the type of output they predict.

You have attempted of activities on this page.

Prev Top Next