Section 7.2 Training Loops & Regression Evaluation

Because steering a robot involves selecting a value along a continuous spectrum (e.g., steering anywhere from \(-30^\circ\) to \(+30^\circ\)), behavioral cloning operates as a regression task.

To determine how accurately our model has cloned the expert’s behavior during a training loop, we must evaluate its performance mathematically. The standard metric used to calculate this error is the Mean Squared Error (MSE). MSE measures the average squared difference between the model’s predicted steering values and the true expert labels across the dataset:

\begin{equation*} \operatorname{MSE} = \frac{1}{n} \sum_{i=1}^{n} \left(y_i - \hat{y}_i\right)^2 \end{equation*}

Where:

\(y_i\) is the actual target command provided by the expert driver.
\(\hat{y}_i\) is the command predicted by the autonomous driving network.
\(n\) is the total number of data samples evaluated.

The Data Collection Loop

A behavioral cloning model is only as good as the data used to train it. The end-to-end training loop follows four main steps: Record Human Drive → Preprocess Data → Train Model (Regression) → Evaluate & Deploy.

Recording: Capture synchronized camera frames (30 Hz) and control actions \((v, \omega)\text{.}\)
Preprocessing: Normalize image pixel values and scale control inputs (e.g., mapping motor values to \([-1.0, 1.0]\)).
Training (Regression Task): Because motor outputs like velocity \(v\) and turn rate \(\omega\) are continuous numerical values, behavioral cloning treats control estimation as a regression problem. The training algorithm minimizes the difference between human actions \(y_{\text{human}}\) and model predictions \(\hat{y}_{\text{model}}\text{.}\)
Evaluation: Calculate error metrics such as Mean Squared Error (MSE) before deploying the model to physical hardware.

Subsection 7.2.1 Section 7.2 Interactive Exercises

Figure 7.2.1. Traditional modular pipelines chain together separate object detection and path planning stages, while end-to-end pipelines map vision input directly to motor control through a single ML model.

Subsubsection 7.2.1.1 Exercise 7.2.1: ActiveCode Exercise — Building a Motor Command Regression Evaluation Loop

Note: Because Runestone CodeLens runs standard Python without external machine learning libraries, we implement a pure-Python regression evaluation loop below!

In this exercise, you will complete a function that evaluates a trained end-to-end control model against a mock dataset of human expert steering actions \((v, \omega)\text{.}\)

Task:

Compute the Mean Absolute Error (MAE) between the human’s linear velocity commands and the model’s predicted commands.
Print whether the model passes safety evaluation (Average Error \(< 0.1\) m/s).

def evaluate_driving_model(human_dataset, model_predictions):
    """
    Evaluates model accuracy against expert human driver commands.
    Both inputs are lists of tuples: (linear_velocity_v, angular_velocity_w)
    """
    total_v_error = 0.0
    total_w_error = 0.0
    num_samples = len(human_dataset)

for i in range(num_samples):
        v_human, w_human = human_dataset[i]
        v_model, w_model = model_predictions[i]

# Calculate absolute errors
        total_v_error += abs(v_human - v_model)
        total_w_error += abs(w_human - w_model)

mae_v = total_v_error / num_samples
    mae_w = total_w_error / num_samples

return mae_v, mae_w

# Mock Dataset: (linear_velocity v in m/s, angular_velocity w in rad/s)
human_expert_data = [
    (0.5, 0.0),   # Drive straight
    (0.5, 0.2),   # Slight right turn
    (0.0, 0.0),   # Stop at obstacle
    (0.4, -0.3),  # Left turn
    (0.0, 0.0)    # Stop at goal
]

# Model Predictions from an End-to-End Network
model_predictions = [
    (0.48, 0.01),  # Close to straight
    (0.52, 0.18),  # Close to slight right
    (0.02, 0.00),  # Almost stopped
    (0.38, -0.28), # Close to left turn
    (0.01, 0.01)   # Almost stopped
]

# Evaluate performance
error_v, error_w = evaluate_driving_model(human_expert_data, model_predictions)

print(f"Linear Velocity MAE (v): {error_v:.4f} m/s")
print(f"Angular Velocity MAE (w): {error_w:.4f} rad/s")

# Safety Threshold Check
if error_v < 0.1 and error_w < 0.1:
    print("STATUS: Model Evaluation PASSED! Safe for robot deployment.")
else:
    print("STATUS: Model Evaluation FAILED! Error too high.")

Subsubsection 7.2.1.2 Exercise 7.2.2: Parsons Problem — End-to-End Data Pipeline Execution

Reorder the steps below to form the correct sequence for training and deploying an end-to-end behavioral cloning robot.

Checkpoint 7.2.2.

Arrange the steps in the order they must occur to collect data, train, and deploy an end-to-end behavioral cloning model.

Mount camera and connect human joystick controller to robot platform.
---
Teleoperate the robot along target paths while recording (Camera Image, Motor Speed) data pairs.
---
Preprocess dataset and normalize motor output targets.
---
Train regression model to minimize difference between predicted and human motor outputs.
---
Deploy trained model to robot onboard computer to drive autonomously from camera feed.

You have attempted of activities on this page.

Prev Top Next