Fine-Tuned YOLO Detector

This page documents how to use scripts/run_models/yolo/yolo.py to run cross-sucking detection using a fine-tuned YOLOv26 model, producing structured JSON event metadata for downstream evaluation and clipping.

Summary

This script runs a fine-tuned YOLOv26 model frame-by-frame on source videos from a test split CSV, detecting cross-sucking events and saving structured metadata. Unlike the baseline script which flags frames based on simple bounding box overlap between any two calves, this script detects cross-sucking directly using a model fine-tuned on labeled cross-sucking clips.

The script runs two detection approaches in a single execution:

YOLO — groups consecutive frames with detections into events using a buffer-based approach. Frames within --buffer seconds of each other are merged into a single event.
Seq-NMS — links detections across consecutive frames using spatial IoU into tubes, then converts tubes into event windows. See the Seq-NMS page for more detail on the algorithm.

Both outputs are saved in separate subfolders under --output_dir, in the same JSON format as baseline.py so evaluation.py works unchanged.

Environment Setup

Before running this script, ensure your local workspace satisfies the following:

.env is populated with ROOT_DIR and LOCAL_DIR (used by config.py).
Source videos exist under SOURCE_VIDEOS_DIR.
A test split CSV exists under data/processed/<split_name>/test.csv, produced by running scripts/split_data/split_data.py.
Fine-tuned YOLO model weights exist at the path specified by --model_path, produced by running scripts/training/training_yolo.py.

Output

Results are saved to two subfolders under --output_dir:

<output_dir>/
├── yolo/
│   ├── ch02_20250913094601_results.json   # YOLO buffer-based events
│   └── ch02_20250913094602_results.json
└── seq-nms/
    ├── ch02_20250913094601_results.json   # YOLO + Seq-NMS events
    └── ch02_20250913094602_results.json

Each JSON file contains event timestamps, confidence scores, and per-frame bounding box coordinates:

{
  "identifier": "ch02_20250913094601.mp4",
  "model": "weights/split_1/best.pt",
  "conf_threshold": 0.1,
  "iou_threshold": 0.0,
  "num_events": 2,
  "events": [
    {
      "start_sec": 12.4,
      "end_sec": 19.1,
      "duration_sec": 6.7,
      "avg_confidence": 0.71,
      "intersection_box": [...]
    }
  ]
}

How it works

collect_frames opens the source video and runs fine-tuned YOLOv26 inference on every Nth frame, collecting all per-frame detections with bounding box coordinates and confidence scores.
extract_events groups consecutive detections into events using a buffer-based approach — frames within --buffer seconds of each other are merged into a single event. Results are saved to yolo/.
build_tubes (from seq_NMS.py) links detections across frames using spatial IoU into consistent tubes. suppress_weak_detections removes low confidence frames within tubes. tubes_to_events converts tubes into event windows. Results are saved to seq-nms/.
build_metadata assembles both sets of events into structured JSON files and saves them to their respective output subdirectories.

Usage

Basic usage

uv run scripts/run_models/yolo/yolo.py \
    --video_path "$ROOT_DIR/raw_cross_sucking_datalog/videos/Pen2/POSTWEANING/Day3/ch02_20251104074042.mp4" \
    --model_path "data/yolo_training_runs/pipeline_demo/demo_01/weights/best.pt" \
    --output_dir "results/metadata/pipeline_demo"

Run with frame skipping for faster processing

uv run scripts/run_models/yolo/yolo.py \
    --video_path "$ROOT_DIR/raw_cross_sucking_datalog/videos/Pen2/POSTWEANING/Day3/ch02_20251104074042.mp4" \
    --model_path "data/yolo_training_runs/pipeline_demo/demo_01/weights/best.pt" \
    --output_dir "results/metadata/pipeline_demo" \
    --frame_skip 30

Show video during inference

uv run scripts/run_models/yolo/yolo.py \
    --video_path "$ROOT_DIR/raw_cross_sucking_datalog/videos/Pen2/POSTWEANING/Day3/ch02_20251104074042.mp4" \
    --model_path "data/yolo_training_runs/pipeline_demo/demo_01/weights/best.pt" \
    --output_dir "results/metadata/pipeline_demo" \
    --show_video

Run with custom thresholds

uv run scripts/run_models/yolo/yolo.py \
    --video_path "$ROOT_DIR/raw_cross_sucking_datalog/videos/Pen2/POSTWEANING/Day3/ch02_20251104074042.mp4" \
    --model_path "data/yolo_training_runs/pipeline_demo/demo_01/weights/best.pt" \
    --output_dir "results/metadata/pipeline_demo" \
    --conf_threshold 0.25 \
    --iou_threshold 0.3 \
    --min_duration 1.0

Arguments

Argument	Type	Default	Description
`--video_path`	str	required	Path to input source video file
`--model_path`	str	required	Path to fine-tuned YOLO weights file
`--output_dir`	str	required	Directory to save metadata output
`--conf_threshold`	float	`0.1`	Minimum YOLO detection confidence to keep a box
`--iou_threshold`	float	`0.0`	Minimum spatial IoU to link boxes across frames into tubes
`--min_duration`	float	`0.0`	Minimum event duration in seconds
`--frame_skip`	int	`10`	Process every Nth frame
`--buffer`	int	`60`	Seconds without detection before ending a YOLO event
`--target_class`	str	`cross-sucking`	Target class for detection
`--show_video`	flag	`False`	Display annotated video during inference

Function Reference

`scripts.run_models.yolo.yolo`

Module for running YOLO models to detect cross-sucking events and output event metadata.

USE THIS TO RUN YOLO AND SEQ_NMS

NOTE: This file is meant to send clips for human review: add pre and post buffer of 10-30s for each video meta-data output to ensure we capture the whole event

NOTE: Redo Docstrings

NOTE: Make Tests

`extract_events(fps, buffer, frame_detections)`

# NOTE: This is not designed for event tracking!

Flagging frames into events with start/end timestamps and collect the intersection box coordinates for every flagged frame within each event.

Args: frame_flags (list[bool]): Per-frame overlap flag — True if overlap detected fps (float): Frames per second of the video, used to convert frame numbers into timestamps in seconds min_duration (float): Minimum event length in seconds to keep confidences (list[float]): Per-frame max YOLO detection confidence frame_boxes (list): Per-frame intersection box [x1, y1, x2, y2], or None if that frame was not flagged frame_indices (list): Actual frame indices in the video (accounting for frame skip)

Returns: list[dict]: Each dict represents one flagged event: - start_sec (float): Event start time in seconds - end_sec (float): Event end time in seconds - duration_sec (float): Total event duration in seconds - avg_confidence (float): Average YOLO detection confidence across all frames in the event - intersection_box (list[dict]): Per-frame intersection coordinates, each entry has 'frame', 'x1', 'y1', 'x2', 'y2'

`collect_frames(model, target_ids, video_path, frame_skip, conf_threshold, show_video=False)`

`build_metadata(video_path, output_dir, model_path, conf_threshold, iou_threshold, min_duration, frame_skip, fps, total_frames, events)`

`run_models(model, model_path, video_path, output_dir, conf_threshold, iou_threshold, min_duration, buffer, frame_skip, target_class='cross-sucking', show_video=False)`

Full YOLOv26 + basic detection pipeline.

Steps: 1. Load fine-tuned YOLOv26 model 2. Open video and process every Nth frame 3. Run YOLO inference on each frame 4. Collect all per-frame detections 5. Convert tubes to event windows 6. Save results as JSON

Parameters:

Name	Type	Description	Default
`video_path`	`str`	Path to input video file.	required
`model_path`	`str`	Path to fine-tuned YOLO weights file (e.g. best.pt).	required
`conf_threshold`	`float`	Minimum YOLO detection confidence to keep a box.	required
`iou_threshold`	`float`	Minimum spatial IoU to link boxes across frames into tubes.	required
`min_duration`	`float`	Minimum event duration in seconds.	required
`frame_skip`	`int`	Process every Nth frame. 1 = every frame.	required

Returns:

Type	Description
`dict`	Full metadata dict, also saved as JSON to results/metadata/seq_nms/_results.json