Skip to content

Seq-NMS Cross-Sucking Detector

This page documents how to use scripts/run_models/seq-NMS/seq_NMS.py to run cross-sucking detection using a fine-tuned YOLOv26 model with Sequential Non-Maximum Suppression (Seq-NMS) for temporal linking.

Summary

This script provides a standalone implementation of the YOLOv26 + Seq-NMS detection pipeline. While yolo.py runs both YOLO and Seq-NMS together on a full test split CSV, this script runs Seq-NMS only on a single video file, making it useful for testing and debugging on individual videos.

The key difference from the baseline and plain YOLO approaches is temporal linking via Seq-NMS:

  1. Fine-tuned YOLOv26 detects cross-sucking bounding boxes per frame
  2. Seq-NMS links detections across consecutive frames into tubes based on spatial IoU overlap
  3. Weak detections within tubes are suppressed
  4. Tubes are converted into event windows with defined start and end times

The output JSON format matches baseline.py so evaluation.py and clipping.py work unchanged.

Environment Setup

Before running this script, ensure your local workspace satisfies the following:

  • .env is populated with ROOT_DIR and LOCAL_DIR (used by config.py).
  • Fine-tuned YOLO model weights exist at the path specified by --model_path, produced by running scripts/training/training_yolo.py.

Output

Results are saved to --output_dir:

<output_dir>/
└── <video_name>_results.json

{
  "identifier": "ch02_20251104074042.mp4",
  "model": "weights/split_1/best.pt",
  "conf_threshold": 0.15,
  "iou_threshold": 0.3,
  "min_duration_sec": 1.0,
  "frame_skip": 30,
  "fps": 30.0,
  "total_frames": 97200,
  "cross_sucking_detected": true,
  "num_events": 3,
  "events": [
    {
      "start_sec": 154.3,
      "end_sec": 167.8,
      "duration_sec": 13.5,
      "avg_confidence": 0.612,
      "intersection_box": [...]
    }
  ]
}

How it works

  1. run_seq_nms_detection loads the fine-tuned YOLOv26 model and opens the source video.
  2. For every Nth frame (controlled by --frame_skip), YOLO inference is run and all detections above --conf_threshold are collected.
  3. build_tubes links detections across consecutive frames using spatial IoU — detections in adjacent frames that overlap above --iou_threshold are grouped into the same tube.
  4. suppress_weak_detections removes frames within each tube where confidence is below --conf_threshold.
  5. tubes_to_events converts surviving tubes into event windows, discarding any tube shorter than --min_duration.
  6. Results are saved as a structured JSON file.

Edge Cases

"Target class not found" If the fine-tuned model does not contain the cross-sucking class (e.g. when testing with pretrained weights), the script falls back to the cross-sucking class if available, or raises a ValueError if neither class is found.

"No detections on a frame" Frames with no detections above --conf_threshold are stored as empty lists. Seq-NMS treats these as breaks between tubes — any active tube is closed when an empty frame is encountered.

Core Pipeline Concepts

Tubes

A tube is a sequence of bounding boxes linked across consecutive frames that all belong to the same detected interaction. Seq-NMS builds tubes by checking whether each new detection overlaps with the last box in any active tube above --iou_threshold. If it does, the tube is extended. If not, a new tube is started.

Why Seq-NMS over baseline

The baseline script flags any frame where two calves overlap sufficiently, treating each frame independently. This produces fragmented detections that don't naturally group into events. Seq-NMS solves this by explicitly linking detections through time, producing cleaner event boundaries and filtering out brief spurious detections via --min_duration.

Limitation

Seq-NMS links detections based purely on spatial overlap between consecutive frames. If two calves shift position significantly during a cross-sucking interaction, the bounding box location may change enough that Seq-NMS breaks the tube into two separate events, inflating event counts and reducing measured duration accuracy.

Usage

Basic usage

uv run scripts/run_models/seq-NMS/seq_NMS.py \
    --video_path "$ROOT_DIR/raw_cross_sucking_datalog/videos/Pen2/POSTWEANING/Day3/ch02_20251104074042.mp4" \
    --model_path "data/yolo_training_runs/pipeline_demo/demo_01/weights/best.pt" \
    --output_dir "results/metadata/pipeline_demo/seq-nms"

Run on a sample clip with video display

uv run scripts/run_models/seq-NMS/seq_NMS.py \
    --video_path "sample_videos/cross_sucking_clip_sample/CS_0276_WEAN_d1_p2_cowT_16102025_ch02-20251016124717_19033_19045.mp4" \
    --model_path "data/yolo_training_runs/pipeline_demo/demo_01/weights/best.pt" \
    --output_dir "results/demo" \
    --show_video

Run with custom thresholds

uv run scripts/run_models/seq-NMS/seq_NMS.py \
    --video_path "$ROOT_DIR/raw_cross_sucking_datalog/videos/Pen2/POSTWEANING/Day3/ch02_20251104074042.mp4" \
    --model_path "data/yolo_training_runs/pipeline_demo/demo_01/weights/best.pt" \
    --output_dir "results/metadata/pipeline_demo/seq-nms" \
    --conf_threshold 0.25 \
    --iou_threshold 0.3 \
    --min_duration 1.0 \
    --frame_skip 30

Arguments

Argument Type Default Description
--video_path str required Path to input video file
--model_path str required Path to fine-tuned YOLO weights file
--output_dir str required Directory to save metadata output
--conf_threshold float 0.15 Minimum YOLO detection confidence to keep a box
--iou_threshold float 0.3 Minimum spatial IoU to link detections into tubes
--min_duration float 1.0 Minimum event duration in seconds
--frame_skip int 30 Process every Nth frame
--target_class str cross-sucking Target class for detection
--show_video flag False Display annotated video during inference

Function Reference

scripts.run_models.seq_NMS.seq_NMS

seq_nms_inference.py

Inference script for cross-sucking detection using a fine-tuned YOLOv26 model with Seq-NMS (Sequential Non-Maximum Suppression) for temporal linking.

Unlike the baseline script which flags frames based on simple bounding box overlap, this script: 1. Runs fine-tuned YOLOv26 on each frame to detect cross-sucking directly 2. Collects all per-frame detections across the full video 3. Applies Seq-NMS to link detections across frames into consistent tubes 4. Converts tubes into event windows with start/end times

The output JSON format matches baseline.py so evaluation.py works unchanged.

Usage: uv run python scripts/models/seq_NMS/seq_NMS.py --video "ROOT_DIR/raw_cross_sucking_datalog/videos/Pen 2 - Group 2/WEANING/Day 2/ch02_20251018035506.mp4" --model weights/split_1/best.pt --output_dir=results/test --frame_skip 30 --show_video

compute_iou(box_a, box_b)

Compute IoU between two bounding boxes.

Used by Seq-NMS to decide whether two detections in consecutive frames belong to the same tube.

Parameters:

Name Type Description Default
box_a list[x1, y1, x2, y2]

Bounding boxes in pixel coordinates.

 required
box_b list[x1, y1, x2, y2]

Bounding boxes in pixel coordinates.

 required

Returns:

Type Description
float

IoU score between 0 and 1.

build_tubes(frame_detections, iou_threshold)

Link per-frame detections into tubes using Seq-NMS.

A tube is a sequence of bounding boxes linked across consecutive frames that all belong to the same detected event. Think of it as tracking one cross-sucking interaction through time.

How it works: For each frame, for each detection in that frame: - Look at all active tubes from the previous frame - If this detection overlaps with the last box in a tube above iou_threshold, extend that tube - Otherwise start a new tube

Parameters:

Name Type Description Default
frame_detections list of list of dict

Per-frame detections. Each entry is a list of detections for that frame. Each detection is a dict with keys: frame (int), x1, y1, x2, y2 (float), confidence (float)

 required
iou_threshold float

Minimum IoU between consecutive boxes to link them into the same tube.

 required

Returns:

Type Description
list of list of dict

Each tube is a list of detection dicts linked across frames. Example: [ [ # tube 1 {"frame": 10, "x1": 100, "y1": 200, "x2": 300, "y2": 400, "confidence": 0.8}, {"frame": 11, "x1": 102, "y1": 201, "x2": 302, "y2": 401, "confidence": 0.75}, ], [ # tube 2 ... ] ]

suppress_weak_detections(tubes, conf_threshold)

Remove low confidence detections from within each tube.

After linking, some frames within a tube may have low confidence detections. This step removes them to clean up the tube, keeping only frames where the model was confident enough.

Tubes that become empty after suppression are discarded.

Parameters:

Name Type Description Default
tubes list

Output of build_tubes().

 required
conf_threshold float

Minimum confidence score to keep a detection within a tube.

 required

Returns:

Type Description
list

Cleaned tubes with low confidence detections removed.

tubes_to_events(tubes, fps, min_duration)

Convert Seq-NMS tubes into event dictionaries.

Takes the linked tubes and converts them into the same event format used by baseline.py so the rest of the pipeline (clipping, evaluation) works without any changes.

Tubes shorter than min_duration are discarded as noise.

Parameters:

Name Type Description Default
tubes list

Output of suppress_weak_detections().

 required
fps float

Frames per second of the video — used to convert frame numbers to timestamps in seconds.

 required
min_duration float

Minimum event duration in seconds. Shorter tubes are discarded.

 required

Returns:

Type Description
list of dict

Each dict is one event with keys: start_sec, end_sec, duration_sec, avg_confidence, intersection_box (per-frame box coordinates)

run_seq_nms_detection(video_path, model_path, output_dir, conf_threshold, iou_threshold, min_duration, frame_skip, target_class, show_video=False)

Full YOLOv26 + Seq-NMS detection pipeline.

Steps: 1. Load fine-tuned YOLOv26 model 2. Open video and process every Nth frame 3. Run YOLO inference on each frame 4. Collect all per-frame detections 5. Apply Seq-NMS to link detections into tubes 6. Suppress weak detections within tubes 7. Convert tubes to event windows 8. Save results as JSON

Parameters:

Name Type Description Default
video_path str

Path to input video file.

 required
model_path str

Path to fine-tuned YOLO weights file (e.g. best.pt).

 required
conf_threshold float

Minimum YOLO detection confidence to keep a box.

 required
iou_threshold float

Minimum spatial IoU to link boxes across frames into tubes.

 required
min_duration float

Minimum event duration in seconds.

 required
frame_skip int

Process every Nth frame. 1 = every frame.

 required

Returns:

Type Description
dict

Full metadata dict, also saved as JSON to results/metadata/seq_nms/_results.json