he-hongjian.com
EN中文

Engineering MVP / Prototype

BibVision

Offline race bib recognition and searchable video events.

Detection → OCR → Temporal Stabilization → Video Indexing → Clip Planning

Multi-person bib-detection video.
AI-generated concept visual.
AI-generated concept image of multiple runners

01 / Problem & constraints

From observations to searchable events.

Race footage introduces several practical challenges:

  • small and blurred bib numbers
  • multiple athletes in the same frame
  • unstable OCR across adjacent frames
  • repeated observations of the same bib
  • fragmented source video files

BibVision turns visual observations into stable, searchable bib events linked to video time.

02 / System design

A layered offline pipeline.

Bib Detection · Iteration V3: the PC workflow connects bib detection and OCR to searchable video events and clip planning. Android experiment · RTMDet-Tiny: a separate Kotlin project explores on-device inference.

INTERACTIVE SYSTEM MAP

BibVision

Offline bib recognition and searchable video events

Engineering MVP / PrototypeBib Detection · Iteration V3

Input

Video or still image

Perception

Locate and validate complete bib candidates

Temporal stabilization

Combine evidence across nearby frames

Indexing & output

Searchable offline event index

Stage 5.5 · Direct route

Per-file observations map to a shared timeline, then to Presence intervals and ClipPlan alongside SQLite indexing.

Key design ideas

  1. Aggregate temporal evidence by complete OCR string
  2. Validate candidates while retaining raw observations
  3. Keep source and time context with each event

Scope boundaries

  1. PC Bib Detection · Iteration V3
  2. The Stage 5.5 Presence / ClipPlan route can bypass SQLite
  3. ClipPlan records source ranges and edit order

03 / Key engineering decisions

Small decisions that make the output auditable.

01

Temporal reasoning over single-frame OCR

Nearby frames add evidence for whole-string candidate voting.

02

Domain-specific bib validation

Event-specific formats refine candidates while raw OCR evidence stays traceable.

03

Logical timeline for fragmented video

Map each source fragment's local timestamps to a shared logical timeline.

04

Traceable evaluation and provenance

Frozen data, fixed protocols, model versions, and artifact records keep comparisons traceable.

04 / Evaluation

A frozen bib-detection benchmark.

SAVED DETECTION BENCHMARK SUMMARY

Core metric comparison

50 images · 66 GT boxes

Benchmark scope

50images

66GT boxes

2negative images

Evaluation protocol

Input size
960
Standard AP confidence
0.001
Fixed confidence threshold
0.25
One-to-one matching IoU
0.50

Core metric comparison

V2 baselineV3 prototype baselineScore range 0–1

Exact metric values for copying
MetricV2 baselineV3 prototype baseline
mAP50 0.544060.92273
mAP50:95 0.300520.57318
Precision @ fixed threshold 0.729730.86765
Recall @ fixed threshold 0.409090.89394
F1 @ fixed threshold 0.524270.88060

GT match transitions (66 boxes)

GT transitionCount
TP in both versions27
V2 miss recovered by V332
V2 TP missed by V30
Missed by both versions7
Matching terms:TP FP FN

Source: saved Stage 5.4 detection comparison summary.

Bib-detection metrics on the fixed dataset.

05 / Temporal OCR stabilization

Combine evidence across nearby frames.

Nearby frames provide OCR observations for whole-string voting. Raw candidates remain traceable.

ILLUSTRATIVE EXAMPLE

Aggregate temporal evidence by complete bib string

A0382 = 3.36 · A038B = 0.43 raw

Select a frame to inspect its raw observation

Raw observed weight by complete string

A03823.36

A038B0.43

Eligible weight after the example format gate

A03823.36

A038BExcluded from eligible candidate total

A038B remains in the raw observation view; the eligible vote total reflects candidates that pass the example format rule.

Format validation

Example format: one letter + four digits. A038B remains visible as raw OCR and is excluded from this example vote.

ShortTrack

Associates nearby-frame observations on a local track.

Whole-string vote

Weights aggregate complete observed bib strings.

STABLE

Marks completion of the configured voting rule; the OCR string remains a recognition result.

Single-runner technical video.
AI-generated concept visual.
AI-generated concept image of a single runner

06 / Video indexing & clip planning

Plan the edit before rendering the video.

Source fragments retain local timestamps on a logical global timeline. The indexed-replay path stores weighted decisions as BibEvent records in SQLite before generating its replay plan.

A separate continuous-file path connects per-file observations directly to Presence intervals and ClipPlan alongside the indexed SQLite workflow.

ClipPlan records source ranges and edit decisions for later rendering.

SCRIPT OUTPUT · DETECTION BOXES + RECOGNIZED BIB LABELS

Browse the bib-recognition output

Drag the timeline or select a thumbnail to seek to that moment.

0:16
Current time 0:00Duration 0:16

OBSERVATIONS FROM THIS SCRIPT OUTPUT

Observed bib time spans

Select a bib to highlight its first-to-last observed span and seek to its first observation.

32 bibs
100First observed: 0:02.302Last observed: 0:02.302
Browse by frameBib-detection script output.

Underlying footage by Miguel Guerra / Pexels

Expand the separate teaching timeline example

Separate timeline example.

ILLUSTRATIVE EXAMPLE

From video segments to a replay plan

Choose an example bib to highlight its events, presence intervals, and clip suggestions

Selected example: Bib A · related items highlighted

  1. 01Segmented video
  2. 02Unified logical timeline
  3. 03Bib presence intervals
  4. 04Clip suggestions & replay preview
01

Segmented video

Keep each segment’s local clock and follow observations across the shared timeline.

3 × 15s
Segment A0s–15s

Local segment time: 0–15s

0s15s
Logical start 0s
Segment B15s–30s

Local segment time: 0–15s

0s15s
Logical start 15s
Segment C30s–45s

Local segment time: 0–15s

0s15s
Logical start 30s
Bib ABib B
02

Unified logical timeline

Map local event positions onto one continuous sequence.

0–45s
0s15s30s45s

Logical time follows segment order and local event positions; source segments remain separate.

Bib A4sBib A11sBib A17sBib B24sBib A33sBib B42s
03

Bib presence intervals

Group observations for the same example bib into time ranges.

45s
Bib A4s–18s
Bib A33s–37s
Bib B24s–27s
Bib B42s–44s
0s15s30s45s
04

Clip suggestions & replay preview

Collect relevant observation ranges into a clip plan.

Clip suggestionBib A · 4s–18s2 source segments
Clip suggestionBib A · 33s–37s1 source segment
Clip suggestionBib B · 24s–27s1 source segment
Clip suggestionBib B · 42s–44s1 source segment
Replay preview conceptEDIT PLAN
Bib ALogical range: 4s–18s

Static edit-plan preview.

Teaching example: map segmented observations into a replay ClipPlan.

07 / Results, limitations & lessons learned

Implemented workflows and current scope.

Implemented

  • offline recognition pipeline
  • temporal bib stabilization
  • SQLite video indexing
  • continuous logical timeline
  • bib presence aggregation
  • ClipPlan and replay workflow
  • frozen detection benchmark

Current scope & open checks

  • Short-term local-track association
  • Offline video ingestion
  • Real-time performance: pending validation
  • OCR quality depends on bib size, blur, and occlusion
  • ShortTrack links nearby observations on local tracks
  • Application-level ground truth and clean-environment reproduction: pending. Android STABLE marks voting-rule completion; OCR output remains a recognized candidate. Controlled motion, occlusion recovery, and multi-bib acceptance: pending.

Lessons

  • Data quality matters alongside quantity.
  • Detection metrics describe detector performance; OCR quality is evaluated separately.
  • Video preserves temporal evidence across frames.
  • Evaluation sets should remain frozen once used as a benchmark.
← Back to Projects