Downstream evaluation: vanishing points

How useful are the segments? Calibrated Manhattan-frame (camera-rotation) estimation on York Urban (outdoor) and NYU (indoor), plus a full error-source decomposition and a fair "best estimator per detector" protocol.

1. Protocol

Hand-labelled line annotations are incomplete and geometrically coarse, so detectors are NOT scored against them. Instead each detector's segments feed the same calibrated Manhattan-frame estimator, and the recovered three orthogonal vanishing directions are compared with the datasets' reliable rotation ground truth. York Urban Database: 102 outdoor images. NYU-VP: 1033 indoor images. (The dataset images are not redistributed here — tools/yud_export.py / tools/nyu_export.py build the manifests from the official downloads; this page is numbers-only.)

2. Results with a fixed common estimator

detector	York Urban (outdoor) median err	NYU (indoor) median err
SweepLSD	1.04°	12.7°
SweepLSD + optional linker	0.99°	10.7°
LSD	0.92°	10.7°
EDLines (ED_Lib)	0.98°	10.4°
ELSED	0.96°	10.4°
hand-labelled GT lines (estimator ceiling)	0.59°	6.0°

ELSED and the linker rows were measured in a verification re-run with the same estimator (the re-run reproduces the other rows within ≤0.2°). The linker row is the interesting one: re-assembling fragments across junction cuts feeds directly through to downstream support — indoors it moves SweepLSD from 12.7° into the LSD/ELSED band.

Read indoor medians with care. The indoor error distribution is bimodal: an image either succeeds (≈1.5°) or the frame search locks onto the wrong structure (≈27°). The median is just the mixing ratio of those two modes — even the hand-labelled ceiling sits at 6°. NYU is a clutter-robustness stress test, not a precision benchmark.

3. Where the differences actually come from

A per-line decomposition against the ground-truth axes (per-line residual angle, inlier support per axis, oracle/refinement splits) shows:

Per-line direction quality is best for SweepLSD (median residual vs the GT axes 0.51° < LSD 0.57° < ED_Lib 0.65° on York Urban) — its measured advantage in segment geometry is real on real photos too.
The ranking is driven by inlier support, especially on the weakest axis. SweepLSD's contrast-gated edge model misses soft low-contrast structure, so it brings ~20% less supporting line length. Subsampling LSD to SweepLSD's line count reproduces SweepLSD's outdoor score almost exactly (0.92°→1.05° vs 1.04°) — the gap is quantity, not quality.
Indoors, all methods fail on largely the same images (bottom-quartile weak-axis support ⇒ 100% failure for every detector). On the 360 images where all three succeed they are statistically indistinguishable (1.56° / 1.53° / 1.48°).

4. The estimator choice flips the ranking (fair protocol)

Because the differences are support-driven, they depend strongly on how the estimator weighs lines. A declared menu of twelve estimator variants, identical for every detector (line weighting × inlier band × search strength × vertical prior), was evaluated with selection and evaluation on separate data (split-half cross-validation within each dataset: variants selected by median error on one half, scored on the other, test halves pooled):

dataset	detector	median err (CV)	<2°	<5°	detect ms	VP est. ms	total ms
NYU (indoor), split-half CV	SweepLSD	6.08°	33.2%	48.6%	1.5	2.1	3.7
	ELSED	7.67°	32.4%	47.1%	4.0	1.6	5.7
	LSD	7.87°	29.2%	46.4%	27.1	3.0	30.1
	EDLines (ED_Lib)	8.09°	30.5%	45.7%	5.8	2.9	8.9
York Urban (outdoor), split-half CV	ELSED	0.82°	84.3%	96.1%	5.5	2.8	8.4
	LSD	0.83°	93.1%	96.1%	32.0	3.7	35.7
	SweepLSD	0.94°	85.3%	95.1%	1.8	2.4	4.2
	EDLines (ED_Lib)	1.00°	89.2%	98.0%	7.3	3.5	11.0

Time columns: medians at 640×480, all detectors built with one toolchain and measured in one window, interleaved image-by-image (detection: median of 15 runs; VP estimation: median of 5 runs of each image's cross-validated pick, timed on the exact segments behind the accuracy columns, with the estimator's scoring stage AVX2-vectorized — output verified bit-identical to the scalar code over all 68,100 menu evaluations).

Indoors SweepLSD wins every fair protocol — now including against ELSED — once the estimator counts each line as one vote (length-weighted voting lets long clutter contours hijack the search; SweepLSD's support is many short-but-accurate lines). The optional linker compounds the indoor lead: under the identical menu and cross-validation it scores 5.23° (outdoor CV median 0.93°, indistinguishable from the linker-less 0.94°). Outdoors the result is a metric-dependent near-tie: ELSED and LSD share the best median while LSD clearly leads recall at 2°. The estimator that measured best overall — unit-weight voting × vertical-prior seed × strong candidate search — is exactly what examples/manhattan_frame.cpp implements, so the recommended configuration ships as working code.

Takeaway. "Which detector is best for vanishing points?" has no detector-only answer: it is a property of the (detector × estimator) pair. Two sets of indoor numbers appear on this page and they are measured differently, so to be explicit: the table above is the cross-validated protocol (variants selected on one half of the data, scored on the other), where SweepLSD is best at 6.08° and 5.23° with the linker. The figures usually quoted for the shipped library are the single recommended estimator run on every image, without selection: there SweepLSD measures 5.98°, and 5.25° with the optional linker — the best of all detector/estimator combinations tested (best non-SweepLSD: ELSED at 6.53°). The two protocols agree on the ordering. Either way SweepLSD stays 4.6× / 5.2× / 25× faster than ELSED / ED_Lib / LSD upstream of the estimator at Full-HD — and 2.6× / 3.8× / 17.6× on the 640×480 frames this study actually runs on (table above). Because the task is the vanishing points, not the segments, the table also prices the downstream. The estimation stage was profiled and optimized for this measurement (its former hotspot, the candidates×lines inlier scoring, is AVX2-vectorized with bit-identical output); even so it costs 1.6–3.7 ms per frame at 640×480 — the remaining work is the multi-start refinement's serial per-seed eigensolves, which cannot be batched without changing the result — and for the fastest detector that still exceeds SweepLSD's own 1.5–1.8 ms detection. End-to-end SweepLSD is the fastest pipeline on both datasets (3.7 ms on NYU against ELSED's 5.7), but the margin over ELSED compresses to 1.5–2.0×: at this resolution the shared estimator, not the detector, bounds the frame time — a property of the task, not of an unoptimized implementation. At high resolution the picture inverts — estimation stays a few milliseconds (5–6 ms at 4K; it scales with surviving lines, not pixels) while detection grows with pixel count (30–580 ms of detection at 4K on the applications page).

Reproduce it. Build with -DSWEEPLSD_BUILD_BENCH=ON; export the manifests with tools/yud_export.py (York Urban) / tools/nyu_export.py (NYU-VP); run

sweeplsd_yud_eval <manifest>
[--gtlines ...] [--edreal-dir ...] [--elsed-dir ...]

. The fair protocol is a committed tool: sweeplsd_vp_bestcfg writes one CSV row per (image, detector, variant) and tools/vp_bestcfg_cv.py performs the split-half cross-validated aggregation (the rule reproduces the original study's published numbers exactly from its surviving records). sweeplsd_vp_bestcfg --time-runs R adds an est_ms column (downstream calibrate + Manhattan estimation, median of R runs), and vp_bestcfg_cv.py --det-dir method=DIR attaches per-image detection times in the runner file format to produce the detect / est / total columns. The recommended estimator ships as sweeplsd_manhattan / examples/manhattan_frame.cpp.

Refreshed 2026-07. The fair-protocol table was re-measured in a single run of all four detectors over the identical 12-variant menu (ELSED added); the linker rows come from the same runs. Under the fixed common estimator the re-run reproduces the §2 table within ≤0.2° — every ranking and takeaway on this page is from internally-consistent, same-run data.

Time columns added 2026-08. Detection was re-measured with all four detectors compiled by the same GCC 15.2, one process per image, interleaved image-by-image in a single window (the cross-window comparison trap). The re-generated segments reproduce the accuracy table exactly for SweepLSD (deterministic), to float rounding for ELSED (≤2×10⁻⁵ px), and to ±1–2 segments per frame for ED_Lib, whose edge chaining is build-sensitive (its NYU CV median moves 8.09°→8.95° between compiler builds of the same source and OpenCV version — the ranking is unaffected; the accuracy columns keep the stored segments' published values, and its estimation time is measured on those same stored segments). LSD's own double-precision detection is also build-sensitive at the last-bit level: recompiling it with a different GCC generation moves its cross-validated York Urban median between 0.83° and 0.94° (ELSED's and SweepLSD's by ≤0.02°; every NYU median reproduces to 0.01°) — so LSD's outdoor lead over SweepLSD is within build jitter, the stable outdoor read is ELSED ≈0.1° ahead of both, and the indoor ordering is unaffected. The estimator's scoring stage was AVX2-vectorized for the timing study, verified bit-identical to the scalar code over all 68,100 menu evaluations, both datasets.