Four stackers, same Seestar subs
What changes when the same subs go through four stackers on defaults?
Within a few percent in depth on four of five targets; the big differences came from culling and one alignment failure.
Version-specific comparison
Siril 1.4.3 · WBPP 3.1.2 (PI 1.9.5) · StackingWizard beta 2026-10-07 · SyQon 1.1.4 CPU
Engines and versions
Version-specific historical comparison. A new engine release makes this a historical recipe. All engines use defaults, except StackingWizard “tick all” for Round B.
| Engine | Version |
|---|---|
| Siril | 1.4.3 with NOVA’s seestar_register.ssf and seestar_stack_only.ssf |
| WBPP | 3.1.2 (PixInsight 1.9.5) |
| StackingWizard | beta 2026-10-07 |
| SyQon Fusion Stack | 1.1.4, CPU-only |
Evidence section: Durability
Data and identity
SeeStar S50; 10 s subs, one night and one field per target, plus the multi-night M 51 finale. Every sub is SHA-256 hashed in its inventory.
| Target | Night | Filter | p95 solve spread | Night total | Ecc gate removed | Round A (gated) | Production cull removed | Round B (same frames) |
|---|---|---|---|---|---|---|---|---|
| M 45 | 2024-09-03 | IRCUT | 2.07′ | 931 | 264 | 667 | 93 | 574 |
| M 101 | 2025-01-23 | IRCUT | 2.39′ | 1,219 | 33 | 1,186 | 232 | 954 |
| M 51 | 2025-02-21 | IRCUT | 1.8′ | 1,142 | 66 | 1,076 | 35 | 1,041 |
| M 1 | 2024-12-19 | LP | 1.88′ | 1,553 (16 manual excl.) | 164 | 1,373 | 237 | 1,136 |
| NGC 281 | 2024-09-07 | LP | 6.53′ | 1,409 | 244 | 1,165 | 150 | 1,015 |
| M 51 all-subs | 19 nights | IRCUT | multi-night | — | — | — | — | 4,659 |
Evidence section: Data and identity
Protocol and scoring
Round A: NOVA’s eccentricity gate, then each tool’s own default selection. Round A is not an engine comparison.
Round B: identical production-culled frames, with tool culling off where possible (StackingWizard “tick all”).
Every master is plate-solved, reprojected to one grid and scored only on common coverage. Flux is matched on signal-dominated pixels, with Siril as the reference scale.
Band-limited noise is measured at 0–1, 1–2, 2–4, 4–8 and 8–16 px. Star FWHM, eccentricity and count use one detector. The 2–4 px band measures depth; 0–1 px reflects interpolation character, not depth. Never rank engines by per-pixel SNR.
Split halves: odd/even frames by capture time, all four engines on each half, 40 masters.
Evidence section: Scoring method (lane 1, the engine verdict)
Round B — measured results
Noise is relative to Siril: negative means less noise. FWHM is in px. Same frames produced depth within a few percent on four of five targets and the finale; NGC 281 is the exception.
| Target | WBPP 2–4 | SW 2–4 | SyQon 2–4 | WBPP 0–1 | SW 0–1 | FWHM Siril / WBPP / SW / SyQon | Common coverage |
|---|---|---|---|---|---|---|---|
| M 45 | −2.2 % | −2.7 % | −1.3 % | +21.5 % | −10.1 % | 2.78 / 2.81 / 2.99 / 2.82 | 0.852 |
| M 101 | −1.2 % | −1.3 % | −0.5 % | +22.4 % | −8.2 % | 3.23 / 3.20 / 3.38 / 3.24 | 0.901 |
| M 51 | −3.6 % | −1.5 % | −1.4 % | +20.7 % | −8.0 % | 2.79 / 2.80 / 2.94 / 2.78 | 0.872 |
| M 1 | +0.3 % | +0.7 % | +0.9 % | +21.1 % | −10.3 % | 2.73 / 2.75 / 2.95 / 2.76 | 0.884 |
| NGC 281 | +7.5 % | +5.8 % | +46.5 % | +39.5 % | −1.5 % | 2.69 / 2.74 / 2.92 / 2.66 | 0.456 |
| M 51 all-subs | −0.5 % | −1.9 % | −2.1 % | +22.1 % | −7.2 % | 2.81 / 2.85 / 3.01 / 2.87 | 0.543 |
Evidence section: Results: round B (same frames)
Split-half outcomes
M 45, M 101 and M 51 gaps of 1–3.6 % reproduce, with H1/H2 spread ≤ 0.7 %.
M 1 and NGC 281 gaps under about 1.5 % are unresolved.
StackingWizard’s wider stars reproduce on all five targets in both halves (+0.11 to +0.28 px, spread ≤ 0.05 px).
On M 51, WBPP’s full-B gap (−3.6 %) lies outside its half range (−2.6 / −2.3 %); half-depth bars only approximate full-depth gaps.
Evidence section: Results: round B (same frames)
NGC 281 — registration failure
SyQon accepted 351 of 1,015 frames. Its log reports “664 frame(s) excluded: Both intensity and stellar registration failed”. This is a registration failure, not stacking quality, and explains the low common coverage (0.456).
NGC 281 had substantially greater pointing spread. Spread alone establishes neither rotation nor the cause of the alignment failure.
Evidence section: Proposed claim ledger (narration candidates; Henry and Advisor to check)
Round A — not an engine comparison
StackingWizard is the only engine that rejects whole subs by default. Selection differs here; these are not pure engine differences.
The gated input was 1,373 for M 1 and 1,165 for NGC 281. Their “gated in” table entries are Siril’s stacked counts; SW removals use the gated input. The √N prediction is √(Siril subs ÷ SW subs) − 1.
On no target did SW’s rejection give smaller stars. Selection, weighting and correlations prevent attributing the smaller-than-baseline noise penalty to weighting.
| Target | Gated in | SW kept | SW removed | SW 2–4 px vs Siril | √N prediction | SW FWHM vs Siril |
|---|---|---|---|---|---|---|
| M 51 | 1,076 | 701 | 375 (35 %) | +18.4 % | +24 % | 2.97 vs 2.79 |
| M 1 | 1,369* | 1,007 | 366 (27 %) | +8.4 % | +17 % | 3.00 vs 2.79 |
| NGC 281 | 1,161* | 954 | 211 (18 %) | +9.9 % | +10 % | 2.93 vs 2.73 |
| M 101 | 1,186 | 1,046 | 140 (12 %) | +4.0 % | +6 % | 3.41 vs 3.29 |
| M 45 | 667 | 593 | 74 (11 %) | −0.5 % | +6 % | 3.01 vs 2.82 |
Evidence section: Results: round A (each tool’s own selection)
Blind rankings — preferences
200 px crops at 3× used one frozen matched-stretch recipe. Letters were reshuffled per crop; Henry ranked twice with fresh shuffles. The key opened only after all six targets were ranked.
7 of 18 crops were four-way ties in round 1 (4 of 18 in round 2). SyQon on NGC 281 was last in all 3 crops in both rounds (6/6).
SW on M 45 was worse in the core and faint crops in both rounds (4/4). SW on M 51 and M 51 all-subs cores was best or tied-best in both rounds.
Not reproducible: M 101 core/faint, M 1 core/stars, and the small M 51 faint/star calls.
Evidence section: Results: blind rankings (lane 2, preference evidence)
Hero picks — showcase only
Each Round-B master went through NOVA’s full pipeline. These finished-image preferences are showcase picks, never stack-quality evidence. The picks span three engines.
| Target | Hero engine |
|---|---|
| M 1 | Siril |
| M 45 | WBPP |
| M 51 | StackingWizard |
| M 51 all-subs | Siril |
| M 101 | WBPP |
| NGC 281 | Siril |
Evidence section: Results: hero shots (lane 3, showcase only, never evidence)
Side results
Laptop vs VM Siril, M 51 Round B (1,041 subs): Siril 1.4.4 for Windows ran NOVA’s script. 0–1 px noise was 1.615 both; FWHM 2.81 vs 2.80; eccentricity 0.378 vs 0.380; stars 503 vs 505; median relative |Δ| 4.5e-5. Agreement is about one part in twenty thousand.
M 1’s weak finished images on 3 of 4 engines came from NOVA’s stretch-picker bug, not the stacks. The grain cap collapsed on a degenerate candidate; Siril’s exactly-zero grain skipped the cap. The bug is fixed.
Pilot, not cohort: an early M 52 test used 20 subs. SW reported SNR 33 vs WBPP 26, with neighbour correlation 0.50 vs 0.27 and equal band-limited noise. This motivated the rule against ranking by per-pixel SNR.
Evidence section: Side results (usable as short beats; each is labelled)
Timings — indicative only
Round B wall time on a shared 4-vCPU VM, with other jobs at times. These figures do not establish general speed.
Finale (4,659 subs): Siril 3,313 s (55 min), WBPP 6,001 s, SW 4,954 s, SyQon 24,711 s (6.9 h).
| Engine | Range | M 51 |
|---|---|---|
| Siril | 224–480 s | 7 min |
| WBPP | 600–1,561 s | 25 min |
| SW | 1,029–1,503 s | 21 min |
| SyQon | 1,668–5,639 s | 84 min |
Evidence section: Side results (usable as short beats; each is labelled)
What we cannot claim
- Any engine is "best" in general: there are five targets, one night each, and default settings.
- Anything about mosaics or drizzle (experiments 2 and 3).
- Anything about speed beyond "indicative on this VM".
- That SW's rejection is always worse. On M 45 it lost nothing in depth.
- Any NOVA grade as a measure of stack quality (#1146).
Reproduce it
- Use the scripts/stackexp/ harness on main: stage.py, verify_inputs.py, run_target.sh, run_finale.sh and score_target.py. Engine wrappers record runs; score.py implements common-grid scoring.
- Archive references are descriptive, not public download links. Request the experiment archive and original subs from Jeff Henry to rerun; this page does not distribute input data or licensed engines.
- Per-target inventories: stackexp-<target>/inventory.json; Round-B scores: B/score_v2/scores.json. M 51 uses stackexp-m51n and stackexp-m51m; the finale uses stackexp-m51all-r2/manifest.json (identity 4eaa24a6) and scores.json.
- Frozen scorer archive: stackexp-archive-20261025. Split halves: stackexp-lanes/halves_summary.json. Blind rankings: stackexp-lanes/lane2_v3/henry_blind_rankings_v3b{,_repeat}.json. Showcase choices: stackexp-lanes/heroes/hero_picks.json.
- These datasets support this version-specific comparison, not a general law. Mosaics are a later experiment.
Pinned evidence record: docs/episodes/stacking_engines_evidence.md at a2e2ca3a2de95ee360a5a97a8ee54861a7d75424 · content revision 1. Evidence and archive access are by request.