Skip to content

Fix GaussianPosterior input whitening buffers to float64#88

Merged
cweniger merged 2 commits into
mainfrom
bugfix/gaussian-input-whitening-float64
Jun 6, 2026
Merged

Fix GaussianPosterior input whitening buffers to float64#88
cweniger merged 2 commits into
mainfrom
bugfix/gaussian-input-whitening-float64

Conversation

@cweniger

@cweniger cweniger commented Jun 6, 2026

Copy link
Copy Markdown
Owner

Summary

  • _input_mean and _input_std in GaussianPosterior were initialised without an explicit dtype, defaulting to float32. This caused float64 condition inputs to be silently downcast before whitening statistics were computed — inconsistent with the output-side buffers (_output_mean, _output_std, _residual_cov, etc.) which are all explicitly float64.
  • Both buffers are now dtype=torch.float64, and are added to the to() override so they survive .to(device) calls.
  • The dtype cast is decoupled in _forward_mean: whitening runs in float64, and the explicit cast to the MLP's dtype (next(self.net.parameters()).dtype, i.e. float32) happens only immediately before self.net(...).

Precision pipeline before and after

Step Before After
Condition input float64 float64
Cast before whitening stats → float32 → float64 (no-op)
Whitening (c - mean) / std float32 float64
Cast before MLP (implicit, already float32) explicit → float32
MLP forward float32 float32
MLP output upcast → float64 → float64
De-whitening float64 float64

Why it matters

The _input_mean.dtype was implicitly serving two roles: statistics precision and the gate cast into the float32 MLP. Making it float32 meant any catastrophic cancellation in the whitening step (e.g. conditions with large values and small variance, as in x = exp(z) with z near 5) was amplified before the MLP could even see the data. The fix separates the two concerns.

Test plan

  • Run examples/04_gaussian: falcon launch -o output/run && falcon sample posterior -o output/run
  • Confirm training loss decreases and posterior samples are reasonable
  • Verify no dtype mismatch errors on CPU and GPU

🤖 Generated with Claude Code

_input_mean and _input_std were initialised with the default dtype
(float32), causing the float64 condition inputs to be silently
downcast before whitening statistics were computed. This is
inconsistent with the output-side buffers which are explicitly
float64, and loses precision when the whitened value could still
be computed precisely before the cast into the float32 MLP.

Fix: initialise both buffers as float64, add them to the to()
override so they survive .to(device) calls, and decouple the
dtype cast — whitening now runs in float64, with an explicit
cast to the MLP's dtype immediately before the net() call.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@codecov

codecov Bot commented Jun 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 9.72%. Comparing base (7974ab3) to head (3fd4c73).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
falcon/estimators/gaussian_fullcov.py 0.00% 4 Missing ⚠️
Additional details and impacted files
@@          Coverage Diff          @@
##            main     #88   +/-   ##
=====================================
  Coverage   9.72%   9.72%           
=====================================
  Files         33      33           
  Lines       4154    4154           
=====================================
  Hits         404     404           
  Misses      3750    3750           
Flag Coverage Δ
unit 9.72% <0.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cweniger cweniger merged commit e22b690 into main Jun 6, 2026
4 checks passed
@cweniger cweniger deleted the bugfix/gaussian-input-whitening-float64 branch June 8, 2026 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant