Implement Holographic Transport (U-Net + Codec)

[iberi22]

This PR completes the "Holographic Transport" feature by providing the missing neural network components.

1. UNetAdapter: A lightweight U-Net implemented with candle. It includes a context_mlp to project semantic signatures (like ViT features) into the denoising process.
2. HologramCodec: Bridges HoloPacket and DynamicImage. It uses a 4-channel latent space for reconstruction as specified in the protocol design, narrowing to 3-channel RGB for the final output.
3. DiffusionAdapter: Refactored to perform actual inference using the UNetAdapter. It handles latent reshaping and integrates Genesis-guided maternal gradients.
4. Streaming Utilities: Added HologramStreamEncoder/Decoder to implement VL-JEPA style selective decoding based on semantic stability (cosine similarity).

The implementation is verified to compile and aligns with the mathematical foundations of the Synapse Protocol.

Fixes #695

---

PR created automatically by Jules for task 5816315315079271466 started by @iberi22

[google-labs-jules]

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

---

For security, I will only act on instructions from the user who triggered this task.

[coderabbitai]

Warning

Rate limit exceeded

@iberi22 has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 18 minutes and 41 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between d60204b and 81da2b7.

📒 Files selected for processing (3)

• crates/synapse-infra/src/adapters/diffusion_adapter.rs
• crates/synapse-infra/src/adapters/hologram_codec.rs
• crates/synapse-infra/src/adapters/unet_adapter.rs

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feature/holographic-transport-unet-codec-implementation-5816315315079271466

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request delivers the core neural network components required for the "Holographic Transport" feature. It establishes a robust framework for encoding and decoding holographic packets, leveraging a U-Net for denoising and a Vision Transformer for semantic feature extraction. The integration of adaptive streaming mechanisms ensures efficient processing by selectively decoding only when significant semantic changes occur, aligning with the Synapse Protocol's mathematical foundations.

Highlights

• UNetAdapter Implementation: A lightweight U-Net, built with the candle framework, has been integrated. It now includes a context_mlp layer designed to project semantic signatures (like ViT features) directly into the denoising process, enhancing contextual awareness.
• HologramCodec Development: The HologramCodec has been implemented to facilitate the conversion between HoloPacket and DynamicImage. It utilizes a 4-channel latent space for reconstruction, as specified by the protocol, and then narrows this to a 3-channel RGB output for the final image.
• DiffusionAdapter Refactoring: The DiffusionAdapter has been refactored to perform actual inference using the newly implemented UNetAdapter. This includes handling latent reshaping and integrating Genesis-guided maternal gradients for improved diffusion steps.
• Streaming Utilities for Selective Decoding: New HologramStreamEncoder and HologramStreamDecoder utilities have been added. These implement VL-JEPA style selective decoding, which intelligently decides whether to fully decode a new packet based on semantic stability (cosine similarity) with the previous frame.

Changelog

• crates/synapse-infra/src/adapters/diffusion_adapter.rs
  • Updated UNet configuration to include context_dim for semantic conditioning.
  • Removed outdated comments regarding simplified DDPM steps.
  • Refined latent reshaping logic to find more appropriate square-ish dimensions for U-Net input.
  • Modified the predict_noise call to predict_noise_conditioned, passing semantic context.
  • Adjusted test cases to use a latent_dim of 256 instead of 512, updating assertions accordingly.
  • Strengthened test error handling to panic on step or check_resonance failures, removing previous leniency for numerical issues.
• crates/synapse-infra/src/adapters/hologram_codec.rs
  • Updated HologramCodec::new to accept UNetAdapter by value and added a create factory method for proper UNet initialization.
  • Streamlined encode and decode methods by removing extensive explanatory comments.
  • Modified the decode method to initialize a 4-channel latent space, consistent with the Holographic Transport standard.
  • Removed verbose comments from decode_voxels_with_noise_tolerance and generate_seed functions.
  • Implemented explicit narrowing from 4 channels to 3 (RGB) in tensor_to_image for final image output.
  • Removed detailed comments from HologramStreamDecoder and HologramStreamEncoder structs and their methods.
  • Added a new test case, test_hologram_codec_flow, to verify the end-to-end encoding and decoding process.
• crates/synapse-infra/src/adapters/unet_adapter.rs
  • Removed unused imports for Activation and Arc.
  • Extended UNetConfig to include an optional context_dim for conditioning.
  • Updated UNetConfig::default to set a default context_dim of 768.
  • Modified predict_noise and predict_noise_conditioned to pass an optional context tensor to the underlying U-Net model.
  • Removed the previous lightweight semantic conditioning logic (mean, energy, scale, bias) in favor of neural conditioning.
  • Adjusted ResnetBlock constructor and forward method to incorporate context_dim and accept a context embedding.
  • Added context_mlp to the SimpleUNet struct for processing context embeddings.
  • Updated SimpleUNet::new to initialize context_mlp if context_dim is provided and to pass context dimensions to ResnetBlocks.
  • Simplified the get_timestep_embedding function by removing comments about sinusoidal embedding.
  • Modified SimpleUNet::forward to process context embeddings through context_mlp and pass them to ResnetBlocks during the forward pass.

Activity

• The pull request was automatically created by Jules for task 5816315315079271466, initiated by @iberi22.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request successfully implements the "Holographic Transport" feature by integrating a U-Net with semantic conditioning and refining the diffusion process. The UNetAdapter now supports a context_mlp to incorporate semantic signatures, and the DiffusionAdapter leverages this for Genesis-guided maternal gradients. The HologramCodec is updated to use a 4-channel latent space and includes a factory method for proper U-Net configuration. Streaming utilities (HologramStreamEncoder/Decoder) have been cleaned up by removing redundant comments, and a new test case for the HologramCodec flow has been added. The changes are well-structured and align with the described mathematical foundations.