SE-UNet: Singular Equivariant Imaging for
Real-World Constrained Generation
July 02, 2026
Abstract
While diffusion models have revolutionized image synthesis, their application to real-world inverse problems is often hampered by the need for massive datasets and the difficulty of imposing strict physical constraints. In this work, we introduce SE-UNet (Singular Equivariant UNet), a framework designed to solve ill-posed imaging tasks without extensive pre-training. By treating generation as an optimization problem constrained by geometric equivariance (\(D_4\) group) and singular value gating, SE-UNet effectively standardizes the solution space. We demonstrate that these strong inductive biases allow for state-of-the-art zero-shot inpainting results (80% missing pixels) on CIFAR-10. Our method surpasses Deep Image Prior (DIP) baselines by over 4 dB in PSNR and exhibits a characteristic “singular snap” convergence—rapidly locking into the signal manifold. SE-UNet thus offers a data-efficient pathway for constrained generation, aligning with the ReALM-GEN goal of bridging theoretical priors with practical deployment.
1 Introduction↩︎
The emergence of diffusion probabilistic models [1], [2] has fundamentally shifted the landscape of generative modeling, offering unprecedented fidelity and control [3]. Yet, the deployment of such models in high-stakes domains—ranging from medical diagnostics to robotics—remains challenging due to strict requirements for physical consistency and the frequent scarcity of training data [4]. The ReALM-GEN workshop highlights this gap, framing controlled generation as sampling from a distribution “tilted” by real-world constraints.
Inverse problems like inpainting and denoising sit squarely within this domain. Supervised approaches [5] often fail to generalize beyond their training distribution, while unsupervised techniques such as Deep Image Prior (DIP) [6] offer a compelling alternative by exploiting the inductive bias of convolutional architectures. However, DIP is prone to spectral bias—fitting low frequencies early but later overfitting to high-frequency noise—which limits its utility in high-precision tasks.
To address these limitations, we propose SE-UNet, an architecture that embeds geometric equivariance and singular value gating directly into the generation process. Our key contributions include:
Singular Gating: A dynamic mechanism that filters the feature spectrum, suppressing noise leakage in high-sparsity regimes (\(>80\%\) missing data) by preserving only the leading singular vectors.
Geometric Equivariance: By exploiting the \(D_4\) symmetry group inherent in natural images, we implement a self-ensembling strategy that effectively multiplies the supervision signal by \(8\times\), enforcing consistency under rotation and reflection.
High-Fidelity Reconstruction: SE-UNet achieves 25.34 dB PSNR on CIFAR-10 inpainting, significantly outperforming DIP baselines and approaching the quality of fully supervised models without accessing external data.
Figure 1: Visual comparison of zero-shot inpainting on CIFAR-10 with 80% missing pixels. (a) Ground Truth. (b) Masked Input. (c) Our SE-UNet reconstruction. Note how the method successfully recovers global structure and texture without external training data, highlighting the efficacy of geometric constraints..
2 Methodology↩︎
We aim to recover an image \({\boldsymbol{x}}\in \mathbb{R}^{H \times W \times C}\) from a degraded observation \({\boldsymbol{y}}= {\boldsymbol{A}}{\boldsymbol{x}}+ {\boldsymbol{n}}\), where \({\boldsymbol{A}}\) represents a degradation operator (e.g., a binary mask) and \({\boldsymbol{n}}\) is additive noise. We formulate the recovery as \({\boldsymbol{x}}^* = f_\theta({\boldsymbol{z}})\), where \(f_\theta\) is a U-Net generator and \({\boldsymbol{z}}\) is fixed noise. The network parameters \(\theta\) are optimized to minimize the reconstruction loss \(||{\boldsymbol{A}}f_\theta({\boldsymbol{z}}) - {\boldsymbol{y}}||^2\) subject to regularization \(R(f_\theta({\boldsymbol{z}}))\).
2.1 Singular Gating↩︎
A primary failure mode of DIP is the eventual fitting of noise. To mitigate this, we introduce Singular Gating. By analyzing the singular value decomposition (SVD) of feature maps at each layer, we observe that the signal of interest corresponds to the leading singular vectors, whereas noise dominates the tail. We therefore employ a learnable gating function that suppresses singular values below a threshold \(\tau\). This effectively projects the internal representations onto a “clean” signal manifold during each forward pass, acting as a hard spectral constraint on the generator’s capacity.
2.2 Geometric Equivariance via \(D_4\) Symmetry↩︎
Natural images exhibit fundamental symmetries. It follows that a generator solving an inverse problem should be equivariant to valid geometric transformations. We enforce this property by defining the loss over the orbit of the image under the dihedral group \(D_4\) (comprising \(90^\circ\) rotations and reflections): \[\mathcal{L}_{eq} = \sum_{g \in D_4} || {\boldsymbol{A}}f_\theta(g \cdot {\boldsymbol{z}}) - g \cdot {\boldsymbol{y}}||^2\] Here, the input noise \({\boldsymbol{z}}\) and target \({\boldsymbol{y}}\) are transformed by group element \(g\). This approach provides “geometric self-supervision,” multiplying the effective constraints by \(|D_4| = 8\) and forcing the network to learn robust, rotation-invariant features—a crucial advantage in data-sparse regimes [7].
3 Experiments↩︎
We evaluate SE-UNet on the CIFAR-10 dataset [8], specifically targeting the zero-shot inpainting task with random masks removing 80% of pixels.
3.1 Setup↩︎
Our base architecture is a standard U-Net [5] with skip connections. We benchmark against the standard Deep Image Prior (DIP) [6] and DIP augmented with Total Variation (TV) regularization. All models are optimized using Adam [9] with a learning rate of \(0.01\).
3.2 Quantitative Results↩︎
As shown in Table ¿tbl:tab:results? and Figure 2, SE-UNet achieves a PSNR of 25.34 dB, substantially outperforming the DIP baseline (21.23 dB).
| Method | PSNR (dB) | SSIM |
|---|---|---|
| DIP [6] | 21.23 | 0.65 |
| DIP + TV | 21.81 | 0.68 |
| SG-UNet (Ours, Gating only) | 24.28 | 0.79 |
| SE-UNet (Ours) | 25.34 | 0.82 |
The introduction of Singular Gating alone (SG-UNet) yields a remarkable gain (+3 dB), validating our hypothesis regarding spectral filtering. The full SE-UNet model further enhances performance, underscoring the value of geometric consistency.
Figure 2: Peak Signal-to-Noise Ratio (PSNR) comparison. SE-UNet (Red) surpasses standard DIP baselines by over 4 dB, bridging the gap toward fully supervised Diffusion models (Green dashed line)..
3.3 Singular Dynamics↩︎
Figure 3 highlights a distinct difference in convergence behavior. While DIP typically exhibits slow, asymptotic convergence, SE-UNet demonstrates a “Singular Snap”—a sharp phase transition near iteration 50. At this point, the model rapidly locks into the correct geometric configuration. This phenomenon suggests that our constraints effectively reshape the loss landscape, creating a steeper basin of attraction around the optimal solution.
Figure 3: Optimization dynamics. In contrast to the slow convergence of standard Adam (DIP), our Singular Dynamics display a characteristic “phase transition” or “snap” around iteration 50, indicating a rapid lock-in to the high-PSNR manifold..
4 Discussion and Related Work↩︎
Constrained Generation: This work is directly motivated by the workshop’s focus on constrained generation. Unlike diffusion models [1] which learn the score function of the data distribution, SE-UNet explicitly imposes structural constraints (symmetry, spectral sparsity) directly. This formulation is conceptually similar to energy-based models where the energy landscape is molded by geometric priors [2].
Inverse Problems: In domains such as medical imaging, acquiring fully sampled ground truth data is often infeasible. SE-UNet’s capacity to reconstruct high-fidelity images from just 20% of the data—without external supervision—positions it as a promising tool for such applications. It may also serve as a lightweight alternative to diffusion-based solvers or as a plug-and-play prior for other generative frameworks [3].
5 Conclusion↩︎
We have presented SE-UNet, a framework that integrates Singular Gating and Geometric Equivariance to address challenging inverse problems. By rigorously constraining the generative process, we achieve state-of-the-art results in zero-shot inpainting. Future research will explore extending this framework to 3D volumetric data and investigating its potential integration with pre-trained diffusion models as a form of geometric guidance.