1 Introduction↩︎

Remote photoplethysmography (rPPG) estimates the blood volume pulse from ordinary facial video, enabling contact-free cardiovascular sensing in telemedicine, neonatal care, and driver-state monitoring [1]–[3]. Modern deep rPPG models [3]–[6] attain sub-bpm (beats per minute) MAE on frontal, stationary video, yet they degrade sharply when the subject turns the head. On MMPD, the FactorizePhys backbone’s MAE grows \(1.60\times\) from frontal (\(|\text{yaw}|{<}15^\circ\)) to large-yaw (\(|\text{yaw}|{\geq}45^\circ\)) frames (Sec. 4.2). Because realistic deployments routinely contain non-frontal frames, pose-induced degradation directly bounds the practical utility of deployed rPPG. Figure 1 visualizes this gap: as yaw grows, image-coordinate pixels drift across anatomy, and the baseline’s MAE rises monotonically, while CanonicalPhys stays flat.

We argue that this is a coordinate-structural problem rather than a data-augmentation one. An image-coordinate pixel \((x,y)\) maps to different anatomy at different poses, which blocks three priors that are otherwise natural for rPPG. First, the dichromatic reflection model [7] predicts that the diffuse, pulse-carrying component of skin reflectance scales as the cosine between the skin normal and the camera axis, a quantity defined per anatomical location rather than per pixel. Second, pulse-phase invariance across forehead and cheeks requires a pose-stable identification of each ROI. Third, classical POS and CHROM [7], [8] project windowed RGB traces onto a chromaticity basis, a projection well-posed only on a fixed skin ROI. A stable anatomy-to-pixel mapping makes all three tractable again.

Motivated by this observation, CanonicalPhys prepends a differentiable four-point homography on the outer eye and mouth corners that maps each frame into a canonical face coordinate system.² On top of an unmodified FactorizePhys backbone, we attach three canonical-space components: a per-pixel Lambertian weight from canonical normals and the per-frame MediaPipe [9] rotation; a cross-ROI temporal consistency loss over fixed canonical forehead and cheek features; and knowledge distillation from POS applied to canonical forehead pixels. None adds trainable parameters, so the parameter count is identical to the backbone.

Pose-stratified evaluation on MMPD (Sec. 4.2) reduces the frontal-to-large-yaw MAE degradation from \(1.60\times\) to \(1.33\times\) and flattens the mild-yaw bin from \(1.32\times\) to \(1.07\times\). Cross-dataset evaluation on UBFC-rPPG [11], PURE [12], OBF [13], and MMPD [14] confirms that the effect transfers across sensors. On the pose-richer UBFC-trained targets, the matched-seed MAE reductions are \(32\%\) on PURE and \(20\%\) on MMPD, PURE seed variance is more than halved, and CanonicalPhys wins \(13\) of the \(16\) cells of the full transfer matrix. In summary, our contributions are:

1. We introduce a differentiable four-point homography, with no trainable parameters, that maps each frame into a canonical face coordinate system, absorbing the dominant rigid component of head pose before any learned processing and approximately restoring a pose-stable anatomy-to-pixel mapping.

2. We propose three canonical-space components enabled by this change of coordinates, none of which adds trainable parameters: a physics-informed per-pixel Lambertian weight from the dichromatic reflection model, a cross-ROI temporal consistency loss that enforces pulse-phase invariance, and a knowledge distillation loss from windowed POS applied to canonical forehead pixels.

3. We evaluate CanonicalPhys with a matched-seed pose-stratified analysis on MMPD that directly isolates pose from other distribution shifts, a full cross-dataset transfer across UBFC-rPPG, PURE, OBF, and MMPD reporting the complete metric suite, and a failure-mode analysis (Sec. 4.6) that delineates the five regimes where the claim does not yet hold: moderate yaw, large yaw, extreme yaw, landmark fallback, and pitch/roll.

2 Related Work↩︎

Deep rPPG. Early deep rPPG methods adapted convolutional architectures from generic video understanding to blood volume pulse regression. PhysNet [4] introduced a 3D-CNN encoder-decoder trained with a temporal Pearson correlation loss. DeepPhys [2] and TS-CAN [15] proposed a two-stream appearance-motion design. EfficientPhys [3] replaced 3D convolutions with temporal shift modules and a self-attention shifted network head. Transformer-based PhysFormer [5] introduced temporal-difference attention. More recently, FactorizePhys [6] computes joint spatial-temporal-channel attention via nonnegative matrix factorization and remains state-of-the-art among lightweight models. All of these methods learn their spatial priors purely from raw appearance, and none consult an external geometric signal. As a consequence, a pixel at coordinate \((x,y)\) plays a different anatomical role at different head poses, and the pose nuisance is left to the backbone to absorb through capacity alone.

Cross-dataset generalization for rPPG. The dominant strategy for closing the cross-dataset gap is distributional alignment in a learned feature space. FreqPhys [16] repurposes an implicit physiological frequency prior as a regularizer during training. HOT [17] formulates cross-dataset adaptation as harmonic-constrained optimal transport between source and target representations. We take an orthogonal route and inject a prior that is domain-invariant by construction, namely the geometry of the human face, directly at the input of the network. Our method is complementary in principle and could be combined with existing methods.

Handcrafted rPPG and physical priors. Classical rPPG pipelines are explicit about their priors. Verkruysse  [1] extract the pulse from a static skin ROI under ambient light. CHROM [8] and POS [7] project windowed RGB traces onto a chromaticity basis derived from the dichromatic reflection model, suppressing specular and illumination components. These projections are well-posed only on a fixed skin ROI and therefore break down when the ROI drifts across anatomy under head motion. Rather than discarding these priors, we make them operable inside a deep network by mapping every frame into a canonical face coordinate system in which the required ROI is fixed by construction. Windowed POS then becomes a stable teacher signal for knowledge distillation, and the dichromatic cosine becomes a per-pixel weight.

Facial landmarks and 3D geometry as visual priors. Facial landmarks are a long-standing tool in face analysis, with mature detectors [9], [18], [19] reaching pixel-level accuracy on unconstrained imagery. They are widely used in face alignment [20], expression recognition [21], and anti-spoofing [22], but they have rarely been used as a geometric prior for physiological sensing. The few works that do involve landmarks use them only to crop a face ROI or to define heuristic skin masks, leaving the backbone to discover pose-invariant features on its own [23]. Closest to our geometric motivation, Cantrill  [24] warp each frame onto an UV texture map from a fitted 3D face mesh and discard oblique surface patches, improving motion robustness on MMPD. Their UV unwrap is more expressive than our planar warp, and it requires per-frame 3D mesh fitting. We instead use a four-point planar homography that adds no trainable parameters and no mesh-fitting step, and we show it suffices for the in-plane-dominated pose regime.

3 CanonicalPhys Method↩︎

3.1 Problem Formulation↩︎

Given a face video \(V\in\mathbb{R}^{(T+1)\times H\times W\times 3}\) of \(T{+}1\) frames at resolution \(H\times W\), the goal is to predict a blood volume pulse signal \(\mathbf{r}\in\mathbb{R}^{T}\), where the one-frame offset absorbs temporal differencing. We additionally consume two auxiliary signals, both pre-extracted once per video and cached alongside the frames. The first is a tensor of normalized 2D landmarks \(\mathbf{L}\in[0,1]^{(T+1)\times K\times 2}\) produced by MediaPipe Face Mesh [9] and subsampled to the \(K{=}68\) dlib-compatible indices used for anchor extraction. The second is a tensor of per-frame facial rotation matrices \(R_t\in\mathbb{R}^{3\times 3}\), taken as the upper-left block of the MediaPipe \(4{\times}4\) facial transform. At inference, canonicalization requires only \(\mathbf{L}\), the Lambertian weight requires only \(R_t\), and the two auxiliary losses are disabled.

3.2 Architecture Overview↩︎

CanonicalPhys is a four-stage pipeline wrapped around an unmodified FactorizePhys backbone: \[V \;\xrightarrow{\text{canon.}}\; V^{c} \;\xrightarrow{\,\odot\,\mathbf{w}^{\text{phys}}}\; V^{c}_{w} \;\xrightarrow{\text{backbone}}\; \hat{\mathbf{r}}. \label{eq:pipeline}\tag{1}\] Canonicalization (Sec. 3.3) warps each frame into a canonical face coordinate system. The Lambertian weight map (Sec. 3.4) multiplies the canonical frame before temporal differencing. The backbone and its BVP head are taken unmodified from FactorizePhys [6]. Two training-time auxiliary losses operate in canonical space: a cross-ROI temporal consistency loss on the backbone’s \(13{\times}13\) feature maps (Sec. 3.5) and a POS knowledge distillation loss on canonical forehead pixels (Sec. 3.6). Figure 2 shows the full pipeline. None of the four canonical-space components adds trainable parameters, so CanonicalPhys has exactly the same parameter count as the backbone.

3.3 Canonicalization via a 4-Point Homography↩︎

We choose four anatomical anchors, namely the outer left eye \(\mathbf{l}_\text{LE}\), outer right eye \(\mathbf{l}_\text{RE}\), left mouth corner \(\mathbf{l}_\text{LM}\), and right mouth corner \(\mathbf{l}_\text{RM}\), and fixed canonical target positions \(\hat{\mathbf{l}}_\text{LE}{=}(0.30,0.40)\), \(\hat{\mathbf{l}}_\text{RE}{=}(0.70,0.40)\), \(\hat{\mathbf{l}}_\text{LM}{=}(0.38,0.72)\), \(\hat{\mathbf{l}}_\text{RM}{=}(0.62,0.72)\). Given these four correspondences, the homography \(H_t\) satisfying \(H_t\mathbf{l}_k{=}\hat{\mathbf{l}}_k\) for \(k\in\{\text{LE,RE,LM,RM}\}\) is the solution of the \(8\)-equation DLT linear system \(A_t\mathbf{h}_t{=}\mathbf{b}_t\) with \(h_{33}{=}1\) fixed. We solve this system in closed form with batched torch.linalg.lstsq. The solution is differentiable with respect to the anchors and the frame. Frames whose anchor quadrilateral is degenerate, detected via the variance of anchor positions, fall back to the identity homography so that the warp is a no-op rather than a singular transform. The warp itself samples the source image at \(H_t^{-1}\)-transformed pixel coordinates via bilinear grid sampling with zero padding. Frames flagged as invalid by the face detector pass through unchanged. The module has no learnable parameters.

3.3.0.1 Why four points and 2D.

Out-of-plane head rotation strictly requires a 3D face model, and a 2D homography is exact only for purely in-plane rotations. In practice the four-anchor quadrilateral is dominated by its rigid in-plane component for the head-pose distribution observed in rPPG datasets. A MediaPipe diagnosis on MMPD shows that more than \(95\%\) of frames have \(|\text{yaw}|{<}45^\circ\). A 2D warp helps the large-yaw regime because it removes the in-plane component of the anatomy-to-pixel reassignment, which dominates for this majority of frames; the residual non-planar shading that a planar warp cannot correct is handled separately by the Lambertian weight (Sec. 3.4). The homography is therefore a deliberately low-capacity warp that corrects the dominant pose component at no added parameter cost. Section 4.2 verifies empirically that it improves moderate and large yaw bins rather than harming them.

3.3.0.2 Temporal stabilization.

A per-frame homography introduces high-frequency jitter when landmarks are noisy. We therefore compute the temporal median \(\bar{H}\) of per-frame homographies within a chunk and use it as the reference warp for all frames in that chunk, which cancels frame-to-frame anchor jitter; degenerate or low-confidence anchors fall back to identity rather than a singular warp. An adaptive variant reverts to per-frame warps when landmark motion exceeds a threshold. A seed-matched sensitivity test at the opposite extreme, a fully unstabilized per-frame warp, does not degrade the recovered pulse, matching \(\bar{H}\) on UBFC and improving on PURE and MMPD, because temporal differencing absorbs the residual warp variation.

3.4 Physics-Informed Per-Pixel Weighting↩︎

Under the dichromatic reflection model [7], the diffuse component of skin reflectance that carries the pulse signal scales as \(\cos\theta\) between the skin normal and the camera axis, whereas the specular component peaks at grazing angles. We therefore upweight camera-facing pixels and downweight grazing ones.

In canonical coordinates every pixel \((x,y)\in[0,1]^2\) maps to a fixed anatomical location, so we precompute once a pixel-to-landmark assignment \[\pi(x,y)=\arg\min_{k\in[K]}\bigl\|(x,y)-\hat{\mathbf{l}}^{(k)}\bigr\|_2,\] where \(\hat{\mathbf{l}}^{(k)}\in[0,1]^2\) are the canonical 2D positions of the MediaPipe face mesh landmarks, and associate each pixel with that landmark’s fixed canonical 3D normal \(\mathbf{n}^{(k)}\in\mathbb{R}^3\). At runtime, the per-frame facial rotation \(R_t\) transforms the canonical normal into camera space: \[\mathbf{n}^{\text{cam}}_{t}(x,y)=R_t\,\mathbf{n}^{(\pi(x,y))}.\] The \(z\)-component of the camera-space normal is exactly \(\cos\theta\) under the camera-axis-as-light convention. We pass it through a sigmoid, \[\mathbf{w}^{\text{phys}}_t(x,y)=\sigma\!\Bigl(\alpha\,\bigl(\mathbf{n}^{\text{cam}}_{t}(x,y)\cdot\hat{\mathbf{z}}+\beta\bigr)\Bigr), \label{eq:physics95weight}\tag{2}\] with sharpness \(\alpha{=}8\) and bias \(\beta{=}0.55\). The weight is applied multiplicatively to the canonical frame prior to temporal differencing. The module has no learnable parameters. The canonical landmark positions \(\hat{\mathbf{l}}^{(k)}\) and their 3D normals \(\mathbf{n}^{(k)}\) are fixed buffers precomputed once from the MediaPipe mesh template, subsampled to the \(K{=}68\) dlib-compatible indices used throughout.

3.5 Cross-ROI Temporal Consistency Loss↩︎

A single heartbeat drives the pulse, so its phase and frequency must agree across skin regions even if the amplitude differs. This self-supervised signal requires a pose-stable ROI definition, which is provided by construction in canonical space. We place three Gaussian soft masks at the feature resolution, which is \(13{\times}13\) for FactorizePhys. The forehead mask is centered at \((0.50,0.20)\) with \(\sigma{=}0.10\), and the left and right cheek masks are centered at \((0.28,0.58)\) and \((0.72,0.58)\) with \(\sigma{=}0.07\). For each ROI \(r\) we extract a per-frame scalar from the backbone voxel embeddings \(\boldsymbol{\varepsilon}\in\mathbb{R}^{B\times C\times T\times H_f\times W_f}\): \[s^{(r)}_{b,t}=\Bigl\lVert \textstyle{\sum_{x,y}}\mathbf{m}^{(r)}(x,y)\,\boldsymbol{\varepsilon}_{b,:,t,x,y}\,/\,\textstyle{\sum_{x,y}}\mathbf{m}^{(r)}(x,y)\Bigr\rVert_2,\] and temporally difference then zero-mean, unit-std normalize the result to produce \(\tilde{s}^{(r)}\in\mathbb{R}^{B\times(T-1)}\). The auxiliary loss is the averaged cosine distance over the three unordered ROI pairs \(\mathcal{P}\): \[\mathcal{L}_\text{cons}=\frac{1}{|\mathcal{P}|}\sum_{(i,j)\in\mathcal{P}}\Bigl(1-\mathbb{E}_{b,t}\bigl[\tilde{s}^{(i)}_{b,t}\tilde{s}^{(j)}_{b,t}\bigr]\Bigr). \label{eq:consistency95loss}\tag{3}\] Operating on backbone features rather than raw pixels routes the gradient through the network, so the backbone learns spatially consistent pulse features at canonical ROIs.

3.6 POS Knowledge Distillation Loss↩︎

POS [7] projects a windowed RGB trace onto a chromaticity basis that suppresses specular and illumination components and yields a pseudo-BVP from a fixed skin ROI. Using POS as a teacher in image coordinates would require either a per-frame ROI detector, which is noisy, or a loose ROI, which degrades quality. In canonical coordinates the forehead is pose-stable by construction. We extract a spatially averaged RGB trace from the canonical forehead, reusing the Gaussian mask at \((0.50,0.20)\) but applied to canonical pixels rather than backbone features: \[\mathbf{c}_{b,t}=\sum_{x,y}\mathbf{m}^{\text{FH}}(x,y)\,V^{c}_{b,:,t,x,y}\,/\,\textstyle{\sum_{x,y}}\mathbf{m}^{\text{FH}}(x,y)\in\mathbb{R}^3.\] We apply POS in its original windowed form [7] with window length \(L{=}\lceil 1.6\,\text{s}\times\text{FPS}\rceil{=}48\) frames at 30 fps, implemented in pure PyTorch so that no gradients flow back to the pseudo-BVP. For each window at offset \(m\), \[\begin{align} C_n &= \mathbf{c}_{m:m+L}\,/\,\bar{\mathbf{c}}_{m:m+L},\\ S &= \mathbf{P}\,C_n^\top \in \mathbb{R}^{2\times L},\quad \mathbf{P}=\begin{bmatrix}0 & 1 & -1\\-2 & 1 & 1\end{bmatrix},\\ h &= S_0 + \bigl(\operatorname{std}(S_0)/\operatorname{std}(S_1)\bigr)\,S_1, \end{align}\] and we overlap-add the zero-mean \(h\) into the full sequence \(\tilde{\mathbf{r}}^{\text{POS}}_{b}\in\mathbb{R}^T\). Degenerate windows, for example those containing only warp-padding pixels, are masked. The pseudo-BVP is detached from the computational graph. The distillation loss is the negative Pearson correlation between the prediction \(\hat{\mathbf{r}}\) and the detached pseudo-BVP: \[\mathcal{L}_\text{dist}=1-\frac{\operatorname{Cov}(\hat{\mathbf{r}},\,\texttt{sg}(\tilde{\mathbf{r}}^{\text{POS}}))}{\sigma_{\hat{\mathbf{r}}}\,\sigma_{\texttt{sg}(\tilde{\mathbf{r}}^{\text{POS}})}+\epsilon}, \label{eq:distill95loss}\tag{4}\] where \(\texttt{sg}(\cdot)\) denotes the stop-gradient operator. As a sanity check on synthetic pulses, we verify that POS applied to canonical forehead pixels recovers the injected pulse with Pearson \(r{\approx}0.96\).

3.7 Training Objective and Cost↩︎

The total training loss is a weighted sum of the primary negative-Pearson BVP loss \(\mathcal{L}_\text{BVP}\) and the two canonical-space auxiliary losses: \[\mathcal{L}=\mathcal{L}_\text{BVP}+\lambda_\text{cons}\,\mathcal{L}_\text{cons}+\lambda_\text{dist}\,\mathcal{L}_\text{dist}, \label{eq:total95loss}\tag{5}\] with \(\lambda_\text{cons}{=}0.1\) and \(\lambda_\text{dist}{=}0.2\) fixed across all experiments. The Lambertian weight is a forward-pass modification rather than a loss term, and both auxiliary losses are disabled at inference.

3.7.0.1 Parameter and runtime cost.

None of the four canonical-space components adds trainable parameters, so CanonicalPhys has the same \(52\)K parameter count as the FactorizePhys backbone. Per frame, canonicalization adds one \(\texttt{lstsq}\) solve and one \(\texttt{grid\_sample}\) call, the Lambertian weight multiplies against a precomputed \(72{\times}72{\times}3\) buffer, and the two auxiliary losses are training-time only. The 2D landmarks and facial-transform matrices are extracted once by MediaPipe and cached in LMDB sidecars, so they add no per-training-step cost and are comparable to the RetinaFace face crop every baseline already computes. The wall-clock overhead over the backbone is under \(10\%\).

4 Experiments↩︎

4.1 Datasets and Protocol↩︎

4.1.0.1 Datasets.

We evaluate on four public rPPG benchmarks that together span a wide range of sensors, illumination conditions, skin tones, and head-pose distributions: UBFC-rPPG [11], PURE [12], OBF [13], and MMPD [14]. MMPD is the most challenging of the four due to its skin-tone and motion diversity, and is used as the primary target for our pose-stratified analysis.

4.1.0.2 Protocol.

For each dataset we split subjects \(80/20\) into train and validation folds, and report results on the official held-out test split; the exact subject partitions follow the rPPG-Toolbox [25] configurations. All frames are face-cropped with RetinaFace [10] to \(72{\times}72\) and chunked into clips of \(128\) frames. MediaPipe Face Mesh [9] produces landmarks, visibility flags, and \(4{\times}4\) facial transforms, which we cache in LMDB sidecars. The best checkpoint is selected by validation loss. All models are trained with the negative Pearson loss using Adam [26] at learning rate \(10^{-3}\), batch size \(4\), for \(30\) epochs. Cross-dataset evaluation trains on one dataset and tests on all four. Both CanonicalPhys and FactorizePhys are re-run over \(3\) seeds under this identical pipeline, so every comparison is matched in preprocessing, splits, optimizer, frame count, and checkpoint selection. We report \(3\)-seed mean \(\pm\) standard deviation for both methods in the primary UBFC-trained comparison (Table 2) and in the full \(4{\times}4\) transfer matrix (Table 3).

4.1.0.3 Metrics.

Following prior work [25], we report mean absolute error (MAE, bpm\(\downarrow\)), root mean square error (RMSE, bpm\(\downarrow\)), mean absolute percentage error (MAPE, %\(\downarrow\)), Pearson correlation (\(r\uparrow\)), and signal-to-noise ratio (SNR, dB\(\uparrow\)). Heart rate is extracted by FFT.

4.2 Pose-Stratified Analysis on MMPD↩︎

Cross-dataset MAE confounds head pose with sensor, illumination, and subject-pool shift, so it can only serve as an indirect probe of pose robustness. We therefore begin with the pose-stratified analysis that most directly tests our thesis. We stratify the MMPD test set by \(|\text{yaw}|\) into four bins, namely frontal (\(<15^\circ\)), mild (\(15\) to \(30^\circ\)), moderate (\(30\) to \(45^\circ\)), and large (\(\geq 45^\circ\)), and evaluate three MMPD-trained models on each bin: FactorizePhys, CanonicalPhys with the canonicalization warp only, and CanonicalPhys with the warp, the Lambertian weight, and the cross-ROI consistency loss. Table 1 reports both absolute MAE and the degradation ratio \(\text{bin\_MAE}/\text{frontal\_MAE}\), which is invariant to a method’s absolute MAE level and is our primary pose-robustness metric.

Three patterns emerge. First, both CanonicalPhys variants deliver their largest absolute MAE reductions at mild and large yaw, the regimes where the image-coordinate baseline is weakest. Second, the degradation ratio from frontal to large yaw drops from \(1.60\) for FactorizePhys to \(1.33\) for canonicalization alone, confirming that canonicalization directly attacks pose-induced generalization loss. Third, adding the Lambertian weight and the consistency loss yields the most uniform degradation across bins (\(1.07/1.36/1.44\)) and removes an anomaly of the canonicalization-only variant at moderate yaw, where the ratio drops from \(1.55\) to \(1.36\). This is the behavior expected of a per-pixel cosine weighting. The \(30\) to \(45^\circ\) band is precisely where the surface-normal-to-camera angle matters most, and the Lambertian weight provides a per-pixel diffuse-reflectance correction there. The gains are not uniform across all bins, however: canonicalization alone regresses at moderate yaw, and the consistency loss is weakest at large yaw. We analyze both effects in Sec. 4.6.

4.3 Cross-Dataset Evaluation↩︎

Different datasets realize different head-pose, motion, and illumination distributions, so cross-dataset MAE probes whether the pose-robustness effect of Sec. 4.2 transfers across sensors and subject pools. Table 2 reports the seed-matched UBFC-trained comparison across the complete metric suite. CanonicalPhys improves or matches FactorizePhys on every metric on every target set (MAE, RMSE, MAPE, \(r\), and SNR), which addresses the concern that the gains might be visible only in HR-MAE. The two largest MAE gains, \(-32\%\) on PURE and \(-20\%\) on MMPD, fall on the pose-richer targets, and CanonicalPhys nearly doubles the Pearson correlation on MMPD (\(0.18{\to}0.33\)) while more than halving PURE’s seed variance.

4.3.0.1 The gains track pose, not sensor.

To connect these cross-dataset improvements to pose rather than to sensor or illumination shift, we measure the fraction of non-frontal frames (\(|\text{yaw}|{\geq}15^\circ\)) in each test set: \(0\%\) on UBFC, approximately \(0\%\) on OBF, \(7.9\%\) on PURE, and \(14.3\%\) on MMPD. The absolute MAE reduction over FactorizePhys rises with this pose-richness (\(-0.22\) bpm on near-frontal UBFC, \(-0.96\) on PURE, \(-2.78\) on MMPD); the relative percentages invert (\(-32\%\) on PURE versus \(-20\%\) on MMPD) only because PURE’s baseline error is far smaller. OBF is near-infrared, outside the regime of the chromatic priors, so its near-zero change is uninformative about pose. Under identical preprocessing, the handcrafted CHROM and POS reach \(13.61\) and \(13.53\) bpm MAE on UBFC\(\rightarrow\)MMPD, matching FactorizePhys’s \(13.69\); this indicates that the residual deep-learning error on this target is dominated by pose- and motion-induced nuisance rather than by pulse-modeling capacity. The deep baselines PhysNet [4] and EfficientPhys [3] reach \(11.52\) and \(19.09\) bpm on this transfer, both above CanonicalPhys’s \(10.91\).

4.3.0.2 Per-video significance.

Beyond aggregate means, the improvement is significant at the level of individual videos: on the MMPD test set, CanonicalPhys’s per-video absolute HR error is below FactorizePhys’s under a matched seed, so the aggregate MAE gap is not an artifact of averaging.

Table 3 reports the full \(4{\times}4\) transfer matrix. The primary seed-matched comparison remains Table 2. CanonicalPhys wins \(13\) of the \(16\) cells against the matched FactorizePhys. Beyond the UBFC-trained gains of Table 2, the largest reductions fall on pose-rich transfers to non-laboratory targets: MAE drops by \(23\%\) on PURE\(\rightarrow\)MMPD and \(18\%\) on MMPD\(\rightarrow\)UBFC, and by more than half on the OBF-trained transfers (e.g., OBF\(\rightarrow\)UBFC), where the matched backbone early-overfits and generalizes poorly. The three cells FactorizePhys wins are the two near-frontal PURE-trained laboratory cells (PURE\(\rightarrow\)UBFC and PURE\(\rightarrow\)PURE, both below \(1.3\) bpm and within cross-seed noise) and MMPD\(\rightarrow\)PURE (\(4.98\) vs \(5.76\) bpm). The in-domain OBF cells are further complicated by an early-overfit training dynamic shared by both methods, which we discuss in Sec. 5.

4.4 Ablation Study↩︎

Table 4 is a per-component ablation over \(3\) seeds that starts from the warp-only model, with no auxiliary loss and no Lambertian weight, and adds each component individually. Because MMPD is the only target whose effect sizes exceed cross-seed noise, we rank components using it.

We observe four trends. First, canonicalization alone accounts for most of the MMPD gain: the warp-only model reaches \(11.67\) bpm versus FactorizePhys’s \(13.69\), closing roughly three quarters of the gap to the full model. Because this variant carries no auxiliary loss and no Lambertian weight, its gain cannot be attributed to regularization, which localizes the mechanism to the change of coordinates. Second, among the auxiliaries the consistency loss is the largest single-component effect on MMPD (\(-0.50\) from warp-only), ahead of the Lambertian weight (\(-0.22\)), consistent with cross-ROI consistency pushing the backbone toward pose-invariant pulse extraction. Third, distillation is individually second-order and slightly negative on aggregate MMPD MAE (\(+0.42\)), yet the three components are super-additive: the full model (\(10.95\)) improves on the sum of individual deltas (\(11.37\)), and distillation still tightens seed variance (the UBFC within-dataset std falls to \(\pm.09\) for the full model). Fourth, effect sizes on the near-frontal targets (UBFC, OBF) are within cross-seed noise, so we do not draw per-component conclusions from those columns and rank components using MMPD only.

4.5 Mechanism Analysis↩︎

A natural concern is that the two auxiliary losses simply act as generic regularizers rather than producing the specific effects they are designed for. We show they do not. For the consistency loss, we read the three canonical-ROI signals it operates on (forehead and the two cheeks) directly from the backbone voxel embeddings and measure their pairwise phase coherence on the UBFC test set. Without the loss the ROI signals are largely out of phase (mean coherence \(0.19\)); with it they lock in phase (mean coherence \(0.68\)), as visualized in Fig. 3. A generic regularizer would not selectively produce cross-ROI phase agreement. The Lambertian weight shows the same module-specific selectivity: with no supervision it repairs precisely the \(30\) to \(45^\circ\) yaw band where the surface-normal-to-camera angle dominates, and it concentrates on camera-facing skin as yaw grows (Fig. 4). Each component therefore acts as claimed.

4.6 Failure Modes under Head Pose↩︎

CanonicalPhys is pose-robust, not pose-invariant. Table 1 itself exposes four regimes in which the method still degrades, and we make no claim on one additional axis.

4.6.0.1 Moderate yaw without the Lambertian weight.

At \(30\) to \(45^\circ\), canonicalization alone regresses above the image baseline, with a ratio of \(1.55\) versus \(1.36\). The reason is that the 2D homography over-stretches the near side of the face, and the backbone sees a distribution it did not train on. The Lambertian weight corrects this anomaly exactly, as the ratio recovers from \(1.55\) to \(1.36\), which confirms its geometric motivation. Canonicalization without the weight should not be expected to help this band uniformly.

4.6.0.2 Large yaw with the consistency loss.

The variant with the Lambertian weight and the consistency loss is the weakest pose-robust variant at \(|\text{yaw}|{\geq}45^\circ\), with a ratio of \(1.44\) versus \(1.33\). The reason is the far cheek is self-occluded, and the single-phase consistency loss then demands agreement with a partially missing region. A visibility-gated cheek pair, weighting ROI by its per-frame landmark visibility, is the natural fix and is left to future work.

4.6.0.3 Extreme yaw above \(60^\circ\).

Beyond \(60^\circ\) the four-anchor quadrilateral is no longer detectable for a non-trivial fraction of frames. These frames fall back to the identity warp and match the image-space baseline by construction. We make no claim of pose-robustness in this regime, and a 3D mesh warp is the natural extension.

4.7 Qualitative Analysis↩︎

4.7.0.1 Canonicalization and physics weighting.

Figure 4 traces the pipeline on MMPD frames of increasing yaw. In the raw frames the observed anchors drift away from their fixed canonical targets, so a given image-coordinate location maps to different anatomy as the head turns. The four-point warp snaps the anchors back onto the targets, stabilizing the forehead and cheek ROIs across poses. On the resulting canonical frame, the Lambertian weight \(\mathbf{w}^{\text{phys}}\) is broad and near \(1\) frontally, concentrates on the camera-facing side of the face as yaw grows, purely from the canonical-normal-times-rotation computation of Eq. 2 .

4.7.0.2 Predicted waveforms on side-face clips.

Figure 5 compares predicted and ground-truth BVP on MMPD test clips with \(|\text{yaw}|{>}30^\circ\). The FactorizePhys drifts or collapses to noise, whereas CanonicalPhys recovers the correct rhythm.

5 Discussion and Limitations↩︎

The gains concentrate where the evaluation contains substantial head pose, whether pose-diverse targets or sources; when both source and target are near-frontal laboratory distributions there is no pose nuisance left to neutralize, and CanonicalPhys sits within cross-seed noise (Table 3, PURE-trained rows). The three auxiliary priors (approximately Lambertian skin, single-phase pulse, POS reliability) can over-regularize on motion-diverse training sets: training on MMPD and testing on PURE, MAE regresses from \(1.15\) bpm with canonicalization alone to \(5.76{\pm}0.91\) with the full recipe. Canonicalization is therefore the unconditionally reliable lift, and the three auxiliaries layer most safely on compact training sets, with Sec. 4.6 detailing the pose regimes where the claim does not yet hold. Finally, OBF training peaks at epoch \(1\) and then overfits for both methods, so all OBF-trained rows use the epoch-\(1\) checkpoint.

5.0.0.1 Toward 3D and more backbones.

A full 3D face model would canonicalize out-of-plane rotation exactly. UV-unwrap approaches such as Cantrill  [24] show the promise of this direction, at the cost of per-frame mesh fitting; extending our parameter-free planar warp toward a 3D-mesh canonicalization and benchmarking against such methods is a natural next step. The canonical-space priors are also backbone-agnostic, since they act on the input and on generic voxel features rather than on FactorizePhys-specific internals; validating them on additional rPPG backbones is left to future work.

6 Conclusion↩︎

We presented CanonicalPhys, a pose-robust rPPG method that treats head pose as a coordinate-system problem. A four-point homography, with no trainable parameters, maps each frame to a canonical face coordinate system. In this frame three parameter-free priors become expressible, and the model has the same parameter count as the FactorizePhys backbone. Pose-stratified MMPD evaluation cuts the frontal-to-large-yaw MAE degradation from \(1.60\times\) to \(1.33\times\) and flattens mild yaw from \(1.32\times\) to \(1.07\times\), though not uniformly across every pose bin; cross-dataset results confirm that the effect transfers across sensors and conditions and holds across the full metric suite. A coordinate change can deliver pose robustness that “more learnable attention” does not, and we expect the same lens to transfer to other face-centric sensing tasks.

Acknowledgements. This work was supported by the Research Council of Finland (former Academy of Finland) Academy Professor project EmotionAI (grants 336116, 359894), HPC project FaceCanvas (grant number 364905), the University of Oulu & Research Council of Finland Profi 7 (grant 352788), and EU HORIZON-MSCA-SE-2022 project ACMod (grant 101130271). As well, the authors wish to acknowledge CSC – IT Center for Science, Finland, for computational resources.

References↩︎