MAME: Multidimensional Adaptive Metamer Exploration with
Human Perceptual Feedback

Mina Kamao
Graduate School of Information Science and Technology
The University of Tokyo, Tokyo, Japan
kmmina0902@g.ecc.u-tokyo.ac.jp
Hayato Ono
Graduate School of Information Science and Technology
The University of Tokyo, Tokyo, Japan
hayato-0628@g.ecc.u-tokyo.ac.jp
Ayumu Yamashita
1. Graduate School of System Informatics
Kobe University, Hyogo, Japan
2. Graduate School of Information Science and Technology
The University of Tokyo, Tokyo, Japan
ayumu722@people.kobe-u.ac.jp
Kaoru Amano
Graduate School of Information Science and Technology
The University of Tokyo, Tokyo, Japan
kaoru_amano@ipc.i.u-tokyo.ac.jp
Masataka Sawayama 1
1. Graduate School of Information Science and Technology
Hokkaido University, Hokkaido, Japan
2. Prometech CG Research, Tokyo, Japan
3. Graduate School of Information Science and Technology
The University of Tokyo, Tokyo, Japan
masataka_sawayama@ist.hokudai.ac.jp

March 17, 2025

Keywords: metamers; texture synthesis; psychophysics; cognitive neuroscience; machine learning

1 Introduction↩︎

2 Methods↩︎

3 Results↩︎

4 Discussion↩︎

5 Conclusions↩︎

6 Acknowledgments↩︎

This work was supported by JSPS KAKENHI Gran Numbers JP21H04909 and JP26K02927. We used ChatGPT (OpenAI) for English language editing. All content was reviewed and verified by the authors.

7 Supplementary Materials↩︎

Figure 1: Examples of image modulation using models with different training regimes but the same architecture as the robust ResNet50 used in this study. The robust ResNet was trained to be resistant to adversarial attacks, whereas the default model was trained on standard ImageNet data, and the stylized model was trained on stylized versions of ImageNet [1]. When the same modulation parameters were applied to each model, we found that, particularly in higher layers, the default and stylized models failed to produce clear modulations from the original image, in contrast to the robust model (cf. Feather, 2023).
Figure 2: Comparison of image modulation using PCA and ICA in high-level representations (computed at the avgpool layer of the robust ResNet50). (Upper) Three principal components were extracted via PCA from vectorized Gram matrices, and image modulation was performed along each component axis. (Bottom) Image modulation using ICA, as employed in the present study. Due to the learning properties of DNNs, high-level features are expected to contain many independent components. The PCA-based modulation, which preserves orthogonality in the Gram matrix space, fails to capture such diversity, making it difficult to generate varied image modulations.
Figure 3: Examples of image modulation for each layer and each component. This figure presents a finer-step version of Fig. [fig:imggen_examples].
Figure 4: Explained variance of ICA components sorted in descending order. (Top) conv1, (Middle) layer3, (Bottom) avgpool.
Table 1: Normalized final loss for each layer, ICA component, and direction (mean \(\pm\) SD, \(n=100\)).The normalized final loss is defined as the ratio between the final loss and the initial loss before optimization.For each layer, target values were sampled uniformly in log-space (i.e., log-spaced sampling) within the specified \(\log_{10}(t)\) range.
Layer \(\log_{10}(t)\) range Component Direction Normalized final loss
conv1 [0.0, 3.0] 0 + 0.006636 \(\pm\) 0.024610
\(-\) 0.007039 \(\pm\) 0.027097
1 + 0.004328 \(\pm\) 0.010553
\(-\) 0.004179 \(\pm\) 0.010673
2 + 0.005512 \(\pm\) 0.010508
\(-\) 0.007136 \(\pm\) 0.014643
layer3 [-2.0, 1.0] 0 + 0.006004 \(\pm\) 0.012682
\(-\) 0.005656 \(\pm\) 0.010533
1 + 0.007383 \(\pm\) 0.014922
\(-\) 0.006431 \(\pm\) 0.014312
2 + 0.005035 \(\pm\) 0.008581
\(-\) 0.006483 \(\pm\) 0.011846
avgpool [-2.0, 1.0] 0 + 0.013961 \(\pm\) 0.055785
\(-\) 0.011262 \(\pm\) 0.044577
1 + 0.006213 \(\pm\) 0.021678
\(-\) 0.007241 \(\pm\) 0.016612
2 + 0.008365 \(\pm\) 0.026241
\(-\) 0.006456 \(\pm\) 0.017010
Table 2: Threshold estimates based on staircase reversal points. In addition to the psychometric-function analysis reported in Table [tab:pse_subjects], thresholds were estimated by averaging the last five reversal points of the staircase trajectory for each condition.
Layer Eccentricity (°) Mean Std
conv1 4 47.915 8.8166
8 63.0404 11.8613
12 63.7918 8.5024
layer3 4 0.486 0.1275
8 0.5983 0.0686
12 0.6407 0.0533
avgpool 4 0.1538 0.0182
8 0.153 0.0253
12 0.1688 0.0203
Figure 5: The ICA threshold values obtained from all subjects (n=8) across nine radar charts. Each row corresponds to a convolutional layer, and each column represents the eccentricity (in degrees) at which the stimulus was presented. The ICA threshold value at each point indicates the mean of reversal points obtained through the staircase method (n \geq 6). Colors distinguish individual subjects, with the same color consistently representing the same subject across all charts.
Figure 6: Example psychometric function fits from a single participant. Thresholds in the main analysis were estimated by fitting sigmoidal functions. Because some fits may not adequately capture the data (e.g., b), we additionally analyzed the data using reversal averages from the classical staircase procedure and obtained similar results (Tables 2). We also confirmed that fit quality was comparable across layer conditions (RMSE: conv1 = 0.11, layer3 = 0.10, avgpool = 0.13).
Table 3: Results of the two-way repeated-measures ANOVA for SSIM metrics. Reported \(p\) values are Greenhouse–Geisser corrected.
Source df \(F\) \(p\) \(\eta_g^2\)
Layer \((2, 14)\) 696.67 \(< .001\) 0.936
Eccentricity \((2, 14)\) 30.91 \(< .001\) 0.346
Layer \(\times\) Eccentricity \((4, 28)\) 7.00 .016 0.218
Table 4: Results of the two-way repeated-measures ANOVA for LPIPS metrics. Reported \(p\) values are Greenhouse–Geisser corrected.
Source df \(F\) \(p\) \(\eta_g^2\)
Layer \((2, 14)\) 826.47 \(< .001\) 0.937
Eccentricity \((2, 14)\) 38.25 \(< .001\) 0.368
Layer \(\times\) Eccentricity \((4, 28)\) 5.30 .044 0.183
Table 5: Results of the two-way repeated-measures ANOVA for DISTS metrics. Reported \(p\) values are Greenhouse–Geisser corrected.
Source df \(F\) \(p\) \(\eta_g^2\)
Layer \((2, 14)\) 1871.61 \(< .001\) 0.980
Eccentricity \((2, 14)\) 23.16 \(< .001\) 0.316
Layer \(\times\) Eccentricity \((4, 28)\) 8.47 .015 0.248

References↩︎

[1]
R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel, “ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness,” arXiv preprint arXiv:1811.12231, 2018.

  1. Corresponding authors: Masataka Sawayama and Mina Kamao↩︎

Subjects

Machine Learning (cs.LG)

Updated on Academus

2026/07/09 11:56 UTC