1 Introduction↩︎

Recent advancements in diffusion-based image generation, exemplified by the Diffusion Transformer (DiT) [@peebles2023scalable] and flow matching [@lipman2022flow; @gat2024discrete; @dao2023flow], have driven significant progress. Beyond diffusion models, other generative paradigms like masked generative models and autoregressive visual models have seen notable developments. For instance, VAR [@tian2024visual] introduces next-scale prediction, where autoregressive visual models surpass diffusion models. Additionally, MAE-tok [@chen2025masked] and MAR [@li2024autoregressive] have improved masked generative modeling, boosting downstream generation performance. Multi-scale designs, as demonstrated by HieraTok [@chen2025hieratok], VAR [@tian2024visual], and FlowAR [@ren2024flowar], further enhance performance. In parallel with quality improvements, recent studies have focused on enhancing generation efficiency. For example, Lightning-DiT [@yao2025reconstruction] accelerates training by optimizing latent representations for image generation, addressing the reconstruction-generation tradeoff. Similarly, REG [@wu2025representation] accelerates SiT [@ma2024sitexploringflowdiffusionbased] training. As generative frameworks advance, image tokenization has become essential for enabling multimodal understanding and generation through compact latent representations. Early methods like VQ-GAN [@esser2021taming] and VQ-VAE [@van2017neural] encoded images as 2D grid latents, preserving spatial relationships. However, the need for a strict one-to-one mapping between image patches and tokens is unclear, driving the exploration of 1D sequential tokenizers, which offer higher compression while retaining key semantic and structural information.

The emergence of 1D image tokenizers to encode images into compact latent sequences provides a promising alternative to conventional 2D grid representations. The key challenge lies in capturing both global semantics and local details within an extremely limited number of tokens. Early efforts such as Titok [@tian2024visual] demonstrated that an image can be effectively represented by as few as 32 one-dimensional tokens, establishing the feasibility of this paradigm. Subsequent developments, including Flextok [@bachmann2025flextok], TA-Titok [@kim2025democratizing], Flowtok [@he2025flowtok], and GigaTok [@xiong2025gigatok], have progressively enhanced token efficiency, flexibility, and semantic richness. Despite these advancements, existing methods still exhibit redundant token interactions and overlap between global and local representations, which constrain scalability and generation efficiency.

In this work, we propose SelfBootTok. It improves the balance at various aspects between image compression and generation, by decomposing conventional visual tokens into high-level semantics-based (i.e., global tokens) and fine-grained visual information-related (i.e., local tokens). The conceptual comparison with existing works is illustrated in 1. The proposed new model has several advantages:

1. Simpler and more efficient generator: Traditionally, visual generators are required to produce all levels of visual details at once when fed with prompts. In contrast, our method bootstraps fine-grained visual information directly from global tokens using unlabeled images, as shown in  1, thereby bypassing the need for extensive text-image pairs. This scheme essentially moves part of the generation process into the compression pipeline. The generator can focus solely on learning high-level semantics from a compact token set, requiring significantly less data and complexity.

2. Scalable image tokenizer: Our tokenizer employs a self-supervised learning strategy to reconstruct local image information from global tokens. It utilizes a hybrid of 1D and 2D local tokens to capture features at varying granularities. A novel optimal transport alignment is introduced to compactly map these 2D features into the 1D token sequence. This paradigm allows the tokenizer to scale efficiently using more data or parameters. This decomposition also minimizes token-level redundancy, thereby reducing the computational burden on the subsequent generator and enhancing the generation efficiency.

3. Parallel optimization of scaled-up tokenizers and generators: This framework enables efficient scaling by sharing only a set of global tokens. After learning these tokens, the global component of the tokenizer is frozen. The generators and the larger, bootstrapped local components of the tokenizers can then be trained in parallel. Crucially, this decouples their design, allowing the tokenizer size to be varied without requiring the generators to be re-trained.

Comprehensive experiments demonstrate that our method achieves state-of-the-art generation performance among 1D tokenizers and exhibits strong scalability of the self-bootstrapping design. Moreover, we propose a training strategy that scales local aligners while generating global tokens only once, reducing total computational cost by approximately 40% and training time by about 54%.

2 Related work↩︎

2.1 Image tokenization↩︎

Image tokenization methods can be broadly categorized by their latent space type (discrete vs. continuous) and spatial structure (2D grid vs. 1D sequence). For discrete modeling, Vector Quantization (VQ) frameworks [@van2017neural; @esser2021taming; @yu2021vector] learn a codebook of discrete visual tokens. In contrast, continuous models such as KL-VAEs [@kingma2013auto] employ the reparameterization trick to constrain latents to Gaussian priors. Most early works, including VQ-VAE and SD-VAE [@rombach2022high], adopt 2D grid latents to preserve spatial structure, a design that underpins diffusion-based models like Stable Diffusion. While highly effective, these 2D formulations impose limitations in compactness and efficiency, prompting recent research into 1D image tokenizers that represent images as highly compressed sequential embeddings.

The study of 1D image tokenization began with Titok [@yu2024image], which demonstrated effective reconstruction using only 32 tokens by representing images as short 1D sequences. Titok uses a two-stage training pipeline for discrete 1D VQ modeling, leveraging codes from pretrained models like MaskGIT-VQGAN [@chang2022maskgit]. Subsequent works have explored various directions, including Flowtok and TA-Titok [@he2025flowtok; @kim2025democratizing] for text-to-image generation, Flextok and OneDpiece [@bachmann2025flextok; @miwa2025one] for variable-length sequences, and GigaTok [@xiong2025gigatok] for combining 1D and 2D structures. Recent work [@beyer2025highly] further investigates 1D sequences for fine-grained image editing. To improve tokenization quality, SoftVQ [@chen2025softvq] introduces a differentiable soft vector quantization mechanism, and MAE-Tok [@chen2025masked] leverages masked autoencoding for semantically enriched latent spaces with strong reconstruction fidelity. However, none of these tokenizers explicitly differentiate or exploit the global-local information. To address this, we propose SelfBootTok, a 1D tokenizer that models global-local interactions for compact, efficient image representation.

2.2 Generative Models↩︎

Generative models are critical for evaluating image tokenizers in downstream generation tasks, and can be broadly divided into diffusion-based and autoregressive approaches. Diffusion-based models generate images by gradually converting Gaussian noise into structured latent codes. Representative methods include DiT, SiT, and MAR [@peebles2023scalable; @ma2024sitexploringflowdiffusionbased; @li2024autoregressive]. DiT adopts a Transformer backbone and models diffusion via stochastic differential equations. SiT leverages stochastic interpolants and optimizes the velocity field of a probability flow ODE. MAR unifies diffusion and autoregression within an encoder-decoder Transformer. Lightning-DiT [@yao2025reconstruction] further boosts efficiency via a lightweight design that balances reconstruction and generation quality. Autoregressive models follow the next-token prediction paradigm from LLMs, enabling unified multimodal generation. Methods incorporating masked modeling [@chang2022maskgit; @weber2024maskbit; @yu2023language; @yu2024image] and next-scale prediction [@tian2024visual; @li2024imagefolder] further improve representation and generation quality.

2.3 Self-Bootstrapped and Scaling Paradigms in Multimodal Learning↩︎

Self-bootstrapping has proven effective for multimodal learning by fully exploiting model capacity. For instance, BLIP-2 [@li2023blip] introduced it for vision–language fusion, OK-VQA [@hao2024self] and VILA [@fang2024vila] applied it to visual question answering and model refinement, and VideoJudge [@waheed2025videojudge] extended it to scalable video assessment. Further works [@xia2025bootstrapping; @ding2024lowis3d] validated its potential in data-efficient reasoning and 3D scene understanding. Nevertheless, its application to image tokenization remains largely unexplored. The success of scaling in LLMs has motivated extensive research in multimodal understanding and generation. In visual understanding, recent studies have focused on scaling vision encoders within multimodal LLMs [@alayrac2022flamingo; @awadalla2023openflamingo; @dai2023instructblip; @li2023videochat; @lin2023video; @maaz2023video; @liu2023visual; @achiam2023gpt; @team2023gemini], with InternVideo2 [@wang2024internvideo2] and CuMo [@li2024cumo] adopting efficient scaling and sparsely-gated Mixture-of-Experts layers. For image understanding, SViT [@zhao2023svit], LongLLaVA [@wang2024longllava], and LLaVA-scale [@lu2023empirical] investigate backbone scaling, while AuroraCap [@chai2024auroracap] and LLaVA-Next [@li2024llava] use multi-stage training to unify visual instruction tuning across image and video. In multimodal generation, scaling remains a critical challenge. Methods such as ViT-VQGAN [@yu2021vector] and ViTok [@hansenestruch2025learningsscalingvisualtokenizers] show that larger tokenizers do not always improve generation performance and often underuse model capacity. GigaTok [@xiong2025gigatok] improves scaling efficiency via hybrid CNN-Transformer tokenizers. However, efficiently scaling local token information is still an open problem, which we target in this work.

3 Method↩︎

Our image tokenization framework consists of three key components: global-local decomposition, self-supervised bootstrapped learning, and 2D to 1D alignment via optimal transport. Our framework employs a hierarchical self-bootstrapped VAE to support efficient global-local representation learning. The global-local decomposition aims to separately optimize global and local token representations, thereby reducing redundant information and improving representational efficiency. The self-supervised bootstrapped learning mechanism is designed to capture the intrinsic relationships between global semantics and fine-grained local details, facilitating hierarchical feature learning. The 2D to 1D alignment module preserves the sequential structure of 1D tokens by transporting the 2D token distribution into a 1D sequence space and minimizing an optimal transport loss. Moreover, our framework naturally supports efficient scaling of local aligners, enabling flexible adaptation to different computational budgets while maintaining high generation performance and significantly reducing training cost.

3.1 Global-local Decomposition↩︎

As shown in 3, after getting the global tokens \(z_G\) from input image \(I\) through a Vision Transformer encoder \(Enc\), our pipeline recovers more local information through local alignment and get local 1D tokens \(z_{L1}\) and local 2D tokens \(z_{L2}\). Then, we apply a multi-codebooks design that distinguishes local entries with global entries and enhances the localized representations with either 1D or 2D information injection: \[\begin{align} & \boldsymbol{Encode:} \quad \mathbf{z}_G = Enc(I), \quad \mathbf{z}_{\mathrm{tot}} = \{ \mathbf{z}_G, \mathbf{z}_{L1}, \mathbf{z}_{L2} \}, \\[4pt] & \boldsymbol{Codebook Set:} \quad \mathcal{C}_{\mathrm{tot}} = \{ \mathcal{C}_{\mathrm{global}}, \mathcal{C}_{\mathrm{local1D}}, \mathcal{C}_{\mathrm{local2D}} \}. \end{align} \label{eq:encoder95codebook}\tag{1}\]

We apply soft vector quantization [@chen2025softvq] such that both global and local quantizations are fully-differentiable, and thus the encoder and codebook can be optimized directly from the reconstruction loss: \[\begin{align} q_{\phi}(Z \mid I) &= \mathrm{Softmax}\!\left( -\tfrac{1}{\tau}\lVert \hat{Z} - \mathcal{C} \rVert_2 \right), \;\hat{Z} \!\in\! \mathbf{z}_{\mathrm{tot}}, \\[-2pt] Z &= q_{\phi}(Z \mid I) \, \mathcal{C}, \;\mathcal{C} \!\in\! \mathcal{C}_{\mathrm{tot}}. \end{align}\]

Under this design, we separate different token groups using distinct codebooks, allowing each group to specialize in learning representations of a specific granularity. The overall training objective combines reconstruction, perceptual, adversarial, alignment, and KL regularization terms: \[\mathcal{L} = \mathcal{L}_{\textrm{recon}} + \lambda_1 \mathcal{L}_{\textrm{percep}} + \lambda_2 \mathcal{L}_{\textrm{adv}} + \lambda_3 \mathcal{L}_{\textrm{align}} + \lambda_4 \mathcal{L}_{\textrm{KL}},\]

The alignment loss \(\mathcal{L}_{\mathrm{align}}\) further decomposes into three components, corresponding to the global, local, and cross-dimensional (2D to 1D) alignment objectives: \[\mathcal{L}_{\textrm{align}} = \mu_1 \mathcal{L}_{\mathrm{repa}}^{\mathrm{global}} + \mu_2 \mathcal{L}_{\mathrm{repa}}^{\mathrm{local}} + \mu_3 \mathcal{L}_{\mathrm{OT}} \label{eq:align95loss}\tag{2}\]

The global alignment loss encourages global tokens to align with the features extracted by a pretrained global semantic encoder (e.g., DINOv2 [@oquab2023dinov2]). This loss is computed as a patch-wise similarity measure: \[\mathcal{L}_{\mathrm{repa}}^{\mathrm{global}} = \text{PatchwiseSim}( \mathbf{z}_G, f_{\mathrm{DINOv2}}(I)) . \label{eq:repa95global}\tag{3}\]

Similarly, the local alignment loss matches each local 1D token to its corresponding localized representation from pretrained encoders (e.g., SigLIP2 [@tschannen2025siglip] or I-JEPA [@assran2023self]): \[\mathcal{L}_{\mathrm{repa}}^{\mathrm{local}} = \text{PatchwiseSim}(\mathbf{z}_{L1} \text{ or } \mathbf{z}_{L2}, f_{\mathrm{local}}(I)). \label{eq:repa95local}\tag{4}\]

Finally, \(\mathcal{L}_{\mathrm{OT}}\) denotes the optimal transport loss that aligns the 2D VAE latent representations of images with their corresponding 1D token sequences, as defined in 10 . This design effectively disentangles global and local information across token groups, leading to more compact and efficient representations, particularly for global tokens.

3.2 Self-supervised Bootstrapped Learning↩︎

After obtaining the global tokens \(\mathbf{z}_G\) from 1 , we employ a self-bootstrapped paradigm to predict local tokens from these global representations. Specifically, to predict local 1D tokens \(\mathbf{z}_{L1}\), we use an MLP module to project \(\mathbf{z}_G\) into a higher-dimensional token space:

\[\mathbf{z}_{L1} = \mathrm{MLP}(\mathbf{z}_{G}), \label{eq:l195prediction}\tag{5}\]

To predict local 2D tokens, we employ several scalable causal transformer blocks that autoregressively map \(\mathbf{z}_G\) to another sequence of tokens \(\mathbf{z}_{L2}\): \[p_{\theta}(\mathbf{z}_{L2} \mid \mathbf{z}_{G}) = \prod_{i=1}^{N_l} p_{\theta}\big( \mathbf{t}^{\mathrm{local}}_{i} \mid \mathbf{z}_{G}, \mathbf{t}^{\mathrm{local}}_{<i} \big), \label{eq:bootstrap95prob}\tag{6}\]

We then align this token sequence with the corresponding sequence transported from the 2D VAE latents, as described in 10 . During the generation process, the generator produces only the global tokens \(\mathbf{z}_G\). The local tokens \(\mathbf{z}_{L1}\) and \(\mathbf{z}_{L2}\) are then automatically predicted through the local 1D and 2D aligners. Finally, after the soft quantization of \(\mathbf{z}_G\), \(\mathbf{z}_{L1}\), and \(\mathbf{z}_{L2}\), all tokens are fused and decoded together: \[\hat{\mathbf{x}} = \mathrm{Dec}\big(\mathrm{Fuse}(\mathbf{z}_G, \mathbf{z}_{L1}, \mathbf{z}_{L2});\mathbf{z}_{L1}; \mathbf{z}_{L2}\big), \label{eq:generation95pipeline}\tag{7}\]

The fused tokens, along with \(\mathbf{z}_{L1}\) and \(\mathbf{z}_{L2}\), are then injected into the cross-attention layers of the decoder. We explore two fusion strategies for decoding: soft residual fusion and concatenation fusion. Further ablation studies comparing these strategies are provided in the experimental section.

\[\small \mathrm{Fuse} = \begin{cases} \alpha\,\mathbf{z}_G + \beta\,\mathbf{z}_{L1} + (1{-}\alpha{-}\beta)\,\mathbf{z}_{L2}, & \!\!\text{(Soft residual)}\\[3pt] \mathrm{Concat}[\mathbf{z}_G;\mathbf{z}_{L1};\mathbf{z}_{L2}], & \!\!\text{(Concatenation)} \end{cases} \label{eq:fusion}\tag{8}\]

Within our self-bootstrapped paradigm, generating only high-level, compact global information is sufficient to achieve high-quality generation. This approach reduces computational costs and optimizes the latent space of global tokens.

3.3 2D to 1D alignment via optimal transport↩︎

To efficiently inject 2D priors into a 1D token sequence and achieve a compact 1D representation, we use a local 2D aligner to predict a 1D sequence and align the 2D VAE latent features with it. This formulation naturally fits into the optimal transport (OT) framework, which constructs mappings between tensors of different dimensions while encouraging compact representations, and can be efficiently solved via the Sinkhorn algorithm. Specifically, let \(\mathbf{X}\in\mathbb{R}^{B\times N_x\times d}\) be the matrix of 2D VAE latents (with \(N_x=H\! \times\! W\) spatial tokens), and \(\mathbf{z}_{L2}\in\mathbb{R}^{B \times N_z\times d}\) be the matrix of 1D tokens (with \(N_z\) latent tokens). Here, \(B\) denotes the batch size, and \(d\) refers to the token dimension. The goal is to find a transport plan \(\mathbf{P}\in\mathbb{R}_{+}^{N_x\times N_z}\) that couples the marginals \(\mathbf{a}\in\Delta^{N_x}\) and \(\mathbf{b}\in\Delta^{N_z}\) (where \(\Delta^N\) denotes the \(N\)-simplex).

Let the squared-Euclidean cost matrix be \[\mathbf{C}_{ij} = \lVert \mathbf{x}_i - \mathbf{z}_j \rVert_2^2, \quad \text{for } i \in [1, N_x], \;j \in [1, N_z]\] measuring pairwise distances between 2D latents \(\mathbf{X}\) and 1D tokens \(\mathbf{z}_{L2}\). We solve the entropy-regularized optimal transport problem \[\mathbf{P}^\ast = \arg\min_{\mathbf{P}\in\Pi(\mathbf{a},\mathbf{b})} \;\langle \mathbf{P}, \mathbf{C}\rangle \;+\; \varepsilon\,\mathcal{H}(\mathbf{P}), \label{eq:ot95objective}\tag{9}\] where \(\Pi(\mathbf{a}, \mathbf{b})\) is the set of transport plans coupling \(\mathbf{a}\) and \(\mathbf{b}\), \(\langle \mathbf{.}, \mathbf{.} \rangle\) represents the Frobenius inner product and \(\mathcal{H}(\mathbf{P})\) is the entropy regularizer. The optimal transport plan \(\mathbf{P}^\ast\) is computed using the Sinkhorn algorithm: \[\mathbf{P}^\ast = \mathrm{Sinkhorn}(\mathbf{K}, \mathbf{a}, \mathbf{b}),\] where \(\mathbf{K} = \exp(-\mathbf{C}/\varepsilon)\). And we use the transport cost as part of the alignment loss shown in 2 : \[\mathcal{L}_{\mathrm{OT}} \;=\; \langle \mathbf{P}^\ast, \mathbf{C}\rangle. \label{eq:transport95loss}\tag{10}\]

5.5pt

3.4 Local Scaling with heavier local aligner↩︎

To explore the potential of the self-bootstrapped paradigm, we conduct experiments using local aligners at different spatial scales. Specifically, we vary the size of the local 2D module while keeping all other components fixed. When the local 2D aligner becomes large, the overall model becomes computationally heavy and difficult to optimize. To address this issue, we adopt a two-stage training strategy. In the first stage, we train the entire pipeline with a smaller local 2D aligner. In the second stage, we scale up the local 2D aligner and fine-tune it while freezing the remaining components. This strategy allows simultaneous training of the generator and the enlarged local aligner, enabling the generator to be trained once for global tokens and subsequently reused across different local aligner scales. Further discussions on the scaling results are provided in Section 4.2 and [fig:architecture11].

4 Experiments↩︎

4.1 Experiments Setup↩︎

Implementation Details of SelfBootTok. In this section, we introduce the experiment settings of our method. Our tokenizer utilizes Dinov2 VIT [@oquab2023dinov2] as the backbone and the encoder is initialized the same as pretrained Dinov2 encoder. All the experiments are conducted on the base size of Dinov2 VIT backbone with a total of 173M parameters, while the sizes of local aligners vary during the scaling experiments, including sizes of 10M, 100M, 200M, 400M and 600M. Most of our experiments are conducted with the number of latent code N = 64 and token dimension 32. The tokenizer is trained on ImageNet at a resolution of \(256\times256\) for 50 epochs with a small local aligner, and up to 75 epochs with larger local aligners in the scaling experiments. For discriminator training, we adopt a setup similar to SoftVQ [@chen2025softvq], using a StyleGAN-like architecture [@karras2019style; @karras2020analyzing] and strategies such as LeCAM [@tseng2021regularizing] and consistency regularization [@zhang2019consistency].

Implementation Details of Generators. We use SiT as the generator for downstream denoising-based image generation tasks. In our setup, where only global tokens need to be generated, we train our tokenizer for 50 epochs and SiT for 800K iterations. For subsequent scaling, we expand the local 2D aligner and train the tokenizer pipeline for an additional 50 to 75 epochs, keeping the ViT encoder frozen. Notably, SiT requires no further training. To accelerate SiT’s training, we apply the representation disentangling method [@wu2025representation] and incorporate a pretrained DINOv2-base encoder [@oquab2023dinov2] to enhance convergence speed.

Evaluation. We adopt reconstruction Frechet Inception Distance (rFID) [@heusel2017gans] on ImageNet validation set to assess the performance of tokenizer. Furthermore, we include PSNR and SSIM metrics to more accurately reflect the reconstruction quality. To evaluate the generation quality, we report generation FID (gFID) under either with or without CFG setting. Additionally, we evaluate the efficiency of generative models with respect to tokenizer GFLOPs.

4.2 Quantitative Results↩︎

We analyze the reconstruction and generation results, along with the training efficiency in this section.

High reconstruction quality with only 64 tokens. As shown in 1, our method achieves superior reconstruction performance (i.e., an rFID score of 0.66) using only 64 tokens, outperforming other baselines trained with the same token budget [@chen2025softvq; @yu2024image] and performing comparably to methods [@esser2021taming; @rombach2022high; @xiong2025gigatok] that rely on 256 tokens. Moreover, after efficient scaling, our method achieves better PSNR and SSIM scores for reconstruction, as illustrated in [fig:architecture11]. This demonstrates the potential for progressively improved reconstruction performance as the local aligner is scaled up within our architecture.

State-of-the-art generation performance among 1D tokenizers. In terms of generation, our approach achieves a state-of-the-art gFID score of 1.56 using only 64 tokens, as reported in 1. This result is comparable to the best-performing 2D tokenizer VA-VAE (i.e., 1.35 gFID with 256 tokens) and the top autoregressive tokenizer MAR-H (i.e., 1.55 gFID with 256 tokens). Moreover, our model attains a lower gFID even without classifier-free guidance (CFG), outperforming other 1D and autoregressive baselines under the same setting. This shows that incorporating multi-granularity information into our model architecture enhances generation quality, leading to superior performance and validating the effectiveness of our design.

Efficient training. Our method also exhibits strong training efficiency, achieving competitive performance without additional generation optimization overhead. As illustrated in 4, once the tokenizer pipeline is trained, our framework supports parallel optimization of the scaled local aligners and the generator. Unlike conventional approaches that require retraining the generator each time the tokenizer is scaled, our design leverages global tokens for generation, allowing the generator to be trained only once. For the overall scaling experiment, this strategy reduces training computation by about 40% (from 4.9B parameters and 60K GFLOPs to 2.95B and 36.6K GFLOPs) and shortens training time by roughly 54% (from 28 to 13 days). Moreover, our method scales efficiently to larger datasets, yielding greater computational savings as the generator size increases.

4.3 Qualitative Results↩︎

As shown in 2, our method generates diverse, high-quality real-world images using only 64 tokens, covering animals (e.g., dogs, tigers), food (e.g., noodles, burgers), and plants (e.g., flowers) sampled from 1000 ImageNet [@deng2009imagenet] classes.

4.4 Ablation Study↩︎

To justify our design choices and validate the self-bootstrapping paradigm, we conduct comprehensive ablation studies on local 1D/2D branches, 1D aligners, and fusion strategies. As shown in [tab:ablation1111], our design outperforms all variants: we use an MLP for the efficient 1D local aligner (not scaled, captures coarse local info) and a transformer-based 2D local aligner (scaled, models rich spatial details).  5 illustrates that progressive integration of local structural info enhances reconstruction quality, with optimal performance when both local branches are included.  [tab:local1d95ablation] investigates local 1D token alignment with frozen visual encoders (I-JEPA, DINOv2, SigLIP): I-JEPA performs best, with DINOv2 and SigLIP comparable, indicating distinct local structural cues affect tokenizer behavior. We compare fusion strategies under Eq. 8 : concatenation-based fusion improves reconstruction but increases decoder size/computational cost, degrading generation quality and revealing a task-dependent trade-off. Finally, [fig:architecture11] shows our method improves with local aligner scaling: at 600M parameters and 64 tokens, it achieves a gFID of 1.56, outperforming baselines and demonstrating architectural scalability.

5 Conclusion↩︎

In this paper, we propose SelfBootTok, an efficient 1D image tokenizer that decomposes image tokens into global and local groups and leverages a self-supervised bootstrapped paradigm to recover sufficient local information from global tokens. Our architecture reduces image token overlap, achieving higher reconstruction quality and better efficiency in downstream generation tasks, while its local self-bootstrapped prediction design enables strong scalability and highlights self-bootstrapping’s potential for image tokenization.