1 Introduction↩︎

Temporal-logic video question answering (VideoQA) asks models to reason about when events occur—before, after, overlap, persistence, and multi-event chains—rather than only what appears in a video. TimeLogic [1] makes this capability explicit through natural-language questions grounded in formal temporal operators. However, standard VLM inference is poorly matched to this setting. Most systems uniformly sample a fixed frame set and answer in one pass, but the decisive evidence for a temporal relation is often localized to a short window, such as the boundary between adjacent actions. Once that evidence is missed, the model has no mechanism to recover it or verify a tentative event ordering. This failure becomes more pronounced for multi-event chains, where the answer depends on localizing several actions on a shared timeline rather than recognizing a single visual state. The central challenge is thus to allocate a limited visual budget to the moments most likely to decide the temporal relation, especially when relevant events are sparse or visually subtle.

We therefore frame TimeLogic inference as an active evidence-seeking problem, in line with recent tool-guided video-agent systems [2]. Our system uses a ReAct-style [3] Think–Act–Observe loop with multi-granular, timestamp-interleaved sampling tools. Instead of committing after one global sample, the agent repeatedly decides where to inspect next, accumulating timestamped observations and reasoning over the full history. The available tools cover coarse overview sampling, structured temporal scans, segment-level skimming, and dense focus around a candidate boundary. Because each returned frame is paired with an absolute timestamp, temporal relations can be checked as comparisons on a common time axis.

The policy is guided by the structure of the benchmark. The released split contains heterogeneous source corpora and many compound three-event categories. We therefore classify each question into a temporal category and adapt the initial tool, iteration depth, frame budget, and reasoning prompt accordingly. Long, action-dense clips receive deeper exploration, while shorter clips can often be handled with denser one-pass coverage.

These choices keep the system simple while encouraging targeted verification before answering. Our contributions are:

• We formulate TimeLogic inference as Think–Act–Observe evidence seeking with multi-granular, timestamp-interleaved sampling.

• We design a training-free, question-type-aware policy that adapts tools, prompts, and budgets to benchmark structure.

• With Gemini 3.1 Pro [4], the final system achieves 77.13 AvgAcc on the official test set.

2 Benchmark Analysis↩︎

We analyze the released TimeLogic benchmark structure [1], focusing on source-corpus characteristics and question templates, to identify where temporal-logic VideoQA is hardest and to motivate the design choices in Section 3. The split contains \(3{,}000\) questions over \(1{,}850\) unique videos, drawn from four source corpora and posed in two answer formats: multiple-choice (MC, \(1{,}878\) questions) and Boolean (bool, \(1{,}122\) questions). Our analysis surfaces two structural properties—heterogeneous corpora and a long tail of compound temporal categories—each of which directly shapes a component of the method.

2.1 Corpus Composition↩︎

The questions are sourced from four datasets with markedly different visual and audio characteristics (Table 1). Each video’s source is recoverable from its identifier prefix. The corpora span a wide range of clip length and action density. Among them, only CrossTask provides effective instructional narration; STAR, AGQA, and Breakfast are treated as having no reliable audio signal.

This heterogeneity is the primary motivation for the corpus-adaptive budgeting in Section 3: a single global frame/step budget is mismatched to inputs that range from narrated CrossTask clips to silent, action-dense recordings. The kitchen-webcam corpus in particular pairs high action density with no effective audio, so it benefits from a larger frame budget, finer temporal segmentation, and visual-only reasoning.

2.2 Temporal-Category Distribution↩︎

We infer each question’s temporal-logic category from its surface template, mapping \(2{,}821\) of the \(3{,}000\) questions onto the benchmark’s operator set; the remaining \(179\) are spatial/object-relation MC questions that do not correspond to a temporal operator. Table 2 shows the resulting distribution.

The distribution is heavily long-tailed toward compound, three-event categories: Loose A,B,C and A always before B,C alone account for over a third of the split, and together with Strict A,B,C they are strongly concentrated in MC format (\(849\) MC questions). These categories require reasoning over a full event chain \(A \!\to\! B \!\to\! C\) rather than a single pairwise relation, and are the hardest to answer without an explicit timeline. This observation motivates two method components: the timeline-first protocol for chain categories and the larger iteration/thinking budgets allocated to them.

2.3 Summary of Design Implications↩︎

The analysis yields a direct mapping from dataset structure to method: (i) corpus heterogeneity \(\Rightarrow\) corpus-adaptive sampling budgets and audio gating; and (ii) the long tail of compound chain categories \(\Rightarrow\) timeline-first reasoning with deeper iteration. Section 3 develops each of these components.

3 Method↩︎

We cast temporal-logic VideoQA as an evidence-seeking problem rather than a fixed-budget classification problem, similar in spirit to long-horizon video agents that seek evidence through tools [2]. Instead of feeding a vision–language model (VLM) a single uniformly sampled set of frames, our agent actively decides where in the video to look, building an explicit action timeline before committing to an answer. The agent follows a Think–Act–Observe loop, following the general ReAct-style pattern of interleaving reasoning with actions [2], [3], equipped with a multi-granular sampling toolkit and a question-type-aware exploration policy. The design is backbone-agnostic: the loop drives any frame-conditioned VLM through a uniform interface, so the same policy applies regardless of the specific model used.

3.1 Overview↩︎

Given a video \(V\), a question \(q\), and (for multiple-choice questions) a set of options \(\mathcal{C}\), the agent produces an answer \(\hat{a}\) through an iterative loop (Figure 1):

1. Classify. \(q\) is mapped to a temporal-logic category, which selects a tailored exploration strategy.

2. Explore. An initial sampling tool produces a first set of timestamped frames.

3. Think–Act–Observe. The VLM reasons over the accumulated observations and either (i) requests a more targeted sampling action, or (ii) commits to a final answer.

4. Finalize. Once confident—or when the step/frame budget is exhausted—the VLM emits a parsed answer.

A state object accumulates the timestamped observations and the per-step reasoning traces, which are re-injected into every subsequent prompt so that the VLM reasons over the full exploration history rather than the latest frames alone.

3.2 Multi-Granular Sampling Toolkit↩︎

Temporal-logic questions vary widely in the temporal precision they demand: an eventually question only needs to confirm an action occurs somewhere, whereas an immediately-after question hinges on the exact boundary between two adjacent actions. To support both, we expose four tools spanning coarse-to-fine temporal granularity (Table 3, Figure 2).

All tools return frames interleaved with their wall-clock timestamps (in seconds). This frame–timestamp interleaving is central to the method: it grounds every observation on an absolute time axis, allowing the VLM to express and verify temporal relations (before/after/overlap) as numerical comparisons rather than relying on implicit frame ordering.

The temporal_scan tool is the workhorse for timeline construction. By dividing \(V\) into \(n\) equal segments and sampling \(k\) frames per segment, it provides denser, structurally labeled coverage than a flat uniform sample, so the VLM can localize each candidate action to a segment before zooming in with skim/focus.

3.3 Question-Type-Aware Exploration↩︎

A lightweight rule-based classifier maps \(q\) to one of the benchmark’s temporal categories (e.g.before, after, immediately-after, until, since, overlap, not-overlap, imply, three-event chains, etc.) via ordered keyword/structure patterns, falling back to a generic strategy when no pattern matches. Each category is associated with an exploration strategy that specializes the loop along several axes:

• Initial tool (temporal_scan vs.overview) and initial frame budget.

• Preferred follow-up tool (focus for boundary-sensitive types such as immediately-after; skim for ordering types such as before/after and chains).

• Iteration depth: the maximum number of steps and whether iteration is needed at all (simple existence checks such as eventually/throughout may answer in a single pass).

• A type-specific reasoning prompt that encodes the formal semantics of the operator (e.g.”A before B means \(A\)’s last occurrence precedes \(B\)’s first occurrence”) together with common pitfalls (e.g.confusing a continuous state such as holding with a point action such as picking up).

• A thinking-token budget scaled to category difficulty, larger for multi-event chains and smaller for simple existence checks.

For the hardest three-event chain categories, the prompt additionally enforces a timeline-first protocol: the VLM must emit an explicit Timeline: [(action,\(t_s\),\(t_e\)), …] before it is permitted to select an answer.

3.4 The Think–Act–Observe Loop↩︎

After the initial exploration, the agent alternates between observation and action. At each step the VLM receives the new timestamped frames, the toolkit description, all prior reasoning, and the type-specific hint, then responds with either a tool call (parsed from a JSON block or an ACTION: tool(args) form) or a final answer (ANSWER:\(\langle\)letter/Yes/No\(\rangle\)).

3.4.0.1 Minimum-step gating.

A recurring failure mode of single-pass prompting is premature commitment: the VLM writes an answer on the first step without ever verifying its tentative timeline. For ordering and chain categories on long videos, we therefore impose a minimum number of exploration steps: any early ANSWER is overridden and replaced by a forced follow-up tool call until the minimum is met. Short videos bypass this gate, since dense sampling already covers them completely in one pass (below).

3.4.0.2 Adaptive frame and step budgets.

Budgets are adapted to the input rather than fixed globally. Short videos (below a per-category duration threshold) are handled by dense uniform sampling of the full clip in a single step, which is both cheaper and more reliable than iterative seeking when the entire video fits in the budget. Longer videos use the iterative loop with temporal_scan initialization. We further apply corpus-adaptive budgeting: clips drawn from sources with denser, more repetitive low-level actions and no effective audio receive a higher minimum frame budget, finer temporal_scan segmentation, and a higher minimum step count, because fine action-boundary discrimination is the bottleneck there; CrossTask clips can additionally use their instructional narration.

3.5 Auxiliary Speech Context↩︎

For CrossTask videos, we attach an ASR transcript as additional temporal evidence: timestamped transcript segments are prepended to the question so the VLM can align spoken narration with visual events. For STAR, AGQA, and Breakfast, the transcript is omitted because these corpora do not provide effective audio for our setting.

4 Experiments↩︎

This section summarizes the experimental setup and official final test-set result. The system is used strictly as an inference-time agent: no task-specific training, fine-tuning, or validation-set model selection is performed. We therefore report only the final official test performance rather than training curves or ablation studies.

4.1 Experimental Setup↩︎

We use Gemini 3.1 Pro [4] as the VLM backbone for all submitted predictions. The model is queried through the evidence-seeking agent described in Section 3: each question is first mapped to a temporal category, the corresponding sampling policy selects timestamped video frames, and the accumulated observations are passed to Gemini 3.1 Pro for reasoning and answer generation. All task adaptation is performed at inference time through the question classifier, prompt templates, sampling schedules, and tool-calling loop; no model weights are updated.

The inference configuration fixes the backbone model, maximum frames per question, maximum agent steps, category-specific minimum steps, corpus-specific budget multipliers, ASR usage, and random seed. For each example, the system records the predicted temporal category, sampled timestamps, tool calls, intermediate reasoning summaries, raw model output, and parsed final answer. The leaderboard submission contains the compact answer file parsed from the model outputs.

4.2 Evaluation Protocol↩︎

We evaluate on the official TimeLogic test split using the challenge’s AvgAcc metric. For multiple-choice questions, the parser extracts a single option letter from the model output; for Boolean questions, it extracts Yes or No. Invalid or unparsable outputs are counted as incorrect. All reported numbers correspond to the final submitted inference run with fixed sampling parameters and random seed. Our final submission obtains an official test-set AvgAcc of 77.13. This is the only quantitative result reported in this challenge submission.

5 Conclusion↩︎

We presented an evidence-seeking agent for the TimeLogic Challenge that converts temporal-logic VideoQA from a single-pass frame classification problem into an iterative search for timestamped evidence. The method combines benchmark-driven question categorization, multi-granular sampling, explicit timestamp interleaving, category-specific reasoning prompts, and corpus-adaptive budgets. This design requires no task-specific training and can be paired with different frame-conditioned VLM backbones.

Our analysis suggests that temporal grounding is the key bottleneck: strong VLMs can often recognize the relevant actions once they are shown the right frames, but they need an explicit mechanism for finding, ordering, and verifying those frames. The final system applies this mechanism with Gemini 3.1 Pro as the VLM backbone and achieves an official test-set AvgAcc of 77.13.

References↩︎