arXiv 每日精选 · 具身智能 / 计算机视觉 / 3D 重建

批次:arXiv 公示批次:Thursday, 30 July 2026(当日新批次未发布,沿用该批并剔除昨日已推送 10 篇,刷新 Top 10)

范围:覆盖 cs.CV / cs.RO / cs.AI / cs.GR / cs.LG / cs.MA 六类新提交:解析约 358 篇,主题匹配约 193 篇,剔除昨日已交付 10 篇后质量排序取 Top 10

说明:排序为三主题混合质量排序(venue 档次 + 作者/团队 + 实证强度 + 方法新颖性 + 代码/基准开源),无领域配额。本期构成:具身智能 6 / 计算机视觉 8 / 3D 重建 2(含交叠),由批次内容决定,非排序偏置。
#1
预印本 (Preprint) Weili Zeng, Yitong Xing, Fulong Liu, Chengqun Yang, Antao Xiang, Feng Tian, Jingnan Gao, Jisong Cai 等
主题

把世界生成模型的前向『计算过程』内化为可从当前视觉上下文与语言指令直接预测的表征,用于高效具身控制。

主要创新点
  • 提出 Enfold:将世界生成器在构造未来时暴露的多层级中间状态作为监督,训练仅依赖当前观测与指令的编码器,把『生成式计算』折叠进当前表征
  • 学到的表征反馈去条件化未来生成、并被任务头读取,但任务梯度不被允许重塑编码器(表示-任务解耦)
  • 部署时动作预测不再执行生成器:LIBERO / RoboTwin2.0 / 真实机器人上动作时延相对 Fast-WAM 降低 3.7×,Enfold-Flash 达 10.1×
  • 表征分析显示其抑制无关变化、优先捕捉长程涌现的变化;人为干预当前场景时,生成续写与执行动作均自适应(区别于固定轨迹回放)
主要内容

把世界生成器重新诠释为『预测式控制表征』的来源——未来的内部计算结构可被折叠进当下,无需在每一步物化未来,为高效 VLA / 世界模型部署提供新视角。

摘要

World generative models are typically used through what they produce: a rendered future, a video-conditioned action, or latent context computed by a costly generative branch. We argue that their more reusable asset is the computation that constructs a future. As a generator transforms a corrupted future into a coherent trajectory, its intermediate states organize appearance, spatial layout, and interaction across levels of abstraction. Can this future-generative computation be internalized in a representation inferred from the present alone? We present Enfold, which transfers this computation into a representation predicted from the current visual context and language instruction. During training, multi-level states exposed as the generator processes the observed future supervise a current-only encoder. The learned representation is fed back to condition future generation and is read by task heads without allowing task gradients to reshape the encoder. At deployment, action prediction no longer executes the generator. Across LIBERO, RoboTwin2.0, and real-robot tasks, Enfold supports strong control while reducing action latency by 3.7times relative to Fast--WAM, Enfold-Flash reaches 10.1times. Representation analyses show that it suppresses nuisance variation and preferentially captures changes that emerge over longer horizons. When the current scene is altered by human intervention, both the generated continuation and the executed actions adapt, which is inconsistent with fixed trajectory replay. These results recast a world generator as a source of predictive control representations: its future need not be materialized at every step if its internal structure can be enfolded into the present.

#2
预印本(代码开源) Wenze Liu, Xintao Wang, Pengfei Wan, Xiangyu Yue
主题

用神经网络把数据矩(moment)学习为分布级训练信号,提出可扩展的条件矩匹配训练目标,改良一步/少步生成。

主要创新点
  • 通过多项式投影将扩散去噪器统一到矩保持框架,证明 n 次投影显式识别至 n+1 阶数据矩
  • 由可解的仿射情形导出 Amortized Fréchet Distance (AMFD) 损失:以交替、无矩阵(matrix-free)优化流水线动态学习条件矩,轻松扩展到高维
  • 在全局表征特征上作为后训练目标,神经形式比精确统计匹配更稳,FDr⁶ 指标大幅超 FD 基线,ImageNet 一步生成更强
  • 扩展到文生图时,条件感知 AMFD 显著提升指令跟随:一步模型在 GenEval 上超多步 FLUX.2 4B teacher,PickScore 持平
主要内容

揭示前两阶矩仅在强语义空间中可定位目标分布,开启在原生生成空间内的直接探索;代码与权重已开源。

摘要

We propose amortized moment matching, utilizing neural networks to learn data moments as distributional training signals. By casting diffusion denoisers through polynomial projections, we establish a general framework for moment amortization, revealing that an n-th degree projection explicitly identifies data moments up to order n+1. Derived from the tractable affine case, we instantiate the Amortized Fréchet Distance (AMFD) loss. Unlike FD-loss which relies on explicit marginal moment calculations, AMFD is able to dynamically learn conditional moments via an alternating, matrix-free optimization pipeline that effortlessly scales to high-dimensional data. When operating on global representation features, AMFD serves as a powerful post-training objective; empirically, its neural formulation yields more robust training dynamics than exact statistical matching, substantially surpassing the FD baseline on the FDr^6 metric and achieving superior one-step generation on ImageNet. Furthermore, it unlocks direct exploration within native generative spaces, suggesting that the first two moments can identify target distributions only in spaces with strong semantics. Finally, when scaled to text-to-image generation, the condition-aware nature of AMFD unlocks massive gains in instruction-following capabilities, enabling our one-step models to outperform their multi-step FLUX.2 [klein] 4B teachers on the GenEval benchmark while achieving on-par performance on PickScore. Code and checkpoints are available at 见项目页.

#3
预印本(世界模型数据集) Yiming Cai, Fangjie Yu, Meiqing Yu, Ziyue Shi, Pengfei Yuan, Yong Guo
主题

源自工业 CG 生产管线的世界态(world-state)大规模数据集与协议,显式记录状态/动作/事件/观测的联合动力学。

主要创新点
  • 显式记录中间态:多模态语义、空间结构、骨骼与控制状态、运动曲线、相机/光照参数、物理缓存、接触事件、多通道渲染
  • v1 约 85 万段 1–5 秒时间对齐片段;将潜态、观测、关系、事件、分支元数据分离并组织为统一时空样本
  • 定义分支谱系(branch lineage)覆盖事实轨迹、观测干预、动作干预、机制干预与严格反事实分支,显式记录干预目标、不变量与替代结局
  • 在几何条件视频生成、动作预测、闭环 VLA 策略迁移上评测,提供可复用的结构化监督
主要内容

面向世界模型 / Physical AI / 具身智能的共享数据基础设施,计划持续扩充并社区共建。

摘要

World models must learn the joint dynamics of states, actions, events, and observations, yet existing video, robotics, and simulation datasets usually capture only part of this structure. We introduce CG-World, a large-scale world-state dataset and protocol derived from industrial computer graphics production pipelines. CG-World explicitly records intermediate states, including multimodal semantics, spatial structure, skeletal and controller states, motion curves, camera and lighting parameters, physics caches, contact events, and multi-pass renderings. CG-World v1 contains approximately 850,000 temporally aligned segments of 1-5 seconds. It separates latent states, observations, relations, events, and branch metadata, and organizes them into unified spatiotemporal samples. To support intervention learning and counterfactual reasoning, CG-World defines a branch lineage covering factual trajectories, observation interventions, action interventions, mechanism interventions, and strict counterfactual branches, with intervention targets, invariants, and alternative outcomes explicitly recorded. We evaluate the dataset on geometry-conditioned video generation, action prediction, and closed-loop vision-language-action policy transfer. Results show that CG-World provides reusable structured supervision for controlled generation, action modeling, and embodied policy transfer. We plan to expand CG-World through continued data collection and community collaboration toward a shared data infrastructure for world models, Physical AI, and embodied intelligence.

#4
预印本(代码开源) Derek Ming Siang Tan, Shailesh Shailesh, Srikrishna Iyer, William Wei Jie Teo, Yuanliang Ju, Qiao Gu, Guillaume Sartoretti
主题

推理期自适应 VLA 转向框架:在 VLA 潜变量上做强化学习,按需组合 steering 流,缓解相关失败模式并仅在失败预测时激活。

主要创新点
  • 训练轻量离线 RL 策略,条件化为从 VLA action expert 抽取的表达性潜变量,推理时将其流速度与冻结 VLA 组合(组合式 steering)
  • 发现推理期转向在成功/失败态遵循根本不同的 scaling law:动作多样性在基 VLA 可能失败时最有益,成功时反而扰动已准动作
  • 据此 RL² 仅在预测失败时激活组合式 steering,既引入超越主导演示模式的动作多样性,又避免无效扰动
  • SIMPLER / PolaRiS 上域外成功率最高 +17.3%,真实机器人实验验证可迁移;模块化、即插即用
主要内容

把大规模模仿学习的 behavioral prior 与离线 RL 带来的动作多样性结合,提供实用的 VLA 部署级转向框架。

摘要

Despite the impressive visuomotor capabilities enabled by Vision-Language-Action (VLA) models, their performance often degrades on challenging and out-of-domain tasks. Recent test-time steering and scaling methods improve performance without extensive data collection and retraining, but action samples often remain concentrated around similar behaviors and therefore inherit correlated failure modes. Moreover, existing methods apply the same intervention strategy at every timestep, regardless of whether the base policy is already likely to succeed. To address these limitations, we introduce RL^2, an adaptive inference-time steering framework that leverages Reinforcement Learning on VLA Latents. First, we train a lightweight offline RL policy conditioned on expressive latents extracted from the VLA action expert and compose its flow velocity with that of the frozen VLA during inference. This compositional steering strategy combines the behavioral priors of large-scale imitation learning with the action diversity induced by offline RL beyond dominant demonstration modes. We further discover that inference-time steering follows fundamentally different scaling laws under success and failure states, revealing that action diversity is most beneficial when the base VLA is likely to fail, but can unnecessarily perturb already-accurate actions when success is likely. Building on this insight, RL^2 activates compositional steering only when failure is predicted. Across the SIMPLER and PolaRiS benchmarks, RL^2 improves success rates by up to +17.3% in out-of-domain settings, while ablations and scaling studies demonstrate the importance of latent representations and RL training. Finally, real-world experiments demonstrate that these gains transfer beyond simulation, establishing RL^2 as a practical and modular steering framework for VLA deployment.

#5
预印本(72 页长文) Davor Dobrota, Vsevolod Skorokhodov, Chenghao Xu, Olga Fink, Malcolm Mielle
主题

从稀疏姿态 RGB 图像直接重建多片 B 样条边界表示(B-rep),产出紧凑、光滑、水密的几何,原生兼容 CAD 与仿真。

主要创新点
  • 提出 FORGE-SIM:直接优化样条表示本身,无需人工介入即得可被数值模拟使用的几何
  • 把观测导出的场(如热态、语义信息)投影到同一 spline 基上,注入后立即可用于仿真
  • 在单一优化框架内统一基于图像的重建与『仿真就绪』建模
  • 演示所得模型质量足以支撑热仿真与模态分析(modal analysis)
主要内容

移除 CV 与数值分析之间的长期壁垒,面向仿真驱动设计、检测与数字孪生新工作流。

摘要

Image-based reconstruction aims to recover three-dimensional geometry from images. Recent advances have enabled the recovery of visually detailed models, yet their representations are not well-suited for numerical simulation. Simulation frameworks typically require explicit, watertight, and smooth geometries to ensure numerical robustness and accuracy, properties that surfaces extracted from image-based reconstructions lack. We propose FORGE-SIM, a method to directly reconstruct a multi-patch B-spline boundary representation from sparse posed RGB images without manual intervention. By optimizing the spline representation itself, our approach produces compact, smooth, and watertight geometries that are natively compatible with both Computer Aided Design and simulation workflows. Additionally, we introduce a strategy to project observation-derived fields, such as a thermal state and semantic information, onto the reconstructed models in the same spline basis, enabling immediate use in simulation. We demonstrate that the obtained models are of sufficiently high quality to enable thermal simulation and modal analysis. By unifying image-based reconstruction and simulation-ready modeling within a single optimization framework, this work removes a long-standing barrier between computer vision and numerical analysis. We anticipate that it will enable new workflows for simulation-driven design, inspection, and digital twin applications.

#6
预印本 (Preprint) Jingxuan Su, Shenglin Wang, Tiesong Zhao, Ge Li, Wei Gao
主题

面向 3D Gaussian Splatting 场景质量评估的多模态框架,结合 3D 感知质量表征学习与接地的多模态推理。

主要创新点
  • 3D 感知质量表征学习:在 VGGT 编码器上增配专用质量头,多视图图像编码为视图特征并聚合以捕捉跨视图一致性
  • 通过联合建模深度与点云相关结构信息纳入几何线索,学到超越外观驱动的结构感知质量表征
  • 接地的多模态推理:将原始图像、深度图、点云渲染与相机参数联合喂入基于 Qwen 的 MLLM
主要内容

突破 2D 感知线索与通用 MLLM 不稳定回归的局限,为 3DGS 新视图合成与场景重建提供可靠质量评估。

摘要

3D Gaussian Splatting (3DGS) has emerged as an effective representation for novel view synthesis and 3D scene reconstruction, creating an increasing demand for reliable quality assessment. Unlike conventional image quality assessment (IQA), the quality of a 3DGS scene depends not only on the perceptual fidelity of rendered views, but also on scene-level factors such as spatial structure and cross-view consistency. Existing IQA methods are limited by their reliance on 2D perceptual cues, whereas general multimodal large language models (MLLMs) are not designed for stable quality regression and may produce unreliable judgments. To address these limitations, a multimodal quality assessment framework is developed for 3DGS scene understanding. First, a 3D-aware quality representation learning framework is introduced by augmenting a VGGT-based encoder with a dedicated quality head. Multi-view images are encoded into view-specific features and aggregated to capture cross-view consistency, while geometric cues are incorporated through joint modeling of depth and point-cloud-related structural information, enabling the learning of structure-aware quality representations beyond appearance-driven features. Second, a grounded multimodal reasoning mechanism is constructed by jointly feeding original images, depth maps, point cloud renderings, and camera parameters into a Qwen-based MLLM.

#7
预印本(代码开源) Jiatong Li, Leo Liang, Linghe Kong, Yulun Zhang
主题

免训练的长视频生成框架,从频域视角诊断自回归 rollout 的误差累积,并提出谱自锚定(Spectral Self-Anchoring)稳定长程视觉。

主要创新点
  • 频域诊断:误差累积表现为低频带能量漂移;分析 attention sink 在频域的作用,发现其部分缓解但无法根除
  • 提出 FreqForcing 与 Spectral Self-Anchoring (SSA):用锚定注意力的低频分量维持长程视觉稳定,用局部注意力的高频分量保留动态运动
  • 将 Self-Forcing(预训练于 5s 片段)扩展到两分钟生成,实现 24× 外推
主要内容

定量与定性均超越现有免训练方法,且与代表性基于训练的方法相当;代码已开源。

摘要

Autoregressive video diffusion models enable real-time streaming video generation. However, errors introduced during self-rollout accumulate over long horizons, manifesting as color drift, motion stagnation, and eventual visual collapse. In this paper, we characterize this phenomenon from a frequency-domain perspective: error accumulation appears as a pronounced energy drift in the low-frequency bands. We further investigate the effectiveness of attention sink in the frequency domain, and find that it improves the video quality by alleviating the spectral energy drift to some extent, but cannot fully resolve it. Motivated by the above analysis, we propose FreqForcing, a training-free framework that addresses error accumulation in long-video generation via Spectral Self-Anchoring (SSA). The proposed SSA leverages the low-frequency components of anchor attention to maintain long-horizon visual stability, while preserving dynamic motion through the high-frequency components of local attention. Our FreqForcing extends Self-Forcing pretrained on 5s clips to two-minute generation, achieving 24x extrapolation. Extensive experiments show that FreqForcing outperforms existing training-free methods quantitatively and qualitatively while remaining competitive with representative training-based approaches.

#8
预印本 (Preprint) Gabe Everett, Brice Gunter, Ryan Vander Stelt, Cleiver Ruiz-Martinez, Blake Hull, Juan Rojas
主题

受并行对称性启发的轨迹级增强框架,超缩放群变换以大幅加速真实机器人(on-robot)学习。

主要创新点
  • 在对称性树(symmetry tree)下建模 MDP,状态-动作对具容许的并行不变变换,形成几何网格结构
  • 用单应(homography)对自我/外部视角图像做空间变换对齐,处理本体感知信息的特殊需求
  • 并行变换产出大量独特对称等价样本,以多样一致经验填满回放缓冲,加速学习
  • 真实机器人接触操作(插销、线缆布线、物体搬运)上,相对 SOTA 墙钟收敛加速 1.37–2.17×,成功率提升 1.09–1.27×,最快收敛 16.6 / 10.9 / 79.3 分钟
主要内容

表明简单分支对称性可因超缩放带来不成比例收益,推动机械臂/人形亚 10 分钟 on-robot 学习。

摘要

Deep reinforcement policy learning directly in physical robots (on-robot learning) remains bottlenecked by slow wall-clock training times. We present SymmGrid, a trajectory level augmentation framework inspired by parallelized symmetries that super-scales group transformations to significantly accelerate on-robot learning in both egocentric and exocentric visual setups. We model a Markov Decision Process (MDP) under a symmetry tree, in which state-action pairs have admissible parallelized invariant transformations that yield a geometric grid structure. The state is modelled with ego- or exocentric images and proprioception information. The latter require special treatment, in the form of homographies, to warp visual scenes in line with their corresponding spatial transformations. These parallelized transformations produce a large set of unique symmetric equivalences that populate the replay buffer with diverse and consistent experiences that speed up learning and improve performance. We present extensive training and evaluations performed directly on real robot manipulation contact tasks including peg-insertions, cable routing, and object relocations. Relative to SOTA, SymmGrid achieved wall-clock training convergence speed-ups of 1.37-2.17x, evaluation success rate improvements of 1.09x-1.27x, fastest training convergence times of 16.6, 10.9, and 79.3 minutes respectively. For trajectory wide assessments, we used normalized area under the curve (nAUC) ratios. SymmGrid achieved improvements of up to 2.59x. These results confirm that simple branch symmetries can have an outsized result due to super-scaling and bring us closer to sub-10 minute on-robot learning training in manipulation tasks suitable for arms and humanoids. The project page is available at 项目页

#9
预印本 (Preprint) Shiyong Meng, Bolei Chen, Ping Zhong, Yang Wan, Rongzhi Wang, Jiazhi Xia, Jianxin Wang
主题

免射线匹配的粗到细视觉楼层平面图定位(FLoc),从不确定性走向确定性,平衡全局多假设与局部亚米级精修。

主要创新点
  • 绕过中间射线匹配范式,提出粗到细框架:粗阶段用图像条件姿态扩散模型参数化连续多模态姿态分布,把随机初始化粒子导向不同候选模态
  • 精修阶段提出局部 refiner,从以候选为中心的平面图裁剪预测有界亚米级姿态残差,消解结构歧义
  • 无需任何离线地图预处理或测试时查找表
主要内容

S3D(full) 与 ZInD 上取得 SOTA 精度与鲁棒性;为室内定位提供全局多假设跟踪 + 局部亚米精修的统一方案。

摘要

Visual Floorplan Localization (FLoc) has emerged as a promising solution for indoor localization by matching egocentric images against minimalist structural maps. However, due to cross-modal information asymmetry and repetitive indoor layouts, visual FLoc is fundamentally challenged by multimodal pose distributions, where visually identical observations map to distinct, spatially separated locations. Existing ray-matching-based methods tackle this by explicitly predicting sparse geometric or semantic rays, which inherently incur information loss and demand resource-intensive preprocessing alongside exhaustive matching during inference. In this paper, we bypass the intermediate ray-matching paradigm and propose a coarse-to-fine visual FLoc framework that progresses from uncertainty to determinism. In the coarse stage, we design an image-conditioned pose diffusion model to parameterize the continuous multimodal pose distribution, effectively routing stochastically initialized pose particles toward distinct candidate modes. In the refinement stage, we propose a localized refiner that predicts bounded sub-meter pose residuals from candidate-centered floorplan crops, where structural ambiguities are largely eliminated. Our method effectively balances global multi-hypothesis tracking and local sub-meter refinement without requiring any offline map preprocessing or test-time lookup tables. Comprehensive results on the S3D (full) and ZInD benchmarks demonstrate that our approach achieves state-of-the-art accuracy and robustness.

#10
IROS 2026 Emma Huang, Zih-Yun Chiu, Neelay Joglekar, Shanglei Liu, Michael C. Yip
主题

用缝合线作为辅助工具的间接拾针框架,实现自主缝合中遮挡/难达针的稳健拾取。

主要创新点
  • 以缝线为辅助工具做间接拾针,避免不必要的工具-组织接触,针被遮挡或不可达时仍可拾取
  • 覆盖完整工作流:线与组织重建、安全抓取点选择、稳定提线、双手跟线直至稳固持针
  • 机器人策略显式计入视觉不确定性以最大化真实环境鲁棒性
主要内容

da Vinci Research Kit 上多真实条件评测:即便线构型复杂或针不可达仍稳健,弥合自主机器人策略在非结构化缝合环境的落地差距。

摘要

Suture-needle pickup is necessary for autonomous suturing, as a needle can be unexpectedly dropped or strategically released to adjust the grasping configuration. Current methods for autonomous needle pickup typically guide a robot to move straight toward the needle and grasp it, limited to conditions where the needle is observable and directly approachable. In addition, grasping the needle lying on tissue can lead to the robot pinching nearby tissue or the needle jumping around due to its slippery surface, posing potential safety issues. This work proposes an autonomous framework that uses a suture thread as an assistive tool for indirect needle pickup, avoiding unnecessary tool-tissue contact and enabling pickup even when the needle is occluded or inaccessible. The framework spans the entire workflow, including thread and tissue reconstruction, safe grasp-point selection, stable thread lifting, and bimanual thread-following until securing needle grasping. The robot policies account for visual uncertainty to maximize robustness in real-world environments. We evaluate the proposed framework on a da Vinci Research Kit under various real-world conditions. The results demonstrate robust performance even with a challenging thread configuration or a non-approachable needle, closing the gap in applying autonomous robot policies to unstructured suturing environments.

自动生成 · 数据来源 arXiv.org · 稳定访问 https://arxiv-daily-search.pages.dev