RIE
Code
Paper <cite>

ϕ-RIE: From Photorealistic Reconstruction to Interactive Environments

1 INSAIT, Sofia University “St. Kliment Ohridski”.

2 vivo Mobile Communication Co., Ltd.

3 Karlsruhe Institute of Technology

Contact: runyi.yang@insait.ai

∗ Corresponding author

Enter the playground PhiView code
THE FULL DEMO

From reconstruction to interaction.

Watch on YouTube ↗
REAL SCANS × OBJECT RECONSTRUCTION × INTERACTIONScroll to explore

01 — PHIVIEW DEMOS

See the interaction
from start to finish.

Nine demonstrations of shooting, grasping, placing, and navigating reconstructed rooms. Compare the same scenes with the harmonizer switched off and on.

PHIVIEW · SIX SHOTS · COLORFUL TRAILS

Spray bottle / shooting

01 / 09

Shoot the spray bottle and watch its physical response.

Download MP4 ↓

02 — THE METHOD

A room is more
than its appearance.

A simulator needs objects with metric geometry, contact surfaces, and visual identities that stay connected during motion. PhiRIE builds these pieces through evidence-guided construction.

ϕ-RIE at a glance. From captured views to editable objects, physical interaction, and harmonized rendering.
Method overview. Reconstruct the scene, build and register objects, complete the background, and connect appearance to simulation.
01Object selected in a reconstructed office

Observe & propose

Start with captured views and metric observations. Specialized generators propose complete object candidates.

Images → candidates
02Generated bottle aligned to its observed geometry

Register & select

Compare proposals in a shared frame. Use alignment evidence to select, retry, or retain a failed attempt.

Evidence → decisions
03Robot grasping a bottle in PhiView

Connect & interact

Link geometry, collision, and Gaussian appearance through object identity. Inspect how they behave together.

Objects → environments

03 — HANDS ON

Go beyond
the still image.

Three interactive experiences, rendered with WebGPU in your browser. Load a demo, orbit the scene, and explore the controls.

Preview of the captured desk scene

Make the scene your own.

Orbit a captured Gaussian desk and reposition its reconstructed bottle.

3D assets load only when you launch.

Ready to exploreDrag to orbit · scroll to zoom

WebGPU needs a compatible browser and a secure connection. If it is unavailable, the recorded demos above remain accessible.

04 — APPEARANCE

Same state.
A different finish.

Slide to compare saved views before and after appearance harmonization. The camera and scene state stay matched.

Harmonization is image postprocessing. These comparisons show appearance, not a change in physics.

Cup scene with harmonizationThe same cup scene without harmonizationWithoutWith harmonizer
↔

Drag the divider to compare

05 — VISUAL ATLAS

A closer look
at the results.

Explore captured environments, reconstruction candidates, background edits, and recorded interactions. Open any image for its full view and details.

FROM THE PAPER FIGURE COLLECTIONDifferent rooms. A shared representation. Gaussian environments across ScanNet++ and LIBERO. Includes original scene fits, saved background completion, and a recorded robot state.

06 — THE RESEARCH

Making rooms
simulatable.

Evidence-driven real-to-sim connects candidate construction with interaction evaluation. The manuscript separates construction quality, coverage, and success under a declared reconstruction scope.

Read the paper ↗

Download PDF ↗ · arXiv:2609.26795 · September 2026

PAPER FIGURE COLLECTIONMatched harmonization views, preserved from the paper’s figure collection.

CITATION

Cite ϕ-RIE.

arXiv:2609.26795 ↗

Download .bib ↓ 
@misc{yang2026phirie,
  title = {{$\phi$-RIE}: From Photorealistic Reconstruction to Interactive Environments},
  author = {Runyi Yang and Deheng Zhang and Xiaoye Wang and Kanzhi Wu and Lei Sun and Ajad Chhatkuli and Kunyu Peng and Luc Van Gool and Danda Pani Paudel},
  year = {2026},
  eprint = {2609.26795},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2609.26795}
}

07 — DEVELOPMENT

How PhiRIE
has grown.

From the first reconstruction prototype to a modular system for building and interacting with reconstructed environments.

  1. 0.0.0

    Reconstruction prototype

    Captured views, camera geometry, and Gaussian scene reconstruction formed the foundation.

    1. 1.0.0

      Object construction and simulation

      Object discovery, 3D asset generation, metric registration, background completion, and physics export connected reconstructed rooms to simulation.

      1. 2.0.0

        Modular pipeline and PhiView

        Independent feature interfaces, composable pipelines, simulator adapters, and PhiView brought construction and interaction into one workflow.

        1. 2.0.1

          PhiRIE release and demonstrations

          Consolidated setup and documentation, the project page, illustrated components, and demonstrations of manipulation, harmonization, and controllable physical parameters.

RELEASE PLAN

What comes next.

Code will be open-source before 10 October 2026. Further releases will follow validation.

  1. Guided setupSimpler installation and a complete first-scene walkthrough.
  2. Downloadable scene packsPrepared scenes and repeatable shooting, manipulation, and harmonizer demos.
  3. Reproducible evaluationDocumented configurations, benchmark instructions, and results as they are finalized.
Follow the release plan ↗