openreality: MCP server turning phone video into interactive 3D scenes
openreality, developed by Reality Opened, is an MCP server that gives AI agents spatial awareness by converting phone footage into queryable 3D data. It converts simple phone videos into interactive 3D scenes and exposes tools so language models can perform segmentation and generate 3D variants. Key capabilities include 3D reconstruction, click-based segmentation, and model-exposed variant generation. The tool targets AI developers and spatial researchers who need programmatic spatial reasoning integrated into LLM workflows.
What tasks can you actually use it for?
The tool maps a clear job-to-be-done: transform mobile video into a queryable 3D scene so LLMs can perform spatial operations. It exposes MCP tools such as scene_segment and generate_variants so agents can request segmentation or produce modified object variants. Use cases include object-level segmentation inside a reconstructed room, automated 3D asset generation from captured footage, and scripted spatial queries routed through an MCP-enabled chat client.
How accurate are the outputs compared to doing it manually?
Reconstruction and segmentation quality depend on the processing pipeline components the project uses, notably VGGT-SLAM and Gaussian Splatting for mapping and SAM-3 for click segmentation. That means output fidelity varies with capture quality and scene complexity; the cited libraries indicate model-based reconstruction rather than hand-modeled geometry. Variant generation uses TRELLIS, producing automated object variants that require manual inspection before use in physical or production contexts.
Does it require technical knowledge to get useful results?
The server targets developers: it requires an MCP-enabled client and a server environment installed via npm or a Python path depending on deployment. Compatibility notes list Claude Desktop, Cursor, and Codex as supported clients, and the repository includes a self-hostable broker for private deployments. Those facts position the tool for integration into developer workflows rather than hobbyist one-click editors.
Who should choose this tool and why
openreality is a practical choice for developers and spatial researchers who need programmatic 3D context inside LLM sessions, particularly where self-hosted control matters. The project is open-source and noted within the MCP community for bringing 3D context to language models. Expect a developer-oriented setup and to validate generated assets before production use; the tool fits teams prepared to integrate spatial pipelines into existing MCP workflows.
