Big Blue Ceiling

iOS LiDAR Scanner

3D scanning and surface reconstruction on LiDAR-equipped iPhones and iPads.

Eddie OffermannDeveloper

Role
Developer
Areas
XR, Computer Vision, Apps, VFX
Concept illustration: a person with purple and teal hair holds up a phone to scan a fossil skull on a museum plinth, and the phone screen shows the skull as a dense, colored point cloud.

Recent iPhones and iPads carry a LiDAR sensor next to the RGB camera. We built a Unity framework on AR Foundation and ARKit that turns that sensor into a 3D scanner, for object scanning, AR visualization and 3D asset creation.

The pipeline

  1. Scanning. Point clouds are captured in real time, and each 3D point is projected into the camera image to pick up its color. There are two modes: freeform, with the device held in the hand, and turntable, with the device fixed and the object rotating, captured a step at a time.
  2. Processing. Modular filters clean the cloud: voxel-grid downsampling, statistical outlier removal, radius outlier filtering and bounding-box clipping, with a one-level undo.
  3. Spatial structure. The cleaned cloud goes into a generic adaptive octree, used for spatial queries, level of detail and meshing. An editor gizmo draws the octree's leaves for debugging.
  4. Reconstruction. Meshes come at three levels of detail:
    • LOD0: a fast voxel-cube preview.
    • LOD1: an optimized cube mesh with hidden faces culled.
    • LOD2: a smooth surface from marching cubes over a voxel signed distance field, with color carried through.
  5. Visualization. The scan and the live mesh preview are overlaid on the camera feed.
Five cards: a voxel grid filter that takes a voxel size, a statistical outlier filter that takes a neighbor count and standard-deviation multiplier, a radius outlier filter that takes a radius and minimum neighbors, a bounding-box clip, and a one-level undo.
We expose each filter as its own call, so a scan can be cleaned step by step and one step undone.

Design

Each stage is its own component with a small public API: capture, processing, octree building and meshing can each be swapped or extended independently. Mesh generation runs asynchronously and raises an event when a result is ready, so the interface stays responsive while it works. The reconstruction architecture is designed to take Poisson reconstruction or a learned mesher as another level of detail.

Comparison of mesh levels: LOD0 is a fast voxel-cube preview, LOD1 an optimized cube mesh with hidden faces culled, LOD2 a smooth colored surface from marching cubes over a voxel signed distance field, with room for Poisson reconstruction or a learned mesher.
We build every level from the same octree, asynchronously, with an event when each mesh is ready.

Scanning real objects into usable meshes connects to our capture work in CameraSmartPlate and to the asset side of b2c3d.

  • Concept illustration of scanning a fossil skull in a museum with a phone, the screen showing a colored point cloud.
    Concept illustration of the scanning experience.
  • UML class diagram of the scanning system: the point cloud capture manager, point cloud processor, octree builder and octree, the fast and adaptive mesh generators, marching cubes and its volume, and the scan mesh renderer.
    The scanning and reconstruction framework, as a class diagram.

  • AR Scuba

    An underwater 360 video performed as a hologram.

    • XR
    • Apps
    • Computer Vision

  • Augmented Clothing

    Clothing as a platform for augmented reality, starting with hats.

    • XR
    • Apps
    • Computer Vision

  • Cardforia and SecureVR

    Unity building blocks for phone-based AR and VR.

    • XR
    • Tools
    • Apps
    • Computer Vision

  • MagicWatch

    Augmented reality that turns context cues into live database views.

    • XR
    • Apps
    • Computer Vision