Log inSign up
user avatar
Daniel Duckworth
@duck
Research Scientist at Google DeepMind, Berlin. stronglyconvex.com
Berlin, DE
stronglyconvex.com
Joined September 2010
149
Following
2,497
Followers
  • Pinned
    user avatar
    Daniel Duckworth
    @duck
    Dec 13, 2023
    Introducing SMERF: a streamable, memory-efficient method for real-time exploration of large, multi-room scenes on everyday devices. Our method brings the realism of Zip-NeRF to your phone or laptop! Project page: smerf-3d.github.io ArXiv: arxiv.org/abs/2312.07541 (1/n)
    Image
    00:00
    188K
  • user avatar
    Daniel Duckworth
    @duck
    Aug 6, 2020
    Our paper, “NeRF in the Wild”, is out! NeRF-W is a method for reconstructing 3D scenes from internet photography. We apply it to the kinds of photos you might take on vacation: tourists, poor lighting, filters, and all. nerf-w.github.io (1/n)
    Image
    00:00
  • user avatar
    Daniel Duckworth
    @duck
    Aug 6, 2020
    Replying to @duck
    For lighting and image post-processing, we introduce a low-dimensional embedding space controlling NeRF’s radiance field. This not only gives NeRF-W the capacity to model photo-specific lighting, it enables us to “relight” a scene from new angles. (3/n)
    Image
    00:00
  • user avatar
    Daniel Duckworth
    @duck
    Aug 6, 2020
    Replying to @duck
    This project wouldn’t have been possible without my amazing coauthors: @rmbrualla, Noha Radwan, Mehdi S. M. Sajjadi, @jon_barron, and Alexey Dosovitskiy. Check out our paper: arxiv.org/abs/2008.02268
  • user avatar
    Daniel Duckworth
    @duck
    Aug 6, 2020
    Replying to @duck
    We build on NeRF, a method for learning a volumetric radiance field from a posed photo collection. We introduce two extensions to soften NeRF’s “static world” assumption: one for lighting/post-processing, the other for transient objects. (2/n)
  • user avatar
    Daniel Duckworth
    @duck
    Aug 6, 2020
    Replying to @duck
    NeRF-W improves on the SOTA by >5dB in PSNR and reduces error on other metrics by 20-50%. Qualitatively, NeRF-W produces consistent, crisp 3D geometry without fog or checkerboard artifacts. Check out the project website for more videos and the paper. nerf-w.github.io (5/n)
    Image
  • user avatar
    Daniel Duckworth
    @duck
    Aug 6, 2020
    Replying to @duck
    For transient objects, we introduce a secondary volumetric radiance field combined with an uncertainty field. The former explicitly captures transient objects; the latter uncertainty about the color of a pixel passing through part of the 3D space. (4/n)
    Image
  • user avatar
    Daniel Duckworth
    @duck
    Dec 13, 2023
    Replying to @duck
    SMERF has the best of both worlds: we produce renders nearly indistinguishable from Zip-NeRF while rendering at 60 fps or more on desktops, laptops, and even recent smartphones, all while scaling to scenes as big as a house! (3/n)
    Image
    00:00
    12K
  • user avatar
    Daniel Duckworth
    @duck
    Apr 24, 2020
    How does one trade-off sample quality and diversity in a language model? Which decoding method is best? We introduce a multi-objective framework maximizing human judgement score subject to a constraint on diversity (entropy). arxiv.org/abs/2004.10450 (1/7)
  • user avatar
    Daniel Duckworth
    @duck
    Aug 6, 2020
    Infinite! We don't use polygons. We learn a "volumetric radiance field" scene representation.
  • user avatar
    Daniel Duckworth
    @duck
    Aug 6, 2020
    We used a few hundred to low-digit thousands. Based on the results of NeRF, if you capture your images in a controlled environment, you might be able to get away with as few as one hundred!
    matthewtancik.com
    NeRF: Neural Radiance Fields
    A method for synthesizing novel views of complex scenes by optimizing an underlying continuous volumetric scene function using a sparse set of input views.
  • user avatar
    Daniel Duckworth
    @duck
    Dec 13, 2023
    Replying to @duck
    This has been a joint work with my amazing collaborators: @PeterHedman3, @ChrisJReiser, @PeterZhizhin, @jfthibert, @MarioLucic_, @RSzeliski, and @jon_barron Learn more and try SMERF out yourself at smerf-3d.github.io (8/n)
    smerf-3d.github.io
    SMERF
    Project page for SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Exploration
    2.4K
  • user avatar
    Daniel Duckworth
    @duck
    Dec 13, 2023
    Replying to @duck
    How do we achieve this? We distill a teacher model into a family of MERF-like student submodels, each of which specializes to a different part of the scene. Each submodel captures the entire scene, so rendering stays fast and GPU memory consumption stays low. (4/n)
    Image
    2.5K
  • user avatar
    Daniel Duckworth
    @duck
    Aug 6, 2020
    Replying to @pradeepviswav @benedictevans and 2 others
    Indeed, we were heavily inspired by Photosynth! I remember being in awe when I first saw it back in high school.