Image
user avatar
Kfir Aberman
@AbermanKfir
Founding Member @DecartAI | Research Scientist | ex-@Snap | ex-@Google | Personalized Generative AI | DreamBooth
Palo Alto, CA
Joined August 2019
Posts
  • Pinned
    user avatar
    1/ Today, we’re announcing Oasis 3: @DecartAI’s interactive world model for Physical AI. For the first time, teams can generate realistic, controllable, multi-view simulation environments in real time, and access them through an API. Thread + links 👇
    Image
    00:00
  • user avatar
    We discovered that imposing a spatio-temporal weight space via LoRAs on DIT-based video models unlocks powerful customization! It captures dynamic concepts with precision and even enables composition of multiple videos together!🎥✨
    Image
    00:00
  • user avatar
    🎉 Today at @DecartAI we're announcing Mirage. The first generative video model that runs in real-time, to infinity. Mirage is a Live Stream Diffusion (LSD), a breakthrough that transforms any video into anything you can imagine, as it plays. 🎮 Try it: mirage.decart.ai
    Image
    00:00
    Image
    00:31
    Introducing MirageLSD: The First Live-Stream Diffusion (LSD) AI Model Input any video stream, from a camera or video chat to a computer screen or game, and transform it into any world you desire, in real-time (<40ms latency). Here’s how it works (w/ demo you can use!):
  • user avatar
    This amazing work takes our prompt-to-prompt, build together with a language model (GPT-3) a large dataset of triplets: (input images, instruction, output image), and use it to train a model that can edit images with instructions!!! 👑
    InstructPix2Pix: Learning to Follow Image Editing Instructions abs: arxiv.org/abs/2211.09800 project page: timothybrooks.com/instruct-pix2p… InstructPix2Pix, trained on generated data, and generalizes to real images and user-written instructions at inference time
    Image
  • user avatar
    We present HyperDreamBooth, our follow-up work to DreamBooth. With a new HyperNetwork architecture, you can now pass your image, count to 10.. 🧨 3.. 2.. 1.. and boom💥 You got your personalized model. Better, cheaper and faster. Play safe 😃 #HyperDreamBooth #DreamBooth
    HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models paper page: huggingface.co/papers/2307.06… Personalization has emerged as a prominent aspect within the field of generative AI, enabling the synthesis of individuals in diverse contexts and styles, while
    Image
  • user avatar
    🚀 Career Update After years pushing the boundaries of Generative AI at some of the world’s top companies -> I’m going startup. I’ve joined @DecartAI as a founding team member, leading the charge to build our San Francisco office from the ground up.
  • user avatar
    Check out our Prompt-to-Prompt implementation with #stabledifussion ! You can now edit images by editing their captions. No more prompt engineering 🤩
    Prompt-to-Prompt: Latent Diffusion and Stable Diffusion implementation with @huggingface diffusers is out github: github.com/google/prompt-…
    Image
  • user avatar
    ⭐️No more prompt engineering!⭐️ Excited to share our “Prompt-to-Prompt” work, an editing framework using text-2-image diffusion models, that enables users to edit images based only on textual edits of a prompt. Check the paper to see how powerful Cross-Attention layers are!
    Prompt-to-Prompt Image Editing with Cross Attention Control abs: arxiv.org/abs/2208.01626 analyze a text-conditioned model and observe that the cross-attention layers are the key to controlling the relation between the spatial layout of the image to each word in the prompt
    Image
  • user avatar
    Back in my PhD days, I really wanted to crack motion transfer between skeletons with totally different topologies. But the tools and data just weren’t there. Crazy to see how far it’s come - these motion transfer (and generation!) results are really exciting 🦖!
    🚀 New preprint! 🚀 Check out AnyTop 🤩 ✅ A diffusion model that generates motion for arbitrary skeletons 🦴 ✅ Using only a skeletal structure as input ✅ Learns semantic correspondences across diverse skeletons 🦅🐒🪲 🔗 Arxiv: arxiv.org/abs/2502.17327
    Image
    00:00
  • user avatar
    📣📣 Thrilled to announce that after 3 incredible years at Google, I'm joining Snap Research to build a world-leading Generative AI research team. More details below 👇🏼
  • user avatar
    Thrilled to share that Break-A-Scene is accepted to #SIGGRAPH Asia 2023 , another amazing leap in Personalized Generative AI! 🍾 You can now take a single image: break it into pieces, re-render them, vary, and extract backgrounds like never before. Big shoutout to @OmriAvr and
    Break-A-Scene: Extracting Multiple Concepts from a Single Image introduce the task of textual scene decomposition: given a single image of a scene that may contain several concepts, we aim to extract a distinct text token for each concept, enabling fine-grained control over the
    Image
    00:00
  • user avatar
    Excited to share that it got accepted to #SIGGRAPH2023! 😍🎉 “Sketch-Guided Diffusion” enables text-to-image models to use spatial maps from other domains (like sketches) for inference guidance without any finetuning! 🙌🏼
    Sketch-Guided Text-to-Image Diffusion Models abs: arxiv.org/abs/2211.13752 project page: sketch-guided-diffusion.github.io
    Image
    00:00
  • user avatar
    ⭐️Open internship position in @GoogleAI ⭐️ If you are: ✅ A last year Ph.D student ✅ Working on image/video synthesis ✅ Willing to spend the summer in the Bay Area and work with us on the next generation of Generative AI. Please ping me!
  • user avatar
    Proud to be part of this legacy. This small lab in Tel Aviv has shaped the landscape of Personalized GenAI - years of innovation behind us, and so much more ahead!
    Thrilled to see this plot in a recent survey on 'personalized image generation' (arxiv.org/abs/2502.13081) — highlighting the impact of our work! Huge congratulations to my fantastic students, whose creativity and dedication continue to drive exciting advances in the field!
    Image