🤖 World Action Models predict the future to act better. However, WAM research remains fragmented in code, ad hoc in design, and unprincipled at scale.
💡 Today, we release 𝗢𝗽𝗲𝗻𝗪𝗔𝗠: 𝗔𝗻 𝗢𝗽𝗲𝗻, 𝗠𝗼𝗱𝘂𝗹𝗮𝗿 𝗘𝘅𝗽𝗹𝗼𝗿𝗮𝘁𝗶𝗼𝗻 𝗧𝗼𝘄𝗮𝗿𝗱𝘀 𝗦𝘆𝘀𝘁𝗲𝗺𝗮𝘁𝗶𝗰
I still don’t get why people think code as policy is so compelling. Language models and their harness can be very useful for robots, but writing programs that directly output for example, eef actions, seems to me not the best interface.
"We’re using language as a crutch to help the deficiencies of our vision systems to learn good representations from images and video." Yann Lecun, Lex Fridman Podcast #416.
Sorry, this thread is very cool, but I still believe in the long run, building models with intrinsic
This one is impressive. "Behavior Prompting", or in-context learning of robotics from seeing demonstrations has a long history, and it's a good time to go down this memory lane. (1/n)
Introducing GEN-1.5, a one-shot learner.
It can learn new tasks in a few seconds. Show it what to do, and it generalizes.
This capability emerged from pretraining on physical data at scale, as a step towards our mission of building general intelligence for the physical world.
@JiangranLyu and @fangfu0830 are two of the best researchers I've known in the intersection of World Models and Robotics. RSI + WM seems the right way to go for solving physical AGI. Congrats on the launch!