Streaming LLM: Handling Infinite Sequences
#14: Generative model for autonomous driving, AI model for robotic learning
Efficient Streaming LMs with Attention Sink
StreamingLLM is an efficient framework that enables LLMs trained with a finite length attention window to generalize to infinite sequence length without any fine-tuning.
StreamingLLM can enable Llama-2, MPT, Falcon, and Pythia to perform stable and efficient language modeling with up to 4 million tokens and more.
Check out the video below (from GitHub Page) and read more here.
GAIA-1: Generative Model for Autonomous Driving
Wayve.ai scaled GAIA-1, 9-billion parameter generative world model for autonomous driving, trained on 4,700 hours of driving data.
GAIA-1 is a model that leverages video, text and action inputs to generate realistic driving videos and offers fine-grained control over ego-vehicle behaviour and scene features.
RT-X: AI Model for Robotic Learning
Google DeepMind introduced 𝗥𝗧-𝗫, a generalist AI model to help advance how robots can learn new skills.
With the help of 33 academic labs across the world, they also created an Open X-Embodiment dataset shows more than 500 skills and 150,000 tasks across more than 1 million episodes.
Read more about the model in their blog.
