Retrieval meets Long Context LLMs
#15: OpenAI fine-tuning on function calls, RekaAI introduces multimodal assistant Yasa-1, RAG vs Long Context LLM's, RA-DIT: Retrieval Augmented - Dual Instruction Tuning
OpenAI introduces fine-tuning on function calls
Including a long list of functions while prompting can impact context length and potential model hallucination
Fine-tuning on function calling can also be used to customize the model's response to function outputs in addition to tuning it on function call accuracy
More details here (under Function Calling header)
RekaAILabs introduces multimodal assistant, Yasa-1
Releases multimodal assistant Yasa-1, a language assistant with visual and auditory sensors that can take actions via code execution.
Also comes with long context document processing, fast natively-optimized retrieval augmented generation, multilingual support (20 languages), a search engine interface, and a code interpreter
Read the blog and the documentation.
Papers
1. Retrieval meets Long Context LLMs
The paper compares retrieval augmentation and long-context windows for downstream tasks, with a focus on combining both methods for optimal performance.
It demonstrates that a 4K context window in a language model with simple retrieval augmentation can achieve similar performance to a fine-tuned model with a larger 16K context window.
Retrieval augmentation significantly enhances language model performance, with a retrieval-augmented LLaMA2-70B-32K outperforming GPT-3.5-turbo-16k on various long-context tasks, while also offering faster generation speed compared to a non-retrieval LLaMA2-70B-32K. Read the paper here
2. RA-DIT: Retrieval Augmented - Dual Instruction Tuning
It proposes to fine-tune both the LLM and Dense neural network retriever
The language model is fine-tuned to better use retrieved texts by training it on top “k” retrieved contexts from a dense retriever
The retriever is fine-tuned with LM’s output to tailor it to the model’s preference. Paper Reference.
