Isaac Wasserman

RAIDD

Retrieval from AI Derived Documents

2024

Retrieval from AI Derived Documents (RAIDD) is new take on retrieval augmented generation (RAG) which leverages the full power of the LLM during the indexing process. During document ingest, we use the LLM to derive new documents which are more information-transparent and explicit in their language than the originals. These "derived documents" act as references for the original documents, effectively increasing the language understanding ability of the retriever.

RAIDD functionality is currently being implemented for the llama_index Python package.

RAIDD Pipeline