Intelli-Docs: An AI-Powered Personal Document Assistant Using Retrieval-Augmented Generation and Multimodal Retrieval

Abstract

Managing personal documents remains a hassle: files accumulate across cloud drives, email, and local devices in formats that mix scanned images with digital text, and conventional retrieval based on filenames and folder hierarchies handles neither ambiguous queries nor cross-format access well. This paper presents Intelli-Docs, a personal document assistant that combines retrieval- augmented generation (RAG) with multimodal retrieval so that users can locate and question their documents through natural language queries. The system ingests documents through three coordinated pipelines. A text pipeline extracts and chunks document text, embeds the chunks, and stores them in a vector index. A query pipeline classifies each request as text retrieval, image retrieval, or general conversation and routes it accordingly. An image pipeline processes scanned and photographic documents with three parallel models: a Tesseract engine for optical character recognition, a BLIP model for caption generation, and a CLIP ViT-B/16 encoder for semantic image embeddings, whose outputs are fused into a single textual representation. Retrieved evidence is passed to a large language model that generates the final grounded answer. On image- text retrieval benchmarks, the retrieval component reached Recall@5 of 92% and mean average precision of 0.89, with a mean query latency of 2.7 s. We describe the architecture, the evaluation protocol, and the observed trade-offs between accuracy and on-device cost, and we discuss the limitations that constrain mobile deployment.