RAG basics

Data preparation for RAG

Web scraping tool — Crawl4AI , youtube video

www.example.com/sitemap.xml e.g. https://www.cisco.com/web/sitemap/www_cisco_com_en_us_index.xml

www.example.com/robots.txt (view allow/disallow list). e.g https://www.cisco.com/robots.txt

other docs incl. PDF, DOCX, PPTX, XLSX, HTML, WAV, MP3, VTT, images (PNG, TIFF, JPEG, …) Docling

https://www.reddit.com/user/NVIDIAOFFICIAL/comments/1piiffw/a_guide_to_agentic_rag_that_runs_locally/?p=1&utm_source=share&utm_medium=android_app&utm_name=androidcss&utm_term=1&utm_content=2