How to Improve LLMs with RAG (Overview + Python Code)

День тому

In this video, I give a beginner-friendly introduction to retrieval augmented generation (RAG) and show how to use it to improve a fine-tuned model from a previous video in this LLM series.
👉 Series Playlist: • Large Language Models ...
🎥 Fine-tuning with QLoRA: • QLoRA-How to Fine-tune...
📰 Read more: medium.com/towards-data-scien...
💻 Colab: colab.research.google.com/dri...
💻 GitHub: github.com/ShawhinT/UKposts-B...
🤗 Model: huggingface.co/shawhin/shawgp...
Resources
[1] github.com/openai/openai-cook...
[2] • LlamaIndex Webinar: Bu...
[3] docs.llamaindex.ai/en/stable/...
[4] • LlamaIndex Webinar: Ma...
--
Book a call: calendly.com/shawhintalebi
Homepage: shawhintalebi.com/
Socials
/ shawhin
/ shawhintalebi
/ shawhint
/ shawhintalebi
The Data Entrepreneurs
🎥 UKposts: / @thedataentrepreneurs
👉 Discord: / discord
📰 Medium: / the-data
📅 Events: lu.ma/tde
🗞️ Newsletter: the-data-entrepreneurs.ck.pag...
Support ❤️
www.buymeacoffee.com/shawhint
Intro - 0:00
Background - 0:53
2 Limitations - 1:45
What is RAG? - 2:51
How RAG works - 5:03
Text Embeddings + Retrieval - 5:35
Creating Knowledge Base - 7:37
Example Code: Improving UKposts Comment Responder with RAG - 9:34
What's next? - 20:58

КОМЕНТАРІ: 35

@ShawhinTalebi Місяць тому

Check out more videos in this series 👇 👉 Series Playlist: ukposts.info/slow/PLz-ep5RbHosU2hnz5ejezwaYpdMutMVB0 🎥 Fine-tuning with QLoRA: ukposts.info/have/v-deo/kKCgf3hqrKmerJc.html -- 📰 Read more: medium.com/towards-data-science/how-to-improve-llms-with-rag-abdc132f76ac?sk=d8d8ecfb1f6223539a54604c8f93d573 💻 Colab: colab.research.google.com/drive/1peJukr-9E1zCo1iAalbgDPJmNMydvQms?usp=sharing 💻 GitHub: github.com/ShawhinT/UKposts-Blog/tree/main/LLMs/rag 🤗 Model: huggingface.co/shawhin/shawgpt-ft Resources [1] github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb [2] ukposts.info/have/v-deo/nZaTomNqq4Z6wIE.html [3] docs.llamaindex.ai/en/stable/understanding/loading/loading.html [4] ukposts.info/have/v-deo/kppmhnmunIt5qp8.html

@saadowain3511 Місяць тому

Thank you Talebi. No one explains the subject like you

@ShawhinTalebi Місяць тому

Thanks :) Glad it was clear!

@ifycadeau Місяць тому

This is so helpful! Thanks Shaw, you never miss!

@ShawhinTalebi Місяць тому

Glad it was helpful!

@GetPaidToLivePodcast Місяць тому

Incredible breakdown Shaw!

@bangarrajumuppidu8354 25 днів тому

superb explanation Shaw !😍

@michaelpihosh5904 Місяць тому

Thanks Shaw!

@deadlyecho 18 днів тому

Very good explanation 👏 👌

@zahrahameed4098 4 дні тому

Thankyou so much. Becoming a fan of yours! Please do a video on Rag with llamaIndex + llama3 if it's free and not paid.

@examore-lite Місяць тому

Thank you very much!

@ariel-dev Місяць тому

Really great

@lplp6961 Місяць тому

good work!

@tomasbusse2410 13 днів тому

Very useful indeed

@nayem5330 Місяць тому

Very useful.

@ShawhinTalebi Місяць тому

Glad it was helpful!

@Blogservice-Fuerth Місяць тому

Great 🙏

@firespark804 Місяць тому

Awesome video, thanks! I'm wondering if instead of using top_k documents/batches one could define a threshold/distance for the used batches?

@halle845 Місяць тому

Thanks!

@ShawhinTalebi Місяць тому

Thank you! Glad it was helpful 😁

@nistelbergerkurt5309 Місяць тому

great video as always 👍 does a reranker improve the quality of the output for a RAG approach? like that we could take the output directly from the reranker, right? or what is your experience with reranker?

@ShawhinTalebi Місяць тому

Great questions! That's the idea. A reranker is typically applied to the top-k (say k=25) search results to further refine the chunks. The reason you wouldn't use a reranker directly on the entire knowledge base is because it is (much) more computationally expense than the text embedding-based search described here. I've haven't used a reranker in any use case, but it seems to be most beneficial when working with a large knowledge base. This video may be helpful: ukposts.info/have/v-deo/jZhqlo-gjaiIwKc.html&ab_channel=JamesBriggs

@TheLordSocke 4 дні тому

Nice Video, any ideas for doing this on PowerPoints? Want to build a kind of knowledge base from previous projects but the grafics are a problem. Even GPT4V is not always interpreting them correctly. 😢

@Pythonology Місяць тому

Happy Nowruz, kheyli khoob! Question: how would you propose to evaluate a document on the basis of certain guidelines? I mean, to see how far it complies with the guidelines or regulations for writing a certain document. Is RAG any good? shall we just embed the guidelines in the prompt right before the writing? or shall we store the guidelines as a separate document and do RAG? Or ...?

@ShawhinTalebi Місяць тому

Happy New Year! That's a good question. It sounds like you want the model to evaluate a given document based on some set of guidelines. If the guidelines are static, you can fix them into the prompt. However, if you want the guidelines to be dynamic, you can house them in a database which is dynamically integrated into the prompt based on the user's input.

@halle845 Місяць тому

Any recommendations or experience on which embeddings database to use?

@ShawhinTalebi Місяць тому

Good question! Performance of embedding models will vary by domain, so some experimentation is always required. However, I've found the following 2 resources helpful as a starting place. HF Leaderboard: huggingface.co/spaces/mteb/leaderboard SentenceTransformers: www.sbert.net/docs/pretrained_models.html

@vamsitharunkumarsunku4583 Місяць тому

So we get top 3 similar chunks from RAG right, We are adding 3 chunks to prompt template?

@ShawhinTalebi Місяць тому

Yes exactly!

@TheRcfrias 5 днів тому

Rag is great for semi-static or static content as knowledge base, but which path do you use for dynamic, time-relevant data like current sales from a database?

@ShawhinTalebi 5 днів тому

That's a great question. The short answer is RAG can handle this sort of data (at least in principle). The longer answer involves taking a step back and asking oneself "why do I want to use RAG/LLMs/AI for this use case?" This helps get to the root of the problem you are trying to solve and hopefully give more clarity about potential solutions.

@TheRcfrias 5 днів тому

@@ShawhinTalebi Its a common use case at work to know how sales have been improving during the current day or week. It would be nice to know how to link the LLM with the corporate database for current information and reporting.