Podcast: Home grown LLMs and Retrieval Augmented Generation (RAG) technologies
This episode of the "Talking with machines" podcast featured a conversation with Dr. Ruben Puentedura. I was joined by Tom Haymes and Bryan Alexander as we discussed “Home grown LLMs and Retrieval Augmented Generation (RAG) technologies.” Ruben acknowledged the limitations of "AI", shared his research and experience with Retrieval Augmented Generation (RAG) technologies, and talked about the affordances of home grown LLMs (Large Language Models) combined with external resources.
Dr. Puentedura is the Founder and President of Hippasus, a consulting practice focusing on transformative applications of information technologies to education.
In our discussion I learned so much about how Ruben thinks about digital technologies and his thoughtful approach to applying these tools to his research. I will definitely be using his methods in the future.
Ruben begins researching a new topic by consulting knowledgeable colleagues for recommendations on where to start, if possible. He then identifies and obtains introductory texts; textbooks, essays, or review articles to gain a basic understanding of the subject. From these initial sources, Ruben creates summaries and extracts key information, sometimes using an LLM to assist with this process, especially if longer texts are in digital form. He organizes these key points and terms on notecards or a note taking app for easy reference.
Next, Ruben searches specialized databases like Google Scholar using the key terms he’s identified, typically collecting around 50 relevant papers. Following the 80/20 rule, he carefully winnows this collection down to the most valuable 20% that contains approximately 80% of the essential content. Then Ruben loads this curated material into a Retrieval-Augmented Generation (RAG) tool like Google’s NotebookLM to identify key points within the texts and generate comprehensive summaries and outlines. He also uses the RAG to question whether some points appear poorly justified.
With this process, Ruben doesn’t then indiscriminately upload everything he’s curated into a new RAG instance. He reviews and determines which sources to include, and adds them incrementally to test and build out the knowledge base. Ruben tries to create only one notebook per topic, but linking several tools is sometimes necessary. Trying to cover too much ground can overwhelm even sophisticated LLM or RAG systems. Ruben’s methodical approach ensures a thorough yet manageable research process that centers human agency and expertise while leveraging LLMs’ information processing and augmentation affordances.
We had a lively and wide ranging discussion from our different points of view on how we are responding to the challenges of using “AI” in higher education and research.
Dickens was mentioned:
The CLiC Dickens project demonstrates through corpus stylistics how computer-assisted methods can be used to study literary texts. https://clic.bham.ac.uk/
And Ruben suggests these resources:
From Local to Global: A Graph RAG Approach to Query-Focused Summarization
https://arxiv.org/pdf/2404.16130
Code a simple RAG from scratch
https://huggingface.co/blog/ngxson/make-your-own-rag
An o1-inspired project by Benjamin Klieger:
https://github.com/bklieger-groq/g1
More from Ruben Puentedura https://www.linkedin.com/in/rubenpuentedura/
Tom Haymes: https://www.linkedin.com/in/tomhaymes/
Bryan Alexander: https://bryanalexander.org/
Mark Corbett Wilson: https://www.linkedin.com/in/markcorbettwilson/