So, I got deeper into LLMs than I initially intended, but I found a lack of tools to simply map a folder with, for example, multiple Excel files and then use a model (I’ve tested a few) to analyze the files and find patterns. I’ve been able to import a single file like Excel or PDF and generate new prompts in Ollama, but I haven’t figured out how to set up a folder with multiple files for analysis.
I’m not looking for training, as I don’t have the compute power for that. I’m just trying to find an offline solution to help manage my personal data and files, which are all text-based for now.
GPT4All, LangChain are both tools I see suggested are you using them and if so pls share the, how and so on if you have time
Ugh 19 days and not even one recommendation. Its something i wander for my self, are we at loss hire no one dos RAG.
Please share your failures if you tried any info would be appreciated.
If you still wanna do langchain, the documentation has some guides
If you don’t want to get too deep into coding, you can also use lmstudio and upload files into a chat, though if you want to use it for work you’ll need to fill out a form
Thanks @progressEdd, I knew it would end up being a long read Thanks for starting me on this journey!
I guess I was hoping for some kind of shortcut like ‘https://ollama.com/install.sh’
My pleasure, feel free to start your own thread documenting the process. At it’s core RAG is effectively a search engine, we want the language model to only use what is most relevant to answer a user’s question.
I linked a bunch, but the llamaindex links should give you a good idea of the concepts, while the huggingface cookbooks give you recipes to try out and is closer to what you are looking for in terms of the reading a shell script.
Yeah, but honestly, it doesn’t work as well as expected. LLama and the WebGUI do offer the ability to load up information, but when I try to feed in my Excel sheets in CSV format—all formatted the same way, with a new sheet each day related to stock trading—they often fail to read them fully, taking in only a few entries. Even then, I’m able to retrieve the relevant info faster manually. I was hoping that over, let’s say, 1,000 sheets, the system might be able to outperform me. It does work if you use PDFs, like feeding in a book and asking related questions. Perhaps I’m missing something
"Thanks again! I will try to document what I did. Gotta put it on the back burner for now though. I’m in the process of designing and procuring a home server with multi-GPUs for AI workloads. Need some serious VRAM for that sweet, sweet context
Yeah this is one of the funny things with llms but it makes sense if you think about how they actually work. That process is now apparently „prompt engineering“ and I’m not always sure when it is used in jest or not.
It goes as far as apple developers asking the lim to „not hallucinate“. I take from this that we’re still far off from so called superintelligence. Not to mention how many kWh OpenAI’s o1 needs to compute the number of r‘s in strawberry…
For RAG, the size of the model prompt window is very important. This will dictate how small the chunks of your data you need to make.
You will need some way to store the chunks and retreive the most relevant chunks; this is trypicaly done with a vector database.
Then, depending on the vector database you choose, there will probably be plugins to feed various types of documents in it and chunk them, and sometimes summarize.
This gets complicated quickly, and the more complicated it gets the less useful the whole idea becomes. Still, by all means, experiment and report your findings, we are all will be courious how well RAG works for you.
no one dos RAG.
Thanks for starting me on this journey!
