Lookin to start developing a AI LLM for CS but also trained on internal documentation for our technicians to consult when they are troubleshooting problems in the field. Is it worth trying to build something ourselves from an open source model or does anyone have any experience with off the shelf solutions that have worked well?
Industry is construction technology so the end users aren’t generally tech savvy and we don’t have the manpower to field calls with human technical experts for both our customer support questions and our field techs that might need support.
Ideally would have a phone number the user can call and have a conversation with the bot as the users have traditionally been pretty resistant to text or email based support.
Conceptually, RAG is similar to going to a search engine and grabbing all the relevant paragraphs for a llm to summarize.
If you want a sample to play around with, I recommend playing around with this code snippet
This guide uses the transformers library, but I use Ollama with the OpenAI api.
If you opt for the build, the biggest challenge is taking your internal wikis and documents and getting them ready for a database and a database solution. If you want a all in one solution, I’d check out Trieve. They have some cool super fast demos.
They also have a version you can self host
Disclaimer I got to chat with the trieve devs they know their stuff and have some super fast demos built on their tech
No bosses, basically trying to build this myself and sell it back to my company to solve the issues we have with a lack of knowledgeable people to answer the phones.
