It could be useful to be able to generate source data pairing Markdown files with the corresponding header-level chunk embeddings, export it for use on other machines running the same application with more limited hardware, then use semantic search and/or RAG on the new machines too depending on their capabilities.
I would likely be able to experiment with this already on my current hardware, perhaps with something small like Project Gutenberg, but moving to PubMed, arXiv, Wikipedia etc. may be unrealistic.
While it would be great to be able to use local models alongside this application, since the API is the same it should not make a huge difference for development; people who can afford the hardware would still be able to use this software with models running locally.