Hello everyone, I am a mathematics student and programmer-in-training and have been a lurker for some time.
A few months ago, I was mulling over two things:
-
The lack of a complete open-source replacement for something like the old PaperPort in the office
-
The necessity for private, local (or self-hosted) AI tools users are actually in control of.
The latter reflection was spurred by the video on using ik_llama.cpp with MoE 1.5 bit quantized models, allowing very large context windows on (relatively) modest hardware, and ameliorating the drawbacks by manually feeding in source information.
I was unable to replicate this at the time with my AMD graphics card, and I failed again months later after attempting to hack things together with ZLUDA.
The idea of being able to extract information from a vast repository of books, personal notes, manuals, papers… stuck with me however, and I also wished to be able to query an entire document base, without having to pick manually.
Clearly this is nothing new, I just need to generate embeddings and use RAG; and really, there is nothing new about what I plan to do.
I only want to tie together open-source tools other people wrote into an application that won’t catch fire and burn when you look at it wrong, making them work in concert together without reinventing the wheel.
Project description
This open-source document management system would use PostgreSQL, Java 25 with Spring and Angular 20.
I’m following a “modular monolith” architecture, using spring modulith to enforce module separation at compile-time through tests. Currently, the modules are dms, storage, security, worker.
Internally, I am trying to imitate “functional core, imperative shell” somewhat. What would normally be the service layer is subdivided into “service” and “logic”, with “service” acting only as orchestrator and “logic” handling business logic through pure functions. JPA entities do not leave the repository layer and service orchestrator, but are first converted to Java records for business logic, communication with other modules, and conversion to DTOs for communication with the Angular front-end.
-
dmshandles the simpler user-facing tasks of a DMS, such as displaying documents. -
storagesaves documents and their artifacts under<data directory>/ab/cd/<document uuid starting with abcd>/. A folder entity exists as metadata for display and user convenience. -
securityhandles users, roles, permissions and a table for revoked refresh token IDs. Uses spring security.
I have a basic draft of the first three running, but worker is the module where the “meat” of it will lie.
Future plans
After reading up on what the available options are, this is the processing pipeline for documents I plan for the worker module to eventually handle:
-
Inspection of uploaded original files with Tika, both for security reasons and for determining needed/available actions
-
Grayscale/deskew with VIPS-FFM
-
OCR with iText pdfOCR
-
Conversion to a “canonical markdown” form with Marker or Docling (Integrated through FastAPI. I was unable to find comparable open-source Java options)
I may need to set up a thread pool to prevent background OCR or deskew also slowing down user-facing tasks such as browsing through folders and downloading documents.
It may be possible to skip the usage of Redis or Kafka entirely for handling these tasks, and instead use Postgres directly, as it allows for a deadlock-free task queue through the FOR UPDATE SKIP LOCKED feature.
All of these files won’t be editable, and will be represented by a “document artifact” entity. Each points to its parent artifact, forming a directed acyclic graph. To save storage, it may be necessary to delete useless files from intermediate artifacts, so a JSONB column in the database could keep track of generation settings even after the file itself has been removed.
The generated files enable the search feature:
-
OCR text, through BM25, allows saving tsvectors in the database for full-text search
-
Markdown text can be split into chunks at header level for embeddings generation with bge-v3
Postgres’ pgvector extension then adds compatibility with the embeddings.
At this point, spring AI significantly simplifies both the process of generating embeddings and that of communicating with the (hopefully self-hosted) llama.cpp or LMStudio API.
I should likely skip to the end and implement embeddings generation, FTS and AI/RAG integration, then test the application with a bunch of markdown files, to first check if this idea has any merit.
Non-exhaustive list of possible issues
(or possible future “opportunities for improvement”)
-
The priority issue with this plan, is that I must actually learn Angular (and Typescript) from scratch. While I have studied C and Java and done some backend work with Java/Spring, I must unfortunately admit I am 100% in vibe-coder territory for the front-end. What little exists now is only a prototype squeezed out of LLMs to check if the backend works.
-
For the storage module, it may be necessary to integrate a root_location entity to let users save files in more than one location, and perhaps let users save files in a remote location (hopefully also self-hosted).
-
Sometimes, users will need to manually fix OCR mistakes. Would this require creating a new child artifact every single time they press CTRL+S? There may be a need for versioning, which can be its own can of worms.
-
A directed acyclic graph for document artifacts does imply that multiple versions of the same kind of intermediate product could exist at the same time, generated with different settings. To avoid issues, perhaps only one should be “active” at any given time. Displaying all this for the user in a sensible way, such that they may find and inspect any version of any intermediate document available, is unlikely to be easy.
-
Marker can also be used for OCR directly, and even be integrated with an LLM to fix small mistakes and formatting. Ignoring this may weaken the application, integrating it complicates the artifact graph.
-
Embeddings generation can be improved by setting a range for desired size (for headers with text that is too long) and a subdivision into child and parent chunks: when the child chunk’s embedding matches, the parent chunk is retrieved to preserve context.
-
Embeddings can be further improved by reordering the results with a ONNX cross-encoder.
-
An interesting experiment would be adding a second BM25 pass on the markdown itself, following the same chunk boundaries as the embeddings instead of page separation. In this manner, one could test out reciprocal rank fusion hybrid search.
Some things which are a very long ways off:
-
Integrating an existing collaborative markdown editor could be a useful extension. For such a purpose, perhaps a mutable DocumentDraft entity could let users clone an existing document artifact for active use, which will then be saved into another non-editable artifact. This may be inappropriate for PDF files, and thus be restricted to markdown files only.
-
I did forget along the way that I was originally thinking of Paperport. Perhaps it’s possible to write a desktop app counterpart reusing as much code as possible with Tauri (So that it weighs ~18 MB rather than ~180 with Electron), which could have added scanning functionality.
-
…tool calling and agentic behaviour?
Conclusion
Given that I am not a software engineer and also very inexperienced, I welcome any and all advice, clarification and criticism.
For instance, is this application’s existence justified? Does something like Paperless-ngx plus other tools/applications I didn’t find out about fulfill this use case?