Help W/ Building an AI Estimating Machine

Hardware Hub
, ,
1

Hello! First time poster long time watcher of Level1Techs and I had some questions about building a local AI system for estimating blueprints/drawings from PDFs. I’ve been theory crafting for over a year now about what parts I’d need through watching videos and using AI to help me with understanding what it is I actually need it to do and I’m more or less at my limit now for information save for talking with knowledgable people.

I have a build list and the budget itself isn’t necessarily important as this is for a small family business, however I am leary of spending oodles of cash unnecessarily. Currently I have widdled the parts down to:

  • AMD Ryzen Threadripper Pro 3955WX 16-Core, 32-Thread 280W WRX8
  • A-Tech 128GB 8x 16GB 1Rx4 PC4-21300R DDR4 2666 ECC REG RDIMM
  • Silverstone RM44 4U Rackmount Server Chassis w/ Enhanced Liquid Cooling Compatibility
  • Pro WS WRX80E-SAGE SE WIFI (already purchased, got a wicked deal at my old job)
  • Silverstone XE360-TR5 Liquid AIO CPU Cooler
  • Seasonic 1600W Platinum Prime 80+ Fully Modular ATX PSU

Where I’m needing help is deciding on the GPU(s) and what my needs are vs what I think they are. Long story short of how it would run is I would give it drawings, say 10 different ones, it would have to read the drawings provided and deduce both pipe and duct sizing, lengths, whether they are exposed or concealed, and then proceed to use that information from tool calling or otherwise to output the estimate of how much say pipe insulation is needed. I have a giant repository of old drawings/estimates and can verify accuracy as I go along so there is zero guess work on if the information it spits out is correct.

Initially, I wanted to go the 6 or 7 B60s route using riser cables and the likes to make it all work, as I’ve seen the various videos Wendell has done on them and they seemed like a really interesting and cost effective way to achieve good enough performance for this kind of task. However, after looking online through forums and the likes I’m almost more so leaning to a “safer” albeit considerably much more expensive route of running 3090s or something similar.

I’m still very much new to the AI sphere and I would love to help my soon to be father in law with his business and I’m sure I’m missing a tonne of information but please feel free to clue me in if I’m looking at this entirely wrong parts wise. I’m sure I will make more posts in the future about the software/AI/OS side at a later date after everythings been done but in terms of the hardware this is where I would love some input. Thank you!

2

Do you have enough of a prototype of the software you want to build working on a more powerful, not local AI? A lot of a tool like that is going to need to be deterministic, or you risk completely blowing an estimate and worst case losing a customer.

IMO, once you have it working reliably it will be much easier to know what you need out of a local AI. A few hundred dollars for a couple months of a subscription seems like a good entry point ahead of the amount of money you’re talking about for dedicated hardware.

3

I need it to run locally as I’m often working with government contracts or other jobs that require NDA’s and the whole point is to run this bad boy offline airgapped entirely for security purposes. The projects I would be testing with it initally would be purely old ones where I know what the amount of square and linear footage of said mechanical work already is so I can verify and test it out. I’m specifically wondering about the hardware I’m choosing for this and how “nice” it all plays together or if there’s someone more knowledgable in general with this kind of stuff that can point me in the right direction

4

I think without the software and pipeline/infra layout it’s hard to recommend anything.

Nvidia has a “better” ecosystem, AMD is catching up fast though. Intel seems slower still.

Get a single 3090 (or 4090) and work on the software side of things before you make further purchases.

Edit: Maybe a DGX Spark could work depending upon your needs.

5

The DGX spark is probably not a bad idea I was looking at that as well, one of the two draw backs were the inability to repair anything should it crap the bed, and the $7,000 Canadian rouble price. I’ve built and customized more pcs than I care to remember as I used to work for Memory Express as a tech and repairability is a huge thing for me. Not a bad idea though I appreciate the input!

When you say nvidia has a better ecosystem are you just talking about how nicely it plays with hardware/troubleshooting problems?

6

I’m actually tinkering with this very same project but for a different trade. We should compare notes!

7

CUDA is what most ML/AI platforms are built on, and is NVIDIAs framework for utilizing GPU. for parallel computing. Pytorch is powered by CUDA for example.

Not too familiar with AMD, but ZLUDA appears to be the drop-in replacement that allows you to run CUDA powered software on AMD hardware. But there’s a conversion step happening there.

I believe CUDA + NVIDIA is the gold standard at the moment

8

More people work with Nvidia hardware, better support out of the box, better documentation, and in general, a more mature ecosystem.

That’s not to say it doesn’t have any issues, there’s plenty but it’s considerably less than other vendors.

DGX Spark is an ok (albeit expensive) platform. The extra memory/VRAM capacity (although memory bandwidth is low) is pretty useful.

9

Gotcha that makes sense, thank you for the help at this point I think Im gonna be going the nvidia route as much as I would love to tinker with the b60’s but who knows maybe price point alone would make it unobtanium lol

10

Yeah I’d be down! I have to get it all built in the first place to even begin to say anything useful lol do you have anything built yet or are you in the putting together stage as well?

11

That’s what everyone’s saying so far, much less headaches but a hefty price to pay indeed.

12

have you tried playing with any local AI on your current machines ? because i think, you may think differently about the setup after trying to test small models first.

13

I have not! I’m going to be testing that this weekend actually I’ve got a 4080s on my gaming rig so I can run a small model and attempt testing this at a really small scale but my concern was how accurate of a representation it’d actually be to a proper setup

14

There are multiple stages to what you’re trying to do and not all of them need a kick-ass AI to handle. Some parts might be handed off to a smaller AI, some parts could be simply a python script.
Let me see if I got right what you wanted: You want to throw a bunch of PDFs with drawings and dimensions, etc on it. You want AI to read it, get the values and the shape, do math and print out a report.

Things you need from your system:

  1. Accurately reading off the info. You might need a check pass to make sure it read off the numbers correctly btw.
  2. Translate the info into a series of math problems.
  3. Do calculations
  4. Print out a report.

For part 1, you need a vision capable model to make sense of what it sees. What you want here is “good enough”. I would suggest using a Qwen model. Qwen3.6-27B is capable, Qwen3.6-35B-A3B is almost as capable and much faster. Qwen3.8-27B will be released shortly as well (in less than a week from today). I haven’t tested Gemma 4, those are vision capable as well. I would use one of these for extracting, the other for checking that it did the job right.

Part 2: Any decently modern model should handle this. Qwen, Deepseek, GLM 4.7 (5.2 is better with significantly higher hardware requirements) Look for ones with good coding capability. Gemma isn’t that good a fit here in my experience

Part 3: This is just a python script. You can fold it into part 2

Part 4: Any decent model should be able to do this.

What I’d do: Put some credits into Openrouter, setup my workflow, try out the models I want to work with using non-sensitive PDFs over openrouter and then select GPUs accordingly.

FYI: Qwen with UD-Q6 quantization (Q6 is almost lossless), GLM 4.7, Gemma 4: These will run with 48 GB VRAM, ie two 3090s. (Go with NVIDIA - i’m an AMD fanboy but i can’t afford the effort required to make AMD work). If you can, put 1 or 2 more, then you can run multiple models simultaneously.

Speed: Speed is sometimes a consideration but think carefully. If you can leave it overnight and find your reports ready in the morning, does it matter if it handles the job in 1 hour or 6?

Hope this helps.