Advice on scaling DIY web app built by hobby programmers for serious business

[vivante]

TLDR;

We need to scale up our poorly designed web app to more manufacturing facilities, not concerned about security aspect too much.

More elaborate background:

Our parent company recently acquired new manufacturing facilities and they are struggling to keep up with us. Main problem is lack of efficient setups, tracking and maintenance.

We were in the same boat about 10 years ago, but hacked it away by putting together internal tools and a web app used for manufacturing. It is relatively simple, Laravel based, and keeps track of products, machine setups, maintenance, tooling development…

We are NOT professional developers, just enthusiastic bunch who develop and manufacture tooling for factory floor (like die cutter tools, injection tooling, thermoforming tooling, glue protocols) and needed a way to make things faster, more accurate and more predictable.

Since we cannot provide any reliable security, IT walled us of to our little corner of the network and port forwarded very limited set of machines. We issue Let’s Encrypt certificates via script that requests NAT to our web server, than renews and port closes 1 minute later or on our request - whichever is first.

The app itself is a bit of a horror show, developed in more naive days where we only have relationships between User and Product models, and each machine setup, tools, any documentation, maintenance logs, error reporting and everything else is crammed inside Product. I know - .

I’m inclined to just port it over to latest Laravel since the framework just sort of clicked for me from very start and it’s easy to build stuff even for noobs.

We have tried several off-the-shelf solutions over the years, but every single one required us to adapt to it instead of the other way around. Since we started using affordable Raspberry Pi Picos to gather live telemetry from our tools and count products as they pass by (and amusingly, doing it more precisely than some machines), no amount of SQL could handle that - we needed a time-series database, and most vendors were clueless when we asked about it.

---

I have few specific questions and some general ones:

1. What is your go to rich editor?
   There is a bunch out there, but a lot appear to be online hosted, AI enhanced and similar bull crap. We need to enable better notes, wikis, error reports…

2. How do you allow for reordering things?
   My instinct is to just add a priority field and space values by, say, 20 (e.g., 20, 40, 60) to allow for easier reordering without a lot of updates.
   I need a way for some users to change priority of manufacturing orders for parts. Am I being naive here? Is there a more standard or robust way of doing this?

3. Data separation between teams.
   Currently, we have roles and permissions baked in, and I’m considering adding a “teams” feature to allow some form of data separation—basically, have a Team model to group users by factory/manufacturing facility.

4. How do you approach designing “models”.
   Do you just go by feel? For example, each Product consists of multiple ProductParts manufactured on separate sets of machinery, and each Machine has its own set of data and shared MaintenanceLog, ErrorReport, etc. How do you map this? Do you just use a whiteboard and a set of colored markers, or do you have better ways to do it? We may need to consult other facilities since we don’t actually know their production very well.

5. What would you wish you knew when you only got started?
   What mistakes have you made when you started building apps? What do you wish you had looked into more in depth before starting your next project?

I did ask AI, but at this point I have a feeling it’s just reinforcing my misguided or just plain bad ideas.

I’m looking to get it done in about a month, and if it works well for other facilities great - I get paid more and piss off SAP people at the same time .

Thank you everyone for your insights!

[retox]

I’ll bite… fair warning - I don’t really know php, but I’ve been building custom software for clients for most of my career.

@vivante

1. What is your go to rich editor?

Anything that works with markdown. There are different implementations, but most can do tables, imaging, headers, etc. The best part, is it’s fairly standardized (so you can switch later) and it’s plain-text.

@vivante

2. How do you allow for sorting things?

Might need more context? Are these tables that should be sort based on a certain column, or a report like interface where you ask for data?

Unless the data is large, I tend to lean on using a little plain old JavaScript to tear down, sort, then redraw the table. If the data size is large, then you should be doing it in the database and returning a subset of the data via sorting, offsets and limits (e.g., postgres)

@vivante

3. Data separation between teams.

Your team model sounds reasonable to me - go for it.

@vivante

4. How do you approach designing “models”.

I probably need more context here as well? Are you asking about how to model something known with code or how to figure out what’s going on in general (relationships, processes, etc). In general though, those are the two hardest problems to solve.

Code structure wise, I prefer to use interfaces to tack on common functionality rather than nesting it in a parent-child structure. Figuring out what’s really going on can be rough, but a white board comes in handy here. Charting out relationships and how data moves/transforms are generally where I start.

@vivante

5. What would you wish you knew when you only got started?

Iteration is the most important thing, imo. I almost never get things right first time, so building something quick that ticks some boxes and actually TESTING it is the way to go. If I think about something too long I generally get stuck, but if I build something I end up learning what the problem actually is and then I can either fix, build upon, or try something new. Having automated tests is really nice for this sort of thing, especially if you know what the expected result should be.

---

In general, it reads like you’re approaching this project in a reasonable way. You’ve already hit some pain points, so that’s a good place to refine things or start trying a different approach. If you’re happy with Laravel, then stick with it - rewriting from scratch in something else just gives you a new set of problems!

Last bit of advice - when replacing functionality with something new, do your best to have a way to test the current state of things and build the new thing in parallel. That way, you can run the “same” tests on the new stuff and know for certain that you didn’t break anything unexpectedly. A lot of times, you’ll find something unexpected with the old stuff too…

[vivante]

Thanks for taking the time to look trough and give your feedback!

About sorting, I meant users reordering things in the database, it was a poor choice of words to call it sorting.
Products are manufactured by producing parts simultaneously. Retooling machines for each individual product part is often inefficient. Instead, it’s more efficient to run batches of similar parts from different products together before retooling for the next set of parts. There are other factors too, so issued manufacturing orders should be reorderable on the fly by people who control the process for each product, part or machine.

About “model”, my terminology may be skewed by Laravel, as far as I understand they use standard MVC pattern.

Model represents the data structure and logic of the application. (Laravel) applications often revolve around the model because it handles data manipulation and interaction with the database. For example User, Product, Part, Team all constitute a model handling data, relationships, data transformation on the fly… I meant like how do you draw clear(ish) lines where one ends and next begins. How do you brainstorm, what tools do you use?

Sorry if I sound like a jackass explaining models, maybe that’s what all MVC frameworks do.

[MadMatt]

@vivante

no amount of SQL could handle that - we needed a time-series database, and most vendors were clueless when we asked about it.

Have you looked into postgresql and timescaledb?

@vivante

Products are manufactured by producing parts simultaneously. Retooling machines for each individual product part is often inefficient. Instead, it’s more efficient to run batches of similar parts from different products together before retooling for the next set of parts. There are other factors too, so issued manufacturing orders should be reorderable on the fly by people who control the process for each

So, your model has products that have specific characteristics, that are composed of multiple parts, parts are built using machines by issuing ‘production orders’ by (I imagine) transforming/composing source materials, by teams that operate on machines

Questions:

• are you tracking/inventorying source materials and products?
• are you tracking time necessary to produce a part?
• do you need to manages batches of products as opposed to batches of parts
• do you have to calculate the time it takes to run a batch of parts on a given machine to optimize the order of batches?

Anyway, the ‘sorting’ question in this context is usually handled by creating production orders for the products, breaking the orders down into parts production and associated machines (I imagine you will have multiple machines that will be able to produce certain parts), then assigning a priority/sort order to these lower level machine orders, then providing a means for the end user to ‘manipulate’ this order value
Assuming the latest sort operation is what you were referring to there are multiple ways of implementing the functionality, each with its own benefits and drawbacks:

• use a column in the machine orders table to keep it’s priority order: easy to query, hard to maintain when priorities are being shifted as you’ll need to update multiple records at once, may become inefficient when the queue of orders grows
• maintain the priority order as a property of each machine by using a json object: easy to insert new priorities and shuffle them around, querying it on the single machine stays easy, tricky to access the value when running queries across multiple machines …

As @retox said, it looks like you are already hitting some interesting pain points … my next one to consider would be ‘leveling up’ in software development practices that you will need if you want other sites to use your software and still have a life
The ideal path would be to befriend your IT guys and understand what they would need to make your application part of their standard operations so that they could learn to deploy/maintain it, especially with regards to backups.
Also, keep in mind that as soon as you will have a ‘customer’ things will change, even if they swear they won’t and you will be expected to write the software the way they are used to work as opposed to what makes sense, so you’ll have to skill up in your ‘soft skills’ to be able to cope with the stress/politics around it

[vivante]

@MadMatt

Have you looked into postgresql and timescaledb?

Right now, we’re using InfluxDB and Grafana. We’re also using Pi Pico devices to gather data like temperature, humidity, pressure, and to count parts as they move through different stages in the machine.

@MadMatt

are you tracking/inventorying source materials and products?

Yes, but it is very rudimentary. Eventually, every manufacturing order goes into SAP. I’ve got a wild idea to use Pi Pico to automatically type out manufacturing orders once we can provide all the necessary data.

@MadMatt

are you tracking time necessary to produce a part?

We create graphs that give us great insights into performance, speed, and the time needed for each task down to the individual operator level. Some folks are mysteriously quicker at building certain products, maybe it’s just bad data but overall, we’re about 15% faster. Our manufacturing director and his henchmen assistants use this data to optimize operations. I’m planning to improve the system to make it less reliant on people, but that is a project for another day.

@MadMatt

do you need to manages batches of products as opposed to batches of parts

Yes. Customers order finished products, and we break those orders down into parts. Some products are basically the same, except for the graphic designs printed on them. Sometimes we work on just one type of product all day, but since some machines are faster than others, the printing pressess might start working on a new, unrelated product while the die cutters catch up.

@MadMatt

do you have to calculate the time it takes to run a batch of parts on a given machine to optimize the order of batches?

We wll be looking into it, humans make decisions based on data and graphs. Reducing human involvement at the machine level is a priority. Pi Picos use a mix of sensors to try and figure out things like retooling, setups, stops, and speed drops (indicating QC issues). But it’s still a work in progress and nowhere near finished…

Anyway, it’s not really possible to put manufacturing into strict, rigid definitions when people are involved. A lot still depends on experience, and the director even notices if some operators look sleepy when he meets them in the morning. But we can still give it a shot.

Thank for the insights, you gave me a lot to think about - especially in the politics department.

[MadMatt]

@vivante

Anyway, it’s not really possible to put manufacturing into strict, rigid definitions when people are involved.

Agreed, what works best, with the right breed of ‘smart people’ is if you provide them with the data to enable them to make decisions, augmented by the human factors and experience. The SAP way of programmatically enforcing a workflow created in Germany in the 70s to ‘optimize’ one specific type of production processes to everything else is a load of crap, in my humble opinion

So things like visual cues as to how much a machine will be working at capacity given the current sort order, and a wider cue as to how much shifting an order priority will impact other products will go a long way to help people improve their and their assigned machines productivity …

[MadMatt]

@vivante

Right now, we’re using InfluxDB and Grafana

Influxdb (disclaimer, haven’t been using it since a couple years, so things may have changed) is perfect for managing timescale data, a little trickier to use if you need to manipulate/aggregate the data in a dynamic way … timescaledb is just an optimized storage and retrieval engine on top of standard postgres, so you get all the benefits of a relational database (that can still use unstructured documents when needed) plus the ability to efficiently store/retrieve the time series data …
Influxdb in my experience is also a pain when it breaks, as long as you don’t care about your historic data it’s not a big deal, but if you do then you should pay attention at having a proper cluster setup and backups …

[teresko]

Honestly, the fists step would be: get rid of Laraval. It is nice for making small projects, but due to use of ActiveRecord antipattern, it tends to get extreme performance issues and malformed DB architecture.

1. Not important, I personally use VScodium and my coworkers have PhpStorm.
2. First idea - keep the order in a separate table, but the question sound very “domain specific”.
3. I think the fancy term for what you want is “hierarchical role based access control”, basically you link permissions to groups. Then each user and group can be part of multiple groups.
4. By implementing data mapper pattern.
5. Global state is source of all evil.