Design Patterns & Tech Debt

Development Software Development
1

Hey all, it’s been a while since I’ve been on here! :stuck_out_tongue:

Introduction

So, I’ve been working on some real enterprise projects, although, I’ve been experiencing one reoccurring thought:

How can I design this software to ensure that the technical debt is kept to a bear minimal?

I think that I feel the needs to ask this purely because of the fact that I’m only just a mid-level developer, I mean let’s be honest, regardless of one’s skill & knowledge, experience pays off massively with discussions such as this.

I mean I do the obvious, using clean, tried & tested design patterns, solid principles, simplicity, etc. I also know that tech debt is inevitable, regardless of how wizardry your skills as a developer are, any tech project will encounter some degree of tech debt at some point or another.

Specific Example

One specific case I have in mind is how one developer that worked on my current team before I joined, he made an app using Apache Camel, but as it turns out, it’s an overkill use case. Apache Camel quite literally just implements the REST API & links the processors together, personally I think of it as a chain of responsibilities at most in this specific case.

However, whilst I’ve established an appropriate design pattern to utilize in place of Camel, I wonder whether it would be better to utilize an iterator based solution or an array list based solution. Personally I favor the list solution since you can see the precise flow of the business logic, where the chain is initiated.

I mean if the business logic needed to add more complexity, I’d think of adding something along the lines of a factory pattern in-front of the different chain of responsibility implementations.

P.S. Even though I’m using Java, I try at all costs to avoid side effects, while using defensive principles.

If you like, I can also include some code snippets to provide a precise example of the logic I’m referring to? :slight_smile:

3 Likes
2

oh boy! quite a question you have there.

tech debt is a very complex topic, and some level of tech debt is acceptable, think of tech dept as this tech debt conceptualizes the trade-off between the short-term benefit of rapid delivery and long-term value.

tech debt has the stigma of being bad and evil, we all been tough at some point that quick hacks are bad and we should have an elegant working code.

that is why a lot of startups play with this, getting fast to market is very useful as it will allow you to test your product and iterate until you get what the customer needs. in this case you can se tech dept as a tool, there is usually a couple of levels of dept

level 1, is manageable. it will hurt but can be worked with:

  • badly designed code
  • lack of tests
  • no code review proccess
  • no ci/cd
  • no docs

level 2: this is a gamble and can hurt the project to the point of killing it:

  • unstable core technology
  • using unmaintained oss projects
  • badly designed algorithms and data structures
  • no design phase for features
  • bad choice of communication protocol

this are examples of tech dept I saw in the past and present. the most important part of working with tech dept is knowing you have and plan for how to work around it.

this get us to the point were engineering can have a lead that is inadvertent and reckless, they just don’t know any better, they simply are not aware their actions are piling up debt, this is the dept you should never take.

On the other side of this coin you find engineering leaders that are prudent and deliberate, they make conscious decisions to take dept, usually, because there is a business purpose, they plan to pay back that dept later, this is the kind of dept you want to take.

how to repay tech dept?
repaying the tech dept is the most important step in the process, it is highly unlikely to convince management to let you stop working on new features and bug fixes to repay debt.

here is where you leave the campground cleaner than you found it this boy scout rule should apply every time, every time your working on something try to find an instance of bad code and clean it up. this usually helps increase the quality of code while delivering value to customers. (any developer that pays dept and cleans up messes without extra resources is very valuable)

Another good practice is repaying a small amount of debt every time you finish a task quicker than you thought.

I for one, every week when we plan our sprints I reserve 5 to 15% of the sprint points to tech dept maintenance (there always a ticket for some of this in the backlog)

it is also a good idea to start introducing a process that will keep dept in control, PR reviews, design phases, test coverage, and make sure nothing is committed until it is thoroughly tested (meaning test suites are written and passing before merges.)

Tech debt is part of every engineering group’s life and is not something you should avoid, it is a tool used to achieve a specific business purpose, and as with any dept it should be repaid in time.

6 Likes
3

You cannot teach experience.

And from reflection on past experiences comes wisdom that is needed to apply better solutions to an issue. In my experience 75% to 95% times/cases/people does not even reflect on past experiences.
That is universal in everything from programming to simply living.

The remaining 25% to %5 of times/cases/people are still pron to yet another cardinal mistake - to implement overkill solution because they were hurt in many past experiences.

Yet another cardinal mistake is to choose solution and keep to it no matter how bad are the results or how much the environment/circumstances have changed (e.g. requirements).

1 Like
4

I agree with @lacion, technical debt it something you need to manage, rather than avoid, since it is inevitable.

As someone who manages, from a technical perspective, a couple of large Java “legacy” applications and is responsible for both reduction in technical debt in these, as well as avoiding it in new applications, here’s a few things that come to mind (in nor particular order):

  • document standards for everything: for code style, for branch naming, for commit flow, for commit messages, for whitespace, and for anything else that might be relevant. And enforce them . It does not matter whether you, for example, pick spaces or tabs, but pick one, and stick to it.
    Though if there are “industry” standards, even de-facto ones, stick to them unless there is a very good reason not to (eg. a corporate standard already in place).
    Consistency is key.
  • enforce good commit messages. A good commit message explains the why of a change. Nobody cares what the commit did, people can read the code to figure that out, but they can’t read the mind of whoever wrote it to see why it was written the way it was. This is extremely important for long lived projects and a skill 90% of the devs do not seem to master, or even understand the importance of.
  • code review. If it all possible do cross team review with different projects (my rule of thumb: one other dev from same team, one senior dev, one dev from other team, preferable to have more reviewers than the bare minimum, leave enough time for everyone to have a reasonable chance to do a review before merging. Expect everyone not on holiday to review everything for at least his own team, ideally).
    Note that here I tend to consider each of the three categories as one vote, so the team gets one, the senior devs get one, and the “external” dev gets one. Need one vote from each, ideally, to pass.
    The dev from the other team is there to make sure the code is clear. (S)He does not know the discussions had within the team about the change, so if he can’t figure out what the code does with the information available in the code, or commit mssages, then the change is bad and needs fixing
  • modularize the codebase. Doing anything in a smaller codebase is just easier (and safer, even if you have tests)
  • be very careful with leaning on large frameworks (eg. J2EE, Spring, Hibernate,…) as they can make large codebases a nightmare to bring up-to-date (ask me how I know)
  • document your APIs. If it is public it needs JavaDoc (has the convenient side-effect that you’ll see a lot less public methods because devs hate writing documentation)
  • per the above, document what a method does if you shove null, or other “weird values” (eg. empty lists/arrays) into it (and have tests to prove it)
  • automate. especially enforcing standards (eg. use server-side commit hooks to enforce commit-message formatting, run code through SonarQube to catch common bugs), this filters out most obvious problems before they get to reviewers. It also saves you from having to choose between seeming like a nitpicking jerk during reviews when some people consistently deliver badly formatted code, or letting crap code slide. A computer rejecting things is accepted a lot better than a human doing so.
  • make sure that your automated quality checks can’t be bypassed (this means you need your management on board, to withstand customer pressure, from within or without)
  • code reviews are a two way process, where mutual understanding, learning, and betterment of the overall codebase is the goal. They are not an exam where a few people are “judging” someone else’s output.
    Note that learning can (and should) be by the reviewers as well. This means that getting junior devs to do reviews is a very good thing.
  • avoid constructs that are hard to read (eg. making people think about the difference between ++i and i++). Simple and readable does it. If readable is not an option (because of, for example performance) document the unreadable mess, in the code.
  • premature optimization is the root of all evil
  • external documentation is mostly useless. If it concerns the code, it needs to be either in the version control system (commit messages), or in the code itself. Everything else is liable to go away at some point (that includes ticketing systems)

Just a few things off the top of my head. If more specific points come up I can probably give some more specific comments.

5 Likes
5

that is the soul of leave the campground cleaner than you found it

very good advice.

6

I totally appreciate everything you said, a lot of those processes/practices I follow to a T, I was just wondering if anyone had a clever & wizardry technique.

Regarding your comment on frameworks… Trust me, I also know that pain, I’ve previously worked on converting an enormous monolithic EAR project into a bunch of micro-services, it was not a fun experience to maintain the precise business logic while using newer technologies, etc. It’s also taught me to never rely massively on a framework for anything, have some layer of abstraction, that way if you want to change from one framework to another the pain will be minimised!

Regarding readability, that’s one thing that I’d go as far as saying I borderline specialize in, using tools like Lombok to ensure the code-base is as readable as humanly possible.

As for documentation, in my current role, that’s a fun topic, there is very little to say the least, but then again, dictating practices, that’s far out of my control, my manager & his manager & so on just continue to argue or debate regarding a number of these details… At the very least, I’ve introduced tools like Swagger, which is a start to some extent I guess, but still far from what you’d expect at a large company… I’ve also introduced as standard practice, forcing other devs to use block comments on all methods in all classes that uses business logic, i.e. it would be a bit overkill to comment a getter/setter IMO, but you could always argue otherwise I guess, but in any case, since we’re now using Lombok, we don’t need to worry about it so much…

As for external documentation, for my team, it’s actually somewhat useful in a number of cases, since there’s a lot to take into consideration that goes beyond the scope of the code base, such as business policies around a specific set of functions, infrastructure & relevant rules, environments, etc…

7

I feel the trick is not just knowing the techniques, but applying them consistently, and knowing when, and when not, to allow exceptions.

And also, of course, managing things for the long haul, both from a project perspective as well as from a technical perspective. Will there be an upgrade path for this in 10 years? Will it still be around? If not, how easy is it to move off of it?

Personally not a big fan of “magic” to fix the shortcomings of the language, but I guess this is easier (especially in the “convince management” department) than just using a better programming language that doesn’t suffer from all this boilerplate, like Scala.

Depends on how hands on your management is, you could always suggest writing a proposal for standardization. It’s what I did, then had it vetted by the team and then enforced by management.

My main concern with tools like these is always their longevity. We have some relics of diagramming tools and the likes in our codebase from over a decade ago, and the tools are long gone.

Well, that’s architectural stuff, kind of hard to keep that inside a repository :wink:
Though I do try to force a Wiki for these things over Word documents or the like, that way things are centralized and everyone has access to the latest version, always, with documents you just end up with lots of different versions floating around eventually.

8

tech debt is the other side of tech investment (ie. spending the time to implement something new).

It’s simple really, you can never get rid of tech debt, and can never get rid of having to maintain the code, it’s always going to be there until you delete the tech, but as long as the payout from the investment covers your maintenance costs you’re good.

Therefore you need to just not ignore the debt, and be able to compare the cost of maintenance debt, with the cost of new development.

As a rule of thumb, for basic cost, I use a my own rule whereby I observed software systems in healthy organizations getting completely rewritten from scratch every 2 years.

Thereby, if the future value brought on by the thing you’re planning to write will not cover the cost of rewriting it again, in 2 years, don’t do it.

On top of that, you might have other concerns. e.g. how much human time will the thing I’m writing save. How much money will it bring, what is the cost of hiring and retaining an engineer. What is the amortized cost of 2T of ram in our cluster over the next 2 years, and so on.

Also, when I say “rewrite” it could be a refactor, it could be “making sure it keeps building” or just reading code to try and remember how it works and re-evaluating if there’s anything to do.

If you’re writing down the design it’s always useful to have “alertnatives considered” section where one of the alternatives is “do nothing”.
It’s also always useful to mention the “expected ongoing maintenance cost” or something like that in that same doc.
Everyone reading the doc who has a clue will understand those numbers are completely made up, but after a lot of code and designs like that you’ll be able to see some patterns.