Over the course of my career, I've seen a lot of engineers come and go. This is not uncommon in the tech industry, but it produces a very unfortunate situation for the people who stick around - complexity increases, but expertise decreases. Every tool, framework, library, and line of code that is introduced tends to outlast the person who introduced it.
This can be (and hopefully is) mitigated with documentation, but documentation is not a substitute for deep knowledge about how something works. At its best, documentation will send you down the right path, but it's up to you to fill in the inevitable gaps.
The solution to this problem is obvious, although difficult to follow in practice - introduce less stuff. As engineers, we tend to always want to introduce more stuff. REST got you down? Use GraphQL instead! Need to store arbitrary data in your database? NoSQL to the rescue! Need to send messages across applications? Time to onboard Kafka!
What's wrong with Kafka and GraphQL?
To be clear, these are only examples. They all exist because they solve real-world problems, and they each have their place. But adding them (or any other tool) to your stack is not free. Each comes with onboarding costs, monetary costs, and maintenance issues. Eventually, someone at your company is going to need to know how these things work at a low level. You might have a Kafka wizard on your team right now, but will they be there in two years? Maybe, maybe not. But even if they're not, chances are Kafka will still be around. And that is true for the vast majority of the stack.
Ask Yourself: Am I solving a real problem?
Have you ever written code that works perfectly fine, but still leaves you with a nagging feeling? An example might be using an ORM that doesn't allow you to write the most performant SQL query, perhaps even forcing you to run multiple queries instead of one. There's nothing wrong with trying to find a solution to a problem like this, but you have to be careful. How far are you willing to go to solve this problem? Is this even a problem that needs to be solved?
Are you doing this for your end users, or are you doing it to make yourself feel better? If you're not careful, you might find yourself burning the next few days test driving alternative ORMs, perhaps even going so far as trying to convince your team of the virtues of adopting a new one. But you need to be able to step back and see the big picture. If everyone on your team likes the current ORM and your less-than-perfect code is still good enough, is introducing a new ORM really the best decision? Is a new ORM a reasonable solution to your original problem?
Can we solve the problem with tools we already use?
Given the astronomical number of tools and libraries that exist today, there is very likely a tool out there built to solve your exact problem. And when you have a perfect match between problem and solution, it's hard not to instinctually reach for that solution.
But are you reaching for a new tool because you actually need it, or because you've heard great things about it and have been looking for a chance to play around with it? Questions that you should ask yourself include:
- What is the surface area of this new tool - can it be used in a small corner of the code base to solve a specific problem, or does it span across the entire stack?
- Is the team excited about using this new tool and eager to learn how about how it works, or is one person doing the lion's share of the work, shielding the team from the underlying complexities?
- Most importantly, is there a good enough solution using your existing tools?
That last question is the most important one to ask. Before reaching for a new tool, you should spend time determining if the tools you're already using can solve this problem for you. It doesn't have to be a perfect solution, but a good enough solution with existing tools is almost always better than a perfect solution with a new tool.
I have been guilty of every misstep outlined above, and I've seen others do the same. I've spent plenty of time imagining how some new tool was going to make a whole host of problems go away. What I didn't spend enough time doing was examining the downsides of introducing such a tool and asking myself and the team if it was really worth the cost. And as I've seen over the course of my career, those costs add up.