No one programming language is prefect for every situation, but some are perfect for specific situations.

Why should you decide to use something?

This is more important than any other decision you make in a project. If you pick the incorrect technology, the rest of the project is at best going to be a slog and slop in production or at worst doomed to failure.

It should be very easy to choose the correct technology to use what you know.

This is a trap that many people fall into. The proverbial "to a hammer everything looks like a nail " trap.' Just because you "know" something does not mean it is the correct tool for a given problem, and if you really "know" your tools, you know when this is true.

The actual problem is too many people do not really know their tools to the depth they need to honestly evaluate them, and thus themselves for using them.

How we got here.

You only really "know" something when you know what it either can not do or is completely unsuited for a given problem.

JavaScript is the current perfect example of this idea and people trying to use it for everything, even things it is the absolutely wrong choice because that is all they know.

NodeJS is almost 100% the reason for this. Previously it was Perl and before that it was Shell Scripting, mostly Bash. Ruby tried to replace Perl but failed as Python beat it out in mindshare and rightly so.

I leave off Python from this list of shame because Python was designed as a support language for other languages, specifically C and C++. The community has mostly stuck to its intended design and very open about writing performance oriented code in more appropriate languages and then calling that code from Python.

When NodeJS first came out, I saw it for what it was. A framework already existed in Python called Twisted, that did every thing NodeJS did. I was unimpressed. I knew it was not going to scale horizontally, the JS runtime it used was single threaded, just like CPython, which was the only production grade Python runtime at the time.

Much like Python, Twisted worked great for what it did. You got exactly what was promised. The problems started when you had to stray past these capabilities. Twisted was a solution to the problem that CPython was, and still is, single threaded.

Twisted introduced an async facility that allowed it to appear that Python was not single threaded and provided features for you to manage this illusion fairly well. With these new features, came new complexities but if you knew you needed to use Twisted, you should be able to understand these new concepts fairly quickly.

When I saw the hype around NodeJS and researched it a bit I realized that NodeJS was the same solution to the same problem with the same limitations, but in JavaScript. And JavaScript was not as capable as language as Python at the time.

NodeJS is a more appropriate technology to use than Python/Twisted in 2024 simply because it is the VHS of single threaded language async solutions. Twisted is the Beta, first to market and at the time a much superior solution in almost every way, but NodeJS rode the JavaScript hype mindshare wave and has more documentation, more support and extensions/features because of it.

Things work until they do not anymore.

Note: Replace Python/Twisted with JavaScript/NodeJS and Hardware/Cloud in the following and you have the same problem and almost the same outcome for 2024.

It was about 2007 and our Python/Twisted project worked really well for what it was intended to do, on the hardware we intended it to run on. Then a Sun sales representative took someone at the company golfing and to an expensive dinner and all of a sudden we were switching from relatively beefy Intel Xenon W5590 based servers that easily scaled vertically to relatively weak Sun Niagara T2 servers that were supposed to scale horizontally.

So when were were scheduled to go to production, we had a bunch of them available because they were not suitable for any of our existing applications. All new applications were required to be provisioned on all these idle servers. This should tell you everything you need to know about what the immediate problem was.

Our application was an I/O bottlenecked application. It just read some data from an RDBMS on first access and then cached that data and updated the server occasionally. It easily scaled vertically, just add more RAM and you can service more clients since most of them were idle. These Intel Xenon machines only had 4 cores and 8 thread but that was plenty, they just needed fast I/O and lots of RAM since the CPU was extremely fast for the time.

The Niagra servers were designed to be web servers, which in many ways were appropriate for our task. Lots of sporadic short lived connections that were I/O bound. The ones we had were about the same cost as the Intel machines were had been using, but were speced with less RAM. This and the fact that each core was approximately as powerful as a Pentium Pro 200 made them the absolutely wrong choice for our Twisted/Python application.

These machines were designed to scale horizontally. A single threaded application no matter how async only scales vertically. These machines were a terrible choice for this application.

I did what any reasonable Principal Engineer would do, I improvised.

The RPM already deployed a start, restart, stop script to the was modified to use a script that started one instance of the app per core on the current machine. This was an incredibly hacky way to solve this problem, but we were not given any time to devise a better solution before it had to go to production. We told ourselves, "this is fine" it will get replaced soon with some more appropriate, I had just gotten comfortable with Erlang at the time and figure, porting this simple app would be a good exercise.

It was, I re-wrote the application in Erlang, it preformed orders of magnitude better and only one instance was required. When we tried to get this version moved to production after testing it reality set in. Management told us we were correct "this is fine", the hacky solution was another teams problem now and we were not allowed to deploy the Erlang version until what was in production broke. It never did, it had 100% uptime for the almost 3 years that application was in service.

I just did not know enough Erlang to write the application eight weeks earlier or I would have just written it in Erlang to begin with. The fact that the other Principal Engineers I worked with tried to convince management that the Python version would never work and after they said that Erlang was a bad choice. They wanted to use C or Java and estimated a 6 - 9 month design and architecture timeline and about as much development and testing. I wrote 2 versions, in series, while running and overseeing three other projects in a span of about six weeks, eight if you include two weeks of learning Erlang.

Constraints are your friend

What it will not do are constraints that actually free you up to come up with better solutions most of the time. Novel solutions, not "clever" ones, that you come up with from working around arbitrarily imposed constraints or limitations forces you to approach problems from directions you would normally not consider.

This is how technology gets used for things they were not designed for, hopefully in a good way. It also forces you to re-think your ideas about what you "know" about solutions to similar problems with slightly different constraints in the future.