Limitations in Software

So I read this: “A history of branch prediction”.

If you just went and read it, don’t worry I didn’t understand it either.

It’s a tome of a blog post on how branch prediction works in CPUs. I started reading it and was fascinated in the beginning, but then I slowly began to fall behind in the logical jumps from one concept to the next and was soon lost. It was kind of like math class in high school, where each concept builds on the last, so if you don’t keep up you’ll soon find yourself bewildered and lost.

Anyway, since I quickly found myself completely lost I started skimming. When I got to the end, there was this interesting observation that I really liked. It speaks about the simplifying constraints of hardware design in comparison to software design. I often feel like we need more limiting constraints in software. It feels like we often exert so much time and effort proving "it can be done" in software that we don’t first ask “should it be done?”

I think that a lot of people have an idea that CPUs are mysterious and hard to understand, but I think that CPUs are actually easier to understand than software. I might be biased because I used to work on CPUs, but I think that this is not a result of my bias but something fundamental.

If you think about the complexity of software, the main limiting factor on complexity is your imagination. If you can imagine something in enough detail that you can write it down, you can make it. Of course there are cases where that’s not the limiting factor …but I think that most of us spend most of our time writing software where the limiting factor is the ability to create and manage complexity.

Hardware is quite different from this in that there are forces that push back against complexity. Every chunk of hardware you implement costs money, so you want to implement as little hardware as possible. Additionally, performance matters for most hardware (whether that’s absolute performance or performance per dollar or per watt or per other cost), and adding complexity makes hardware slower, which limits performance. Today, you can buy an off-the-shelf CPU for $300 which can be overclocked to 5 GHz. At 5 GHz, one unit of work is one-fifth of one nanosecond. For reference, light travels roughly one foot in one nanosecond. Another limiting factor is that people get pretty upset when CPUs don’t work perfectly all of the time. Although CPUs do have bugs, the rate of bugs is much lower than in almost all software, i.e., the standard to which they’re verified/tested is much higher. Adding complexity makes things harder to test and verify. Because CPUs are held to a higher correctness standard than most software, adding complexity creates a much higher test/verification burden on CPUs, which makes adding a similar amount of complexity much more expensive in hardware than in software.

I feel like, as an industry (or maybe just as humans), we still haven’t grasped that last part. With physical hardware (really any physical product), it’s obvious that more adjustments, corrections, enhancements, and complexity means more money. But for some reason in software, we seem to have this notion that more adjustments, corrections, enhancements, and complexity is easy (granted there are times when everyone involved knows the arduous task at hand). Too often it feels like an ask such as “just add another form field, it can’t be that hard” are misinformed. Very often it’s not that easy. What’s the field? Is it free-form text? Is it an enumeration? Should it be a string? An integer? Should it be validated? On the client or on the server? What should the form field look like? What should the interaction be like? Does this already fall into an established pattern in our software? What are UI patterns? Do we have them? Why should we have them? What is software? What are computers? What is life? You get it.

Perhaps this is our fault as designers/developers that we haven’t properly pushed back and communicated the complexity required in building software. But I do think that it’s just easier to understand the limitations of the physical because people live in the physical world. They intuitively understand the constraints and limitations of the physical because they encounter them everyday. However, limitations of the metaphysical, like software, are more difficult to grasp. And that’s not just for product managers or executives; that’s for those of us actually writing the code too.