Throwing in More Hardware?

In a recent blog entry, Jeff Atwood makes the case that “hardware is Cheap, Programmers are Expensive”, basically arguing that in a lot of cases, it’s cost-efficient to add cheap, faster, hardware to a performance problem rather than investing time to actually enhance the code at the algorithmic and implementation level.

I do, however, disagree with him on many points.

First, throwing in more hardware makes sense only until a certain point. For exactly one project and one installation, it may make perfect sense to just add an extra blade to keep the system running without major tweaks. After a number of blades, it stops making economic sense for your costumers. Clearly, the more blades you ask them to buy to run your system, the more they’ll look at the competition. Why buy the solution with 32 server blades when there’s one with only 4? Second, you (and your customer) will quickly run into power consumption and heat dissipation problems. While this sounds all cute and funny when you have a 5 blade rack, it’ll be quite less so when you get your local hydro company to install a 500KW transformer in your backyard and that half of it will be for powering the cooling systems.

Second, adding extra hardware (or waiting for Moore’s law to provide faster computers) may not be realistic at all for your customer. If your software stops being cost-effective for your customers, they’ll turn to other solutions (unless you are already in a dominant position and keep your customers in some kind of lock-in). Indeed, it’s not realistic to ask your customers to get a quad-core computer to use your spreadsheet or word processor application. Your only solution is to provide better software.

Providing superior software means having superior design from the start. And I’m not talking about premature optimization where all kinds of wacky micro-optimization hacks are pointlessly included in early development. I am talking about committing to better programming, that is, taking time to think out the right abstractions, taking time to think how your tasks can exploit thread– or core-level parallelism (or even outright distribution), taking time to carefully select the right data structures and algorithms for your use-cases, making room for hardware acceleration and advanced instruction sets, fighting feature creep and over-engineering, and even resorting to use-case directed minimalism.

4 Responses to Throwing in More Hardware?

  1. Lodle says:

    I think both you and Jeff have valid points, however, they are valid for different situations. In Jeff case, the software is running internally (and the companies servers, not the client) and thus hardware is cheaper. On the other hand, your talking about running the software client side, and all the points you raised apply.

    • Steven Pigeon says:

      I think even if you run it internally, it may be problematic to maintain a large-scale installation to service your internal needs or your customers’. It’s a delicate trade-off situation where you must balance between Jeff’s solution (a relatively low load and saving work hours by just adding a few more CPUs) and mine (where you’d better have better software because just adding more CPUs is just not feasible).

      Of course, you may have to do both: I don’t think Google just keeps piling up blades without tweaking its software. I think they do both.

  2. zakimirza says:

    IMHO, there is no reason why you should not squeeze the hardware to its best.the days are here when we rely on garbage collection and fresh grads dont know what a heap or stack is, or what dynamic memory allocation really means. True that you have 4Gb of RAM, but that doesnt mean a simple IM software should take 100mb of it. To what value? What is your audience? its normal people, so normal people sometimes have 512mb ram too.

    And like you said, then people look for alternatives.

    In pure terms of software, the whole scenerio is an ill assignment of responsibilities. It is the developer’s responsiblity, and thats what they are paid for, to write good code in the first place. Its his responsiblity to ensure performance. Now, what happens is that CTO intervenes and says okay fine, lets add more blades to this. CTO is not responsible for post-development performance increase, though he should have seen into alternatives before.

  3. Steven Pigeon says:

    The fact that most freshmen do rely on the garbage collector (especially in Java) is one huge problem. Not only are they not aware too much of what’s going on behind the scene, the garbage collector itself may be problematic. I know that while it may be atypical for applications in general, I have seen a JVM garbage collector being unable to reclaim massive amounts of memory because it did’nt have time too. I am not a Java programmer but I often discuss with guys that are; and from what I understand, the JVM as two modes for garbage collection: workstation and server. The workstation profile tries to reclaim as much memory as possible to the detriment of computation speed. The server profile tries to give the most speed to the application, to the detriment of memory.

    Because the JVM garbage collector runs periodically and tries to reclaim memory. However, if the program is computation-intensive, the JVM garbage collector gets a tiny time slice and basically gives up too early in each iteration, leaving the JVM heap to grow rather than reclaiming more memory.

    As for “their responsibility to write good code in the first place”, I totally agree with you. However, reality being suboptimal (to say the least), very often programmers are hard-pressed to get something done quick as a prototype, and the prototype is deployed as-is. I know of almost no business that will build a prototype version, test it, understand where the problems are (in terms of usability but also in terms of scalability… if it works for 100 records in the database, does it still work with 10 millions?) and then re-implement a version 1.0 with the problem addressed and deploy this (new) version.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: