Automated transparent scalability

It's amazing how bad many software system designs are. It's as if nobody had even imagined that scalability wouldbe important. The problem facing us is that people do recognize that sclability is more important now, but the approach is almost always the same: throw more bodies at the problem, rather than to pursue more modern approaches to design of the software itself.

Sure, companies like Google and Amazon have the big bucks to throw at the problem and burn through prodigious numbers of talented software professionals, all in a struggle to compensate for the fact that we're using really bad software system designs.

The key technology needed for future software system design is that scalability needs to be done transparently and it needs to be be automatically, at the system software level. If a developer messes up, we need to have software in place that will detect such mistakes.

The solution to scalability is not more hardware, more money, or more people, but simply better software tools and more enlightened software design. The measure of a good developer is not how many lines of code they write or how complex their code is, but how simply it goes about solving the problem at hand. A better software infrastructure architecture will eliminate much of the burden that is currently misguidedly placed on the shoulders of application developers.

We need to shift more of the current application burden back where it belongs, on better system software.

One of the keys to enhancing application performance is to exploit the parallel and concurrent processing that is inherent in networks. Applications can sort of do so via brute force techniques, but that makes the applications more unwieldly, fragile, and less easy to update. Only by shifting the burden back onto system software, where it belongs, can we ever hope to dramatically reduce the cost to design, develop, deploy, and maintain sophisticated distributed computing applications.

In short, scalability is of paramount importance, but it needs to be done transparently and automatically, and not dumped in the laps of applications developers and systems administrators.

-- Jack Krupansky