Thursday, July 14, 2011

The big problem with storage

As I continue to ponder the question of how to make real progress with software agent technology and a knowledge web, the big problem I keep coming back to is what I will call "The Big Problem with Storage", namely, how to achieve a degree of persistence in the digital networking domain comparable in robustness and reliability to storage in the physical world, and then to go a leap beyond that to achieve truly robust and reliable digital storage. Ultimately this includes communications reliability as well, but we can tolerate a little connectivity flakiness, but storage flakiness is not so tolerable since it generally cannot be recovered. What is needed is a fully redundant and diversified network storage scheme that is 100% robust and reliable so that people can have complete confidence that information and media stored on a digital network is even safer than the best storage in the real world.
I have written a proposal for a Vision for a Distributed Virtual Personal Data Storage, but it certainly doesn't appear as if even my limited proposal will happen any time soon.
I would add to this requirement that we are in desperate need of connectivity options that are far more reliable than the best offered today. Wired connectivity is probably the most reliable connectivity we have, but has diversity problems. Wireless has greater potential for diversity, but has coverage issues. The sad fact is that if you truly want "always-on" connectivity to your data you need to maintain a local copy on your local computer/network.