README_may26.pdf Draft Worksheet - rewritten 18-26/05/08.

The improvements to the design above try to:-

Maximise:- Storage Capacity, availability of data, data restoration and recovery speed, scalability, modularity, cross-platform deployment, resilience and robustness.
Minimise:- Operating bandwidth overhead, impact of network outages, management overheads, support costs.
Simplify:- Development cycle, deployment, data recovery, operation, integration with existing systems.

README_march08.pdf Worksheet - rewritten 24-25/03/08.

DIAP describes a de-centralised, self-contained and managed storage utility.

DIAP describes the emergence of a DVTL - Distributed Virtual Tape Library

The system design uses round robin allocation between ideally three (optionally two planned) nodes either between sites say between offices, homes, on a campus or over WAN's, which could be dedicated to storage or used for existing services, have a round robin synchronisation of FULL - differential backup pools where the source of data ranges from a personal laptop to a file store over unused band-width where the data rate is dynamically controlled, including compression, according to load and availability.

Three for simplicity and because the probability of failure beyond three is so small the extra coding to accommodate more nodes would be self-defeating. In real life use of three nodes for a DIAP pool is strong enough. Chaining together DIAP pools to extend data retention periods is a future aim of the project.

DIAP Rule of Thumb
Circumstantial observation of my email archive, at 272MBytes, having never deleted an email permanently and the file, ../mail, has been in use for 4 years. During this time my available xDSL line Bandwidth has increased, 2004 500MBits/sec to 1GBit/sec, 2008 1GBit/sec to 6GBits/Sec this is about 150% yearly increase whereas my mailbox has increased yearly by about 50%. It is this difference which DIAP attempts to use classing my email record as 'mission critical' - see Diagram to the left. Other record types will increase at different rates, as will bandwidth depending on location, but probably less than the average yearly bandwidth increase. This idea needs expanding but forms the foundation for the usefulness of DIAP, describing a DIAP rule of thumb. DIAP can also be viewed as a technique.

DIAP isn't...
Peer to peer file sharing
A secure file system like OpenAFS
Data Mining
Storage over Network (SAN) iSCSI or ATA over Ethernet
and takes a much broader view than simply creating mirrors.

Open Source and Open standards are a way of developing strong security standards based on the principle that many eyes make strong security possible. While DIAP is not a cryptographic algorithm I feel the principle still holds true for backup software.

Here is a quote from the Open Source Watch UK:-

"Trade-secret cryptographic algorithms have a long history of being both weak and leaked to the public. The tendency to weakness is due to the fact that the only known reliable way to determine the strength of cryptographic algorithm is to have a large number of cryptographers examine it which is not possible to do while keeping it a trade-secret."

oss-watch-uk

© Site construction: DIAP ® Damian Brasher 2008