Image: Grap, Wikimedia Commons
In 2009, A&E Television Networks began broadcasting a reality-based program, Hoarders. Each week the show examines the case of an individual who suffers from a symptom of obsessive compulsive disorder or obsessive compulsive personality disorder that causes him or her to experience extreme distress at the prospect of discarding specific items, or in some cases, anything at all.
According the show's website: Hoarders not only captures the drama as experts work to help each person get on the road to recovery, but also highlights the individual's inner challenges and triumphs. Although cleaning marks the first step of tackling this disorder, success is not definite. For some individuals, throwing away even the tiniest object is so traumatizing that they are not able to allow the cleaning process to continue, no matter how it may impact their lives.
Extreme examples included episodes chronicling a woman with 76 cats, a man who saved every copy of National Geographic ever printed, and a woman who could not part with a collection of 50,000 dolls. The audience watches in amazement at a person who is seemingly unable to make a rational decision to throw junk away in order to improve his or her life.
Unfortunately, the phenomenon of this condition is not confined to individuals, but wreaks havoc on large organizations as well. Just as this condition carries significant consequences for people, the toll it extracts on corporations is equally destructive, if not readily apparent. There are significant costs associated with the tendency to save nearly everything, regardless of its value.
Indeed, the failure to dispose of anything is itself a decision that all data is of equal value, imposes the same risk on an organization, and is justified by the costs imposed on the organization. Much like hoarders, an organization cannot be as productive in this state, because no one can find, use, or protect what is actually valuable to the organization.
DO THE MATH
A recent article in Science magazine, "The World's Technological Capacity to Store, Communicate, and Compute Information," stated that collectively we have accumulated 295 exabytes of information. While legal and corporate IT departments are finally getting a grip on managing terabytes and moving on to petabytes, exabytes are lurking and ready to be thrust into reality. According to the "Gartner IT Key Metrics Data 2012" report, the total cost of storing and managing a petabyte of information is nearly $5 million per year. Loosely, this translates to about $5,000 per terabyte. However, this is only part of the story. If we assume an organization that stores 10 petabytes of data might have about 1 petabyte of email throughout its IT infrastructure — including production email, PSTs, or Lotus Notes files (NSFs), or other loose email files on individual hard drives or file shares — and an email archive. (We are purposely avoiding the issue of backup tapes.)
Further assume that this size of organization might pay upwards of $20 million a year on electronic data discovery. From this figure, it is possible to back into the EDD "tax" that must be assessed to a given terabyte of data from a target-rich environment such as email. The RAND Institute for Civil Justice issued a study report, "Where the Money Goes: Understanding Litigant Expenditures for Producing Electronic Discovery," this year that showed a median cost for collection of $910 per gigabyte, $2,931 per gigabyte for processing, and $13,636 for reviewing a gigabyte of data.
Plugging these numbers into the $20 million spent for a corporation above, we arrive at a "probability of review" for a given message of about .1 percent. Therefore, for every terabyte of key data, we see an EDD "tax" of 1 gigabyte or about $15,000 when the costs of collection, processing, and review are tallied. If we add the $5,000 in hard costs from the IT figure above, we arrive at about $20,000 per year.
However, for the purposes of this analysis let's set aside the EDD costs. Finance departments often struggle to properly account for projected costs that are probabilistic, discounting these costs to the "best case scenario." Considering solely the IT costs of $5,000 per terabyte, some rather ominous mathematical calculations begin to take shape.
According to the Compliance Governance and Oversight Council, the amount of data that an organization could defensibly dispose of is staggering. The Council's postulate is that information must be retained for three reasons: 1) it is subject to legal hold, 2) it is subject to a regulatory requirement, or 3) it is valuable for business purposes. According to CGOC, about 5 percent of information is subject to regulatory obligations, about 25 percent of corporate data is of business value, and only about 2 percent is subject to legal hold.
Subscribe to Law Technology News
-
Dan Woodard
The author asserts that all stored data costs $5000 per terabyte-year to manage. If this were true Google would be bankrupt overnight, since Google charges less than $1000 per terabyte-year for secure, instantly-accessible, backed-up cloud storage ( https://developers.google.com/storage/docs/pricing-and-terms ).
Simply reviewing emails and deciding which ones to delete or keep costs a company about $.40 worth of loaded employee time per message, or about $1 cent per kilobyte. This adds up quickly, $10 per megabyte, $10,000 per gigabyte, $10 million per terabyte, pretty steep when a 1 terabyte drive costs less than $100. Then there is the cost of not having the data accessible securely, online, from any location. Copying emails to the desktop PC costs more than keeping it on the server since every copy of mass email must be separately stored by every user.
Users should be encouraged to delete emails they don't need when this makes it easier to keep information accessible and organized. But to require them to delete files to stay within arbitrarily small allocations of storage space wastes time and money.
Finally, if you were representing a client who made a legitimate request for information and it was denied with the claim that it had been deleted, would you accept that result without action? Surely it would appear improbable and make it appear that the organization had something to hide.
— Dan Woodard
-
Paul Robinson
I once pointed out to someone that when he said he didn't want to admit to his girlfriend, if he was collecting MP3s, that he'd be a data horder. I said it doesn't matter, an associate of mine had his collection of 500 CDs, (which would be between 500 and 10,000 songs depending on how many per disc) and it took racks the size of a couch. Today, we can store 50,000 songs on a box the size of a textbook and the cost is around $100. The cost is so low that a gigabyte, which could hold anywhere from 50 to 500 songs depending on compression, is about 10c worth of storage, and for individuals, until you get above a block of 9 figures of data (100 meg) it ain't even worth bothering to take the time to check because that's only 1c worth of storage.
If you've got, say, 4-5000 songs on storage, you're never going to listen to all of them at any time but you might listen to any of them at any time, and unless you can actually monitor usage, there's no real way to tell what you would or wouldn't listen to, so basically you have to keep everything. Same thing for corporate documents. You're never going to use all of it, but you'll use some of it, and in rare cases you might go through all of it, but absent usage monitoring you do not know what is or isn't going to be used, is being used, or how often.
With large corporations, all they're going to recognize is the cost per terabyte (now down to less than $75, 2 tb drives are $150, retail) and forget the cost of managing that storage. Not to mention, is it properly indexed, or are you simply keeping 50 copies of the same data, and some of it is not the same, and you may not know which is the most accurate or most definitive copy of the data. You might even grab the wrong data if it's a dataset of an entire database and you use an older backup or something not marked as not the latest copy.
-
David Obarowski
Thank you, Jake and Anthony, for a particularly insightful article. Thanks especially for the entertaining, but still revealing, comparison of individual and corporate "hoarders". Here's a follow-on thought with this regard:
The individual hoarder knows exactly what s/he owns, what's in the "boxes". The irrationality lies, not in ignorance of what s/he owns, but in the inability to apply distinctive values to the stuff in those boxes. It all is valuable and needed.
The corporate "hoarder" on the other does not completely know what's in its various data containers, the "boxes". The company then over-retains in the very rational context that the unknown contents of any given "box" MAY be valuable or needed.
Reality television shows invariably have happy endings...the irrational hoarder is "cured". The rational corporate hoarder however can never be so "cured" until it is enabled to know what is in those "boxes".
Comments are not moderated. To report offensive comments, click here.
Reader Comments