The technology community is all abuzz about this year's new, new thing: Big Data. What is Big Data, and should in-house counsel even care? Yes, I believe in-house counsel does need to know what their company is doing. But first an explanation of Big Data is warranted.

At the simplest level, Big Data is the ability to analyze very large amounts of data that may be stored in different formats or come from different sources. Historically, data have been analyzed on a single system, typically within a single application such as a relational database. But the sheer quantity and variety of data pouring in have increased dramatically, straining the ability of single-system applications to hold and analyze these large quantities. New technologies with strange-sounding names like Hadoop have been developed to analyze very large, dynamic data sets that are frequently updated.

Real-world examples of Big Data are numerous. For example, Wal-mart Stores Inc. collects and analyzes millions of customer purchases every hour, looking for buying trends. Credit card companies constantly analyze tens of millions of incoming transactions searching for indicators of fraud. Social communication applications like Facebook and Twitter generate massive amounts of message traffic that, once analyzed, can be valuable to online advertisers. Perhaps the best example is Google's search engine, which tracks, indexes and ranks billions of constantly changing web pages, performing this analysis across thousands of computers in Google's data centers located across the world. That's Big Data.

Big Data has replaced social media as the new, new thing in the technology world. And every technology vendor that remotely touches data is repositioning itself as a Big Data company.

Should inside counsel care? Yes, because Big Data has the potential to introduce new compliance and legal risks. A few of the risks include:

Privacy: An advantage of Big Data management technology is that it can analyze information from multiple systems, public record repositories, social media sites, and other sources to create a detailed profile of consumers. Some have argued that under this 1 + 1 = 3 scenario, the “3” results in too much private information that, in the aggregate, must be protected from unauthorized access.

Compliance: Under Financial Industry Regulatory Authority (FINRA) rules, companies in the securities industry are required to monitor all communications between salespersons, stockbrokers and their clients. At one time, such rules applied to paper documents and single systems such as e-mail. The company-to-client data stream is now much wider, including Facebook, Twitter, e-mail and many other systems. The old rules don't apply very well. Similar examples exist in many other industries.

Liability: Sometimes companies make decisions based on vast data sets and complex, computer-driven analysis beyond the comprehension of a single individual. These Big Data-based decisions put a lot trust into automated systems. When the decisions go wrong they can create disastrous results. In the lead-up to the recent financial crisis, many financial firms created complex financial models based on large collections of information that no one understood, and some turned out to be quite wrong. Chalk up an early failure for Big Data.

Discovery: What happens when your company faces litigation, and there is potentially relevant information in your Big Data system? How do you preserve, process, review, analyze and produce this constantly changing data? Would you even know where to start?

Big Data will be a boon for most organizations, allowing them to better understand their customers, optimize their supply chains and make better decisions in many parts of the business. This opportunity also comes with new risks. In-house counsel should be aware of how their organizations are using Big Data, and ideally should be involved in their setup and execution. The challenge is that in the rush to deploy new Big Data systems, compliance and legal risks will not be identified or may be ignored.

Ironically perhaps the best solution to Big Data's risks is Big Data itself, teaching these systems to not only analyze data, but in the process, use the analysis to ensure and monitor compliance.

Key Takeaways

  • Big Data is an emerging group of technologies that allow the analysis of large, varied and constantly changing data sets
  • Big Data presents potential risks around privacy, compliance, discovery and other areas
  • Many of the existing rules and regulations governing data are ill-equipped to address Big Data
  • In-house counsel should be, at the very least, aware of and ideally involved in the design and deployment of Big Data projects within their companies

Legal information is not legal advice

Contoural provides information regarding business, compliance and litigation trends and issues for educational and planning purposes. However, legal information is not the same as legal advice — the application of law to an individual or organization's specific circumstances. Contoural and its consultants do not provide legal advice. Readers should consult with competent legal counsel for professional assurance that our information, and any interpretation of it, is appropriate to each reader's particular situation.