3 big data security challenges

Big data has made a big impact on the business intelligence industry over the last decade. 

What makes data “big?” Literally, it’s the sheer size of it. The term describes data that’s too large for us to easily grasp, according to Peter Purcell, cofounder of Trenegy.

Anyone who has broken a spreadsheet dabbles in big data because big data describes data sets so massive that our current forms of processing (e.g., Excel) are incapable of making sense of them.

This information enables companies to get greater insight into their customers and refine their processes, all for the purpose of gaining a competitive advantage. Below are some examples of what you can do with big data: 

• Retailers can analyze and predict customer preferences and develop new products or services to address them. 
• Manufacturers can use sensors to monitor their machinery and implement predictive maintenance strategies that will help them save money. 
• Restaurants can more easily determine the best sites for expansion based on demographic data, traffic patterns, and consumer behavior.

Of course, as with anything that’s hard to grasp, there are challenges associated with big data; one in particular is security. In this post, we’ll highlight security challenges big data is facing and the steps you can take to remedy them.

Big data security: 3 challenges and solutions

1. Lost or stolen data

Data loss can occur for a number of reasons. It could be a hardware or system failure, human error, or a virus. While the last two are difficult to stave off, the first is easily avoidable. Statistics vary on just how much hardware or system failures are to blame for data loss, but one source notes that nearly a third of IT pros cite such failures as the main culprits.

Purcell says that’s where modern-day cloud storage solutions come in handy. Storing data in the cloud offers distinct benefits over physical backups and local storage. For example, files stored in the cloud can’t be lost due to damage or theft of your computer. You also have access to file histories to recover prior versions or deleted copies of documents. 

“Dropbox and Microsoft OneDrive are great options for lots of files,” Purcell says. “All files stored in these accounts are available on all of your devices and on the web, synchronized in real time. For large data processing and analytics, you can look into enterprise options like Amazon Web Services and Microsoft Azure.”

Pro Tip

Ensure that data is secure across your entire organization — try Jotform for free.

2. Evolving security practices

The move to the cloud does create a new challenge. Ray Walsh, digital privacy expert at ProPrivacy, tells us why: “The boundless scalable and affordable nature of cloud environments means that big data is no longer stored onsite and, as a result, traditional security measures don’t always meet the needs of big data security.”

Walsh says that security measures need to be multifaceted. Big data comes from a wide array of sources, and the unstructured nature of this data makes it challenging to secure as it moves from these disparate sources to whatever centralized platform a company uses. 

“The sheer volume of data being handled creates a challenge in itself,” Walsh says. “Because companies are increasingly dealing with petabytes of data, conducting routine security audits can be challenging.”

The solution, as Walsh explains, includes three parts:

• Implement robust policies that take into account the increasingly hybrid and often cloud-only nature of big data storage and analysis.
• Employ strong encryption for data in transit and at rest.
• Adopt strict user access controls so the flow of information is easily manageable and auditable.

3. Determining access privileges

Walsh’s point about access brings up a third challenge. Big data has disparate methods of identifying an end user, defining the access control rights for each data store and type, and filtering data access. Dave Mariani, cofounder and chief strategy officer at AtScale, adds that big data is often distributed across multiple, heterogeneous datastores, and managing user access across multiple systems can be complicated.

“There are really only two options to address user access,” Mariani says. The first option is to standardize data consumption through one set of access tools so that user identification and access controls can be managed consistently. But, he notes, this is unrealistic for larger enterprises that have a myriad of business intelligence and data access tools. 

“The second, more viable option,” he says, “is to implement an enterprise-wide data governance solution that acts as a proxy for all data access. By intercepting, filtering, and logging each data access attempt, along with authenticating users and applying access control lists consistently, an enterprise can ensure data is accessed and audited in one place.”

Purcell uses this story to underscore the importance of access privileges: “In a recent security breach at a large, well-known investment banking firm, data from more than 350,000 customers was stolen — by an employee. The SEC found the firm responsible, citing a failure to employ ‘written policies and procedures that are designed to protect customer data.’ Limiting access is crucial to securing your data.

Curious about other areas of data security? We created a lengthy guide on the topic to help you ensure your data is as safe as possible.