Big Data or Big mess

After the year of 2012, people are becoming more and more aware of the term “big data” or “mega data”. Even a lot of my friends around me read about the book “Big Data:A Revolution That Will Transform How We Live, Work, and Think”. However, does the big data would really impact our lives and help us do research.

In the field of social science, there are generally two typically methods of doing research. The first one is quantitative research and the other one is qualitative research. Whether big data is going to impact the research is a question.

When we carry out a  quantitative research, the first step after we read about a lot of other people’s research is that we establish our own hypothesis. When we have a hypothesis, the next step is that we would start doing research to prove the hypothesis or falsify this statement. And this is the reason why we need data to tell other people and persuade them.

However, in the age of big data, no matter the data is useful or not, the data is gathered by certain software. The research steps are reversed. We first get the data, then we try to figure out what the data means. For example, we are experiencing a economic recession. The big data shows that women’s skirts are getting shorter. Is there a real connection between this two phenomenon. So in the age of mega data, it is difficult to distinguish what is the cause and what is the reason.

Moreover, when people are doing qualitative research, people want to find the most weird things out of ordinary. When ordinary things happen, we cannot find anything different among the data. Doing a qualitative research, we need to control some variables. For example, control group is a very good method. However, when it comes to the big data, what can we get? We are receiving a lot of noise. In order to find out the useful data, we may set the threshold so high that only the most weird data is shown.

So here is the question. Is the big data a really useful data or just a mess?images


This entry was posted in Uncategorized. Bookmark the permalink.

One Response to Big Data or Big mess

  1. sydhavely says:

    Kiwen–you have successfully asked the primary question about the role and relationship of big data and scientific inquiry. What big data does is subsume the importance of sampling by allowing n=all. As such all the outliers are included with the rest. This reduces the margin of error but as you say, reverses the process of starting with a hypothesis. Big data says, this is what is happening. The scientists then has to correlate the what and the why. Great post. BTW, we read the book you referenced.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s