What is Big Data?
Big Data is any data set that is too large or complex to analyze via traditional means. Every day over 2 ½ exabytes of data are created. An exabyte is a billion gigabytes, and would require a stack of CDs 1000 miles high to store. That’s a lot more information than you can put on a spreadsheet!
All this data is coming from the smartphones, credit cards, store sensors, and highway monitors that are part of everyday life. Think you can get off the grid? You’d have to live in a log cabin in the Yukon to get away from the data collection, and no one can escape the implications.
Big Data can create big opportunities. Stores can analyze real-time information to run experiments, adjust prices, reorder hot items, and move merchandise around. By testing, bundling, synthesizing, and processing information from the sales floor to the corner office, businesses can become much better at what they do.
Data scientists are in demand. They command high salaries because they can add so much value. But it takes wisdom to transform information into insight. Besides technical issues of data quality and integrity, there can be ethical implications. Sometimes it’s just wrong to gather all the data you want.
Big Data analysis is part of the world we live in. It can make an enterprise more effective. But be careful! The first rule of data processing is, “Garbage in, garbage out.”
Douglas R. Tengdin, CFA
Chief Investment Officer
Leave a comment if you have any questions—I read them all!