Summary of The Business of Big Data Workshop

I just attended The Business of Big Data workshop which was organized by General Assembly in the Flatiron district of Manhattan.  * GA told me not to attend because the event had sold out but luckily they let it slide when I showed up at the beginning of the presentation (how could they turn away another $20?).  The presentation was by Ben Siscovick and Andrew Cove of IA Ventures which is a dedicated Big Data venture fund located in NYC.  Here is a summary of my notes from the event which was interrupted three times by a faulty smoke alarm:


Managing data is difficult for three reasons:

1. Size - the amount of data generated today is astronomical and growing which requires creative solutions for storage.  "Every two days now we create as much information as we did from the dawn of civilization up until  2003."  - Eric Schmidt, Google
2. Unstructured - text, pictures, and video aren't as easy to catalog, sort, and analyze as clicks or numeric data.
3. Real time - data is produced so quickly making it difficult to keep up.

Google has laid the groundwork to make data management easier through the APIs and Open Source projects that have developed from their search tools.  They have manifested in services such as Amazon Cloudspace which make distributed computation and distributed storage possible so two entrepreneurs in a Starbucks can create the next generation data driven business.  

IA Ventures has a taxonomy for organizing their investments based on two characteristics of the data being used:  ** This section was slightly confusing for the audience because some companies have a number of business models that blur their lines.  You should consider companies as having two fronts or verticals: their consumer product front (Facebook profiles, sharing, and games) and their data front (how Facebook uses the data collected to sell advertising). ** 

1. What is the data?  Is it data that you produce (Twitter, NYSE) or data from another source (Bloomberg)?   
2. What is your product?  Do you produce data (Yipit), sell data (Data Sift), or do you have a data driven product (Recorded Future)?  The latter company sounds/looks really cool and I can't wait to write about them.

Ben commented at this point that, "Every product will be data driven at scale whether it is simply collecting or driving it."

The third section of their presentation was about Network Effects and Data Economies of Scale.  Big data lacks network effects until nodes (servers) have the capability to share information and analysis (THE CLOUD).  Securing data in a private "cloud" is more secure but it doesn't benefit from the mass amounts of data and tools that could make it valuable.  Data economies of scale was a topic that I didn't take any notes on because I thought it was too similar of a concept to Network Effects.  They use Google as an example but it was more a flow chart of their product/development process which sounded an awful lot like Network Effects.  

The final section was about building a Data Dream Team and the need for having two distinct data experts.  The first is a database expert who knows how to start a data set and effectively populate it with data.  This person should be dedicated to the integrity and accessibility of the data.  The second is a Data Scientist who is capable of hacking, has programming/data expertise, and is fluent in math/science (specifically statistics).  This person should be well balanced in each of these subjects otherwise bad things can happen...

One remark I enjoyed from Ben after the presentation was, "A business is 5% the idea and 95% how you execute," in reference to companies starting in Stealth Mode.  I think there is a delicate balance between gaining publicity (traction) and racing to have a product in your users hands which is obviously why some companies launch in stealth but quickly move to Beta.  Any suggestions for the optimal amount of time spent in stealth?   

I hope this summary helps and that I've accurately summarized the thoughts and knowledge expressed by Ben and Andrew.