Planning Databases and Dashboards For Big Data

Pierre DeBois – founder of Zimana, a small business digital analytics consultancy

Pierre DeBois is the founder of Zimana, a small business digital analytics consultancy. Since 2009, Pierre has conducted analysis for various small and medium sized businesses. He has also been involved in lead specialty projects such as being a technical editor for Pearson/Que Publications and developing an analytic metrics whitepaper for Pitney Bowes. Previously, he has provided his business and engineering acumen at various corporations such as Ford Motor Company. As a guest blogger for CODE_n he shares his insights related to web analytics and its impact for small businesses.

The scale of unstructured data is certainly as astounding as it is far reaching. Just a few years ago IBM referenced that the majority of data is unstructured. In 2013, Gooddata Vice President of Marketing Mike Smitheman made a similar claim, with a higher percentage – 90% vs. IBM’s 80%. Much of this growth is from social media related unstructured data elements – millions of tweets, likes, and of course, images from over 6 billion mobile devices worldwide.

But what is the impact beyond the increase in percentages? Modeling quality based on the data is one impact. Companies are increasingly dependent on internal and external data to refine operation decisions as well as sales and marketing initiatives. But because of unstructured data, companies are struggling to adopt the right database architecture and ongoing administration.

The resulting challenge is to understand how data from various sources are monitored. The ability to develop a dashboard that supports fast data-driven decision-making can be the prize at stake.

To ease development concerns a few steps should be considered during the dashboard development. Understanding what should be reported can inform the database administration needed. Here are a few ideas:

• Roadmap the behavior expected from a digital property related to your start-up – apps, website, mobile pages, etc. This consideration will spark discussion on the tagging effort required by an analytics solution. The data from the tags can at least be a starting point to confirm what digital exposure is working for your firm and what should be included in the dashboard.
• The next steps should lead to roadmaping the metrics expected against the data source behind the metrics. This may also lead into database discussion as well – you may find that as an analytics tool can yield initial metrics, other missing metrics may need additional sources. A dashboard should be a brief reporting document containing a journal, a list of next steps as a response to the metrics reported.
• Incorporate technical reviews alongside regular reports during the first reporting. Assign double the assumed time for reviewing a first analysis so that any concerns, particularly with query results can be identified early. Database administration concerns such as cluster configuration, and designing patterns for data processing runs can be addressed as well.
• Metrics that seemed important at the start of a campaign can be unimportant by the end. You should be prepared to revise accordingly. The team can then decide if metrics and data have become irrelevant due to company strategic changes.
• Use analytics to identify social media platforms that influences the unstructured data encountered. For example, a consistent Foursquare traffic over a sporadic Twitter traffic can imply a likelihood need to access customer data over sharing white paper pdfs from storage. This can inspire database architecture needs – flexibility for scale over storage volume (or a blend of both qualities).

While much of the database technologies remains to be sorted out, opportunity exists to make the best plans possible. Assessing database options alongside dashboard concerns can help make even the small steps towards big data more assured.