How industry leaders are making the best of big data analytics?

‘Big Data’ has been at the center of many businesses in the past and it has gained prime importance in today’s context as brand and data management are collaborating far more towards pragmatic decision making approach. We are living in a data driven world and there is no denying the fact that now organizations have more data at their disposal than ever before. But the biggest challenge is how to convert those meaningful data into knowledge for action.

Though the path to successful implementation of ‘Big Data’ is not an easy task but some organizations have been successful in converting insightful data into meaningful knowledge for action that have truly transformed their business. It has certainly paved way for some of the most amazing advancements in how we perceive data.

The sheer volume of data that is being churned (courtesy mobile, IoT etc.) is humungous. Big Data Analytics have made it possible to churn insights out of the collected data and has provided critical business insight, forcing organizations to embrace a culture that is more objective and data driven. Surely, Big Data represents a paradigm shift in today’s decision making approach.

have already seen evidence that Big Data can deliver. According to a study done by Capgemini, wherever ‘Big Data’ analytics was applied, businesses saw a 26 percent improvement in performances over the last three years and the respondents hoped to witness a 41 percent rise of the next three years.

‘Big Data’ Analytics has disrupted each and every business – ranging from fashion, cosmetics, farming, banking, online portals, beverages and what not. To recall, an iconic British fashion brand has been using Radio Frequency Identification ("RFID") tech in some of its stores to enhance customer experience. As soon as a customer walks past a display screen with an item, the RFID tag attached to the item will trigger a video on the screen showing how the item was created. When a customer walks by a display screen with an item in hand, the RFID tag triggers a video showing how the item was made and offering up other products complementary to it. In future, the brand also plans to use the RFID tags to create a customer profile, which would inform the customer what he/she has tried on in the past.

Down Under, a famous bank disrupted Australian banking sector with data analytics by creating an innovative personal relationship program with its 9 million customers that resulted in an incremental revenue of Aus $7.8 million. Over a period of two-and-a-half- year, the bank analyzed data on the basis of customers ATM usage and call center interactions and based on the analysis, targeted customers with better offerings.

One of the world’s largest cosmetic player have made the best use of tons of data to make a big transformation in its marketing approach. The company moved on from its advanced marketing plan, where it would plan for two-five years, to real-time marketing. It didn’t take long for the company to realize that marketing is all about real-time decision making and with Big Data Analytics the company not only helped its retailers do a better job but also simplified beauty goals for the customers.

Likewise in Magicbricks, ‘Big Data’ remains at the heart of our business. From just being a provider of information, we have redefined the purpose of information, serving it exactly the way how customers need it and when they need it. All this has been made possible due to the ‘Big Data.’

In a cluttered market of real estate portals, Magicbricks.com has remained in the forefront of technological advancements with an entire gamut of innovative products that has made the home buying process not only easy but scientific as well.

While we have had multiple successes in areas spanning from duplicate properties to duplicate images to call center sentiment analysis, one area where we have seen ‘Big Data’ making a substantial difference is in property price prediction.

We recently launched an app called “PropWorth” which is a price-discovery app and in a very short time it has become the Google for property prices. It is the de-facto benchmark app for all buyers/sellers & Agents across the country before finalizing any property transaction. PropWorth uses complex algorithm based census data, market trends and home listings to determine the estimated value of a house/apartment/project or a locality.

Magicbricks, took up this challenge to put some ‘Data Science’ behind it. Being the custodian of the largest number of property listings data, both from owners and advertisers, provided Magicbricks a unique ability to put in place, a big data analysis system, which can use statistical and predictive algorithms to accurately identify the price of any property covered by our data set, which covers 12+ cities, 2000+ localities and 8500+ Apartment Projects.

The Data Science team at Magicbricks, came up with a set of algorithms and rules to accurately compute the price of a property. It required the application to run analytics over massive property database containing information on millions of properties belonging to different cities, localities, projects and unit types. The algorithm uses multiple attributes of the property along with recent price changes to compute the worth of the property.

Below is a high level architecture of how we placed the Big Data tech stack pieces:

Property Input

PropWorth REST API

Spark master Driver Program

Spark slave 1

Spark slave 2

Spark slave n

Historical property data in Hadoop

PropWorth uses smart algorithms using comprehensive statistics and data science to come up with an estimation of a certain apartment/project or a locality. All the computations of the app are based on the real user generated data points which are collected and updated daily, thus making the predictions richer every single day. The intelligent checks in the algorithm weed out the anomalies in the data, thus showing trends based on last 10 years of property related data.

This application of Big Data, solves a big problem for property buyers and sellers. Most of the time people had no clue about the existing market rates and trends and were dependent completely on local agents to set a benchmark figure. With this application, a user can keep himself updated with the latest benchmark figure of any property prices across multiple cities. It serves a great purpose of providing a benchmark figure to all three sets of users - the buyer, the seller and the agent/broker. It also helps in providing direction to the negotiation process which is the key milestone in the home buying process.

While Big Data helped us solving the property price prediction problem, system interaction with end users remains primarily driven by inputs and outputs. What user exactly wants cannot be very easily captured by a form so we need to take a more conversational approach to understand the user’s need. This is where chat bots come in context. From Facebook’s messenger bot framework to Microsoft’s bot framework to IBM Watson’s conversation APIs to mention few, application of AI in bots using machine learning has been icing on the cake. Chat bots has opened many more channels to reach out to end users via some of the non-traditional business ways.

In near future every object around us will be generating data, be it our homes, cars or our bodies. We will be overwhelmed by the amount of data that we would be exposed to as our smart devices are turning us into human censors. We are leaving behind a huge trail of digital exhaust, in forms of search patterns, text, locations and other information data. In today’s world, data is the new soil that we all need to nurture to transform key corporate decision-making.

Subodh Kumar
Tech-Head, Magicbricks