8 Key Elements for A Successful Data Marketplace

This article was co-authored by Nirav Krishnakant Gandhi, Senior Principal at Infosys Consulting and Dominic Launder, Senior Consultant at Infosys Consulting.

In the ‘New normal’ of low oil prices, oil and gas companies are accelerating their digitization efforts to boost operational efficiencies and reduce costs. The oil and gas industry has always been data-intense. However, the vast majority of the companies do not effectively use the growing flood of data.

Typically, data resides in silos. Access to this data is often restricted to certain departments within the organization, leading to lost opportunities and poor data practices as well as orphaned data, duplication and rework.

To exploit the data to its full potential, oil and gas organizations need to democratize data to break down siloes and make data easily available and consumable for the whole organization. Data democratization changes how organizations manage data access, usability, data ownership, and data culture. Making data available to non-data specialists and removing barriers to data access not only expedites decision making, but also uncovers opportunities for the organization.

Data marketplaces play a key role in enabling data democratization. Similar to Amazon Marketplace, an e-commerce platform that lets any seller bring its products to market, a Marketplace for data is a platform on which producers and data consumers collaborate to search and discover enterprise data assets, network with data experts and collectively improve the value of data. This omnichannel app is accessible to all stakeholders within the organization, revolutionizing the way data is discovered and accessed.

Here are 8 key success factors oil and gas companies should consider to build a successful data marketplace:

1. Robust search and mature data catalog – A robust and flexible search capability built upon a mature data catalog is critical to achieving user satisfaction and focused search results. To get the right dataset, a data-asset should sit within structured category hierarchies and be tagged with relevant search terms from a business glossary and user tags. The search should also capture ontological nuances but it needs to be focused enough to show the most relevant results on the first page. The catalog must accommodate searching by free text, filters/facets and deliver results parsed by a relevance algorithm.
2. Effective change management – Effective change management is vital for creating awareness among the user community. Anticipation of the potential benefits and early feedback upon release for feature development specific to various data communities is key. Working with users on real use-cases brings the product closer to them and demonstrates tangible benefits and improvement potential. Early adopters generate ideas which evolve into more ideas, therefore the sooner the users experiment with what the Marketplace provides and can appreciate the benefits, the better the outcome and the greater the return on the investment will be. There’s no better champion or promoter than users to other users. Robust change management and a well-planned, consistently executed communication program both from top-down and bottom-up is crucial to the success of the Marketplace and the concept of data democratization.
3. User adoption – Developing a brand and creating the perception of a one-stop-shop to find and use data is essential in engaging active and enthusiastic data communities. More users enhance the Marketplace content further by collaboration and publishing insights back to the Marketplace. Also, crowdsourcing for metadata enrichment and glossary improves search quality. Simply speaking, if users find what they want, then adoption goes up.
4. Re-imagined data ownership and governance – Data ownership and custodianship are the foundation of data management. Data owners must be willing to come along on the Marketplace journey, to manage and promote their data, and to open access to it across the business. Tight access controls would result in unhappy customers. Finding the right balance between what needs to be controlled and what can be opened for general access determines the success of a marketplace. Governance mechanisms need to be defined and implemented to ensure meta data is evergreen for old and new data.Security classification (e.g. general, confidential, secret, or public) should be defined at data source level. This rating is stored as technical metadata and will hasten any access requests for data not already available in the Marketplace. Ideally, most data should be available to discover and use across the organization. To attain an open data eco-system the principle of ‘most access’ is useful rather than the more traditional and still commonplace approach of ‘least access’ to data. With the exceptions of ‘secret’ data, personally identifiable information (PII), content restricted by company policy or regulation and law, all other datasets must be discoverable.

   A robust security model is an essential component of data marketplace. Such models could include data licensing implementation to open the data based on a user’s persona/role. Automating security via APIs is required so that users can request access to non-general access datasets.

5. Greater trust in the content – A guiding principle and benefit of a Marketplace is to improve trust in data which encourages usage. A Marketplace attempts to overcome trust issues by making the data journey transparent. That includes data lineage, any transformation, applied quality measures, curation, ownership etc. If there are quality issues or anomalies, a Marketplace can provide information with metadata and annotations to users. Conversely users themselves can highlight issues to fellow users. When people find trusted data sooner, they leapfrog over the typical search torment to work on good data with the tools provided and get to insight faster.
6. Simple connectivity with the right tools – After finding high-quality data, the next logical step in a user journey is to easily connect the data to their tool of choice for performing analysis. The ability to seamlessly connect various data science and analytical tools is a critical success factor to improve the user journey. A Marketplace experience should include connecting people, data and tools without having to leave the platform. Common pain points include the need to manipulate, wrangle and otherwise prepare data with tools before analysis can begin; hence, the data Marketplace can assist by making possible data discovery, preparation and analysis in one place. Success might be the ability to find and open trusted fit-for-purpose data in a tool of the users’ choice in three clicks.
7. Collaboration between data experts and users (contributors and consumers) – Collaboration among users, data experts, or groups helps in sharing insights or improves the dataset collectively. The socialization of data encourages data science culture and data openness. Organizations can utilize the platform itself with comments and ratings or use social networking platforms like Yammer groups to discuss datasets (non-confidential), tools, processes and even suggest feature enhancements and new data needs.
8. Onboarding new data sources – As oil and gas organizations’ data maturity grows, the data variety and sources will broaden. As data from different functions and disciplines migrate to the cloud, it is important to ensure that data silos are not propagated, and the Marketplace catalog is refined. As sources turn on and off, a seamless plug & play process (figure below) is necessary to reduce technical integration efforts. A more mature organization will go beyond provisioning raw source data and begin building business data models and other products that the more discerning data consumer desires. Similarly, advanced data groups and formalized data management processes compile and curate data to be published via the Marketplace, creating a more valuable consumer-ready data estate.

Data empowerment is the key to digital transformation initiatives for oil and gas organizations wishing to effectively tap into the enormous data pools. Democratizing data gives these organizations a competitive advantage in leveraging their data assets, technology and people and ultimately leads to new opportunities, revenue streams and business growth.

Want to learn more? Read the full PoV here.