Starburst has updated its enterprise data management and analytics platform, dubbed Starburst Enterprise, with new features designed to help companies create data products from any source, using the data mesh distributed architecture approach to data management.
The data mesh concept embraces decentralized management and governance of heterogeneous, distributed data. The goal of data mesh architecture is to allow management and analysis of data regardless of where it resides — on-premesis; in public cloud or multicloud environments; or whether it is stored in SQL or NoSQL databases.
“Enterprises have found moving large amounts of data to be cumbersome, expensive and time consuming delaying major business decisions. The data mesh concept not only solves this problem but also is a step towards ensuring that data is treated as a first-class product as it is no longer a by-product of an organisation’s operations,” said Colleen Tartow, director of engineering at Starburst.
Starburst launches Data Products module
At its second annual Datanova Conference Tuesday, Starburst unveiled Starburst Data Products, a new module in the Starburst Enterprise web UI designed to let data administrators and developers create and maintain data products, tapping the ability of the Starburst platform to connect to multiple data sources. Data products are essentially applications, such as dashboards and predictive analytics, that allow data to be analyzed in order to gain insights for business.
Starburst Enterprise allows the data application itself to be defined as a schema within a catalog, and the datasets are views within that schema, according to the company. The software lets end users rate and bookmark datasets, use a built-in query editor to directly query the data, or link to a third-party BI (business intelligence) tool.
Benefits of data mesh architecture
Data mesh is an emerging approach to data management that can benefit large and far-flung organizations that have never been able to fully centralize data management using platforms such as data warehouses or lakes, Doug Henschen, principal analyst at Constellation Research, said.
“Data fabric, data hub and data virtualization approaches, which have been around for many years, also recognize and attempt to bring management and governance to such heterogeneous and distributed data landscapes, but they attempt to do so with centralized metadata management schemes. Data mesh is a decentralized approach that provides some common governance provisions, but it also encourages local, decentralized control over data definitions, governance and usage,” Henschen said.
The principle, according to the analyst, is that local data owners and stewards know their data best and are in a better position to create and deliver valuable data products in a secure, governed manner.
Starburst’s Tartow and customers claim that the new platform has reduced the time to generate insights and brought down costs.
“We were struggling to connect our data sources. We wasted too much time copying and transferring our data. By deploying Starburst, we’ve reduced our time-to-insight from months to hours, strengthened our security, reduced costs by 40%, and increased our overall conversion rate by over 10%,” said Shen Wang, principal data engineer at AssuranceIQ, in a press release accompanying the launch of the updated Starburst Enterprise.
Data mesh allows for analytics at source
While efforts to gather real-time data from multiple sources for business analysis is not new, Starburst’s underlying technology is designed to enable data to be accessed and analyzed at source, and is based on data connectors designed to access varied storage locations as well as Trino, formerly Presto SQL — an open source, distributed SQL query engine for big data that allows users to query data from multiple data sources, including NoSQL databases, within a single query, Tartow said.
Other approaches include using other databases and architectures such as Online Transaction Processing (OLTP), Online Analytical Processing (OLAP) and Hybrid transaction/analytical processing (HTAP).
OLTP captures, stores and process data from transactions in real-time, while OLAP uses complex analytical queries to analyse historical data gathered in OLTP systems. Alternatively, HLTP, as the name suggests, uses a hybrid architecture approach wherein a database gathers data which is later copied to another system for analytics.
Rivals and market outlook
While there is a lot of interest in the new approach that Starburst is taking, it is very early days for these types of deployments, Henschen said.
“Snowflake acknowledged last year that it, too, is seeing some of its customers embracing data mesh approaches and it [said] it will deliver capabilities to support these deployments,” Henschen said, adding that Starburst’s foray into decentralized data management is expected to further the seesaw battle between the different approaches.
In terms of rivals, there are different products that address the data mesh concept but in particular, Intenda, a South Africa-based company, is eyeing the same market and solutions as Starburst, according to Henschen.
As of the end of 2021, Starburst had about 325 employees and 200 customers. At its DataNova conference Tuesday, Starburst also announced it had secured a $250 million funding round, at a $3.35 billion valuation. The financing round was led by Alkeon Capital, with participation from Altimeter and B Capital Group, as well as existing investors Andreessen Horowitz, Coatue Management, Index Ventures and Salesforce Ventures, Starburst said.