Google Cloud makes significant upgrade to BigQuery in pursuit of ‘ultimate data cloud’

Google Cloud is making upgrades to BigQuery that it hopes will lead to the ‘ultimate data cloud’, the tech giant revealed at its Next conference this week.

Starting today, data teams can analyze structured and unstructured data in BigQuery, with easy access to Google Cloud’s capabilities in machine learning, speech recognition, computer vision, translation, and text processing, using familiar SQL BigQuery interface.

In the past, data teams worked with structured data, using BigQuery to analyze data from operational databases and SaaS applications such as Adobe, SAP, ServiceNow, and Workday as well as semi-structured data such as JSON log file.

This comes after it announced a public preview last January for the BigQuery native JSON data type, a capability that brings support for storing and analyzing semi-structured data in BigQuery. With this, it said semi-structured data in BigQuery is easy to consume and query in its native format.

The company is making changes to how structured and unstructured data work in BigQuery, its data warehouse platform. It underlines that unstructured data can account for up to 90% of all data today, including video from television archives, audio from call centers or radio, and documents in various formats.

It also adds support for the major data formats in use today. BigLake, its storage engine, will add support for Apache Iceberg, Delta Lake, and Apache Hudi, the leading open-source table formats commonly used in data lakes. Support for Apache Iceberg is entering preview today while support for the other two formats is coming soon, but at an unspecified date.

The company hopes that by supporting these widely adopted data formats, it will help remove barriers that prevent organizations from getting the full value from their data.

The tech giant is also combining two of its business intelligence products, Looker and Google Data Studio, under the new Looker umbrella to create a deep integration of Looker, Data Studio, and core Google technologies such as AI and ML. This means Data Studio has now become Looker Studio, and Google hopes it will help customers make better data-driven decisions.

It says Looker Studio helps make performing self-service analytics easier. It currently supports more than 800 data sources with a catalog of more than 600 connectors, which it says makes exploring data from different sources simple.

Looker data models from Looker Studio are currently available in preview. This allows customers to explore trusted data through the Looker modeling layer, and for the first time they will be able to combine both self-service analytics from ad-hoc data sources with trusted data already analyzed and modeled in Looker .

Additionally, customers upgrading to Looker Studio Pro will get new enterprise management features, team collaboration capabilities, and SLAs. The tech giant underscored that this is just the first release, and that it has developed a roadmap of capabilities, including Dataplex integration for data lineage and metadata visibility, that enterprise customers are demanding.

Google Cloud also shared that Looker (Google Cloud core) is in preview. This is a new version of Looker available in the Google Cloud Console and is deeply integrated with key cloud infrastructure services, such as key security and management services.

Enhancements for Looker and BigQuery were also introduced to Microsoft Power BI, with Google Cloud branding the move as a significant step in providing customers with the most open data cloud. It said this means Tableau and Microsoft customers can easily analyze trusted data from Looker and seamlessly connect to BigQuery.

Google Cloud also said that a data cloud should enable organizations to bring all their data together with confidence, helping to ensure that data is of high quality and enabling strong, flexible management and management skills.

To address this, the company is updating Dataplex that will automate common processes related to data quality. For example, users can more easily understand data lineage — where the data came from and how it changed and moved over time — reducing the need for time-consuming manual processes.

“The ability to let our customers work with all kinds of data, in the formats they want, is the hallmark of an open data cloud,” said Gerrit Kazmaier, VP and GM of Data Analytics at Google Cloud. “We’re committed to delivering the support and integration customers need to remove limits on their data and avoid data lock-in.”

Google Cloud has confirmed its ambition to create “the most open, scalable and powerful data cloud” on the market. It wants customers to be able to use all their data from as many sources and in as many formats as needed.