Integrating Google BigQuery with Streamline lets you leverage cloud-native analytics at scale. This guide walks you through how to connect a BigQuery data source and create reusable datasets for your workflows.
Note: This article is specific to Google BigQuery. If you're working with another database, see our Data Source Library for the appropriate guide.
Part 1: Connect to a BigQuery Data Source
Step 1: Go to the Integrations Page
From your dashboard, navigate to the Integrations section. Click + Add New connection.
Step 2: Select BigQuery as Your Data Source
In the list of data sources, select BigQuery.
Name your connection clearly (e.g., Marketing_BQ, BQ_Analytics_Prod).
Tip: Use a name that helps team members identify the connection’s data purpose or source project.
Step 3: Authenticate Your Account
To connect BigQuery, you'll need to authenticate using Project ID and Sa Key:
Security Tip: Limit access to only the necessary datasets or projects via IAM roles in GCP.
Step 4: Select the Schema
Select your Schema & click Connect to complete setup. Your BigQuery source will now appear in your Integrations list, ready to use.
Step 5: Explore Your BigQuery Data
Navigate to the Data Catalog to see the available BigQuery tables and fields. Use Entity View to browse datasets and tables.
Use Data Field View to explore individual fields, including types and classifications.
Tip: You can filter by Data Source type, search, and sort through tables to quickly find what you need for dataset creation.
Part 2: Create and Manage a Dataset from BigQuery
With BigQuery connected, you can organize cloud-based analytics data into structured datasets for use in projects and workflows.
Step 1: Access the Datasets Section
Go to the Datasets section from the platform’s navigation. Click + Create New Dataset.
Step 2: Define Your Dataset
Enter a descriptive name for the dataset & select your BigQuery data source.
Choose a Primary Entity (a BigQuery table) that will serve as the main data source.
Optionally, assign a label for easier understanding (e.g., rename customer_id to Client ID).
Step 3: Configure Field Access
Review and select fields to include in the dataset. Configure write access if applicable (BigQuery is generally read-only, unless you're using advanced write-back tools).
Click Create when ready.
Alternative: Create a Dataset from the Data Catalog
You can also create a dataset directly from the schema browser:
In the Data Catalog, select a BigQuery table. Click New Dataset from Entity.
Choose the fields and optionally update classifications as covered in previous steps.
Optional: Add Related Entities
You can link additional BigQuery tables to your dataset:
While building your Dataset, click Add Related Entity. Select from available BigQuery tables that have logical relationships with your primary table (e.g., join orders to customers).
Note: BigQuery doesn't enforce foreign keys, but common key fields (like customer_id) are used to relate data.
Optional: Edit an Existing Dataset
To update a dataset:
Go to Datasets and click Edit on the one you want to modify.
You can:
- Add/remove related entities
- Update labels or classifications
- Reorder or hide fields
Click Next & Update to save changes.
Summary
By connecting to BigQuery, you can tap into scalable, cloud-native datasets and organize them for streamlined use in your workflows. Whether you’re building datasets from the catalog or refining them through the dataset manager, you’re in control of how cloud data fits into your project’s structure.
Need help troubleshooting? Check out the BigQuery FAQ or reach out to Support for assistance with credentials, IAM roles, or schema setup.
Comments
0 comments
Article is closed for comments.