Connecting a Snowflake database to your project is a key step in leveraging the powerful capabilities of Streamline. This guide walks you through integrating a Snowflake data source and creating datasets for use in your workflows.
Note: This article is specific to Snowflake. If you're connecting a different data source, check our Data Source Library for platform-specific instructions.
Prerequisites
Before you begin, ensure the following:
- You have network access to your Snowflake account (e.g., correct region URL and access permissions).
- Your Snowflake user account has appropriate roles and privileges (e.g.,
USAGEandSELECTon the database, schema, and tables). - Required firewall or network configurations are in place to allow secure outbound HTTPS traffic to Snowflake.
If unsure, check with your database administrator or Snowflake account admin.
Part 1: Connect to a Snowflake Data Source
Step 1: Go to the Integrations Page
- From your dashboard, navigate to the Integrations tab on the left.
- Click + New Connection at the top of the page or the centre.
Step 2: Select Snowflake as Your Integration Type
- In the pop-up window, choose Snowflake from the list of supported databases.
Give your connection a recognizable name & description (e.g., Snowflake - Prod Warehouse, Snowflake Reporting).
Tip: Use names that help your team quickly understand the connection’s purpose or environment.
Step 3: Enter Snowflake Connection Details
Provide the following details:
- Host
- Warehouse
- Database (target Snowflake database)
- Username
- Private Key
Click Next to test the connection.
Step 4: Choose Your Database and Schema
Once authenticated, you'll be prompted to select the database and schema you want to connect to.
Step 5: Finalize the Connection
Click Connect to complete the integration. Your Snowflake connection will now appear in the Integrations list.
Step 6: Explore Your Snowflake Data
Once connected:
- Navigate to the Data Catalog to browse tables and fields in your Snowflake schema.
- Use search or filters to find specific entities.
- The Entity View displays tables; the Data Field View shows individual columns with metadata (data type, nullability, etc.).
Part 2: Create and Manage a Dataset from Snowflake
Once your Snowflake connection is active, you can organize your data into reusable datasets for your projects and workflows.
Step 1: Access the Datasets Section
- Go to Datasets via the left-hand panel.
- Click + Create New Dataset.
Step 2: Define Dataset Basics
- Give your dataset a clear and intuitive name.
- Select your Snowflake data source.
- Choose a Primary Entity (table) from your schema.
-
Add a label to make database terms more user-friendly.
Example: Use “Sales Orders” instead of
STG_ORDERS_VW.
Step 3: Configure Field Access
- Review available fields (columns).
- Configure field visibility and write access where applicable.
- Click Create to save the dataset.
Optional: Create a Dataset Directly from the Data Catalog
- From the Data Catalog, select a table.
- Click New Dataset from Entity.
- Customize the dataset by choosing fields or editing labels.
Optional: Add Related Entities
To enrich your dataset:
- During dataset creation, click Add Related Entity.
- The platform will automatically detect foreign key relationships where available (if constraints are defined in Snowflake).
- You can manually add joins as well if needed.
- Add as many related tables as needed before finalizing.
Optional: Edit an Existing Dataset
To update an existing dataset:
- Go to Datasets and locate the dataset.
- Click Edit to:
- Add/remove related entities
- Change field names or labels
- Adjust data access settings
- Click Next & Update to save your changes.
Troubleshooting Tips
Encountering issues? Here are some common solutions:
- Connection fails: Check account URL and warehouse/database names.
- Authentication errors: Ensure your Snowflake credentials are valid and that your user has the correct role.
-
Schema access issues: Confirm your role has
USAGEandSELECTon the database and schema. - Firewall/network problems: Ensure outbound HTTPS traffic to Snowflake is allowed.
Contact your Snowflake admin or IT support if issues persist.
Performance Tips
- Choose only necessary columns to keep datasets efficient.
- Use filters or SQL WHERE clauses for targeted data retrieval.
- Regularly audit and update datasets to remove unused or redundant data.
- Consider using materialized views or external functions for complex transformations.
Connecting: Best Practices and Requirements
Currently, only private key authorization is supported for connecting to Snowflake.
The private key must be in unencrypted PKCS#8 format.
Generate Your Key
Use the following command to generate a private RSA key:
openssl genpkey -algorithm RSA -out rsa_key.p8 -pkeyopt rsa_keygen_bits:2048
Extract the Public Key
Then extract the corresponding public key:
openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub
Register Public Key in Snowflake
Open a SQL Worksheet in Snowflake and run the following command, replacing your_user with your username and ...with the actual public key content:
alter user your_user set rsa_public_key='...';
Note: If you do not have the required permissions, you may need assistance from your Snowflake
ACCOUNTADMINto run this command.
Create Streamline Integration
When creating the integration in Streamline, upload your rsa_key.p8 file from your local machine to authenticate the connection.
Enabling New Record Insert from Streamline to Snowflake
Snowflake uses bulk loading by default, so enabling row-by-row inserts requires additional configuration. Follow this guide to set up Snowflake for INSERT commands from Intellistack Streamline.
Limitations
Supported Data Types
Streamline currently supports a subset of Snowflake's data types. The following are supported (including their synonyms):
VARCHARTIMESTAMPTZTIMESTAMPNTZTIMESTAMPLTZDATEDOUBLEBOOLEANTIMENUMBERBINARY
For full reference, see the official Snowflake Data Types documentation.
Summary
By connecting your Snowflake environment and building datasets, you ensure your data is clean, accessible, and workflow-ready. Whether you're building datasets from scratch or using the Data Catalog, Streamline provides control over your Snowflake data.
Need help? Check our FAQs/articles or contact Support.
Comments
0 comments
Article is closed for comments.