Snowflake (dagster-snowflake)¶

This library provides an integration with the Snowflake data warehouse.

Presently, it provides a snowflake_resource, which is a Dagster resource for configuring Snowflake connections and issuing queries.

To use this library, you should first ensure that you have an appropriate Snowflake user configured to access your data warehouse.

dagster_snowflake.snowflake_resource ResourceDefinition[source]¶

A resource for connecting to the Snowflake data warehouse.

A simple example of loading data into Snowflake and subsequently querying that data is shown below:

Examples:

from dagster import execute_pipeline, pipeline, DependencyDefinition, ModeDefinition
from dagster_snowflake import snowflake_resource

@solid(required_resource_keys={'snowflake'})
def get_one(context):
    context.resources.snowflake.execute_query('SELECT 1')

@pipeline(
    mode_defs=[ModeDefinition(resource_defs={'snowflake': snowflake_resource})],
)
def snowflake_pipeline():
    get_one()

result = execute_pipeline(
    snowflake_pipeline,
    {
        'resources': {
            'snowflake': {
                'config': {
                    'account': {'env': 'SNOWFLAKE_ACCOUNT'},
                    'user': {'env': 'SNOWFLAKE_USER'},
                    'password': {'env': 'SNOWFLAKE_PASSWORD'},
                    'database': {'env': 'SNOWFLAKE_DATABASE'},
                    'schema': {'env': 'SNOWFLAKE_SCHEMA'},
                    'warehouse': {'env': 'SNOWFLAKE_WAREHOUSE'},
                }
            }
        }
    },
)

class dagster_snowflake.SnowflakeConnection(context)[source]¶

execute_queries(sql_queries, parameters=None, fetch_results=False)[source]¶

execute_query(sql, parameters=None, fetch_results=False)[source]¶

get_connection(raw_conn=True)[source]¶

load_table_from_local_parquet(src, table)[source]¶