Starburst/Trino setup
profiles.yml
file is for dbt Core users onlyIf you're using dbt Cloud, you don't need to create a profiles.yml
file. This file is only for dbt Core users. To connect your data platform to dbt Cloud, refer to About data platforms.
- Maintained by: Starburst Data, Inc.
- Authors: Marius Grama, Przemek Denkiewicz, Michiel de Smet, Damian Owsianny
- GitHub repo: starburstdata/dbt-trino
- PyPI package:
dbt-trino
- Slack channel: #db-starburst-and-trino
- Supported dbt Core version: v0.20.0 and newer
- dbt Cloud support: Supported
- Minimum data platform version: n/a
Installing dbt-trino
Use pip
to install the adapter. Before 1.8, installing the adapter would automatically install dbt-core
and any additional dependencies. Beginning in 1.8, installing an adapter does not automatically install dbt-core
. This is because adapters and dbt Core versions have been decoupled from each other so we no longer want to overwrite existing dbt-core installations.
Use the following command for installation:
Configuring dbt-trino
For Starburst/Trino-specific configuration, please refer to Starburst/Trino configs.
Connecting to Starburst/Trino
To connect to a data platform with dbt Core, create appropriate profile and target YAML keys/values in the profiles.yml
configuration file for your Starburst/Trino clusters. This dbt YAML file lives in the .dbt/
directory of your user/home directory. For more information, refer to Connection profiles and profiles.yml.
The parameters for setting up a connection are for Starburst Enterprise, Starburst Galaxy, and Trino clusters. Unless specified, "cluster" will mean any of these products' clusters.
Host parameters
The following profile fields are always required except for user
, which is also required unless you're using the oauth
, oauth_console
, cert
, or jwt
authentication methods.
Field | Example | Description |
---|---|---|
host | mycluster.mydomain.com Format for Starburst Galaxy:
| The hostname of your cluster. Don't include the http:// or https:// prefix. |
database | my_postgres_catalog | The name of a catalog in your cluster. |
schema | my_schema | The name of a schema within your cluster's catalog. It's not recommended to use schema names that have upper case or mixed case letters. |
port | 443 | The port to connect to your cluster. By default, it's 443 for TLS enabled clusters. |
user | Format for Starburst Enterprise or Trino:
Format for Starburst Galaxy:
| The username (of the account) to log in to your cluster. When connecting to Starburst Galaxy clusters, you must include the role of the user as a suffix to the username. |
Roles in Starburst Enterprise
If connecting to a Starburst Enterprise cluster with built-in access controls enabled, you must specify a role using the format detailed in Additional parameters. If a role is not specified, the default role for the provided username is used.
Schemas and databases
When selecting the catalog and the schema, make sure the user has read and write access to both. This selection does not limit your ability to query the catalog. Instead, they serve as the default location for where tables and views are materialized. In addition, the Trino connector used in the catalog must support creating tables. This default can be changed later from within your dbt project.
Additional parameters
The following profile fields are optional to set up. They let you configure your cluster's session and dbt for your connection.
Profile field | Example | Description |
---|---|---|
threads | 8 | How many threads dbt should use (default is 1 ) |
roles | system: analyst | Catalog roles can be set under the optional roles parameter using the following format: catalog: role . |
session_properties | query_max_run_time: 4h | Sets Trino session properties used in the connection. Execute SHOW SESSION to see available options |
prepared_statements_enabled | true or false | Enable usage of Trino prepared statements (used in dbt seed commands) (default: true ) |
retries | 10 | Configure how many times all database operation is retried when connection issues arise (default: 3 ) |
timezone | Europe/Brussels | The time zone for the Trino session (default: client-side local timezone) |
http_headers | X-Trino-Client-Info: dbt-trino | HTTP Headers to send alongside requests to Trino, specified as a YAML dictionary of (header, value) pairs. |
http_scheme | https or http | The HTTP scheme to use for requests to Trino (default: http , or https if kerberos , ldap or jwt ) |
Authentication parameters
The authentication methods that dbt Core supports are:
ldap
— LDAP (username and password)kerberos
— Kerberosjwt
— JSON Web Token (JWT)certificate
— Certificate-based authenticationoauth
— Open Authentication (OAuth)oauth_console
— Open Authentication (OAuth) with authentication URL printed to the consolenone
— None, no authentication
Set the method
field to the authentication method you intend to use for the connection. For a high-level introduction to authentication in Trino, see Trino Security: Authentication types.
Click on one of these authentication methods for further details on how to configure your connection profile. Each tab also includes an example profiles.yml
configuration file for you to review.
- LDAP
- Kerberos
- JWT
- Certificate
- OAuth
- OAuth (console)
- None
The following table lists the authentication parameters to set for LDAP.
For more information, refer to LDAP authentication in the Trino docs.
Profile field | Example | Description |
---|---|---|
method | ldap | Set LDAP as the authentication method. |
user | Format for Starburst Enterprise or Trino:
Format for Starburst Galaxy:
| The username (of the account) to log in to your cluster. When connecting to Starburst Galaxy clusters, you must include the role of the user as a suffix to the username. |
password | abc123 | Password for authentication. |
impersonation_user (optional) | impersonated_tom | Override the provided username. This lets you impersonate another user. |
Example profiles.yml for LDAP
trino:
target: dev
outputs:
dev:
type: trino
method: ldap
user: [user]
password: [password]
host: [hostname]
database: [database name]
schema: [your dbt schema]
port: [port number]
threads: [1 or more]
The following table lists the authentication parameters to set for Kerberos.
For more information, refer to Kerberos authentication in the Trino docs.
Profile field | Example | Description |
---|---|---|
method | kerberos | Set Kerberos as the authentication method. |
user | commander | Username for authentication |
keytab | /tmp/trino.keytab | Path to keytab |
krb5_config | /tmp/krb5.conf | Path to config |
principal | trino@EXAMPLE.COM | Principal |
service_name (optional) | abc123 | Service name (default is trino ) |
hostname_override (optional) | EXAMPLE.COM | Kerberos hostname for a host whose DNS name doesn't match |
mutual_authentication (optional) | false | Boolean flag for mutual authentication |
force_preemptive (optional) | false | Boolean flag to preemptively initiate the Kerberos GSS exchange |
sanitize_mutual_error_response (optional) | true | Boolean flag to strip content and headers from error responses |
delegate (optional) | false | Boolean flag for credential delegation (GSS_C_DELEG_FLAG ) |
Example profiles.yml for Kerberos
trino:
target: dev
outputs:
dev:
type: trino
method: kerberos
user: commander
keytab: /tmp/trino.keytab
krb5_config: /tmp/krb5.conf
principal: trino@EXAMPLE.COM
host: trino.example.com
port: 443
database: analytics
schema: public
The following table lists the authentication parameters to set for JSON Web Token.
For more information, refer to JWT authentication in the Trino docs.
Profile field | Example | Description |
---|---|---|
method | jwt | Set JWT as the authentication method. |
jwt_token | aaaaa.bbbbb.ccccc | The JWT string. |
Example profiles.yml for JWT
trino:
target: dev
outputs:
dev:
type: trino
method: jwt
jwt_token: [my_long_jwt_token_string]
host: [hostname]
database: [database name]
schema: [your dbt schema]
port: [port number]
threads: [1 or more]
The following table lists the authentication parameters to set for certificates.
For more information, refer to Certificate authentication in the Trino docs.
Profile field | Example | Description |
---|---|---|
method | certificate | Set certificate-based authentication as the method |
client_certificate | /tmp/tls.crt | Path to client certificate |
client_private_key | /tmp/tls.key | Path to client private key |
cert | The full path to a certificate file |
Example profiles.yml for certificate
trino:
target: dev
outputs:
dev:
type: trino
method: certificate
cert: [path/to/cert_file]
client_certificate: [path/to/client/cert]
client_private_key: [path to client key]
database: [database name]
schema: [your dbt schema]
port: [port number]
threads: [1 or more]
The only authentication parameter to set for OAuth 2.0 is method: oauth
. If you're using Starburst Enterprise or Starburst Galaxy, you must enable OAuth 2.0 in Starburst before you can use this authentication method.
For more information, refer to both OAuth 2.0 authentication in the Trino docs and the README for the Trino Python client.
It's recommended that you install keyring
to cache the OAuth 2.0 token over multiple dbt invocations by running python -m pip install 'trino[external-authentication-token-cache]'
. The keyring
package is not installed by default.
Example profiles.yml for OAuth
sandbox-galaxy:
target: oauth
outputs:
oauth:
type: trino
method: oauth
host: bunbundersders.trino.galaxy-dev.io
catalog: dbt_target
schema: dataders
port: 443
The only authentication parameter to set for OAuth 2.0 is method: oauth_console
. If you're using Starburst Enterprise or Starburst Galaxy, you must enable OAuth 2.0 in Starburst before you can use this authentication method.
For more information, refer to both OAuth 2.0 authentication in the Trino docs and the README for the Trino Python client.
The only difference between oauth_console
and oauth
is:
oauth
— An authentication URL automatically opens in a browser.oauth_console
— A URL is printed to the console.
It's recommended that you install keyring
to cache the OAuth 2.0 token over multiple dbt invocations by running python -m pip install 'trino[external-authentication-token-cache]'
. The keyring
package is not installed by default.
Example profiles.yml for OAuth
sandbox-galaxy:
target: oauth_console
outputs:
oauth:
type: trino
method: oauth_console
host: bunbundersders.trino.galaxy-dev.io
catalog: dbt_target
schema: dataders
port: 443
You don't need to set up authentication (method: none
), however, dbt Labs strongly discourages people from using it in any real application. Its use case is only for toy purposes (as in to play around with it), like local examples such as running Trino and dbt entirely within a single Docker container.
Example profiles.yml for no authentication
trino:
target: dev
outputs:
dev:
type: trino
method: none
user: commander
host: trino.example.com
port: 443
database: analytics
schema: public