16. Databricks Icon Description automatically generated
16.1. Connection Options
Property
Description
The authentication scheme used. Accepted entries are PersonalAccessToken, AzureServicePrincipal.
The host name or IP address of the server hosting the Databricks database.
The Protocol Version used to authenticate with Databricks.
The name of the Databricks database.
The path component of the URL endpoint.
The token used to access the Databricks server.
Property
Description
Your AWS account access key. This value is accessible from your AWS security credentials page.
Your AWS account secret key. This value is accessible from your AWS security credentials page.
The hosting region for your Amazon Web Services.
The name of your AWS S3 bucket.
Property
Description
The name of your Azure storage account.
The storage key associated with your Databricks account.
The name of your Azure Blob storage container.
Property
Description
The Tenant id of your Microsoft Azure Active Directory.
The application(client) id of your Microsoft Azure Active Directory application.
The application(client) secret of your Microsoft Azure Active Directory application.
The Subscription id of your Azure Databricks Service Workspace.
The Resource Group name of your Azure Databricks Service Workspace.
The name of your Azure Databricks Service Workspace.
16.1.5. SSL
Property
Description
The TLS/SSL client certificate store for SSL Client Authentication (2-way SSL).
The type of key store containing the TLS/SSL client certificate.
The password for the TLS/SSL client certificate.
The subject of the TLS/SSL client certificate.
The certificate to be accepted from the server when connecting using TLS/SSL.
16.1.6. Firewall
Property
Description
The protocol used by a proxy-based firewall.
The name or IP address of a proxy-based firewall.
The TCP port for a proxy-based firewall.
The user name to use to authenticate with a proxy-based firewall.
A password used to authenticate to a proxy-based firewall.
16.1.7. Proxy
Property
Description
This indicates whether to use the system proxy settings or not. This takes precedence over other proxy settings, so you'll need to set ProxyAutoDetect to FALSE in order use custom proxy settings.
The hostname or IP address of a proxy to route HTTP traffic through.
The TCP port the ProxyServer proxy is running on.
The authentication type to use to authenticate to the ProxyServer proxy.
A user name to be used to authenticate to the ProxyServer proxy.
A password to be used to authenticate to the ProxyServer proxy.
The SSL type to use when connecting to the ProxyServer proxy.
A semicolon separated list of destination hostnames or IPs that are exempt from connecting through the ProxyServer .
16.1.8. Logging
Property
Description
A filepath which designates the name and location of the log file.
The verbosity level that determines the amount of detail included in the log file.
Core modules to be included in the log file.
A string specifying the maximum size in bytes for a log file (for example, 10 MB).
A string specifying the maximum file count of log files.
16.1.9. Schema
Property
Description
A path to the directory that contains the schema files defining tables, views, and stored procedures.
This property restricts the schemas reported to a subset of the available schemas. For example, BrowsableSchemas=SchemaA,SchemaB,SchemaC.
This property restricts the tables reported to a subset of the available tables. For example, Tables=TableA,TableB,TableC.
Restricts the views reported to a subset of the available tables. For example, Views=ViewA,ViewB,ViewC.
Set this property to define primary keys.
16.1.10. Caching
Property
Description
Automatically caches the results of SELECT queries into a cache database specified by either CacheLocation or both of CacheConnection and CacheProvider .
The database driver used to cache data.
The connection string for the cache database. This property is always used in conjunction with CacheProvider . Setting both properties will override the value set for CacheLocation for caching data.
Specifies the path to the cache when caching to a file.
The tolerance for stale data in the cache specified in seconds when using AutoCache .
Use offline mode to get the data from the cache instead of the live source.
This property determines whether or not to cache the table metadata to a file store.
16.1.11. Databricks
Property
Description
Determine which cloud storage service will be used.
This option specifies whether Databricks server will create and save tables in cloud storage.
Specifies whether to use DESCRIBE FORMATTED ... to query detailed table information, the query will take a long time if set it to True.
This option specifies whether the Databricks Upload API will be used when executing Bulk Insert operations.
This option specifies whether to use CloudWatch to improve query efficiency when the data volume of the table is large.
This option specifies whether to support Unity Catalog.
16.1.12. Miscellaneous
Property
Description
The application name connection string property expresses the HTTP User-Agent.
The timeout for asynchronous requests issued by the provider to download large result sets.
The maximum size of each batch operation to submit.
The maximum lifetime of a connection in seconds. Once the time has elapsed, the connection object is disposed.
This property specifies whether to connect to the Databricks when the connection is opened.
The describe command used to communicate with the Hive server. Accepted entries are DESCRIBE and DESC.
Specifies whether to use DESCRIBE FORMATTED ... to detect the specified table is view or not.
Limits the number of rows returned rows when no aggregation or group by is used in the query. This helps avoid performance issues at design time.
These hidden properties are used only in specific use cases.
The allowed idle time for a connection before it is closed.
The maximum connections in the pool.
The minimum number of connections in the pool.
The max seconds to wait for an available connection.
This property indicates whether or not to include pseudo columns as columns to the table.
This option passes the query to the Databricks server as is.
You can use this property to enforce read-only access to Databricks from the provider.
The runtime key used for licensing.
A name-value list of server configuration variables to override the server defaults.
The value in seconds until the timeout error is thrown, canceling the operation.
This property enables connection pooling.
This option specifies whether the columns will be retrieved using a DESC TABLE query or the GetColumns Thrift API.The GetColumns Thrift API works for the Apache Spark 3.0.0 or later.
Specifies whether to use an INSERT INTO SELECT statement.
A filepath pointing to the JSON configuration file containing your custom views.