40. Parquet 
40.1. Connection Options
40.1.1. Authentication
Property | Description |
AuthScheme | The type of authentication to use when connecting to remote services. |
AccessKey | Your account access key. This value is accessible from your security credentials page. |
SecretKey | Your account secret key. This value is accessible from your security credentials page. |
ApiKey | The API Key used to identify the user to IBM Cloud. |
SharePointEdition | The edition of SharePoint being used. Set either SharePointOnline or SharePointOnPremise. |
40.1.2. Connection
Property | Description |
URI | The Uniform Resource Identifier (URI) for the Parquet resource location. |
DataModel | Specifies the data model to use when parsing Parquet documents and generating the database metadata. |
Region | The hosting region for your S3-like Web Services. |
ProjectId | The id of the project where your Google Cloud Storage instance resides. |
OracleNamespace | The Oracle Cloud Object Storage namespace to use. |
StorageBaseURL | The URL of a cloud storage service provider. |
UseVirtualHosting | If true (default), buckets will be referenced in the request using the hosted-style request: http://yourbucket.s3.amazonaws.com/yourobject. If set to false, the bean will use the path-style request: http://s3.amazonaws.com/yourbucket/yourobject. Note that this property will be set to false, in case of an S3 based custom service when the CustomURL is specified. |
40.1.3. AWS Authentication
Property | Description |
AWSAccessKey | Your AWS account access key. This value is accessible from your AWS security credentials page. |
AWSSecretKey | Your AWS account secret key. This value is accessible from your AWS security credentials page. |
AWSRoleARN | The Amazon Resource Name of the role to use when authenticating. |
AWSPrincipalArn | The ARN of the SAML Identity provider in your AWS account. |
AWSRegion | The hosting region for your Amazon Web Services. |
AWSSessionToken | Your AWS session token. |
MFASerialNumber | The serial number of the MFA device if one is being used. |
MFAToken | The temporary token available from your MFA device. |
40.1.4. Azure Authentication
Property | Description |
AzureStorageAccount | The name of your Azure storage account. |
AzureAccessKey | The storage key associated with your Parquet account. |
AzureSharedAccessSignature | A shared access key signature that may be used for authentication. |
AzureTenant | The Microsoft Online tenant being used to access data. If not specified, your default tentant will be used. |
AzureEnvironment | The Azure Environment to use when establishing a connection. |
40.1.5. SSO
Property | Description |
SSOLoginURL | The identity provider's login URL. |
SSOProperties | Additional properties required to connect to the identity provider in a semicolon-separated list. |
40.1.6. OAuth
Property | Description |
InitiateOAuth | Set this property to initiate the process to obtain or refresh the OAuth access token when you connect. |
OAuthVersion | The version of OAuth being used. |
OAuthClientId | The client ID assigned when you register your application with an OAuth authorization server. |
OAuthClientSecret | The client secret assigned when you register your application with an OAuth authorization server. |
OAuthAccessToken | The access token for connecting using OAuth. |
OAuthAccessTokenSecret | The OAuth access token secret for connecting using OAuth. |
OAuthSettingsLocation | The location of the settings file where OAuth values are saved when InitiateOAuth is set to GETANDREFRESH or REFRESH. Alternatively, this can be held in memory by specifying a value starting with memory://. |
CallbackURL | The OAuth callback URL to return to when authenticating. This value must match the callback URL you specify in your app settings. |
OAuthGrantType | The grant type for the OAuth flow. |
OAuthPasswordGrantMode | How to pass Client ID and Secret with OAuthGrantType is set to Password. |
OAuthIncludeCallbackURL | Whether to include the callback URL in an access token request. |
OAuthAuthorizationURL | The authorization URL for the OAuth service. |
OAuthAccessTokenURL | The URL to retrieve the OAuth access token from. |
OAuthRefreshTokenURL | The URL to refresh the OAuth token from. |
OAuthRequestTokenURL | The URL the service provides to retrieve request tokens from. This is required in OAuth 1.0. |
OAuthVerifier | The verifier code returned from the OAuth authorization URL. |
AuthToken | The authentication token used to request and obtain the OAuth Access Token. |
AuthKey | The authentication secret used to request and obtain the OAuth Access Token. |
OAuthParams | A comma-separated list of other parameters to submit in the request for the OAuth access token in the format paramname=value. |
OAuthRefreshToken | The OAuth refresh token for the corresponding OAuth access token. |
OAuthExpiresIn | The lifetime in seconds of the OAuth AccessToken. |
OAuthTokenTimestamp | The Unix epoch timestamp in milliseconds when the current Access Token was created. |
40.1.7. JWT OAuth
Property | Description |
OAuthJWTCert | The JWT Certificate store. |
OAuthJWTCertType | The type of key store containing the JWT Certificate. |
OAuthJWTCertPassword | The password for the OAuth JWT certificate. |
OAuthJWTCertSubject | The subject of the OAuth JWT certificate. |
OAuthJWTIssuer | The issuer of the Java Web Token. |
OAuthJWTSubject | The user subject for which the application is requesting delegated access. |
40.1.8. SSL
Property | Description |
SSLClientCert | The TLS/SSL client certificate store for SSL Client Authentication (2-way SSL). |
SSLClientCertType | The type of key store containing the TLS/SSL client certificate. |
SSLClientCertPassword | The password for the TLS/SSL client certificate. |
SSLClientCertSubject | The subject of the TLS/SSL client certificate. |
SSLMode | The authentication mechanism to be used when connecting to the FTP or FTPS server. |
SSLServerCert | The certificate to be accepted from the server when connecting using TLS/SSL. |
40.1.9. SSH
Property | Description |
SSHAuthMode | The authentication method to be used to log on to an SFTP server. |
SSHClientCert | A certificate to be used for authenticating the user. |
SSHClientCertPassword | The password of the SSHClientCert certificate if it has one. |
SSHClientCertType | The type of SSHClientCert certificate. |
40.1.10. Firewall
Property | Description |
FirewallType | The protocol used by a proxy-based firewall. |
FirewallServer | The name or IP address of a proxy-based firewall. |
FirewallPort | The TCP port for a proxy-based firewall. |
FirewallUser | The user name to use to authenticate with a proxy-based firewall. |
FirewallPassword | A password used to authenticate to a proxy-based firewall. |
40.1.11. Proxy
Property | Description |
ProxyAutoDetect | This indicates whether to use the system proxy settings or not. This takes precedence over other proxy settings, so you'll need to set ProxyAutoDetect to FALSE in order use custom proxy settings. |
ProxyServer | The hostname or IP address of a proxy to route HTTP traffic through. |
ProxyPort | The TCP port the ProxyServer proxy is running on. |
ProxyAuthScheme | The authentication type to use to authenticate to the ProxyServer proxy. |
ProxyUser | A user name to be used to authenticate to the ProxyServer proxy. |
ProxyPassword | A password to be used to authenticate to the ProxyServer proxy. |
ProxySSLType | The SSL type to use when connecting to the ProxyServer proxy. |
ProxyExceptions | A semicolon separated list of destination hostnames or IPs that are exempt from connecting through the ProxyServer . |
40.1.12. Logging
Property | Description |
Logfile | A filepath which designates the name and location of the log file. |
Verbosity | The verbosity level that determines the amount of detail included in the log file. |
LogModules | Core modules to be included in the log file. |
MaxLogFileSize | A string specifying the maximum size in bytes for a log file (for example, 10 MB). |
MaxLogFileCount | A string specifying the maximum file count of log files. |
40.1.13. Schema
Property | Description |
Location | A path to the directory that contains the schema files defining tables, views, and stored procedures. |
BrowsableSchemas | This property restricts the schemas reported to a subset of the available schemas. For example, BrowsableSchemas=SchemaA,SchemaB,SchemaC. |
Tables | This property restricts the tables reported to a subset of the available tables. For example, Tables=TableA,TableB,TableC. |
Views | Restricts the views reported to a subset of the available tables. For example, Views=ViewA,ViewB,ViewC. |
FlattenObjects | Set FlattenObjects to true to flatten object properties into columns of their own. Otherwise, objects nested in arrays are returned as strings of JSON. |
FlattenArrays | By default, nested arrays are returned as strings. The FlattenArrays property can be used to flatten the elements of nested arrays into columns of their own. Set FlattenArrays to the number of elements you want to return from nested arrays. |
40.1.14. Caching
Property | Description |
AutoCache | Automatically caches the results of SELECT queries into a cache database specified by either CacheLocation or both of CacheConnection and CacheProvider . |
CacheDriver | The database driver to be used to cache data. |
CacheConnection | The connection string for the cache database. This property is always used in conjunction with CacheProvider . Setting both properties will override the value set for CacheLocation for caching data. |
CacheLocation | Specifies the path to the cache when caching to a file. |
CacheTolerance | The tolerance for stale data in the cache specified in seconds when using AutoCache . |
Offline | Use offline mode to get the data from the cache instead of the live source. |
CacheMetadata | This property determines whether or not to cache the table metadata to a file store. |
40.1.15. Miscellaneous
Property | Description |
AggregateFiles | When set to true, the provider will aggregate all the files in URI directory into a single result. With this option enabled, the AggregatedFiles will be exposed which can be used to query the dataset. |
BatchSize | The maximum size of each batch operation to submit. |
Charset | Specifies the session character set for encoding and decoding character data transferred to and from the Parquet file. The default value is UTF-8. |
ClientCulture | This property can be used to specify the format of data (e.g., currency values) that is accepted by the client application. This property can be used when the client application does not support the machine's culture settings. For example, Microsoft Access requires 'en-US'. |
Compression | Specifies which compression encoding to be used when creating .parquet files using Create Table Statement and Bulk Inserts. |
ConnectionLifeTime | The maximum lifetime of a connection in seconds. Once the time has elapsed, the connection object is disposed. |
ConnectOnOpen | This property species whether to connect to the Parquet when the connection is opened. |
Culture | This setting can be used to specify culture settings that determine how the provider interprets certain data types that are passed into the provider. For example, setting Culture='de-DE' will output German formats even on an American machine. |
DeleteDownloadedFiles | When set to true, the provider will delete parsed .parquet files downloaded from cloud sources. |
DirectoryRetrievalDepth | Limit the subfolders recursively scanned when IncludeSubdirectories is enabled. |
EnableDictionary | When set to true, the provider will enable dictionary encoding when creating .parquet files using Create Table Statement and Bulk Inserts. |
ExcludeFiles | Comma-separated list of file extensions to exclude from the set of the files modeled as tables. |
IncludeDropboxTeamResources | Indicates if you want to include Dropbox team files and folders. |
IncludeFiles | Comma-separated list of file extensions to include into the set of the files modeled as tables. |
IncludeSubdirectories | Whether to read files from nested folders. In the case of a name collision, table names are prefixed by the underscore-separated folder names. |
InsertMode | The behavior when using bulk inserts to create Parquet files. |
MaxRows | Limits the number of rows returned rows when no aggregation or group by is used in the query. This helps avoid performance issues at design time. |
MetadataDiscoveryURI | Used when aggregating multiple files into one table, this property specifies a specific file to read to determined the aggregated table schema. |
Other | These hidden properties are used only in specific use cases. |
PageSize | (Optional) PageSize value. |
ParallelPagingSizeLimit | Parquet file size limit (MegaBytes) for which to use parallel paging. |
PathSeparator | Determines the character which will be used to replace the file separator. |
PoolIdleTimeout | The allowed idle time for a connection before it is closed. |
PoolMaxSize | The maximum connections in the pool. |
PoolMinSize | The minimum number of connections in the pool. |
PoolWaitTime | The max seconds to wait for an available connection. |
PseudoColumns | This property indicates whether or not to include pseudo columns as columns to the table. |
RTK | The runtime key used for licensing. |
SupportEnhancedSQL | This property enhances SQL functionality beyond what can be supported through the API directly, by enabling in-memory client-side processing. |
TemporaryLocalFolder | The path, or URI, to the folder that is used to temporarily download parquet file(s). |
Timeout | The value in seconds until the timeout error is thrown, canceling the operation. |
UseConnectionPooling | This property enables connection pooling. |