33. Kafka 
33.1. Connection Options
33.1.1. Authentication
Property | Description |
AuthScheme | The scheme used for authentication. Accepted entries are Auto,Plain,Scram,Kerberos. |
User | The user who is authenticating to Apache Kafka. |
Password | The password used to authenticate to Apache Kafka. |
BootstrapServers | The address of the Apache Kafka BootstrapServers to which you are connecting to. |
Topic | The name of the topic to produce into. |
UseSSL | This field sets whether SSL is enabled. |
33.1.2. Connection
Property | Description |
ConsumerGroupId | Specifies which group the consumers created by the driver should belong to. |
AutoCommit | Specifies if the Apache Kafka consumer should autocommit after each poll. |
33.1.3. Kerberos
Property | Description |
KerberosKeytabFile | The Keytab file containing your pairs of Kerberos principals and encrypted keys. |
KerberosSPN | The service principal name (SPN) for the Kerberos Domain Controller. |
KerberosServiceName | The Kerberos service name you will authenticate. |
UseKerberosTicketCache | In case that you don't want to use a keytab file but instead a ticket cache with the logged in user. |
33.1.4. SSL
Property | Description |
SSLCert | Path to the Keystore location or to the client's certificate location. |
SSLClientCert | The TLS/SSL client certificate store for SSL Client Authentication (2-way SSL). |
SSLClientCertType | The type of key store containing the TLS/SSL client certificate. |
SSLCertPassword | The password of the Keystore or of the client's private key. |
SSLClientCertPassword | The password for the TLS/SSL client certificate. |
SSLClientCertSubject | The subject of the TLS/SSL client certificate. |
SSLServerCert | The certificate to be accepted from the server when connecting using TLS/SSL. |
SSLIdentificationAlgorithm | The endpoint identification algorithm used by the Apache Kafka data provider client app to validate server host name. |
SSLKey | Set either the password to decrypt the client private key or the path of the client's private key depending on the data provider edition. |
TrustStorePassword | The password of the Truststore, 'ssl.truststore.password'. |
TrustStorePath | Path to the Truststore or Certification Authorities (CA) location. |
33.1.5. SchemaRegistry
Property | Description |
RegistryVersion | Version of the schema read from RegistryUrl for the specified topic. |
RegistryUrl | The server for the schema registry. When this property is specified, the driver will read Apache Avro schema from the server. |
RegistryUser | Username to authorize with the server specified in RegistryUrl . |
RegistryPassword | Password to authorize with the server specified in RegistryUrl . |
RegistryType | Type of the schema specified for the a specific topic. |
RegistryService | The Schema Registry service used for working with topic schemas. |
33.1.6. Firewall
Property | Description |
FirewallType | The protocol used by a proxy-based firewall. |
FirewallServer | The name or IP address of a proxy-based firewall. |
FirewallPort | The TCP port for a proxy-based firewall. |
FirewallUser | The user name to use to authenticate with a proxy-based firewall. |
FirewallPassword | A password used to authenticate to a proxy-based firewall. |
33.1.7. Proxy
Property | Description |
ProxyAutoDetect | This indicates whether to use the system proxy settings or not. This takes precedence over other proxy settings, so you'll need to set ProxyAutoDetect to FALSE in order use custom proxy settings. |
ProxyServer | The hostname or IP address of a proxy to route HTTP traffic through. |
ProxyPort | The TCP port the ProxyServer proxy is running on. |
ProxyAuthScheme | The authentication type to use to authenticate to the ProxyServer proxy. |
ProxyUser | A user name to be used to authenticate to the ProxyServer proxy. |
ProxyPassword | A password to be used to authenticate to the ProxyServer proxy. |
ProxySSLType | The SSL type to use when connecting to the ProxyServer proxy. |
ProxyExceptions | A semicolon separated list of destination hostnames or IPs that are exempt from connecting through the ProxyServer . |
33.1.8. Logging
Property | Description |
Logfile | A filepath which designates the name and location of the log file. |
Verbosity | The verbosity level that determines the amount of detail included in the log file. |
LogModules | Core modules to be included in the log file. |
MaxLogFileSize | A string specifying the maximum size in bytes for a log file (for example, 10 MB). |
MaxLogFileCount | A string specifying the maximum file count of log files. |
33.1.9. Schema
Property | Description |
Location | A path to the directory that contains the schema files defining tables, views, and stored procedures. |
BrowsableSchemas | This property restricts the schemas reported to a subset of the available schemas. For example, BrowsableSchemas=SchemaA,SchemaB,SchemaC. |
Tables | This property restricts the tables reported to a subset of the available tables. For example, Tables=TableA,TableB,TableC. |
Views | Restricts the views reported to a subset of the available tables. For example, Views=ViewA,ViewB,ViewC. |
33.1.10. Caching
Property | Description |
AutoCache | Automatically caches the results of SELECT queries into a cache database specified by either CacheLocation or both of CacheConnection and CacheProvider . |
CacheDriver | The database driver to be used to cache data. |
CacheConnection | The connection string for the cache database. This property is always used in conjunction with CacheProvider . Setting both properties will override the value set for CacheLocation for caching data. |
CacheLocation | Specifies the path to the cache when caching to a file. |
CacheTolerance | The tolerance for stale data in the cache specified in seconds when using AutoCache . |
Offline | Use offline mode to get the data from the cache instead of the live source. |
CacheMetadata | This property determines whether or not to cache the table metadata to a file store. |
33.1.11. Miscellaneous
Property | Description |
AggregateMessages | Specifies whether or not to return the message as a whole string. |
BatchSize | The maximum size of each batch operation to submit. |
CompressionType | Data compression type. Batches of data will be compressed together. |
ConnectionLifeTime | The maximum lifetime of a connection in seconds. Once the time has elapsed, the connection object is disposed. |
ConnectOnOpen | This property species whether to connect to the Apache Kafka when the connection is opened. |
EnableIdempotence | If set to true, the Apache Kafka will ensure messages are delivered in the correct, and without duplicates. |
FlattenArrays | By default, nested arrays won't show up if TypeDetectionScheme is set to SchemaRegistry. The FlattenArrays property can be used to flatten the elements of nested arrays into columns of their own. Set FlattenArrays to the number of elements you want to return from nested arrays. |
GenerateSchemaFiles | Indicates the user preference as to when schemas should be generated and saved. |
MaximumBatchSize | Specifies maximum batch size to gather before sending a request. |
MaxRows | Limits the number of rows returned rows when no aggregation or group by is used in the query. This helps avoid performance issues at design time. |
MessageKeyColumn | If specified, the message key sent to Apache Kafka will be read from this column. |
MessageKeyType | If MessageKeyColumn is specified, this property must be set to the expected type for the pertinent column. |
OffsetResetStrategy | Specifies an offset for the consumer group. |
Other | These hidden properties are used only in specific use cases. |
PoolIdleTimeout | The allowed idle time for a connection before it is closed. |
PoolMaxSize | The maximum connections in the pool. |
PoolMinSize | The minimum number of connections in the pool. |
PoolWaitTime | The max seconds to wait for an available connection. |
ProduceMeta | Specifies whether or not to send a meta message while producing the outgoing message. |
PseudoColumns | This property indicates whether or not to include pseudo columns as columns to the table. |
ReadDuration | The duration which additional messages are allowed. |
Readonly | You can use this property to enforce read-only access to Apache Kafka from the provider. |
RowScanDepth | The maximum number of messages to scan for the columns available in the topic. |
RTK | The runtime key used for licensing. |
SerializationFormat | Specifies how to serialize/deserialize the incoming or outgoing message. |
SupportEnhancedSQL | This property enhances SQL functionality beyond what can be supported through the API directly, by enabling in-memory client-side processing. |
Timeout | The value in seconds until the timeout error is thrown, canceling the operation. |
TypeDetectionScheme | Comma-separated list of options specifying how the provider will scan the data to determine the fields and datatypes for the bucket. |
UseConnectionPooling | This property enables connection pooling. |