Configuration Settings
Note
Possible values of Configuration Settings having Type “Log Level” are TRACE, DEBUG, INFO, WARN, ERROR, FATAL, or OFF.
The following table lists the Hyperscale Compliance properties with their default values.
Group | Property name | Type | Description | Default value |
---|---|---|---|---|
Controller Service | API_KEY_CREATE | Boolean | This property is by default uncommented to have the container create a new API key and print it in the logs when starting. Since the value is in the logs, this API key should only be used to bootstrap the creation of other - more secure - API keys and be discarded. Comment it once bootstrap key is available. |
true |
load.service.requirePostLoad | Boolean | Set if the Post Load step needs to be executed. | true | |
source.key.field.names | String | Dataset configuration. These fields/columns are used to uniquely identify source data. | schema_name,table_name | |
execution.status.poll.duration | Milli-seconds | Time duration in which execution status is collected from different services | 120000 | |
logging.level.root | Log Level | Logging configuration. This spring boot configuration controls the logging of all the packages/libraries getting used in application. NOTE: Increasing this Log Level will produce too many logs. It is recommended to keep this log level to WARN or below. | WARN | |
logging.level.com.delphix.hyperscale | Log Level | This configuration controls the logging level of Hyperscale specific packages. This log level can be increased if Hyperscale service specific actions needs to be monitored closely. NOTE: It is recommended to keep this log level to INFO. Increasing the log level can impact application’s performance. | INFO | |
logging.file.name1 | String | Log file location & name | /opt/delphix/logs/hyperscale-controller.log |
|
logging.pattern.file | String | Logging pattern for file | %d{dd-MM-yyyy HH:mm:ss.SSS} \[%thread\] %-5level %logger{36}.%M - %msg%n |
|
logging.pattern.console | String | Logging pattern for console | %d{dd-MM-yyyy HH:mm:ss.SSS} \[%thread\] %-5level %logger{36}.%M - %msg%n |
|
logging.pattern.rolling-file-name1 | String | Archived file location & name | /opt/delphix/logs/archived/hyperscale-controller-%d{yyyy-MM-dd}.%i.log |
|
logging.file.max-size | File Size (String) | Max individual file size | 5MB | |
logging.file.max-history | Number of Days | History in days (i.e. keep 15 days’ worth of history capped at 5GB total size) | 15 | |
logging.file.total-size-cap | File Size (String) | Max limit the combined size of log archives | 5GB | |
logging.level.org.springframework.web. filter.CommonsRequestLoggingFilter |
Log Level | This configuration controls the logging information of the HTTP requests received by Hyperscale. This is by default set to DEBUG level for logging request URIs of the Incoming Requests. | DEBUG | |
api.version.compatibility.strict.check | Boolean | These properties are used to check the version compatibility. Setting this as true will enable strict comparison of API versions of different services. In strict comparison, the complete version i.e x.y.z is compared while in other case when this property is set to false, only major version(x out of x.y.z) of APIs will be compared. | true | |
api.version.compatibility.retry.count | Number | These properties are used to check the version compatibility. Number of times to retry the comparison if the services are not compatible. | 3 | |
api.version.compatibility.retry.wait.time | Time in milli-seconds | These properties are used to check the version compatibility. Time to wait before next retry if the services are not compatible. | 10000 | |
Unload Service | unload.fetch.rows | Number | Number of rows to be fetched from the database at a time. | 10000 |
logging.level.root | Log Level | This spring boot configuration controls the logging of all the packages/libraries getting used in application. NOTE: Increasing this Log Level will produce too many logs. It is recommended to keep this log level to WARN or below. | WARN | |
logging.level.com.delphix.hyperscale | Log Level | This configuration controls the logging level of Hyperscale specific packages. This log level can be increased if Hyperscale service specific actions needs to be monitored closely. NOTE: It is recommended to keep this log level to INFO. Increasing the log level can impact application’s performance. | INFO | |
logging.file.name1 | String | Log file location & name | /opt/delphix/logs/hyperscale-unload.log |
|
logging.pattern.file | String | Logging pattern for file | %d{dd-MM-yyyy HH:mm:ss.SSS} \[%thread\] %-5level %logger{36}.%M - %msg%n |
|
logging.pattern.console | String | Logging pattern for console | %d{dd-MM-yyyy HH:mm:ss.SSS} \[%thread\] %-5level %logger{36}.%M - %msg%n |
|
logging.pattern.rolling-file-name1 | String | Archived file location & name | /opt/delphix/logs/archived/hyperscale-unload-%d{yyyy-MM-dd}.%i.log |
|
logging.file.max-size | File Size in String | Max individual file size | 5MB | |
logging.file.max-history | Number of Days | History in days (i.e. keep 15 days’ worth of history capped at 5GB total size) | 15 | |
logging.file.total-size-cap | File Size in String | Max limit the combined size of log archives | 5GB | |
logging.level.org.springframework.web. filter.CommonsRequestLoggingFilter |
Log Level | This configuration controls the logging information of the HTTP requests received by Hyperscale. This is by default set to DEBUG level for logging request URIs of the Incoming Requests. | DEBUG | |
Masking Service | logging.level.root | Log Level | This spring boot configuration controls the logging of all the packages/libraries getting used in application. NOTE: Increasing this Log Level will produce too many logs. It is recommended to keep this log level to WARN or below. | WARN |
logging.level.com.delphix.hyperscale | Log Level | This configuration controls the logging level of Hyperscale specific packages. This log level can be increased if Hyperscale service specific actions needs to be monitored closely. NOTE: It is recommended to keep this log level to INFO. Increasing the log level can impact application’s performance. | INFO | |
logging.file.name1 | String | Log file location & name | /opt/delphix/logs/hyperscale.log |
|
logging.pattern.file | String | Logging pattern for file | %d{dd-MM-yyyy HH:mm:ss.SSS} \[%thread\] %-5level %logger{36}.%M - %msg%n |
|
logging.pattern.console | String | Logging pattern for console | %d{dd-MM-yyyy HH:mm:ss.SSS} \[%thread\] %-5level %logger{36}.%M - %msg%n |
|
logging.pattern.rolling-file-name1 | String | Archived file location & name | /opt/delphix/logs/archived/hyperscale-%d{yyyy-MM-dd}.%i.log |
|
logging.file.max-size | File Size in String | Max individual file size | 5MB | |
logging.file.max-history | Number of Days | History in days (i.e. keep 15 days’ worth of history capped at 5GB total size) | 15 | |
logging.file.total-size-cap | File Size in String | Max limit the combined size of log archives | 5GB | |
logging.level.org.springframework.web. filter.CommonsRequestLoggingFilter |
Log Level | This configuration controls the logging information of the HTTP requests received by Hyperscale. This is by default set to DEBUG level for logging request URIs of the Incoming Requests | DEBUG | |
remove.splitted.masked.files | Boolean | Flag to indicate retaining or removing staging splitted masked files. This is only applicable when merging of files is done post Masking and before Loading into the target data source. |
true | |
monitor.initial.delay | Time(in seconds) | Time to delay the first monitoring call to Masking Engines. This is related to Monitoring of Masking Engine Jobs. |
1 | |
monitor.poll.duration | Time(in seconds) | Polling period between subsequent monitoring calls to Masking Engines. This is related to Monitoring of Masking Engine Jobs. |
10 | |
scheduler.poll.duration | Time(in seconds) | Rate at which scheduler polls for available data to schedule load balancing. Configured in seconds. | 60 | |
intelligent.loadbalance.enabled | Boolean | Set this to false if need to enable round robin load balancing in place of intelligent load balancing. | true | |
job.monitor.pool.corePoolSize | Number | Job monitor thread pool related configuration Number of core threads. This is related to Monitoring of Masking Engine Jobs. |
5 | |
job.monitor.pool.maxPoolSize | Number | Job monitor thread pool related configuration Maximum number of threads in thread pool. This is related to Monitoring of Masking Engine Jobs. |
20 | |
job.monitor.pool.keepAliveSeconds | Time in seconds | Job monitor thread pool related configuration Maximum idle time of threads. This is related to Monitoring of Masking Engine Jobs. |
300 | |
job.monitor.pool.queueCapacity | Number | Job monitor thread pool related configuration Maximum queue capacity. This is related to Monitoring ofMasking Engine Jobs. |
50 | |
Load Service | sqlldr-success-message | String | Message printed by sqlldr on successful loading of data. | ‘successfully loaded.’ |
logging.level.root | Log Level | This spring boot configuration controls the logging of all the packages/libraries getting used in application. NOTE: Increasing this Log Level will produce too many logs. It is recommended to keep this log level to WARN or below. | WARN | |
logging.level.com.delphix.masking | Log Level | This configuration controls the logging level of the Masking Driver Support package. This Log Level can be increased when Driver Support Steps of Load Process need to be monitored closely. | INFO | |
logging.level.com.delphix.hyperscale | Log Level | This configuration controls the logging level of Hyperscale specific packages. This log level can be increased if Hyperscale service specific actions needs to be monitored closely. NOTE: It is recommended to keep this log level to INFO. Increasing the log level can impact application’s performance. | INFO | |
logging.file.name1 | String | Log file location & name | /opt/delphix/logs/hyperscale-load.log |
|
logging.pattern.file | String | Logging pattern for file | %d{dd-MM-yyyy HH:mm:ss.SSS} \[%thread\] %-5level %logger{36}.%M - %msg%n |
|
logging.pattern.console | String | Logging pattern for console | %d{dd-MM-yyyy HH:mm:ss.SSS} \[%thread\] %-5level %logger{36}.%M - %msg%n |
|
logging.pattern.rolling-file-name1 | String | Archived file location & name | /opt/delphix/logs/archived/hyperscale-load-%d{yyyy-MM-dd}.%i.log |
|
logging.file.max-size | File Size in String | Max individual file size | 5MB | |
logging.file.max-history | Number of Days | History in days (i.e. keep 15 days’ worth of history capped at 5GB total size) | 15 | |
logging.file.total-size-cap | File Size in String | Max limit the combined size of log archives | 5GB | |
logging.level.org.springframework.web. filter.CommonsRequestLoggingFilter |
Log Level | This configuration controls the logging information of the HTTP requests received by Hyperscale. This is by default set to DEBUG level for logging request URIs of the Incoming Requests. | DEBUG | |
sqlldr.blob.clob.char.length | Number | SQLLDR properties | 20000 |
Info
- For each service, the file path(absolute) configured for
logging.file.name
and forlogging.pattern.rolling-file-name
has to be the same. This path is a path inside the respective container. - For each service, if the log files(configured through
logging.file.name
andlogging.pattern.rolling-file-name
) need to be accessed outside the container, respective log path has to be mounted by adding volume binding of that path indocker-compose.yaml
for that service.
- For each service, the file path(absolute) configured for