-
Notifications
You must be signed in to change notification settings - Fork 16
Description
Specifications
- Client Version: influxdb3-python==0.3.6
- InfluxDB Version: 3.0 serverless
- Platform: mac
Code sample to reproduce problem
Any file in examples that provides the following code
wco = write_client_options(success_callback=callback.success,
error_callback=callback.error,
retry_callback=callback.retry,
WriteOptions=write_options
)
with InfluxDBClient3.InfluxDBClient3(token=token,
host=url,
org=org,
database=database, enable_gzip=True, write_client_options=wco) as _client:Examples:
https://github.com/InfluxCommunity/influxdb3-python/blob/main/Examples/batching_example.py
https://github.com/InfluxCommunity/influxdb3-python/blob/main/Examples/pokemon-trainer/write-batching.py
https://github.com/InfluxCommunity/influxdb3-python/blob/main/Examples/pokemon-trainer/write-batching-flight-calloptions.py
https://github.com/InfluxCommunity/influxdb3-python/blob/main/Examples/batching_example.py
The easiest way to observe this is by modifying the provided callback to measure the amount of data that is being sent:
class BatchingCallback(object):
def __init__(self):
self.lines = 0
def success(self, conf, data: str):
lines = len(data.decode().split('\n'))
self.lines += lines
print(f"Written batch: {conf}; {lines} lines; {self.lines} total")
def error(self, conf, data: str, exception: InfluxDBError):
print(f"Cannot write batch: {conf}, data: {data} due: {exception}")
def retry(self, conf, data: str, exception: InfluxDBError):
print(f"Retryable error occurs for batch: {conf}, data: {data} retry: {exception}")Expected behavior
Documented examples of batch sizing should send the batch sizes as expected.
Actual behavior
batch_size defaults back to 1000
Additional info
This appears to be a result of passing in keyword arguments in the form of **self._write_client_options, where the WriteAPI would expect write_options instead of the provided key of WriteOptions.
This can be seen in the output from the callback.
This is with the following code in place:
write_options = WriteOptions(batch_size=6_000,
flush_interval=10_000,
jitter_interval=2_000,
retry_interval=5_000,
max_retries=5,
max_retry_delay=30_000,
exponential_base=2)
wco = write_client_options(success_callback=callback.success,
error_callback=callback.error,
retry_callback=callback.retry,
WriteOptions=write_options
)$ python3 code/testing_client.py
Written batch: ('mig_test', '', 's'); 1000 lines; 1000 total
Written batch: ('mig_test', '', 's'); 1000 lines; 2000 total
Written batch: ('mig_test', '', 's'); 1000 lines; 3000 total
Written batch: ('mig_test', '', 's'); 1000 lines; 4000 total
Written batch: ('mig_test', '', 's'); 1000 lines; 5000 total
Written batch: ('mig_test', '', 's'); 1000 lines; 6000 total
Written batch: ('mig_test', '', 's'); 1000 lines; 7000 total
Written batch: ('mig_test', '', 's'); 1000 lines; 8000 total
Written batch: ('mig_test', '', 's'); 1000 lines; 9000 total
Written batch: ('mig_test', '', 's'); 1000 lines; 10000 total
Compared to the following:
write_options = WriteOptions(batch_size=6_000,
flush_interval=10_000,
jitter_interval=2_000,
retry_interval=5_000,
max_retries=5,
max_retry_delay=30_000,
exponential_base=2)
wco = write_client_options(success_callback=callback.success,
error_callback=callback.error,
retry_callback=callback.retry,
write_options=write_options
)$ python3 code/testing_client.py
Written batch: ('mig_test', '', 's'); 6000 lines; 6000 total
Written batch: ('mig_test', '', 's'); 4000 lines; 10000 total