Looks like its working nicely here
I had to put str() round the setting to start with to get it to work but after that it worked fine:
self._log.info("Setting " + self.name + " compress: " + str(setting))
In order to test I added additional logging to check the compression ratio and verify that it was running the compression, the url encoded output was also confusion so I changed the log to show the content prior to url encoding as you suggested.
I found that in the case of a single item in the buffer the compressed size is larger than the non compressed size so have added the check to use compression only when it does reduce the data size:
# Construct post body
post_body_data = {"data": data_string, "sentat": sentat}
if self._settings['compress']:
# Compress data and encode as hex string.
compressed = zlib.compress(post_body_data["data"]).encode("hex")
compression_ratio = 100 * len(compressed) / len(post_body_data["data"]);
if compression_ratio<100:
post_body_data["data"] = compressed
# Set flag.
post_body_data["c"] = 1
# Log compression ratio
self._log.info("compressed data size " + str(compression_ratio) + "% of original")
post_body = urllib.urlencode(post_body_data)
# logged before apikey added for security
self._log.info("sending url:" + post_url + "E-M-O-N-C-M-S-A-P-I-K-E-Y, body data:")
self._log.info(post_body_data)
The result in emonhub.log:
Buffer size: 1
2018-12-19 11:36:20,240 DEBUG emoncmsorg Buffer size: 1
2018-12-19 11:36:20,244 INFO emoncmsorg compressed data size 162% of original
2018-12-19 11:36:20,245 INFO emoncmsorg sending url:http://192.168.0.132/emoncms/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y, body data:
2018-12-19 11:36:20,247 INFO emoncmsorg {'data': '[[1545219379.938908,10,166,928,0,0,252.49,21.75,300,300,77281,41181,0,0,0,-55]]', 'sentat': 1545219380}
Buffer size: 5
2018-12-19 11:36:50,354 DEBUG emoncmsorg Buffer size: 5
2018-12-19 11:36:50,356 INFO emoncmsorg compressed data size 77% of original
2018-12-19 11:36:50,357 INFO emoncmsorg sending url:http://192.168.0.132/emoncms/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y, body data:
2018-12-19 11:36:50,358 INFO emoncmsorg {'c': 1, 'data': '789c75d0eb0dc3200c04e0855ccb3edb606689b2ff1a7589449236e5f1e704a70fb64dc3033a2c07e768dd9554489bd34092d444803de51c2028f7201399bb7724c855d3e605a157c44eab79184b58c209568b5de4bb0db7c8c885b57ab49afcda54c666185846d3656cfad7d59e5c2e51ef02e470e1c725e4c1760beb28e72d296c299b5c7b3f4a54f5a1d4fac9762aed4169533996d2f7fd0dccec50be', 'sentat': 1545219410}
Buffer size: 166
2018-12-19 11:56:23,625 DEBUG emoncmsorg Buffer size: 166
2018-12-19 11:56:23,629 INFO emoncmsorg compressed data size 43% of original
Would you be happy with these changes?