Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically set mqtt buffer size #637

Merged
merged 4 commits into from
Jun 18, 2020
Merged

Automatically set mqtt buffer size #637

merged 4 commits into from
Jun 18, 2020

Conversation

balvig
Copy link
Contributor

@balvig balvig commented Jun 16, 2020

Thank you for creating (and maintaining!) this fantastic project!

I happened to notice that current versions of pubsubclient allow the buffer size to be set
dynamically: knolleary/pubsubclient#282

We can take advantage of this to avoid having to modify one of the
dependencies and instead automatically increase the buffer size when a
powerful board is detected.

It also reduces a step in the documentation.

My first PR to an Arduino project so let me know if I'm doing something
wrong! 😅 🙏

I suppose this would mean changing the version of the library that is bundled with the
project (PubSubClient_ID89?) as well, but I'm not sure how to go about doing so...? 🙇

Current versions of pubsubclient allows the buffer size to be set
dynamically: knolleary/pubsubclient#282

We can take advantage of this to avoid having to modify one of the
dependencies and instead automatically up the buffer size when a
powerful board is detected.
@1technophile
Copy link
Owner

Thank you for creating (and maintaining!) this fantastic project!

Thanks for the feedback!

We can take advantage of this to avoid having to modify one of the
dependencies and instead automatically increase the buffer size when a
powerful board is detected.

Well seen, this will avoid confusions regarding the MQTT MAX PACKET SIZE modifications.

I suppose this would mean changing the version of the library that is bundled with the
project (PubSubClient_ID89?) as well, but I'm not sure how to go about doing so...?

It will be changed when we release a new version, and done automaticaly by the github actions setup made by @Legion2

I begin the review, thanks for your participation!

@1technophile 1technophile self-requested a review June 16, 2020 17:50
@@ -401,6 +401,7 @@ void connectMQTT() {
char topic[mqtt_topic_max_size];
strcpy(topic, mqtt_topic);
strcat(topic, will_Topic);
client.setBufferSize(mqtt_max_packet_size);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok for this change,

below you can find also a warning

#if defined(ESP8266) || defined(ESP32) || defined(__AVR_ATmega2560__) || defined(__AVR_ATmega1280__)
  if (MQTT_MAX_PACKET_SIZE == 128)
    Log.error(F("WRONG PUBSUBCLIENT LIBRARY USED PLEASE INSTALL THE ONE FROM RELEASE PAGE" CR));
#endif

I think it could be removed and the trace just before log mqtt_max_packet_size instead of MQTT_MAX_PACKET_SIZE

Comment on lines 102 to 107
# define mqtt_max_packet_size 1024
#else
# define parameters_size 15
# define mqtt_topic_max_size 50
# define mqtt_max_packet_size 128
#endif
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should apply clang format to the files, (we didn't updated the dev docs yet sorry).
Install the Clang-format extension to PIO and do a right click inside the main window of the file and click "format document"

@1technophile 1technophile added this to the v0.9.4 milestone Jun 16, 2020
@balvig
Copy link
Contributor Author

balvig commented Jun 16, 2020

Thank you @1technophile!

I actually discovered cd3a2a0 which seemed to suggest the feature was already there at some point, but then later removed?

I therefore started having doubts whether maybe there was a problem with it I'm overlooking...if you think it looks ok however, I will take a look at the comments above and apply the changes to this PR 👍

@1technophile
Copy link
Owner

I actually discovered cd3a2a0 which seemed to suggest the feature was already there at some point, but then later removed?

Yes later removed due to the fact it was not working, the #define was not taken into account.

I therefore started having doubts whether maybe there was a problem with it I'm overlooking...if you think it looks ok however, I will take a look at the comments above and apply the changes to this PR 👍

No doubt your PR is still relevant ;-)

@balvig
Copy link
Contributor Author

balvig commented Jun 17, 2020

@1technophile I found these as well 🙇 :

## You don't see the messages appearing on your broker but they appears on the serial monitor
This is due to a too small mqtt packet size, open pubsubclient.h from pubsubclient library and set:
try to modify in pubsubclient.h:
`#define MQTT_MAX_PACKET_SIZE 1024`
The other option is to use the [pubsubclient library](https://github.com/1technophile/OpenMQTTGateway/tree/master/lib/pubsubclient) provided in the OMG repository folder.

And into platformio.ini
`MQTT_MAX_PACKET_SIZE=1024`
by
`MQTT_MAX_PACKET_SIZE=1280`

-DMQTT_MAX_PACKET_SIZE=1024

-DMQTT_MAX_PACKET_SIZE=1024

I suppose we should do something about those as well? I assume they will get ignored if we begin using setBufferSize() 🤔 What do you think?

@1technophile
Copy link
Owner

Well seen ! 😉
All these files should be modified also to take into account the automatic setting you've added.
You can remove MQTT_MAX_PACKET_SIZE from the .ini definitions and also from the docs.

@balvig balvig marked this pull request as ready for review June 18, 2020 10:19
@balvig
Copy link
Contributor Author

balvig commented Jun 18, 2020

@1technophile I had a crack at updating the docs in 50819d3, seemed that some of the tips in there were still relevant, so I simply updated the place to make changes, but let me know if I'm off track!

@1technophile
Copy link
Owner

No, you are on tracks ;-). Thanks

@1technophile 1technophile merged commit 1e59941 into 1technophile:development Jun 18, 2020
@balvig balvig deleted the mqtt_buffer branch June 18, 2020 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants