-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[feature request] support ESP-IDF platform #925
Comments
Using multiple SPI slave devices on the same bus shouldn't be a problem if
|
I looked at LennartF22/RF24@nRF24:RF24:master...master and found some breaking changes that wouldn't be welcome here, specifically c'tor and Our HAL is primarily done through utility drivers that try to use a platform's native implementations wrapped in an Arduino-like API (see utility folder and RF24_config.h). Some platforms need special invocations (in RF24.cpp) to behave as expected, thus all the |
I just saw that @stefan123t created this issue. Here's some input from my side: We have a little bit of a special situation with the "OpenDTU Fusion" board mentioned above. On that board, the nRF24 and the CMT2300 are routed to two different sets of GPIO pins of the ESP32-S3, mainly because the CMT2300 uses 3-wire SPI, while the nRF24 uses standard SPI. Recently, an Ethernet shield with PoE has been designed for the "OpenDTU Fusion" board, but due to the ESP32-S3 not having an integrated Ethernet RMII controller anymore, we had to use an external SPI Ethernet controller. This is problematic, because we are already using both available SPI controllers of the ESP32, and the externally available pins of the "OpenDTU Fusion" board do not expose the SPI buses of the two RF ICs. Basically, we have 3 SPI slaves that are connected to 3 different sets of pins on the ESP32. I solved this by re-routing the internal signals to the SPI peripheral inside the ESP32 on demand in software (i.e. when a register of an SPI slave needs to be accessed), which works quite well. To get this to work with your RF24 library, I created a virtual HAL class with methods for doing SPI transfers and stuff like toggling the CE pin. A user of the library could just implement the virtual class to match their hardware, and pass an instance of that class to the RF24 "begin" method. In my fork (https://github.com/LennartF22/RF24) I just implemented a quick and dirty proof of concept, which obviously does not maintain compatibility with the existing interface, so for it to be merged, I would make this feature optional (for example by using a define). Our main question is, whether something like this would be merged (if implemented properly), or whether you have a different idea/approach that could help us. |
@2bndy5 That's correct, my fork is just a quick proof of concept. If @stefan123t had informed me that he was about to create this issue/question (maybe he did and I overlooked it), I would have told him to mention this upfront 😅 |
The way I see it, this is your main problem. Is there a pin compatible alternative that uses a CS signal? I see there's a CMT2300A chip, but I'm not familiar with this OpenDTU Fusion PCB.
TL;DR: We are not looking to overhaul our current HAL approach. With implementation aside, the HAL concept proposed is not likely to be merged because we already have a bunch of |
On a kind of related note, I've been considering namespacing our utility drivers for various reasons. |
@LennartF22 well I thought it is better to ask this as a question first as Brendan and @TMRh20 are quite responsive anyway, before you are going through the troubles of creating a PR forth-and-back again.
e.g.
@2bndy5 what do you have in mind considering namespacing ? |
I did some experimenting with this in my CirquePinnacle library when porting it to Linux and PicoSDK (similarly to how HAL is done here but with I2C support). My main motivation with namespacing is that our HAL drivers shouldn't be used by other Arduino libraries on Linux because the Arduino-wrapping API can be quite library-specific (and there's always a concern for naming conflicts of structures like EDIT: see #926 for a more practical proposal.
There is a template folder that is supposed be used as a starting off point for new HAL drivers... The idea is that the compiler would respect the following steps:
If this doesn't suite your project's needs, then it might be more feasible to
I've already voiced my hesitance to revising our current HAL implementation, but I'm not the only maintainer here. It would be nice to have only radio-specific code in RF24.cpp, but I'd rather not fix what isn't broken. |
@LennartF22 I would be more enthusiastic about a utility driver that is meant for ESP-IDF-only projects. But I'm not sure if such a driver would conflict with the Arduino framework on ESP boards. |
I already had a look at how you realize hardware abstraction yesterday (but didn't have time to answer), and we should be able to make that approach work for us. I must admit that I had not looked much into that (back when I modified the RF24 library for our needs), mainly because I just wanted to quickly get our 3 SPI devices to work, and had no intention of getting it merged here in that form. Regarding our somewhat special hardware situation: Although there might be a comparable RF IC with normal SPI, we cannot really change that anymore as well above 100 boards have already been made and distributed, so unfortunately software remains the only solution. My plan right now is to implement a driver that directly uses the ESP-IDF (instead of the Arduino layer), with the additional feature of being able two specify a pair of functions that is called before/after SPI transfers. Ideally, the I don't know when I'll have time for that though, as I'm quite busy right now (and I'm also still waiting for feedback from the OpenDTU maintainer). |
Hi, HW dev of the board here. regarding the RF-IC: No, there is unfortunately no alternative. The CMT2300A for 868MHz with the needed functionality is the only IC out there we can use. Unfortunately they use this stupid 3-wire SPI. That is also the reason why we ran them on seperate ESP pins as doing otherwise and just switching slave select surely could cause electrical trouble. Thanks for this fruitful discussion here and considering this, but I admit as a hardware guy who has just some firmware knowledge this architectural discussion for the driver is a bit above my head^^ |
Yeah! 🎉 These additional functions sound like Arduino's
The
Thanks @markusdd for answering my query there. I wish Texas Instruments had an easier to use RF development kit... Then, you'd surely be able to market your own sub-GHz RF module or embed it into the Fusion PCB. |
@2bndy5 thanks for the suggestions using a generic flag, Regarding the RF-IC we have used the Nordic Semiconductor NRF24L01+ simply because the other side of the DTU is a common Solar Micro-Inverter here in EU, which has a communication module built-in (enclosed in housing/thermal jelly mass) that sepcifically relies on 250kBps Enhance ShockBurst (ESB) protocol in their communication. In order to open source this we already have reverse engineered the majority of the Gen2 & Gen3 protocol's on top of that, but emulating Nordic's ESB protocol on some other RF-IC seems overkill for our purpose. Have in mind that the vendor switched to Sub-1GHz communication using a CMT2300A for the models which came out last year and further modified the latest model range to include a WiFi connection since this year. We are attempting to serve all three communication methods using the so called OpenDTU Fusion board sic, hence the name. |
I was wondering about the various forms of wireless used... That historical insight seems like a good tidbit for the readme section on compatible inverters. Their design decisions may have something to do with wireless regulations imposed by the UN's ITU. |
For my own reference, this guide outlines how to integrate third-party libs (called "components") into an ESP-IDF based project. Following this design when implementing RF24 HAL wrappers for the ESP-IDF should make it easier for devs to integrate RF24* libs into their ESP-IDF project(s). |
Another hint toward extending support for ESP-IDF: https://github.com/LennartF22/OpenDTU/blob/spi-rework/lib/Hoymiles/src/nrf_hal.cpp |
Well, I started a branch (named esp-idf) to begin supporting the ESP-IDF as a separate platform (independent of the Arduino platform). I haven't written any examples or performed any test builds, but I did fill out the HAL API to follow our current conventions... It's going to take me a while to understand how to use the ESP-IDF to build apps. I think I'll just use PlatformIO in CI since it already has support for the
|
@LennartF22 does the branch Brendan started yesterday already qualify for integration into OpenDTU ? I just cross checked with the ESP-IDF component registry and could only spot the LICENSE & README files missing for conformity. |
Thanks, I can use whatever help I can get. I am a completely new to ESP-IDF. Currently, the code compiles but it doesn't work. One of the calls to I pushed my platformIO project for inspection (located in examples_esp/getting_started/).
Thanks for the tip. I'm planning to do that when the branch is ready to merge into master. The next release will likely be v1.5.0. For now, you can simply use git specification in the idf_component.yml. dependencies:
nRF24/RF24:
version: esp-idf
git: https://github.com/nRF24/RF24.git The example project(s) will instead use: RF24/examples_esp/getting_started/src/idf_component.yml Lines 5 to 9 in d048b2d
so we are running the examples_esp/ projects with the current repo changes. |
In the US (California). This is public info as far as I'm concerned. I do have solar panels on top of my house, but there's no battery or monitoring capability enabled. I think the new tankless water heater uses a ESP32 to control the recirculating pump, but I haven't tinkered with it because it isn't broken. |
@2bndy5 i asked the OpenDTU Fusion board resellers and they would be able to send you one to CA/USA. However @markusdd may tell us if the latest board rev would allow you to access the Serial Debug Port on the OpenDTU Fusion ? @LennartF22 has added a new PR to make use of the SPIManager as an abstraction between the user code and other libraries used, ie nRF24L01+, CMT2300A, W5500. This is possible by passing an |
@stefan123t I already got a slightly modified version of @2bndy5's RF24 driver for the ESP-IDF running. The main issue probably was that |
The PR LGTM, but it still uses Arduino API. I'm not sire how it is relevant to this feature request. If you want more eyes reviewing your code/contributions, might I suggest coderabbitai? It can summarize and review PRs for you (and you can set it to output in German 😉 ). @LennartF22 Thank you for taking the initiative! 🚀 Would you mind if I cherry-pick (and squash) commits from your fork's esp-idf branch? I see there are merge conflict that would make a PR to upstream difficult. |
Yeah, the OpenDTU PR currently does not contain any interesting changes regarding the usage of the RF24 library, so there we do not use a shared SPI bus for the nRF yet. There is another commit that‘s not included in the PR that changes this: LennartF22/OpenDTU@1b9f181 I'm still not particularly happy with how I solved the issue that we are using the Arduino Framework in OpenDTU, but still need to use a „downgraded“ version of the RF24 ESP-IDF that does not redefine the functions already defined by the Arduino framework. Apart from the specific, aforementioned hacks (like the renaming of the I've already thought about a better/simpler approach for our situation, but more on that later. |
This commit explains the I'm working to merge the useful parts now. I was unable to save the commit author info after I squashed the numerous commits you pushed to your fork. I'll tag you in the PR that will merge into our esp-idf branch (not ready for master yet), and I'll add you as a co-author in the merge commit's trailer. First I'll have to test it... 🤞🏼 |
Yikes, that git history is a mess. I think that there were a lot of LF -> CRLF changes (undesirable). It looks like you I think I'm going to try to just take some improvements from your branch manually. I especially like the use of semaphore mutex in
I also don't see this change in your remote. Maybe it hasn't been pushed from your local? |
@stefan123t Sounds good. Although, if there's no exposed DAP, then I'm afraid it would be pointless to send me one. I have several ESP32 boards lying around, but none of them have a DAP exposed. I have a few debug probes sitting around: STLink V2 and a Segger J-link (for non-commercial purposes). Having no DAP is what halted my progress months ago. |
My I would expect cherry picking to work as usual, although there might be so merge conflicts at locations where there have been changes in the master that are not in your In general though, I think it makes more sense for you to just look through my commits, and just apply relevant changes to your branch, anyways, since some commits contain „desired“ and „undesired“ changes at once. I guess having the mutex does not hurt, but ideally, the caller would have to make sure that they are not calling RF24 functions from different tasks/threads in parallel, and I‘m not sure whether the rest of the library is thread safe already. I had to (at least temporarily) add it, because in OpenDTU, the RF24 library is used from different tasks in parallel (although I think it's wrong to do so and that it must be fixed in OpenDTU in the future). I did not change the |
The ESP-IDF docs were somewhat confusing about that. My limited understanding is that field was specific to Interrupt or DMA usage. Maybe I'm using DMA without knowing it. Would a higher value like 3 be better (because there's 3 slots in the radio's FIFOs)? |
The ESP32-S3 on the OpenDTU Fusion board has an integrated USB serial/JTAG controller, which can be accessed via USB-C. Regular JTAG pins are also exposed via 2.54mm headers. |
I've only got ESP32-S2 or older. I tend to spend money on boards that I can also test with CircuitPython. |
Since we are not using queued transactions, setting the
|
As far as I could determine the ESP32-S3 can be debugged via the USB-C Port using the D-/D+ USB Data lines connected to GPIO19 / 20 respectively, connect GND and VCC (if not already powered via the USB Serial !). See here: Configure ESP32-S3 Built-in JTAG Interface and here: JTAG Debugging As hardware you may use a whole range of JTAG devices like STLink V2, Segger J-Link, OpenOCD, an Arduino, etc. The ESP32-S2 may also be used with JTAG, though this comes under several CAVEATs, see here JTAG Debugging - Configure and Connect JTAG Interface, here for the Espressif DevBoard Configure ESP32-S2-Kaluga-1 JTAG Interface and here to Configure Other JTAG Interfaces. Tim from Alliance App's will contact you on your mail to ask for the address details. |
I can report that setting the queue to 1 gets past the My USB Serial output
Above I'm using GPIO17 for the CE pin and GPIO9 for the CSN pin. I'm guessing the lines about these pins are the result of the wrapper's @stefan123t I did respond to Tim from Alliance App, but I have not heard back since. I'm guessing things are in the works and there's nothing to report there. |
@2bndy5 Did you push your latest commits somewhere already? I could do some debugging later today. |
I already rebased the esp-idf branch on master. I just pushed the changes (most of which are about the WIP example). |
@2bndy5 the OpenDTU-Fusion board is already heading your way. Tim will send you a reply with the Shipping ID later. |
The user in #1004 referenced a lib call ESP-IDF-MIRF which kind of does exactly what we're trying to do here. This could be a good reference/influence; it is MIT licensed. |
@stefan123t Just got the OpenFusionDTU board in the mail today 🎉. I am extra surprised to have the add-on module (equipped with a Ethernet port) 👀. I have my LAN piped through my power lines, but I don't have PoE capable switch... I'll investigate this later. I'm still getting my bearing though. I think I found the pinout for the nRF24 builtin to the board. The site https://www.opendtu.solar seems to be down, so I had to build and serve the docs locally. I'm also still learning to navigate the various openDTU repos. |
@2bndy5 You can switch between PoE and USB power with a jumper. You can find further documentation for the Fusion board here: https://github.com/markusdd/OpenDTUFusionDocs |
Yeah I'm exploring that repo as well. Lots to take in here. |
@2bndy5 good to read that it arrived, Tim already notified me yesterday that it should arrive soon. You may change the config by editing and uploading the pin_mapping.json file under Settings > Config Management. |
Oh, my network's DNS sink hole was blocking the site. Usually, it just blocks ads and analytics across the entire network. opendtu.solar got blocked under the rule
He did not, but that's ok. I have a ST7789 display that I used to test our new scannerGraphic Arduino example. I found a driver lib for the ST7789 chip that uses ESP-IDF, which is authored by the same person that wrote that ESP-IDF-MIRF lib referenced in #1004 . Unfortunately, I couldn't find a driver lib for the CMT2300A that is compatible with the ESP-IDF. @LennartF22 Did you plan on using a home brew solution for the CMT2300A when using only ESP-IDF? |
Quick updateI managed to get the example building for the openDTU fusion board, but the debug option (in windows) doesn't work (likely a USB driver problem). Unfortunately, the current WIP branch still doesn't work. I changed the way default SPI pins are defined using either
I'm feeling a bit burned out after last night. I dreamed about debugging the ESP32 with an external probe, but woke up suddenly when I fried the board (in my dream). Even in my dream, I was unsuccessful 😠 I pushed my work and rebooted into Ubuntu, hoping the debug option in pio works better than on Windows. I'll give it another try shortly, but that dream was a bit of a nightmare. |
@2bndy5 There definitely were some issues with the configuration in your example. I fixed them and now USB serial and USB JTAG is working as expected. When putting the board into bootloader mode via the physical buttons, does it show up as "USB JTAG/serial debug unit"? On my PC, "usbser.sys" is used for "Interface 0" and "libusb0.sys" is used for "Interface 2" of the "USB JTAG/serial debug unit". If that's not the case for you, you could try loading different drivers for the interfaces via Zadig. You would choose "USB Serial (CDC)" for "Interface 0" and "libusb-win32" for "Interface 2". |
I'm in Linux right now (building a node.js binding for my new rust project: rf24-rs). I'll try to get some sleep without dreaming about a debugging probe, and try again @LennartF22 Thanks for the Windows USB advice. That's pretty much the same solution I was seeing in my Google searches. I'm glad to have a definitive answer though 🙏🏼 ; most threads on the internet are vaguely concluded. |
@2bndy5 I'll push my changes soon. With some additional fixes, the example is running now:
|
Now I'm very eager to see what you did (& play around with it)! |
@stefan123t @LennartF22 I noticed the openDTU project is still using RF24 v1.4.9. We recently released v1.4.10 which has improved SPI throughput (via #988) 😉. The esp-idf branch already has these changes (after recently rebasing the branch). |
Hi @TMRh20 & @2bndy5
We are using the library you maintain in https://github.com/tbnobody/OpenDTU
As we need/want to adress mulitple devices via the same SPI a user has created a fork for of your library which introduce a HAL (Hardware Abstraction Layer) to use the same SPI configuration for both NRF24L01+ as well as an CMT2300A module on the same SPI. This way we can make use of the comm modules on the same SPI and switch between them using the CE/CSN and IRQ lines.
Would you kindly review the changes/commits in the following repo to give us a hint how we could best get this HAL abstraction into upstream RF24, ie changing the public interfaces to accept the hal object ?
master...LennartF22:RF24:master
https://github.com/LennartF22/RF24
https://github.com/LennartF22/OpenDTU
@LennartF22 how do you imagine to propagate your great contributions needed by the OpenDTU Fusion PoE board (by @markusdd) forward into OpenDTU and RF24 masters ?
The text was updated successfully, but these errors were encountered: