Fetch LED types from Bus classes (dynamic UI) #4129

blazoncek · 2024-09-05T20:51:34Z

The idea was sparked by @netmindz (#4056) to allow dynamic creation of enabled (compiled) LED types in LED settings UI instead of relying on editing HTML file to represent them.

This PR implements @netmindz principles while also improves JavaScript handling of implemented bus types in UI (no longer relies on magic numbers).

As a bonus BusPwm is enhanced to support "dithering" and "phase shifting" on ESP32. LEDC supports 4 fractional bits that enhance resolution by 1 bit using temporal statistical distribution (see ESP32 technical reference manual, LEDC chapter).
Phase shifting can be used to distribute load on PSU and, more importantly, drive H-bridge used for reverse-polarity PWM CCT LED strips (thanks goes to @DedeHai for debugging signals on the oscilloscope and adjusting calculations).

The downside of this PR is increased stack buffer size for oappend() function as LED types are pushed into it. When the number of LED types reaches the limit of usable stack that information will have to be split into a separate HTTP/XHR request.

What remains to be implemented is:

passing RAM and power requirements for each LED type (partially implemented by is16bit() function)
detection of parallel I2S vs. RMT buffers on ESP32 (RAM and available outputs)
better handling of available HW (avoid const.h and use HAL)
possible additional true "virtual" or usermod buses (see Usermod bus type #4123)
pass information if LED type requires GPIO and how many (currently length of "t" JSON key determines it)
there is only one "virtual" type (currently used for network DDP/ArtNet)
if color orders are supported, off refresh is required, etc (could be stored in capabilities "c" JSON key)

Closes #4112 #4117

@netmindz

- provide LED types from BusManager for settings Credit: @netmindz for the idea.

@DedeHai

- move macros to constexpr methods - introduce type capabilities for UI - add phase shifting (POC) to PWM - replace PWM CIE LUT with calculated curve CIE & phase shifting credit @DedeHai

- LEDC: allocate same timer for CCT PWM - use SOC constants BusManager update - use allocateMultiplePins for BusPwm

- phase shifting correction (limit to PWM CCT)

Clarifications

@DedeHai

Thanks to @DedeHai & @zalatnaicsongor

Add semaphoring Determine CCT overlap

- fixed maxBri value - fixed overflow in dead time subtraction - deadtime and offset now also work if signals are inverted (_reversed)

- renamed variables - some tuning

fixed offsets and inverted signal plus dead time

@netmindz

- (credit @netmindz & @willmmiles) Tuning.

willmmiles · 2024-09-06T01:25:28Z

Re 43cc4ec, a quick thought offhand regarding oappend: I think there's really little value in using a stack buffer at all, since the two places where it's used immediately turn around and copy the contents to a heap allocated buffer anyways.

With that in mind and since this limitation keeps coming up, I sketched a proof-of-concept "remove oappend" patch last week -- there's some more cleanup to be done, the API change would need to be applied to all usermods, and it probably wants to be extended to use a rope-like data structure like DynamicBufferList, but it should give the general idea for a possible future direction.

willmmiles

High level thoughts:

This is an excellent step in the right direction
Big thumbs up for moving from macros to constexpr
It really makes me want to take a stab at modularizing Bus type handling/config management/construction

wled00/bus_manager.h

wled00/bus_manager.cpp

blazoncek · 2024-09-06T05:33:16Z

a quick thought offhand regarding oappend

oappend is there since ever. If you intend to work on it it may be easiest just to replace char[] buffer with String so it can grow as needed.

willmmiles · 2024-09-06T13:52:39Z

a quick thought offhand regarding oappend

oappend is there since ever. If you intend to work on it it may be easiest just to replace char[] buffer with String so it can grow as needed.

Every time I see String::operator+, I weep for the allocator. ;) I went with a Print interface instead, since it's got a rich API, can be backed by different allocator strategies depending on the context, and AsyncWebServer already had a Stream response available.

wled00/data/settings_leds.htm

wled00/const.h

blazoncek · 2024-09-11T04:29:28Z

@netmindz & @willmmiles would you think this is good to be merged? @softhack007 do you have any second thoughts?
This (principle), along with #4134, may make expanding settings (including and where most beneficial - audioreactive settings) much easier.
@Aircoookie please review (or at least look at) this PR and #4134 if they clash somehow with what you planned for settings rewrite.

willmmiles · 2024-09-11T12:51:01Z

wled00/bus_manager.h

      return &(_mappings[n]);
    }

    uint8_t getPixelColorOrder(uint16_t pix, uint8_t defaultColorOrder) const;

  private:
-    uint8_t _count;
-    ColorOrderMapEntry _mappings[WLED_MAX_COLOR_ORDER_MAPPINGS];
+    std::vector<ColorOrderMapEntry> _mappings;


Is the switch to std::vector intended to reduce memory usage?

One of the challenges is that std::vector has an exponential allocation strategy. Depending on the relationship between WLED_MAX_COLOR_ORDER_MAPPINGS and the number of entries actually used, it may end up using more memory. I checked the code for std::vector for ESP32 -- this implementation of push_back uses the classic "double the array length if it needs more memory" strategy. So for WLED_MAX_COLOR_ORDER_MAPPINGS == 10 (esp32), when you push the 9th element on to the list, the buffer will expand to 16 -- leaving 6 elements you pay for in RAM, but will never use.

What you can do to mitigate this is any of:

You could ensure that the limits are powers of 2

When loading the vector, figure out how many elements you need, then call reserve() up front to allocate exactly the right amount of memory

Alternately, you can preallocate the maximum memory you plan to use with reserve() at construct time; won't save any RAM but also guarantees you'll never allocate more than you need.

You could call shrink_to_fit() after filling the vector, or make a copy and discard the original std::vector object. (Cost is the same, a complete copy and temporarily holding more RAM than it needs)

Is the switch to std::vector intended to reduce memory usage?

No. It is to allow variable size (in the future) without the expense of pre-allocated memory. The question was "why 10 and why only 10?".
reserve() could be done, yes. That'll go into cfg.cpp and set.cpp. Thanks.

blazoncek · 2024-09-14T13:02:32Z

@jw2013 please rebase your branch(es). I'm going to delete bus-config soon.

blazoncek and others added 20 commits August 22, 2024 17:15

Dynamic bus config

6f3267a

- provide LED types from BusManager for settings Credit: @netmindz for the idea.

Bus manager rework

dbb47d5

- move macros to constexpr methods - introduce type capabilities for UI - add phase shifting (POC) to PWM - replace PWM CIE LUT with calculated curve CIE & phase shifting credit @DedeHai

Remove erroneous debug

8818a9e

Merge branch '0_15' into bus-config

f55e2fc

getNumberOfPins

865e3dc

PinManager update

820df0c

- LEDC: allocate same timer for CCT PWM - use SOC constants BusManager update - use allocateMultiplePins for BusPwm

Introduce network type (previous virtual)

b2e00eb

- phase shifting correction (limit to PWM CCT)

Fix C/P error

c51ce2e

Clarifications

Dithering support & bugfix in UI

0d035a0

Thanks to @DedeHai & @zalatnaicsongor

Remove ledc_set_duty_and_update

2ca034b

Add semaphoring Determine CCT overlap

Dirty fix for newer framework

a3b2887

fixed offsets and inverted signal plus dead time

85e2f7e

bugfixes, removed debug printout

be7475f

Bugfixes

2cc0ace

- fixed maxBri value - fixed overflow in dead time subtraction - deadtime and offset now also work if signals are inverted (_reversed)

Optimisations

b1e58dd

- renamed variables - some tuning

found a better way of deadTime subtraction, fixed inverted signals

d3c31c5

Unsigned fix

092a5a1

Merge pull request #4126 from DedeHai/PWM-fixes

55d98ff

fixed offsets and inverted signal plus dead time

Move LED types to individual bus type

9cc333b

- (credit @netmindz & @willmmiles) Tuning.

Increase oappend buffer on ESP8266

43cc4ec

blazoncek added enhancement javascript Pull requests that update Javascript code labels Sep 5, 2024

willmmiles reviewed Sep 6, 2024

View reviewed changes

wled00/bus_manager.h Outdated Show resolved Hide resolved

wled00/bus_manager.h Show resolved Hide resolved

wled00/bus_manager.h Outdated Show resolved Hide resolved

wled00/bus_manager.cpp Show resolved Hide resolved

Remove array, un-inline calculateCCT

ac689a9

Bugfix

c5b31e2

netmindz reviewed Sep 7, 2024

View reviewed changes

wled00/data/settings_leds.htm Show resolved Hide resolved

wled00/data/settings_leds.htm Outdated Show resolved Hide resolved

wled00/const.h Show resolved Hide resolved

Add numPins()

b431515

netmindz approved these changes Sep 11, 2024

View reviewed changes

willmmiles reviewed Sep 11, 2024

View reviewed changes

blazoncek added 3 commits September 11, 2024 16:45

Pre-allocate COM vector

5df1a22

Remove void

6169c45

Merge branch '0_15' into bus-config

5b8d9d1

blazoncek self-assigned this Sep 12, 2024

blazoncek added 3 commits September 12, 2024 15:08

Merge branch '0_15' into bus-config

f12de61

Merge branch '0_15' into bus-config

ec7f69d

Hot

49d51c5

blazoncek merged commit df24fd7 into 0_15 Sep 13, 2024
40 checks passed

blazoncek deleted the bus-config branch September 15, 2024 11:50

willmmiles mentioned this pull request Sep 28, 2024

Implement phase shifted PWM #4034

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fetch LED types from Bus classes (dynamic UI) #4129

Fetch LED types from Bus classes (dynamic UI) #4129

blazoncek commented Sep 5, 2024

willmmiles commented Sep 6, 2024

willmmiles left a comment

blazoncek commented Sep 6, 2024

willmmiles commented Sep 6, 2024

blazoncek commented Sep 11, 2024

willmmiles Sep 11, 2024

blazoncek Sep 11, 2024

blazoncek commented Sep 14, 2024

Fetch LED types from Bus classes (dynamic UI) #4129

Fetch LED types from Bus classes (dynamic UI) #4129

Conversation

blazoncek commented Sep 5, 2024

willmmiles commented Sep 6, 2024

willmmiles left a comment

Choose a reason for hiding this comment

blazoncek commented Sep 6, 2024

willmmiles commented Sep 6, 2024

blazoncek commented Sep 11, 2024

willmmiles Sep 11, 2024

Choose a reason for hiding this comment

blazoncek Sep 11, 2024

Choose a reason for hiding this comment

blazoncek commented Sep 14, 2024