[Feature request]: Prediction: Take into account the current screen brightness if working solely off of the display luminosity i.e. without an ALS

[salman-farooq-sh]

Please describe your feature request

I think in this case the prediction value should be a delta (instead of an absolute value) to be applied on top of the current brightness level.

I tried to understand predict() and src/als/none.rs and I think it is not this way right now. Please correct me if I am wrong, otherwise I can submit a PR for this :-)

wluma/src/predictor/controller.rs

Line 153 in d646247

fn predict(&mut self, lux: &str, luma: u8) {

An easy to do this might be to return a value from the following function that is directly proportional to the current brightness, basically fake-sense a high lux value when brightness is high and vice versa.

wluma/src/als/none.rs

Line 7 in d646247

fn get(&self) -> Result<String, Box<dyn Error>> {

This is based on the assumption that if the current brightness manually set by the user was high, then the ambient environment was probably very bright (hence the user increased the brightness manually), and in that case a real als sensor would have reported a high value too. Likewise for the low value.

Maybe it's a hack? Maybe the logic is circular? Comments and guidance welcome.

If this can work, then maybe this can be a separate [als.relative] or [als.fake] or something?

Thanks!

[maximbaz]

Hello! When you use als=none, that essentially means that als values do not participate in the decision, so if user manually increases brightness, that is just because they like a higher brightness value at the given screen contents; and similarly when user manually decreases brightness, this just means they like a lower screen brightness given the current screen contents. Does this help clarify something? I'm not yet sure what you are trying to solve... If you could give an example of some scenario that you'd like to work better, that would help too 🙂

[salman-farooq-sh]

For the same luma screen contents, the user will desire different brightness values depending on whether they are in sunlight or in a pitch dark room. The obvious solution is using an actual als sensor's help but if that is unfortunately not available then this feels like the next best thing.

The screen luma alone is not the only factor that determines the optimal brightness to be set, so it cannot be predicted with that alone.

For example, consider I am watching a well-lit bright movie at night, brightness value of even 5% in a dark room for such content will be enough. So wluma learns that preference alright.

But let's say I go to work the next day and look at white web pages all day, and screen content luma is basically the same as that extra-bright movie, so the current wluma logic tries to set the brightness to around 5%. But this is a problem because my office is very well lit with big windows and any laptop's 5% absolute brightness will never be enough in such a well lit environment. It will just be too dim.

The current prediction algorithm fails when an als sensor is not available in such situations. The user will manually increase the brightness and wluma will now have learned the wrong values once again for the next movie night.

Enter deltas (or the trick I am proposing, same outcome? at least in theory I think, debatable). The user will set the baseline themselves and wluma will only try to change the brightness relative to that baseline.

The data learning might need adjustments for storing deltas as well but I will need to take a closer look at what the learning data currently being saved looks like.

I hope I have explained the problem well enough at least. The potential solution/s and whether any of them are viable are still up for debate and experimentation though.

[maximbaz]

Thanks for the context! Have you already tried to play with als.time? It was made exactly for this kind of scenario - you define what value you "fake" ambient light sensor is supposedly reporting at any given hour of the day, and then you train wluma accordingly, and wluma then takes that fake als value together with screen contents for its prediction purposes.

[salman-farooq-sh]

While als.time is useful too, it is not quite sufficient for me because the two scenarios I just gave can happen in the space of just a few seconds: the lights in the room are turned off, and then turned on when someone comes in.

I mean, wluma can't do magic or read your mind in the end. All kinds of presets are going to have pros and cons. The point of this is giving the user ample choice of selecting which pros and cons are best suited for their most desired use-case.

This feature for example is focusing on user giving manual input for setting the baseline and using wluma largely only for finer adjustments required when switching between apps.

Ummm, maybe this should be a completely separate project if this feels out of scope for wluma to you. Because this can be very simple to do if we take out the automatic learning thing. A utility that simply applies deltas to the current brightness, whatever that is, based on constant content luma ranges. The simplest version being something like just reduce the current brightness by 5% if it is very bright and increase by 5% if it is very dark. There are obviously several other things to consider but I am just giving a rough idea.

No wonder smartphones took so long to get good auto-brightness features, even with per-device tuned sensors etc.

[maximbaz]

I don't mind exploring ideas, especially because the scenario you mention now is definitely valid - I believe people are solving it using als.webcam (have you tried it btw?), but it has its own downsides.

I suspect adding delta would be very difficult / invasive, because so much logic is currently based on the absolute values, but it's worth having a look in any case.

I'm wondering if another idea could be something like "presets", where you have multiple presets to pick from (e.g. "movie-night" and "evening-with-lights-on"), and then user picks the preset manually. The workflow can easily be prototyped (to experience how it feels switching presets manually), e.g. simply using two wluma configs with different als.time using different values for the same evening hours, and then switching between them is like a symlink one or another into ~/.config/wluma/config.toml and restarting wluma.

[salman-farooq-sh]

I believe people are solving it using als.webcam (have you tried it btw?), but it has its own downsides.

Yeah I tried, idk it just feels too hacky tbh. Plus battery life seemed like it took a small hit at least, can't say for sure though. My laptop already has a small battery that is considerably worn out on top of that. The privacy concerns too. Blinking led raises questions by other people too. I am the same person who made the last comment in the PR that makes the sleep interval configurable. Btw I commend you for having such a detailed discussion on that.

I suspect adding delta would be very difficult / invasive, because so much logic is currently based on the absolute values

Yup, I already took a look at the codebase and I am afraid it's as you say, difficult and invasive. What I have envisioned in my previous comment effectively feels like a separate utility anyways even if implemented within wluma. Maybe I can extract the common parts in a separate lib which my simple utility can consume? I am otherwise totally clueless about stuff like running the luma calculation on the gpu, graphics/gpu stuff in general really, and also the part which deals with the required wayland protocols. There is one problem with this though, I don't have nearly enough know-how to be the maintainer of such a lib. Will you be willing to be the maintainer of it?

presets

I think presets makes it too complicated. Having the need to constantly switch between them. The controls should be only two buttons ideally: brightness-up and brightness-down.

[maximbaz]

I agree on all the points about webcam 😁 😅

I think it's a good idea if you try to implement your proposal! What do you think if we discuss how to share code once you have something working? Start with a fork, change what you need to change, lets see the amount of changes. The idea is, maybe we can find a way to integrate this into wluma itself (say if it's only the "predictor" that is changed, maybe we introduce multiple predictors that you can pick via config, similar to multiple "als"). If we judge that for whatever reason integrating doesn't make sense, maybe we can publish wluma itself as a crate, and you can depend directly on wluma (I think that would require changing layout a bit, into lib and bin, but I'm up for that if there's a need). If we judge that an external shared lib is better, we can go that route. Or something else entirely.

[salman-farooq-sh]

Great idea!

I just had in the back of my mind for some reason that you wouldn't prefer to lose the automatic learning smarts for this in keeping with the rest of the project where I had something like a fixed toml table in mind like fixed deltas for fixed luma ranges. That is why I was gravitating towards something separate so as to not disturb the wluma codebase too much. But in the end, it makes complete sense to configure this exactly instead of relying on the learning capability to eventually arrive at the same thing and that too only when it worked correctly. Dumb is smarter sometimes :-)

Let me think a bit about the implementation in another reply.

[salman-farooq-sh]

So I think this should be implemented as a simple als.manual which uses it's own separate logic different from the usual predict() flow alongside some config.

Other als.* don't need this separate logic because:

• als.iio and als.webcam already predict well because they have a real lux reading (or equivalent) plus luma values to auto learn from.

• als.none is useful for screens whose environments don't change, it's predictions based on luma alone are enough for that use case while still being smart and automatic in learning, it also predicts well already if used for that.

• als.time uses its own config and learning combo and related logic.

als.manual neither uses nor saves any learning data. It's dumb. It works solely based on a toml table specified in the config which maps absolute luma ranges to brightness reduction values (i.e. only negative deltas can be configured). Let's use positive numbers for these reduction values for a simpler explanation:

It will work by maintaining a separate internal state, call it B, for keeping track of a pre-reduction brightness value. When luma changes, the corresponding reduction value is subtracted from B and the resulting brightness value is applied. When the user changes brightness manually, then the just set brightness value plus the luma-corresponding reduction value is set as the new B. So at all times:

B - (reduction value as determined from luma) = (the physical brightness value)

Let me know your thoughts on this. Thanks.

[maximbaz]

I'll let your creativity flow, I think my main feedback point at this stage is that maybe we are confusing some terms, as the actual prediction of the best brightness value happens in predictor/controller.rs, which is independent of any als - it just uses als as a number, as an additional input to make a decision, but that predictor doesn't really know or care about how exactly a particular als implementation got that value.

I might be totally wrong and simply not realizing yet your idea, but with my current understanding I suspect what you really have to do is to forget about als altogether, and instead create a new predictor, which will have a totally different algorithm to decide which brightness value to set, and as inputs maybe it doesn't need any als value at all?

[salman-farooq-sh]

Yes, you are right. This has nothing to do with an ambient light sensor and as such should not be configured under an als.manual nor should the implementation have anything to do with any als. Because we are actually looking to completely eliminate any als input for the prediction only rely on the contents' luma. Thanks for pointing me into the right direction!