Implement Lorenz et al. POA and GHI QC methods

[abhisheksparikh]

• Closes QA methods for plane-of-array irradiance #123
• Added tests to cover all new or modified code.
• Clearly documented all new API functions with PEP257 and numpydoc compliant docstrings.
• Added new API functions to docs/api.rst.
• Non-API functions clearly documented with docstrings or comments as necessary.
• Adds description and name entries in the appropriate "what's new" file
  in docs/whatsnew
  for all changes. Includes link to the GitHub Issue with :issue:`num`
  or this Pull Request with :pull:`num`. Includes contributor name
  and/or GitHub username (link with :ghuser:`user`).
• Pull request is nearly complete and ready for detailed review.
• Maintainer: Appropriate GitHub Labels and Milestone are assigned to the Pull Request and linked Issue.

This is a draft PR. I need help editing the .rst files.

[kandersolar]

Thanks @abhisheksparikh, this is looking pretty good. This PR has some merge conflicts, largely due to editing the same files as #163 did. If you need help resolving those let us know.

Also, the reference says:

Additionally, we limit the maximal step between two consecutive irradiance measurements to 1000 W/m2 for both GHI and GTI as proposed for also in Espinar et al. (2011).

Do we need to do that as well in order to claim we have fully implemented the method as published?

[kandersolar]

Friendly ping @abhisheksparikh about this PR :)

[AdamRJensen]

The code seems good, so well done!

My main concern is that it seems like an awful lot of code to implement the integer flags. I would prefer to just have the individual functions, but it's not a strong preference.

[AdamRJensen]

I'm still in strong favor of not flagging values at nighttime. I recently had a chat with the main author of the paper who did not have a recommendation for night time. Lorenz pointed me toward one of the co-authors who has yet to respond to my request.

[abhisheksparikh]

Not having thought about the significance of nighttime data, I don't have a strong opinion on this. But if I were to think of irradiance as a part of operational analysis of a PV plant, I would like to have some distinction between daytime and nighttime data.

[AdamRJensen]

I think this deserves it's own function.

The step change limit is fairly common:

For 1-min data Shafer et al. (2000) recommended a step change limit of 800 W/m2, although Espinar (2011) suggested 1000 W/m2 for GHI. Lorenz et al. (2022) also suggested using the 1000 W/m2 threshold for tilted irradiance measurements.

[AdamRJensen]

It could be done in a separate pull request.

[abhisheksparikh]

Good point. I think one good reason to have the step change limit in here is to stay true to the list of checks the paper suggests.

[cwhanse]

we could have a helper function that returns the step threshold, and pass it a kwarg 'pvlive' to return 1000.

[cwhanse]

However, I am completely fine with separating out the filter as a separate function ☺️.

As am I, let's do this and have a step_limit kwarg or something.

[abhisheksparikh]

we could have a helper function that returns the step threshold, and pass it a kwarg 'pvlive' to return 1000.

Can somebody help me with this? After some reading, I think I understand kwargs, but have never written a function with them. Some more guidance on how the functions' design would be helpful.

[cwhanse]

@abhisheksparikh let's do that in a follow-on pull request. I feel like we are good enough here and we can improve later. Agree @AdamRJensen ?

[abhisheksparikh]

The code seems good, so well done!

My main concern is that it seems like an awful lot of code to implement the integer flags. I would prefer to just have the individual functions, but it's not a strong preference.

Do you have any recommendations? For e.g. would you combine the upper and lower limit functions? Individual functions provide clarity, but I am okay with combining them.

[abhisheksparikh]

The code seems good, so well done!

My main concern is that it seems like an awful lot of code to implement the integer flags. I would prefer to just have the individual functions, but it's not a strong preference.

Yeah, I agree, the code is definitely longer for what it's doing.

[cwhanse]

@abhisheksparikh I think we should explain the integer flag values. I don't know if my suggestions will render properly as a web page.

[AdamRJensen]

The authors have responded to our question concerning what to do for nighttime values:

Regarding the night values: in our implementation we set all night values to zero and set a Flag with value 1 for all values changed. But actually, the method is not made for night values. So as you wrote, not checking is also an option which is maybe better for general applications. For some applications night values might be of interest, we actually do not use them anywhere.

So for the integer output, I think we should set nighttime values to 1 True. But for the bool series returned, I suggest not setting any nighttime limits but rather setting a flag of True or False depending on the outcome of #191.

Additionally, the authors also wrote:

One other thing I have to point out. While checking your implementation we found a mistake in the publication, which is very unfortunate. We write I0 as solar constant with a fixed value. But actually I0 should be the Normal extraterrestrial irradiance corrected from Sun-Earth distance, which has a yearly cycle. We use the formula proposed by Spencer 1971 with a simplification suggested in “Energy Meteorology, Lecture Notes from Detlev Heinemann, 2002 Formula 2.16”

@abhisheksparikh This should be relatively straightforward to correct. You can look at the existing tests which do something very similar.

[abhisheksparikh]

So for the integer output, I think we should set nighttime values to True. But for the bool series returned, I suggest not setting any nighttime limits but rather setting a flag of True or False depending on the outcome of #191.

@AdamRJensen - I am confused by this: 'So for the integer output, I think we should set nighttime values to True'.

I will change the Normal extraterrestrial irradiance based on the formula.

[AdamRJensen]

@AdamRJensen - I am confused by this: 'So for the integer output, I think we should set nighttime values to True'.

A typo on my behalf - the flag for the integer should of course be 1 as stated by the authors 😄

[AdamRJensen]

@adriesse I wanted to bring your attention to this QC check for GTI.

[adriesse]

Thanks @AdamRJensen. We have read Lorenz 2022.