Probabiltiy of Feasibility ACQF by jduerholt · Pull Request #580 · experimental-design/bofire

jduerholt · 2025-05-12T10:31:41Z

Motivation

This PR adds the probability of feasibility ACQF from botorch. It is automatically used in MOBO and SOBO optimizations when no feasibile solution is there.

Update 23.05.2025:

It is not anymore used automatically in the SOBO and MOBO. Instead it is now a selectable ACQF for SoboStrategy. To use it dynamically it can be applied in the StepwiseStrategy together with respecting condition introduced in this PR.

Have you read the Contributing Guidelines on pull requests?

Yes.

Test Plan

Unit tests.

bertiqwerty

Thanks Johannes! Can you please add an example that shows how this can be used? I think this is especially important since a step-wise strategy with the FeasibleExperimentCondition should be combined with the pf-acquisition-function.

bertiqwerty · 2025-05-26T06:54:46Z

bofire/data_models/acquisition_functions/acquisition_function.py

+    constraints in the next batch. It can be only used in the SoboStrategy
+    and is especially useful in combination with the FeasibleExperimentCondition
+    within the StepwiseStrategy.
+


Is there a literature reference that you can add?

No, this is based on the BoFire PR, in which they implemented it: meta-pytorch/botorch#2815

Would it make sense to add a link to this paper mentioned in the BoTorch API reference and a link to the BoTorch API reference itself?

The paper does not really fit, it does not mention the new acqf and is just explaining why one should use log based acqfs ;)

Now I am confused. Ok, it is not the main topic of the paper. But at least there is this section.

Or is this PR about something else and I missed it?

This is the formula for constrained expected improvement, which we were using automatically when we use EI in combination with output constraints. This PR introduced the qLogPF (PF = Probabiliy of feasibility) acquisition function which is just the feasibility term without EI.

Aaaah. Got it. Now your new documentation (strategies.md) makes sense to me :D.

bertiqwerty · 2025-05-26T06:56:17Z

bofire/data_models/acquisition_functions/api.py

+    qLogPF,
 ]

 AnySingleObjectiveAcquisitionFunction = Union[


didn't you say above this was a single objective acquisition function? shouldn't it appear here?

It is kind of a special case of a single objective acqf, as it should be only used for SoboStrategy but not for the other SoboStrategys like AdditiveSoboStrategy etc. This is why I implemented it in this way. Do you have a better idea? I know, it is not ideal ...

bertiqwerty · 2025-05-26T06:57:43Z

bofire/data_models/strategies/predictives/sobo.py

-    def validate_is_singleobjective(cls, v, values):
-        if len(v.outputs) == 1:
-            return v
+    acquisition_function: Union[AnySingleObjectiveAcquisitionFunction, qLogPF] = Field(


see my comment above.

bofire/data_models/strategies/stepwise/conditions.py

bertiqwerty · 2025-05-26T07:01:45Z

bofire/data_models/strategies/stepwise/conditions.py

+                relevant_constraints.is_fulfilled(valid_experiments)
+            ]
+
+        # TODO: have a is fulfilled for input features --> work for future PR


rather use issues than comments in the code.

done: #593

I let the comment in the code so that one knows where to add it in the code.

bertiqwerty · 2025-05-26T07:03:34Z

bofire/strategies/predictives/sobo.py

    def __init__(
        self,
-        data_model: SoboBaseDataModel,
+        data_model: SoboDataModel,


why did you change this? multiplicative or additive pass their data model here which is an heir of SoboBaseDataModel.

It was due to some typing errors, which were occuring due the change on the type of the ACQF. And of course, changing it how it is now, also again introduces tying errors. As told above: I am very open for a better solution.

Co-authored-by: Behrang Shafei <50267830+bertiqwerty@users.noreply.github.com>

jduerholt · 2025-05-27T11:05:46Z

Thanks for the review @bertiqwerty, I have let comments on your comments. Maybe you have an idea regarding one typing issue.

Can you please add an example that shows how this can be used?

I find adding another notebook for every new feature a bit too much. What do you think about examples in the docstring? Or what would you recommend?

bertiqwerty · 2025-05-27T11:10:36Z

Thanks for the review @bertiqwerty, I have let comments on your comments. Maybe you have an idea regarding one typing issue.

Can you please add an example that shows how this can be used?

I find adding another notebook for every new feature a bit too much. What do you think about examples in the docstring? Or what would you recommend?

I didn't say you need a new notebook. I like examples in the doc-strings. But we should then also test them. You could also add this to another existing notebook. But without example it is always hard to understand what something is for.

jduerholt · 2025-05-27T11:49:12Z

@bertiqwerty: will make up my mind where to add the example. Any idea regarding the typing etc.?

bertiqwerty · 2025-05-27T14:09:50Z

@bertiqwerty: will make up my mind where to add the example. Any idea regarding the typing etc.?

What happens if you role this back?

…om/experimental-design/everest into feature/probability_of_feasibility

jduerholt · 2025-06-06T14:52:22Z

I didn't say you need a new notebook. I like examples in the doc-strings. But we should then also test them. You could also add this to another existing notebook. But without example it is always hard to understand what something is for.

Hi @bertiqwerty, I created a new documentation page, which gives a still very rough overview on strategies, in which also the concept of stepwise strategies is documented with an example for the new acqf. This documentation can be still improved, but I did not found time to do so. So there is enough material for future PRs ;)

I also switched to a different solution regarding exclusion of the ACQF for the other strategies. It is not perfect, but better then the old one which was causing a lot of typing problems. And honestly, I do not have time for more there ...

So it is ready for re-review ;)

bertiqwerty · 2025-06-10T08:26:06Z

bofire/data_models/strategies/predictives/sobo.py


+class _ForbidPFMixin:
+    """
+    Mixin to forbid the use of qLogPF acquisition function in strategies that are not


You mean single-objective botorch strategies that are not sobo, right?

Good point, I will make it more clear and merge.

bertiqwerty

Thanks, Johannes.

jduerholt added 5 commits May 12, 2025 12:28

first implementation

144b83a

Merge branch 'main' into feature/probability_of_feasibility

0c45764

finalize implementation

3cbf237

fix tests

15dd67a

fix typing

748bcd3

jduerholt marked this pull request as ready for review May 23, 2025 10:11

jduerholt requested a review from bertiqwerty May 23, 2025 10:11

jduerholt added 4 commits May 23, 2025 12:15

fix typing

91e5b87

fix typing

3b4bc5a

fix typing

8b29993

update botorch version

957b71e

bertiqwerty requested changes May 26, 2025

View reviewed changes

Update bofire/data_models/strategies/stepwise/conditions.py

bdebf7f

Co-authored-by: Behrang Shafei <50267830+bertiqwerty@users.noreply.github.com>

jduerholt mentioned this pull request May 27, 2025

[FEATURE REQUEST]: is_fulfilled method for Inputs #593

Closed

jduerholt added 5 commits June 2, 2025 14:37

Merge branch 'main' into feature/probability_of_feasibility

5c069ed

add doc on strategies

3dd72ab

finish documentation

3729322

Merge branch 'feature/probability_of_feasibility' of https://github.c…

8b55230

…om/experimental-design/everest into feature/probability_of_feasibility

add to yaml

57efae2

jduerholt added 2 commits June 6, 2025 17:10

add mixin functionality

c5f2011

add tests

f97a2eb

jduerholt requested a review from bertiqwerty June 6, 2025 15:22

fix it

b4f3697

bertiqwerty reviewed Jun 10, 2025

View reviewed changes

bertiqwerty approved these changes Jun 10, 2025

View reviewed changes

update doc string

a0a4dd9

jduerholt merged commit b3bca78 into main Jun 10, 2025
12 checks passed

jduerholt deleted the feature/probability_of_feasibility branch June 10, 2025 11:47

Conversation

jduerholt commented May 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Have you read the Contributing Guidelines on pull requests?

Test Plan

Uh oh!

bertiqwerty left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bertiqwerty May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jduerholt commented May 27, 2025

Uh oh!

bertiqwerty commented May 27, 2025

Uh oh!

jduerholt commented May 27, 2025

Uh oh!

bertiqwerty commented May 27, 2025

Uh oh!

jduerholt commented Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bertiqwerty left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jduerholt commented May 12, 2025 •

edited

Loading

bertiqwerty May 30, 2025 •

edited

Loading

jduerholt commented Jun 6, 2025 •

edited

Loading