Setting bound in kernel parameters

[Yvonne-Ng]

Hi experts,

Thanks for this wonderful package. I was wondering if there are any functionalities on how to set bound on the kernel parameter space when I do optimize without going into messing up functions COptimiser.cpp? Thanks!

Yvonne

[lawrennd]

Hi Yvonne,

The parameter transformations are actually handled in CTransform,

https://github.com/SheffieldML/GPc/blob/master/CTransform.cpp

These different classes (exponential transforms, negloglogit (which today people call soft-relu!) and sigmoid).

My memory is that a sigmoid transformation should enable you to fix bounds.

But while I wrote the code and was once an expert, that was a while ago!

Neil

[Yvonne-Ng]

Hi Neil,

I tried using the addTransformation of sigmoid in _init to add in the bound, however, it appears that the bound not applied as the optimized kernel printed out later is not of the correct bound. Wondering if made a mistake I tried to apply the defaultZeroOne bound to the code. but it appears that the optimized kernel params is out of the 0,1 bound as well. I wonder if I misunderstood the logic and the addTransformation needs to be applied somewhere else?
Thanks, any pointers from you will be greatly appreciated.

Yvonne

[lawrennd]

Hi Yvonne,

Can you provide a gist for me to have a look at showing how you've done it?

(https://www.youtube.com/watch?v=wc2NlcWjQHw)

I can't promise I'll remember how it all works, but I can take a look at least!

Neil

[Yvonne-Ng]

Hi Neil,

https://github.com/Yvonne-Ng/gpc_root
Check out branch: dev

to compile: make dummy_main
execute: ./dummy_main

You will likely need this tool to compile, likely a package manager from your OS will work. But maybe the logic on its own is not too hard to read.
https://root.cern.ch/downloading-root

The lines are here:
dummy_main.cpp: line 80 (the learn function) ->
gp_TH1.cpp: line 353 (only testing on rbf) ->
and CKern.cpp: line 1096

The biggest mystery to me right now is why setting transformation with defaultZeroOne in CKern.cpp stills gives me a post-optimise rbfinverseWidth=1.51499 (out of the 0/1 bound? )
I wonder if I am missing something obvious...

I am also attaching an output.log from running ./dummy_main.

Sorry that the code is messy, I hope to rewrite once I figure out how it works. I greatly appreciate your offering to take a look!

output.log

Yvonne

[lawrennd]

Missing TH1D.h ...

[Yvonne-Ng]

Did you install the Root package? Yvonne

…

On Apr 8, 2020, 4:19 AM +0800, Neil Lawrence ***@***.***>, wrote: Missing TH1D.h ... — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[lawrennd]

Doh! Should read further in instructions ;-)

[Yvonne-Ng]

Let me know if it works :) Yvonne

…

On Apr 8, 2020, 4:34 AM +0800, SheffieldML/GPc ***@***.***>, wrote: Missing TH1D.h ...

[lawrennd]

Hmmm ... struggled to get Root to compile on my machine (I'm on a mac, but it doesn't seem to like my C++ compiler .. it seems to want a newer one ...)

Can you point me at the lines in the code where the problem is?

[Yvonne-Ng]

Ah yes! it's here (also in the comment above) :
The lines are here:
dummy_main.cpp: line 80 (the learn function) ->
gp_TH1.cpp: line 353 (only testing on rbf) ->
and CKern.cpp: line 1096

The biggest mystery to me right now is why setting transformation with defaultZeroOne in CKern.cpp stills gives me a post-optimise rbfinverseWidth=1.51499 (out of the 0/1 bound? )
I wonder if I am missing something obvious...

I am also attaching an output.log from running ./dummy_main.
output.log

Sorry that the code is messy, I hope to rewrite once I figure out how it works. I greatly appreciate your offering to take a look!

Yvonne

[lawrennd]

So to be clear, these are line numbers in gpc_root (the other repo).

Looking at your line 353 in gp_TH1.cpp

    kernels[i]->setBound(0, min(LS2IW(lower_LS), LS2IW(upper_LS)), max(LS2IW(lower_LS), LS2IW(upper_LS)));

My first question is: are you sure that 0 is the right index for the parameter (the lengthscale)? Maybe we started changing this practice in later code bases ... but normally index the kernel variance first because it's always there ...

[Yvonne-Ng]

Hi Neil,

I think it's right since in line 1143 in CKern.cpp, case 0 = inverseWidth and case 1 is variance. In any case I have tried putting a bound on the variance as well, but the variance optimized still ended up going beyond. Does the effect of addTransform gets propagated into optimise()? Thanks!

Yvonne