Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slice sampler shrank to zero #9

Open
JohnReid opened this issue Oct 4, 2013 · 7 comments
Open

Slice sampler shrank to zero #9

JohnReid opened this issue Oct 4, 2013 · 7 comments
Assignees

Comments

@JohnReid
Copy link

JohnReid commented Oct 4, 2013

I'm trying a relatively vanilla optimisation over 9 dimensions, 5 of which are categorical (2 values), the other numeric. I'm getting the following exception:

Exception: Slice sampler shrank to zero!
in
File "/home/john/src/spearmint/spearmint/spearmint/util.py", line 69, in direction_slice

What does it mean and do you have any ideas what I can do to get around this? I've tried with and without noise. I'm using GPEIOptChooser with a local driver, max concurrent of 1

Thanks,
John.

@JasperSnoek
Copy link
Owner

Aha, is it possible that your code is returning NaNs or infs? This error basically means that the sampling algorithm is not finding any reasonably valid setting for the hyperparameters (e.g. everything returns a NaN).

Would you mind trying to set the categorical parameters to ints (with two values) and seeing if that changes anything?

Jasper

@ghost ghost assigned JasperSnoek Oct 4, 2013
@JohnReid
Copy link
Author

JohnReid commented Oct 6, 2013

I haven't had time to re-run this yet but I notice you put a fix in for cases where the first two results are the same. Was the fix for this issue? I suspect my values don't change for some large parts of parameter-space.

@JohnReid
Copy link
Author

I fetch/merged your latest changes and switched the categorical parameters to ints. Now I get a different error after two evaluations (-0.775600 and -0.790467) have completed:

----------------------------------------
Current best: -0.775600 (job 0)
19998 candidates   1 pending   1 complete
Maximum number of jobs (1) pending.

----------------------------------------
Current best: -0.790467 (job 1)
19998 candidates   0 pending   2 complete
Choosing next candidate...
Traceback (most recent call last):
File "/home/john/src/spearmint/spearmint/bin/../spearmint/main.py", line 319, in <module>
    main()
File "/home/john/src/spearmint/spearmint/bin/../spearmint/main.py", line 165, in main
    while attempt_dispatch(experiment_config, expt_dir, chooser, driver, options):
File "/home/john/src/spearmint/spearmint/bin/../spearmint/main.py", line 244, in attempt_dispatch
    job_id = chooser.next(grid, values, durations, candidates, pending, complete)
File "/home/john/src/spearmint/spearmint/spearmint/chooser/GPEIOptChooser.py", line 217, in next
    self._real_init(grid.shape[1], values[complete])
File "/home/john/src/spearmint/spearmint/spearmint/chooser/GPEIOptChooser.py", line 173, in _real_init
    self.hyper_samples = state['hyper_samples']
KeyError: 'hyper_samples'
Exception TypeError: "'NoneType' object is not callable" in <bound method Locker.__del__ of <Locker.Locker instance at 0x45f20e0>> ignored

My config is:

language: PYTHON
name:     "MotEvo"

variable {
name: "EMPRIOR"
type: INT
size: 1
min: 1
max: 0
}

variable {
name: "UFEWMPRIOR"
type: FLOAT
size: 1
min: 0.
max: 800.
}

variable {
name: "BGPRIOR"
type: FLOAT
size: 1
min: 0.5
max: 0.9999999
}

#variable {
# name: "MARKOVORDERBG"
# type: INT
# size: 1
# min:  0
# max:  3
#}

and I'm running spearmint with these options:

    --max-concurrent=1 \
    --max-finished-jobs=10000 \
    --method-args=noiseless=0 \
    --driver=local \
    --method=GPEIOptChooser \

@JohnReid
Copy link
Author

I've since changed the max value for BGPRIOR from 0.9999999 to 0.99 and spearmint seems happier now.

@Quanticles
Copy link

Is there a known solution to the "Slice sampler shrank to zero" error? Many of my variables are log-domain, like 0.0001-0.1. I'm considering making the variable an exponent (the spearmint parameter is -5 to -1, and I use it as 10^parameter in the script).

@JasperSnoek
Copy link
Owner

Ah, so there are solutions that will be rolled out hopefully very shortly,
depending on the cause of this. One potential cause is that there are
invalid values being thrown in as results (like NaNs, Infs, etc), in which
case we have a solution but that will break the code currently. In other
cases, in the meantime, usually deleting the *Chooser.pkl and rerunning
spearmint usually clears the issue (this causes the sampler to reset and
reburn).

It is definitely a good idea to log transform your parameters if you think
that's a sensible transformation. We're also about to roll out code to
learn the transformation automatically (i.e. according to this paper:
http://people.seas.harvard.edu/~jsnoek/bayesopt-warping.pdf).

On Thu, Jan 9, 2014 at 5:13 PM, Quanticles [email protected] wrote:

Is there a known solution to the "Slice sampler shrank to zero" error?
Many of my variables are log-domain, like 0.0001-0.1. I'm considering
making the variable an exponent (the spearmint parameter is -5 to -1, and I
use it as 10^parameter in the script).


Reply to this email directly or view it on GitHubhttps://github.com//issues/9#issuecomment-31982666
.

@Quanticles
Copy link

Deleting the *Chooser.pkl file worked for me. My cleaning script wasn't removing it properly due to a bug in VMWare Workstation. Thanks for continuing to maintain the code and whatnot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants