0

This is an example, of hyper-parameters search in a hypothetical python code. My question is How can I be sure that RandomSearch really will find the best parameters that lies within the top 5% of the true maximum, with 95% probability.

I read this answer, and its claims that with 60 iterations of random search I will find that 5% with 95% of probability. But that looks like a magic number and I don't know if that works just for any case or any type of problems without considering any other characteristic of the problem or the amount of hyper-parameters 

import numpy as np
from sklearn.model_selection import ParameterSampler
#... lot of imports and code that doesn't matter right now
parameters = {
        "sequence_length": np.arange(50, 300, 50),
        "vocabulary_size": np.arange(1000, 50000, 1000),
        "embeddings_size": np.arange(64, 257, 64),
        "dropout": np.arange(0.2, 0.8, 0.1)
    }
    param_grid = ParameterSampler(parameters, n_iter=60)

    for grid in param_grid:
        my_awesome_lstm_model.fit(parameters=grid)
carpinchosaurio
  • 137
  • 8
  • 1
    You might want to read the paper proposing random search. http://jmlr.csail.mit.edu/papers/volume13/bergstra12a/bergstra12a.pdf – Sycorax Aug 30 '17 at 16:48
  • As I mention, I already read that answer. But seems too magical the number 60 as general solution for random search, without considering any characteristic of the problem or the amount of hyper-parameters to search, as I wrote. – carpinchosaurio Aug 30 '17 at 17:58
  • You'll have to clarify what part you don't understand. The block quote in that answer describing how to arrive at 60 seems perfectly clear to me: solve for $n$. If you read the paper about random search, you will find that random search is problem-agnostic and only assumes that the optimum of the function under optimization is finite. – Sycorax Aug 30 '17 at 18:16
  • Ok, about how to arrive mathematically to 60 you are right it is clear. I suppose the paper will be the answer. – carpinchosaurio Aug 30 '17 at 18:19

0 Answers0