I would like to fit a Gaussian Process Regressor from sk-learn to predict values of an the objective function in order to explore behaviour and interplay between inputs.
Data points are difficult to obtain (hours of computation on hundreds of cores). Thus I would like to use an effective DoE to get data to fit the regression model.
Which DoE method should I use? How many data points are needed based on number of inputs?
Let me try. I have little practical experience with such experiments. Lets say you have 5 continuous input variables. First, decide on the range of interest for each variable. Then, for each variable, take max and min of range as first inputs. This might make a problem if the behavior of the function is untypical near the edges, so maybe first make a somewhat smaller range, and depend on continuity to extend outside the range (at least stay away from singularities at first). Run a fractional factorial experiment, maybe a $2^{5-2}$ which have eight runs, plus a centerpoint (a centerpoint is a cheap way to get some information on curvature). That gives 9 points for a first model. So few points only makes possible to use a linear model, fitting interactions will need more points, unless you have some strong prior information to use in a Bayes model. If time permits, maybe extend that to a $2^{5-1}$-design which will give also information on interactions (still with centerpoint).
Use this first model to help find point of interest for further runs.
I would at least start along some such lines, but maybe first read some papers such as Use of Kriging Models to Approximate Deterministic Computer Models (Kriging is just another name for gaussian process models), the books Engineering Design via Surrogate Modelling and Evolutionary Operation: Statistical Method for Process Improvement seems directly relevant.
Also some ideas and references at Function Approximation vs. Regression. In some cases you could look at ideas of optimal experimental design, see this presentation
The answer here depends on whether you want to explore the space by collecting a bunch of data points at once, or whether you'd like to quickly converge to an optimal point in the space.
If you want to collect a bunch of points at once that are informative about the rest of the space, then you want to maximize entropy of the points you collect. If you know what covariance function you're using in your GP model, then you can calculate the entropy from the training covariance before you actually make the measurements at those points. The entropy of a multivariate Gaussian is:
$${\frac {1}{2}}\ln \operatorname {det} \left(2\pi \mathrm {e} {\boldsymbol {\Sigma }}\right)$$
where $\Sigma$ is the covariance.
On the other hand, if you want to find an optimal point, then you want a Bayesian black box optimization algorithm. One simple example is the upper confidence bound algorithm:
I believe one of the other commenters linked you to more resources on Bayesian black box optimization. scikit-optimize implements some of these with Gaussian processes.
