
Hi,
BO is used by some people for (hyper) parameter tuning when computing cost function is expensive or parameter space is large.
Suppose in my case, such computation is inexpensive and a grid search is computationaly feasible, is there any reason I should use BO instead of grid search?




nikol


Total Posts: 850 
Joined: Jun 2005 

 
nikol


Total Posts: 850 
Joined: Jun 2005 


I don't like an embarrassing "I guess". It was an exclamation from my intuition.
Here it is in full:
P(H) is your grid over set of parameters of the model. Consider it flat, because you scan paramspace with equal probability. P(DataH) is usual likelihood of your Data fit to a particular model on every point of the grid
as result you get
P(HData) = P(DataH)*P(H)
which defines probability of having certain set of model parameters for observed set of Data.
I assumed P(Data)=1, but it can be that P(Data) = Sum_i(P(DataH_i)*P(H_i)) < 1 for example, if you already have some idea about P(H) distribution from previous step and now you have knew data (Data). So, it becomes
P(H_iData) = P(DataH_i)*P(H_i) / (Sum_i(P(DataH_i)*P(H_i))) 


