Questions regarding optimize_acqf_mixed_alternating

[jduerholt]

Hi all,

we are currently planning to adopt optimize_acqf_mixed_alternating into our workflows. For this reason, I have a few questions regarding its interface:

• From my understanding one provides the discrete dims as a list of indices, the actual discrete values that are tried by the optimizer are then read from the bounds, or? This means if one has a dicrete dimension with lower bound zero and upper bound two, the discrete values [0,1,2] can be tried by the optimizer. Correct? My main question is how to facilitate in the best way the mapping from the discrete values that the optimizer tries to the actual discrete options in the optimization process which are different from the integer options that are used by the optimizer. For example the actual search space for the discrete degree of freedom is [5, 10, 20] (note the non-equally spaced values). So which kind of input transforms should be used to facilitate the mapping? And would it not be easier to introduce a keyword discrete_values: List[List[float]] instead of using the bounds? Then one does not need any input transform to facilitate the mappings and have an easier handling of non-equally spaced discrete options.
• Regarding categorical degrees of freedoms, also here the options are read from the bounds. What is the best way how to handle non-allowed categories. For example, I have three categories for a categorical degree of freedom: [a, b, c] and all three have been used in the past but for future experiments only options [a, c] are allowed. For me it would be also the best way to have here a keyword categorical_options: List[List[int]] and in the example case described here, it would be [0, 2].

If you like the proposed changes, I would volunteer for providing a PR. I think, it would make the methon much easier to use, if one models the allowed discrete/categorical options explicitly.

Best,

Johannes

cc: @TobyBoyne

[saitcakmak]

Hi @jduerholt! That's actually pretty close to how I was thinking of modifying this optimizer while implementing categorical support, but @TobyBoyne beat me to it. The current version works fine for us in Ax, since we tend to use transforms that convert "choice" (could be ordinal or categorical) parameters to a sequence of integers.

What you're proposing makes a lot of sense and opens the door to using the optimizer without requiring such a transform (but doesn't prevent using it with one either). I am guessing some of the neighbor generation code would be slightly more expensive, but that's likely negligible compared to evaluating the acquisition function.

Regarding the input format, what do you think of discrete_values: dict[int, list[float]] rather than list[list[float]]? This way all information about the discrete parameters could be contained on one input, rather than being split between indices and values.

@SamuelGabriel is working on batchifying optimize_acqf_mixed_alternating in #2895. We should wait for that to land before making any more changes to the optimizer :)

[Balandat]

Great suggestion and discussion; not a lot to add here other than voicing my support for it :)

[jduerholt]

@saitcakmak: thanks for your detailed explanation. I would like to have the option while using the alternating to compute the covariance over the original values for cases with non-equally spaced values or those where this kind of warping is not possible as I want to make sure that all available information can be seen by the surrogate, despite this being rather esoteric use cases :D

So, I will try to make the changes to the optimizer after Samuel is done, could be that I will not be able to start before end of July due to being on pto.

In general my vision for the alternating optimizer would be to support also the following functionalities (despite the ones being discussed above)

• linear equality constraints
• non-linear constraints
• linear interpoint constraints (I assume that this could already work with the changes regarding batch optimizations in the open PR by Samuel)
• Homotopies as in SEBO
• categorical exclude constraints

When one would get all this to work in the optimizer, this would be really powerful for a lot of usecases and one would get rid of some ugly workarounds ;)

Best,

Johannes

[jduerholt]

As #2895 is now in, so it should be ready to go with the changes discussed above, or?

[Balandat]

@jduerholt yes should be good to move ahead

[jduerholt]

I am already on it :-)