Sampling + DOE Analysis

[shor-ty]

Hey everybody,
its quite a while ago that I was using Dakota (almost 7 years). If I remember correctly, Dakota offers to work with different interface pointers one can use choose which pointer (method+model) is evaluated and in which order, right?

Hence, it should be no big deal to start a sampling of my space first (lets say a hyper-cube sampling) which is followed by evaluating all points. All these points are feed into a DoE investigation and the evaluation starts. With the output of the DoE model new parameters are evaluated and depending on the settings I will get (after some iterations) a good final results.

Should be actually straight forward. I do have the following Input file in which we can see the two methods:

• sampling
• doe by using surrogate

What I am missing at the moment is how dakota executes the code. Sure, the code does not work as expected and I probably miss some other things but wasn´t there a "main-key" that handles which method is executed first (so the pointers)? I will search again inside the help because I know it it there. Probably I need some small break :)

Anyway, if you want to anwer, feel free to do so. If I will find my solution, I will also report it back.
Best, Tobi

# DAKOTA Input File

environment
  tabular_data
    tabular_data_file = 'dakota_tabular.dat'

method
  sampling
    sample_type = lhs
    samples = 20

model
  single
    variables
        uniform_uncertain = 2
        descriptors = 'x1' 'x2'
        lower_bounds  = -10.0 -10.0
        upper_bounds  =  10.0  10.0

interface
      fork
       analysis_drivers = 'my_analysis_driver'
        parameters_file = 'params.in'
        results_file    = 'results.out'

responses
  response_functions = 1
  no_gradients
  no_hessians

# Second stage: DOE
method
  surrogate_based_global
    gaussian_process
    samples = 4

model
  single
    variables
      continuous_design = 2
        descriptors = 'x1' 'x2'
        lower_bounds  = -10.0 -10.0
        upper_bounds  =  10.0  10.0

interface
   fork     
      analysis_drivers = 'my_analysis_driver'      
        parameters_file = 'params.in'
        results_file    = 'results.out'

responses
  response_functions = 1
  no_gradients
  no_hessians

[thenectarine]

The example below builds a polynomial surrogate model, then an uncertainty quantification is performed by sampling the polynomial surrogate model. The original blackbox function is the textbook problem.

Here is a brief summary of what each block is doing.

• The environment block has keyword top_method_pointer that points to block 'UQ' and will execute that method first. If there is no keyword top_method_pointer, the first method block is used.
• Method block 'UQ' is configured to perform an uncertainty quantification via the sampling method. This method block points to model 'UQ_M'
• Model block 'UQ_M' is a surrogate model created by method 'UQ_DACE'
• Method 'UQ_DACE' uses LHS to query the blackbox function and acquire the necessary training data.

In listing 2, it is shown the black box function (textbook) was evaluated 6 times. The uncertainty quantification involved running the surrogate model 5,000 times.

This example could be modified to use a Gaussian process surrogate.

Sometimes it helps to see branches. In listing 3 I added branches to help see how each keyword is associated. I then print out this input file on physical paper, and I use highlighters to mark how each keyword block is associated. See this example.

Listing 1

environment
   top_method_pointer 'UQ'
   error_file 'dakota.err'
   output_file 'dakota.out'
   tabular_data
      tabular_data_file 'tabular.data'

method
   id_method 'UQ'
      sampling
         model_pointer 'UQ_M'
         distribution
               complementary
         response_levels  0.1  0.2  0.6  0.1  0.2  0.6  0.1  0.2  0.6
            compute
                  probabilities
            num_response_levels  3  3  3
         sample_type
               lhs
         samples 5000
         seed 12347

model
   id_model 'UQ_M'
   responses_pointer 'UQ_R'
   variables_pointer 'UQ_V'
      surrogate
            global
                  dace_method_pointer 'UQ_DACE'
                  polynomial
                        quadratic

variables
   id_variables 'UQ_V'
   lognormal_uncertain 2
      descriptors  'x1'  'x2'
         means  1  1
               std_deviations  .5  .5

responses
   id_responses 'UQ_R'
   descriptors  'r1'  'r2'  'r3'
      no_gradients
      no_hessians
      response_functions 3

method
   id_method 'UQ_DACE'
      sampling
         model_pointer 'UQ_DACE_M'
         sample_type
               lhs
         seed 12347

model
   id_model 'UQ_DACE_M'
   responses_pointer 'UQ_R'
   variables_pointer 'UQ_DACE_V'
      single
         interface_pointer 'UQ_ACTUAL'

variables
   id_variables 'UQ_DACE_V'
   active
         all
   lognormal_uncertain 2
      descriptors  'x1'  'x2'
         means  1  1
               std_deviations  .5  .5

interface
   id_interface 'UQ_ACTUAL'
   analysis_drivers = 'text_book'
      direct

Listing 2

<<<<< global_polynomial approximation updates completed.
Constructing global approximations with no anchor, 6 DACE samples, and 0 reused points.

<<<<< global_polynomial approximation builds completed.
Beginning Approximate Fn Evaluations...
<<<<< Function evaluation summary (APPROX_INTERFACE_1): 5000 total (5000 new, 0 duplicate)
<<<<< Function evaluation summary (UQ_ACTUAL): 6 total (6 new, 0 duplicate)
-----------------------------------------------------------------------------
Statistics based on 5000 samples:

Sample moment statistics for each response function:
                            Mean           Std Dev          Skewness          Kurtosis
            r1  2.0811387700e-01  6.2787019985e-01  1.0581426295e+01  2.0846983144e+02
            r2  7.5036841032e-01  1.5205184636e+00  4.7377733695e+00  4.6534701603e+01
            r3  7.4941195823e-01  1.5068047567e+00  4.4815034161e+00  3.9949291962e+01

95% confidence intervals for each response function:
                    LowerCI_Mean      UpperCI_Mean    LowerCI_StdDev    UpperCI_StdDev
            r1  1.9070630805e-01  2.2552144595e-01  6.1580151974e-01  6.4042483185e-01
            r2  7.0821235640e-01  7.9252446423e-01  1.4912916410e+00  1.5509221199e+00
            r3  7.0763611394e-01  7.9118780251e-01  1.4778415338e+00  1.5369341994e+00

Listing 3

environment
├──top_method_pointer 'UQ'
│  
├──error_file 'dakota.err'
│  
├──output_file 'dakota.out'
│  
└──tabular_data
   └──tabular_data_file 'tabular.data'

method
├──id_method 'UQ'
│  
└──Method (Iterative Algorithm) (Group 1)
   └──sampling
      ├──model_pointer 'UQ_M'
      │  
      ├──distribution
      │  └──Distribution Type (CDF/CCDF) (Group 1)
      │     └──complementary
      ├──response_levels  0.1  0.2  0.6  0.1  0.2  0.6  0.1  0.2  0.6
      │  ├──compute
      │  │  └──Statistics to Compute (Group 1)
      │  │     └──probabilities
      │  └──num_response_levels  3  3  3
      ├──sample_type
      │  └──Sample Type (Group 1)
      │     └──lhs
      ├──samples 5000
      │  
      └──seed 12347

model
├──id_model 'UQ_M'
│  
├──responses_pointer 'UQ_R'
│  
├──variables_pointer 'UQ_V'
│  
└──Model Type (Group 1)
   └──surrogate
      └──Surrogate Category (Group 1)
         └──global
            ├──Build Data Source (Group 3)
            │  └──dace_method_pointer 'UQ_DACE'
            └──Global Surrogate Type (Group 1)
               └──polynomial
                  └──Polynomial Order (Group 1)
                     └──quadratic

variables
├──id_variables 'UQ_V'
│  
└──lognormal_uncertain 2
   ├──descriptors  'x1'  'x2'
   │  
   └──Lognormal Characterization (Group 1)
      └──means  1  1
         └──Spread (Group 1)
            └──std_deviations  .5  .5

responses
├──id_responses 'UQ_R'
│  
├──descriptors  'r1'  'r2'  'r3'
│  
├──Gradient Type (Group 2)
│  └──no_gradients
├──Hessian Type (Group 3)
│  └──no_hessians
└──Response Type (Group 1)
   └──response_functions 3

method
├──id_method 'UQ_DACE'
│  
└──Method (Iterative Algorithm) (Group 1)
   └──sampling
      ├──model_pointer 'UQ_DACE_M'
      │  
      ├──sample_type
      │  └──Sample Type (Group 1)
      │     └──lhs
      └──seed 12347

model
├──id_model 'UQ_DACE_M'
│  
├──responses_pointer 'UQ_R'
│  
├──variables_pointer 'UQ_DACE_V'
│  
└──Model Type (Group 1)
   └──single
      └──interface_pointer 'UQ_ACTUAL'

variables
├──id_variables 'UQ_DACE_V'
│  
├──active
│  └──Active Variables (Group 1)
│     └──all
└──lognormal_uncertain 2
   ├──descriptors  'x1'  'x2'
   │  
   └──Lognormal Characterization (Group 1)
      └──means  1  1
         └──Spread (Group 1)
            └──std_deviations  .5  .5

interface
├──id_interface 'UQ_ACTUAL'
│  
└──analysis_drivers  'text_book'
   └──Interface Type (Group 1)
      └──direct

[shor-ty]

Wow. Thank you very much for that valuable answer and especially in that detail. That is exactly what I was looking for. I guess you refreshed a lot of things and I will be able to proceed. Sorry for the late reply - a lot on my desc :)

Thanks very much.
Best, Tobi

[shor-ty]

Hey @thenectarine , may I ask why we need the UQ and UQ_M method? I am bit struggled why we cannot directly start with the UQ_DACE?

[thenectarine]

Method 'UQ' is required because it tells Dakota what to do, e.g. sampling, optimization, sensitivity analysis, etc. If 'UQ' is absent, then Dakota will not know what to do.

Model 'UQ_M' takes inputs and returns outputs. If UQ_M is absent, Dakota won't have access to any outputs.

Model UQ_M can point to one of many different models. Below are some options for the model, but are not the only options avaialble.

Option A - The model can point directly to black box function, e.g.

model
   id_model 'UQ_M'
   responses_pointer 'UQ_R'
   variables_pointer 'UQ_V'
      single
         interface_pointer 'UQ_ACTUAL'

interface
   id_interface 'UQ_ACTUAL'
   analysis_drivers  'text_book'
         direct

Option B - The model can point to a surrogate model, which then requires training data that is provided by UQ_DACE.

model
   id_model 'UQ_M'
   responses_pointer 'UQ_R'
   variables_pointer 'UQ_V'
      surrogate
            global
                  dace_method_pointer 'UQ_DACE'
                  polynomial
                        quadratic
                        
method
   id_method 'UQ_DACE'
      sampling
         model_pointer 'UQ_DACE_M'
         sample_type
               lhs
         seed 12347

[...]

interface
   id_interface 'UQ_ACTUAL'
   analysis_drivers = 'text_book'
      direct

For new users, I advise using option A just to get used to Dakota's workflow, then move on to option B. Option B is more complex, and due to limits on computational resources, option B is practical for 1-10 variables. If option B is the first example you saw, then you might be tempted to commit to option B, but option A is an acceptable alternative.

I took the original example I provided, which used option B, and I reworked it so it uses option A. I did this to demonstrate that 'UQ' and 'UQ_M' are required, but UQ_DACE is optional.

environment
   top_method_pointer 'UQ'
   error_file 'dakota.err'
   output_file 'dakota.out'
   tabular_data
      tabular_data_file 'tabular.data'

method
   id_method 'UQ'
      sampling
         model_pointer 'UQ_M'
         distribution
               complementary
         response_levels  0.1  0.2  0.6  0.1  0.2  0.6  0.1  0.2  0.6
            compute
                  probabilities
            num_response_levels  3  3  3
         sample_type
               lhs
         samples 5000
         seed 12347

model
   id_model 'UQ_M'
   responses_pointer 'UQ_R'
   variables_pointer 'UQ_V'
      single
         interface_pointer 'UQ_ACTUAL'

variables
   id_variables 'UQ_V'
   lognormal_uncertain 2
      descriptors  'x1'  'x2'
         means  1  1
               std_deviations  .5  .5

responses
   id_responses 'UQ_R'
   descriptors  'r1'  'r2'  'r3'
      no_gradients
      no_hessians
      response_functions 3

# model
#    id_model 'UQ_M'
#    responses_pointer 'UQ_R'
#    variables_pointer 'UQ_V'
#       surrogate
#             global
#                   dace_method_pointer 'UQ_DACE'
#                   polynomial
#                         quadratic
#         
# method
#    id_method 'UQ_DACE'
#       sampling
#          model_pointer 'UQ_DACE_M'
#          sample_type
#                lhs
#          seed 12347
# 
# model
#    id_model 'UQ_DACE_M'
#    responses_pointer 'UQ_R'
#    variables_pointer 'UQ_DACE_V'
#       single
#          interface_pointer 'UQ_ACTUAL'
# 
# variables
#    id_variables 'UQ_DACE_V'
#    active
#          all
#    lognormal_uncertain 2
#       descriptors  'x1'  'x2'
#          means  1  1
#                std_deviations  .5  .5

interface
   id_interface 'UQ_ACTUAL'
   analysis_drivers = 'text_book'
      direct