ActiveLearningGPDecision

Evaluates a GP surrogate model, determines its prediction quality, launches full model if GP prediction is inadequate, and retrains GP.

Description

The ActiveLearningGPDecision is a Reporter class that performs the following functions to facilitate the active learning process in a parallelized fashion:

  • For every new batch of input samples produced by the ActiveLearningMonteCarloSampler, the corresponding Gaussian Process (GP) predictions are made.

  • These GP predictions are assessed for their quality by using the uncertainty information. If any of the predictions are of poor quality in the batch, those samples are marked and this information is sent back to the ActiveLearningMonteCarloSampler.

  • Once a user-specified batch size of inputs for running the high-fidelity model evaluations (i.e., those input samples for which the GP predictions are poor) is met and these evaluations are executed in parallel, re-train the GP model via the ActiveLearningGaussianProcess Surrogate class.

  • Output the GP mean predictions and standard deviations and corresponding inputs to a json file. Also indicate those input samples for which a high-fidelity model evaluation was necessary during the active learning.

Why Gaussian Process surrogate for active learning?

A GP is a Bayesian surrogate model. This means, in addition to making predictions of the high-fidelity model outputs, a GP produces uncertainty estimates of its outputs. Higher the uncertainty, lesser the confidence in the GP predictions. An illustration of the GP predictions and associated uncertainties is presented in Figure 1. In addition, many studies (e.g., Xiu et al. (2019) and Dhulipala et al. (2022)) have found that the uncertainty estimates produced by a GP are robust. This in-built uncertainty information permits us to create an active feedback loop between the training data and GP in that we can use the GP uncertainty estimates to select the next best training data point(s). As such, we can even start the active learning process using a GP that is trained with little data.

Figure 1: Schematic of active learning in Monte Carlo simulations with parallel computing. The interaction between the three objects, ActiveLearningMonteCarloSampler, ActiveLearningGaussianProcess, and ActiveLearningGPDecision, is presented.

Of the spectrum of Bayesian surrogate models, a GP is the most straight-forward to train. First, during active learning, the training data set size would typically be small (in the order of hundreds) for most problems. Therefore, re-training the GP should be associated with a low computational cost. Even in those extreme cases where the training data set size becomes large, we can adopt stochastic optimization methods like Adam optimization Kingma and Ba (2014) (this feature is available in MOOSE GaussianProcessTrainer) to select a mini-batch of the training data set and substantially reduce the computational expense.

Owing to these reasons, a GP surrogate is usually preferred for active learning. However, for the sake of completeness, we note that there have been other studies that have used different surrogates like a polynomial chaos expansion for active learning (e.g., Marelli and Sudret (2018)).

Procedure for active learning in MOOSE

In general, active learning has two phases: (1) initial training phase for the GP; and (2) the active learning phase.

Figure 2 presents the initial training phase of the GP surrogate. The ActiveLearningMonteCarloSampler sends a batch of input samples for high-fidelity model executions to the SamplerFullSolveMultiApp. These high-fidelity models executions are performed in parallel using separate sets of processors. Outputs from the model executions will be collected by the ActiveLearningGPDecision reporter. Once a required number of training data size is reached, ActiveLearningGPDecision will call the ActiveLearningGaussianProcess to initially train the GP model.

Figure 2: Schematic of the initial training phase of the GP surrogate.

Figure 3 presents the subsequent active learning phase. The ActiveLearningMonteCarloSampler sends a batch of input samples to the ActiveLearningGPDecision reporter. Then, the ActiveLearningGaussianProcess is called to make GP predictions. In addition, ActiveLearningGPDecision assesses the quality of the GP predictions through user-defined acquisition (or learning) functions. Some popular acquisition functions are currently available, and it is easy to include additional functions in the ActiveLearningGPDecision class. If one or more of the GP predictions are poor (as indicated by the acquisition function), the corresponding input samples are marked and this information is communicated to the ActiveLearningMonteCarloSampler. This process continues until a user-defined batch size is met for the maximum number of poor quality GP predictions. Once this threshold is reached, the ActiveLearningMonteCarloSampler sends the batch of those book-marked input samples whose GP predictions are poor for high-fidelity model evaluations. The batch of high-fidelity models can be run in parallel. Then, the high-fidelity model outputs are acquired by the ActiveLearningGPDecision reporter where the GP model is re-trained. This process then repeats until the user-specified number of Monte Carlo samples is met.

Figure 3: Schematic of active learning in Monte Carlo simulations with parallel computing. The interaction between the three objects, ActiveLearningMonteCarloSampler, ActiveLearningGaussianProcess, and ActiveLearningGPDecision, during the active learning phase.

Input file syntax

The full input file driver for active learning is presented below. The three main components of the input file are ActiveLearningMonteCarloSampler, ActiveLearningGaussianProcess, and ActiveLearningGPDecision. These are discussed in detail below.

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Uniform
    lower_bound = 5
    upper_bound = 20
  []
  [q_dist]
    type = Uniform
    lower_bound = 7000
    upper_bound = 13000
  []
  [Tinf_dist]
    type = Uniform
    lower_bound = 250
    upper_bound = 350
  []
[]

[Samplers]
  [mc]
    type = ActiveLearningMonteCarloSampler
    num_batch = 1
    distributions = 'k_dist q_dist Tinf_dist'
    flag_sample = 'conditional/flag_sample'
    seed = 5
    num_samples = 20
    execute_on = PRE_MULTIAPP_SETUP
  []
[]

[MultiApps]
  [sub]
    type = SamplerFullSolveMultiApp
    sampler = mc
    input_files = 'sub.i'
    mode = batch-reset
    should_run_reporter = conditional/need_sample
    execute_on = TIMESTEP_END
  []
[]

[Transfers]
  [sub]
    type = SamplerParameterTransfer
    to_multi_app = sub
    sampler = mc
    parameters = 'Materials/conductivity/prop_values Kernels/source/value BCs/right/value'
    to_control = 'stochastic'
    check_multiapp_execute_on = false
  []
  [reporter_transfer]
    type = SamplerReporterTransfer
    from_reporter = 'avg/value'
    stochastic_reporter = 'conditional'
    from_multi_app = sub
    sampler = mc
  []
[]

[Reporters]
  [conditional]
    type = ActiveLearningGPDecision
    sampler = mc
    parallel_type = ROOT
    execute_on = 'timestep_begin'
    flag_sample = 'flag_sample'
    inputs = 'inputs'
    gp_mean = 'gp_mean'
    gp_std = 'gp_std'
    n_train = 6
    al_gp = GP_al_trainer
    gp_evaluator = GP_eval
    learning_function = 'Ufunction'
    learning_function_parameter = 349.345
    learning_function_threshold = 2.0
  []
[]

[Trainers]
  [GP_al_trainer]
    type = ActiveLearningGaussianProcess
    covariance_function = 'covar'
    standardize_params = 'true'
    standardize_data = 'true'
    tune_parameters = 'covar:signal_variance covar:length_factor'
    num_iters = 1000
    learning_rate = 0.005
  []
[]

[Surrogates]
  [GP_eval]
    type = GaussianProcessSurrogate
    trainer = GP_al_trainer
  []
[]

[Covariance]
  [covar]
    type = SquaredExponentialCovariance
    signal_variance = 1.0
    noise_variance = 1e-4
    length_factor = '1.0 1.0 1.0'
  []
[]

[Executioner]
  type = Transient
[]

[Outputs]
  file_base = 'single_proc_single_row_ufunction'
  [out]
    type = JSON
    execute_system_information_on = none
  []
[]
(contrib/moose/modules/stochastic_tools/test/tests/reporters/ActiveLearningGP/main_adam.i)

ActiveLearningMonteCarloSampler

The Sampler block of the input file is presented below. As with the regular MonteCarloSampler, this block requires the distributions of the input parameters to the subApp. The "num_batch" parameter specifies the number of input samples to sample per step and also the number of allowed poor GP predictions before launching the high-fidelity model executions in parallel. The "flag_sample" is a Reporter object of type vector bool which indicates which GP predictions in the batch are poor to the Sampler.

[Samplers]
  [mc]
    type = ActiveLearningMonteCarloSampler
    num_batch = 1
    distributions = 'k_dist q_dist Tinf_dist'
    flag_sample = 'conditional/flag_sample'
    seed = 5
    num_samples = 20
    execute_on = PRE_MULTIAPP_SETUP
  []
[]
(contrib/moose/modules/stochastic_tools/test/tests/reporters/ActiveLearningGP/main_adam.i)

ActiveLearningGaussianProcess and GaussianProcess

The surrogate Trainers block of the input file is presented below. As with the regular GaussianProcessTrainer, this block has the required parameters for training a GP model. The key difference with ActiveLearningGaussianProcess is that it permits re-training of the GP model on the fly; this functionality is used by the ActiveLearningGPDecision Reporter. The below block presents ActiveLearningGaussianProcess relying on Adam optimization for re-training the GP model.

[Trainers]
  [GP_al_trainer]
    type = ActiveLearningGaussianProcess
    covariance_function = 'covar'
    standardize_params = 'true'
    standardize_data = 'true'
    tune_parameters = 'covar:signal_variance covar:length_factor'
    num_iters = 1000
    learning_rate = 0.005
  []
[]
(contrib/moose/modules/stochastic_tools/test/tests/reporters/ActiveLearningGP/main_adam.i)

The below block presents ActiveLearningGaussianProcess relying on TAO optimization for re-training the GP model.

[Trainers]
  [GP_al_trainer]
    type = ActiveLearningGaussianProcess
    covariance_function = 'covar'
    standardize_params = 'true'
    standardize_data = 'true'
    tune_parameters = 'covar:signal_variance covar:length_factor'
    num_iters = 1000
    learning_rate = 0.005
  []
[]
(contrib/moose/modules/stochastic_tools/test/tests/reporters/ActiveLearningGP/main_adam.i)

Once the GP model is re-trained, the GaussianProcess Surrogate is used by ActiveLearningGPDecision Reporter for making the GP predictions and uncertainty quantification. This block of the input file is presented below.

[Surrogates]
  [GP_eval]
    type = GaussianProcessSurrogate
    trainer = GP_al_trainer
  []
[]
(contrib/moose/modules/stochastic_tools/test/tests/reporters/ActiveLearningGP/main_adam.i)

ActiveLearningGPDecision

The Reporters block of the input file is presented below. It requires the name of the ActiveLearningMonteCarloSampler sampler. It also requires the names of the Reporter variables (which are all vectors) "flag_sample", "inputs", "gp_mean", and "gp_std" which, respectively, correspond to GP predictions being good/poor, input samples, GP mean predictions, and associated standard deviations. The values of these Reporters are all stored in an output json file.

In addition, the "n_train" parameter specifies the number of initial steps after which the GP model is first trained. "al_gp" specifies the name of ActiveLearningGaussianProcess for facilitating GP re-training and "gp_evaluator" specifies the name of GaussianProcess for facilitating GP predictions and uncertainty quantification.

Finally, "learning_function" specifies the name of the acquisition (or learning) function. "learning_function_parameter" specifies a parameter for the acquisition (or learning) function; it may or may not be required depending upon the type of the acquisition (or learning) function used. For example, the U acquisition (or learning) function requires "learning_function_parameter" but the COV acquisition (or learning) function does not. "learning_function_threshold" specifies the acceptable threshold for the acquisition (or learning) function beyond which the GP predictions are marked as poor.

[Reporters]
  [conditional]
    type = ActiveLearningGPDecision
    sampler = mc
    parallel_type = ROOT
    execute_on = 'timestep_begin'
    flag_sample = 'flag_sample'
    inputs = 'inputs'
    gp_mean = 'gp_mean'
    gp_std = 'gp_std'
    n_train = 6
    al_gp = GP_al_trainer
    gp_evaluator = GP_eval
    learning_function = 'Ufunction'
    learning_function_parameter = 349.345
    learning_function_threshold = 2.0
  []
[]
(contrib/moose/modules/stochastic_tools/test/tests/reporters/ActiveLearningGP/main_adam.i)

Output format

The recommended output format for using active learning is a json file. For each time step, the json file contains vectors of input samples, need_sample bool variables (indicating good/poor GP predictions), GP mean predictions, and associated standard deviations. The total number of samples equals "num_samples", including the "n_train" samples used for training. The total number of times need_sample value is True is the total samples for which GP re-training is required after the initial training.

commentnote:Number of output samples

When processing the json file, the samples whose need_sample value is True need to be ignored to avoid repeated results. Therefore, the number of output samples with need_sample set to False should be equal to "num_samples", as any re-training step does not count as an accepted sample.

Other supported sampler types

The following samplers are also supported with the ActiveLearningGPDecision reporter:

  • Active Learning in Adaptive Importance Sampling (AIS-AL)

    See AISActiveLearning for more information.

Input Parameters

  • al_gpActive learning GP trainer.

    C++ Type:UserObjectName

    Unit:(no unit assumed)

    Controllable:No

    Description:Active learning GP trainer.

  • gp_evaluatorEvaluate the trained GP.

    C++ Type:UserObjectName

    Unit:(no unit assumed)

    Controllable:No

    Description:Evaluate the trained GP.

  • learning_functionThe learning function for active learning.

    C++ Type:MooseEnum

    Unit:(no unit assumed)

    Options:Ufunction, COV

    Controllable:No

    Description:The learning function for active learning.

  • learning_function_thresholdThe learning function threshold.

    C++ Type:double

    Unit:(no unit assumed)

    Controllable:No

    Description:The learning function threshold.

  • n_trainNumber of training steps.

    C++ Type:int

    Unit:(no unit assumed)

    Controllable:No

    Description:Number of training steps.

  • samplerThe sampler object.

    C++ Type:SamplerName

    Unit:(no unit assumed)

    Controllable:No

    Description:The sampler object.

Required Parameters

  • execute_onTIMESTEP_ENDThe list of flag(s) indicating when this object should be executed. For a description of each flag, see https://mooseframework.inl.gov/source/interfaces/SetupInterface.html.

    Default:TIMESTEP_END

    C++ Type:ExecFlagEnum

    Unit:(no unit assumed)

    Options:NONE, INITIAL, LINEAR, NONLINEAR_CONVERGENCE, NONLINEAR, POSTCHECK, TIMESTEP_END, TIMESTEP_BEGIN, MULTIAPP_FIXED_POINT_END, MULTIAPP_FIXED_POINT_BEGIN, FINAL, CUSTOM

    Controllable:No

    Description:The list of flag(s) indicating when this object should be executed. For a description of each flag, see https://mooseframework.inl.gov/source/interfaces/SetupInterface.html.

  • flag_sampleflag_sampleFlag samples.

    Default:flag_sample

    C++ Type:ReporterValueName

    Unit:(no unit assumed)

    Controllable:No

    Description:Flag samples.

  • gp_meangp_meanThe GP mean prediction.

    Default:gp_mean

    C++ Type:ReporterValueName

    Unit:(no unit assumed)

    Controllable:No

    Description:The GP mean prediction.

  • gp_stdgp_stdThe GP standard deviation.

    Default:gp_std

    C++ Type:ReporterValueName

    Unit:(no unit assumed)

    Controllable:No

    Description:The GP standard deviation.

  • inputsinputsThe inputs.

    Default:inputs

    C++ Type:ReporterValueName

    Unit:(no unit assumed)

    Controllable:No

    Description:The inputs.

  • learning_function_parameter1.79769e+308The learning function parameter.

    Default:1.79769e+308

    C++ Type:double

    Unit:(no unit assumed)

    Controllable:No

    Description:The learning function parameter.

  • parallel_typeDISTRIBUTEDThis parameter will determine how the stochastic data is gathered. It is common for outputting purposes that this parameter be set to ROOT, otherwise, many files will be produced showing the values on each processor. However, if there are lot of samples, gathering on root may be memory restrictive.

    Default:DISTRIBUTED

    C++ Type:MooseEnum

    Unit:(no unit assumed)

    Options:DISTRIBUTED, ROOT

    Controllable:No

    Description:This parameter will determine how the stochastic data is gathered. It is common for outputting purposes that this parameter be set to ROOT, otherwise, many files will be produced showing the values on each processor. However, if there are lot of samples, gathering on root may be memory restrictive.

  • prop_getter_suffixAn optional suffix parameter that can be appended to any attempt to retrieve/get material properties. The suffix will be prepended with a '_' character.

    C++ Type:MaterialPropertyName

    Unit:(no unit assumed)

    Controllable:No

    Description:An optional suffix parameter that can be appended to any attempt to retrieve/get material properties. The suffix will be prepended with a '_' character.

  • use_interpolated_stateFalseFor the old and older state use projected material properties interpolated at the quadrature points. To set up projection use the ProjectedStatefulMaterialStorageAction.

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:For the old and older state use projected material properties interpolated at the quadrature points. To set up projection use the ProjectedStatefulMaterialStorageAction.

Optional Parameters

  • allow_duplicate_execution_on_initialFalseIn the case where this UserObject is depended upon by an initial condition, allow it to be executed twice during the initial setup (once before the IC and again after mesh adaptivity (if applicable).

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:In the case where this UserObject is depended upon by an initial condition, allow it to be executed twice during the initial setup (once before the IC and again after mesh adaptivity (if applicable).

  • control_tagsAdds user-defined labels for accessing object parameters via control logic.

    C++ Type:std::vector<std::string>

    Unit:(no unit assumed)

    Controllable:No

    Description:Adds user-defined labels for accessing object parameters via control logic.

  • enableTrueSet the enabled status of the MooseObject.

    Default:True

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:Yes

    Description:Set the enabled status of the MooseObject.

  • execution_order_group0Execution order groups are executed in increasing order (e.g., the lowest number is executed first). Note that negative group numbers may be used to execute groups before the default (0) group. Please refer to the user object documentation for ordering of user object execution within a group.

    Default:0

    C++ Type:int

    Unit:(no unit assumed)

    Controllable:No

    Description:Execution order groups are executed in increasing order (e.g., the lowest number is executed first). Note that negative group numbers may be used to execute groups before the default (0) group. Please refer to the user object documentation for ordering of user object execution within a group.

  • force_postauxFalseForces the UserObject to be executed in POSTAUX

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:Forces the UserObject to be executed in POSTAUX

  • force_preauxFalseForces the UserObject to be executed in PREAUX

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:Forces the UserObject to be executed in PREAUX

  • force_preicFalseForces the UserObject to be executed in PREIC during initial setup

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:Forces the UserObject to be executed in PREIC during initial setup

  • outputsVector of output names where you would like to restrict the output of variables(s) associated with this object

    C++ Type:std::vector<OutputName>

    Unit:(no unit assumed)

    Controllable:No

    Description:Vector of output names where you would like to restrict the output of variables(s) associated with this object

  • use_displaced_meshFalseWhether or not this object should use the displaced mesh for computation. Note that in the case this is true but no displacements are provided in the Mesh block the undisplaced mesh will still be used.

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:Whether or not this object should use the displaced mesh for computation. Note that in the case this is true but no displacements are provided in the Mesh block the undisplaced mesh will still be used.

Advanced Parameters

References

  1. S. L. N. Dhulipala, M. D. Shields, B. W. Spencer, C. Bolisetti, A. E. Slaughter, V. M. Labouré, and P. Chakroborty. Active learning with multifidelity modeling for efficient rare event simulation. Journal of Computational Physics, 468:111506, 2022. URL: https://doi.org/10.1016/j.jcp.2022.111506.[BibTeX]
  2. D. P. Kingma and J. Ba. Adam: a method for stochastic optimization. arXiv:1412.6980 [cs.LG], 2014. URL: https://doi.org/10.48550/arXiv.1412.6980.[BibTeX]
  3. S. Marelli and B. Sudret. An active-learning algorithm that combines sparse polynomial chaos expansions and bootstrap for structural reliability analysis. Structural Safety, 75:67–74, 2018. URL: https://doi.org/10.1016/j.strusafe.2018.06.003.[BibTeX]
  4. Y. Xiu, D. Barajas-Solano, G. Tartakovsky, and A. M. Tartakovsky. Physics-informed cokriging: a gaussian-process-regression-based multifidelity method for data-model convergence. Journal of Computational Physics, 395:410–431, 2019. URL: https://doi.org/10.1016/j.jcp.2019.06.041.[BibTeX]