ParameterStudy

Builds objects to set up a basic parameter study.

Overview

ParameterStudy is meant to be a simplified syntax for running common parameter studies like the one described in Parameter Study. The idea with this syntax is users do not need to understand or go through the modular design of the stochastic tools module to run a basic parameter study. The ParameterStudyAction builds the necessary objects programmatically, objects including distributions, samplers, multi-apps, and transfers. The following sections describe in detail the capabilities of this system, but the basic functionalities include:

Defining parameters and quantities of interest (QoIs) within the given physics input.
Perturbing the parameters using the specified sampling method.
Efficiently running each instance of the perturbation.
Gathering the QoI results.
Computing statistics on the QoIs
Outputting the QoIs and statistics.

Defining the Study

The problem containing the physics is defined via another input file and specified using the "input" parameter. The idea is that this input defines the "nominal" case of the problem; as such, it should be able to run independently (without the ParameterStudy block).

The parameters being perturbed in the study are specified using the "parameters" parameter. This is a list of parameters in the physics input that use CommandLine syntax.

The QoIs for the study are specified using the "quantities_of_interest" parameter. This is a list of reporter values with <object_name>/<value_name> syntax. Postprocessor values use <pp_name>/value syntax. Vector-postprocessor values use <vpp_name>/<vector_name> syntax. This parameter is optional as users may want to do their own analysis using outputs, such as CSV or Exodus, generated from the physics input.

Whether or not to compute statistics on the QoIs is defined by the "compute_statistics" parameter. The default for this parameter is true, so statistics will be computed by default if QoIs are specified with the "quantities_of_interest" parameter. A StatisticsReporter object is created, which by default computes the mean and standard deviation of the QoIs with 90%, 95%, and 99% confidence intervals. The type of statistics can be specified with "statistics" and the confidence interval computation can specified with "ci_levels" and "ci_replicates", see StatisticsReporter for more details on confidence interval computation.

Sampling Method

There are various methods of sampling included with the ParameterStudy syntax, including random and pre-defined techniques described below. This method is specified using the "sampling_type" parameter.

Random Sampling

The monte-carlo and lhs options in "sampling_type" implement random sampling using the MonteCarlo and LatinHypercube samplers, respectively. "num_samples" is a required parameter that defines the total number of samples. "distributions" is also a required parameter that defines the probability distribution for each parameter. The available distributions are described in the following subsection. Each distribution type have their own set of required parameters that are specified via a list of values. For example, defining a random sampling with three parameters using uniform, normal, and uniform distributions is shown below:

[ParameterStudy]
  input = sub.i
  parameters = 'p1 p2 p3'
  sampling_type = monte-carlo
  num_samples = 10
  distributions = 'uniform normal uniform'
  uniform_lower_bound = '1 100'
  uniform_upper_bound = '2 200'
  normal_mean = '0'
  normal_standard_deviation = '1'
[]

Uniform Distribution

The uniform option in "distributions" builds a Uniform distribution. The "uniform_lower_bound" and "uniform_upper_bound" parameters are required if one or more of these options are specified.

Normal Distribution

The normal option in "distributions" builds a Normal distribution. The "normal_mean" and "normal_standard_deviation" parameters are required if one or more of these options are specified.

Weibull Distribution

The wiebull option in "distributions" builds a Weibull distribution. The "weibull_location", "weibull_scale", and "weibull_shape" parameters are required if one or more of these options are specified.

Lognormal Distribution

The lognormal option in "distributions" builds a Lognormal distribution. The "lognormal_location" and "lognormal_scale" parameters are required if one or more of these options are specified.

Truncated Normal Distribution

The tnormal option in "distributions" builds a TruncatedNormal distribution. The "tnormal_mean", "tnormal_standard_deviation", "tnormal_lower_bound", and "tnormal_upper_bound" parameters are required if one or more of these options are specified.

Cartesian Product Sampling

The cartesian-product option in "sampling_type" implements Cartesian product sampling using the CartesianProduct sampler. "linear_space_items" is a required parameter with this option.

User-Defined Matrix Sampling

A pre-defined matrix for sampling can be implemented using the csv or input-matrix options in "sampling_type".

csv builds a CSVSampler sampler with the CSV file being specified using the "csv_samples_file" parameter. The choice and order of the matrix columns can be specified using the "csv_column_indices" or "csv_column_names" parameters.

input-matrix builds a InputMatrix sampler with the matrix being specified using the "input_matrix" parameter.

Execution Mode

The stochastic tools module has a number of different ways to perform stochastic sampling, as described in Stochastic Tools Batch Mode. The permissible and most efficient mode is largely based on the type of parameters being perturbed and the problem's physics. There are five different mode options using the "multiapp_mode" parameter, which are described in the following subsections.

Additionally, it is often useful to define the minimum processors to use when running the samples. Typically this is done for large models in batch mode to avoid excessive memory usage. The "min_procs_per_sample" will utilize this capability.

Normal mode

The normal option for "multiapp_mode" runs the study in "normal" mode, which creates a sub-application for each sample. The sub-applications are created upfront and run sequentially. This mode is arguably the least efficient option as it can become extremely memory intensive. This option mainly exists to replicate the typical execution mode for other MOOSE MultiApps. It also serves as a useful debugging tool for stochastic tools module development.

Batch-Reset Mode

The batch-reset option for "multiapp_mode" runs the study in "batch-reset" mode, which creates a sub-application per processor, or set of processors, and re-initializes it for each sample. This mode is the recommended mode for general problems where the parameters are not (or not known to be) controllable. The controllability of objects can be found in their documentation; for instance, BCs/DirichletBC/"value" is controllable, but Mesh/GeneratedMeshGenerator/"xmax" is not controllable.

Batch-Restore Mode

The batch-restore option for "multiapp_mode" runs the study in "batch-restore" mode. Similar to batch-reset, batch-restore creates a sub-application per processor, but does not re-initialize. Instead it "restores" the sub-application to its state before the solve took place. This mode can be more efficient than batch-reset since the application's initialization is skipped, reusing the mesh and system vectors already built. However, the parameters being perturbed must be controllable. Parameters involved in mesh generation and initial conditions are usually not controllable.

Batch-Restore Mode – Keep Solution

The batch-keep-solution option for "multiapp_mode" runs the study in "batch-restore" mode. The difference with batch-restore is that the solution from the previous sample is reused as the initial guess. This can reduce the solve time for subsequent samples since the previous sample's solution is most likely a better initial guess than the default or user-defined one. This mode is not recommended for transient physics since it will alter the initial condition, which will change the resulting solution. This mode is recommended for pseudo transients, where a steady-state solution is the result of the simulation.

Batch-Restore Mode – No Restore

The batch-no-restore option for "multiapp_mode" runs the study in "batch-restore" mode, except the sub-application is not restored. Instead, the sub-application's solve is simply repeated with the newly perturbed parameters. This mode provides another efficiency by skipping this restoration step. This mode only works with steady-state problems like those run with the Steady and Eigenvalue executioners, as transient problems need to be restored to the original time step.

Automatic Mode Detection

While the execution mode can be explicitly specified using the "multiapp_mode" parameter, if this parameter is unspecified, the action will attempt to run in the most efficient mode by reading the physics input file. First, it will determine if the perturbed "parameters" are all controllable. If not all the parameter are controllable, the action will use batch-reset mode. This detection is a bit rudimentary, so it might not detect all controllable parameters. Next, it will determine what type of execution is being performed. If the Executioner/type is Steady or Eigenvalue, then it will use batch-no-restore mode. If the executioner is Transient with steady_state_detection = true, then it will assume the simulation is pseudo-transient and use batch-keep-solution. All other cases with controllable parameters will use batch-restore. This automatic detection is not meant to be exhaustive, so it is recommended that users fully understand the problem and use the "multiapp_mode" parameter.

Outputs

The ParameterStudy syntax provides ways to output the results of the study in CSV and/or JSON format. The sampler matrix (the values of the perturbed parameters) and the resulting QoI values can be output by setting the "output_type" parameter to csv and/or json. However, the CSV output will not contain vector-type QoIs, like those from vector-postprocessors. The default for both of this parameter is json. A JSON output is automatically created if QoI statistics are computed.

List of Objects

Typically performing parameter studies only requires the ParameterStudy block. However, ParameterStudy does not prevent the addition of more objects by using other valid input syntax. This section gives advanced users more information on how to make other syntax work with ParameterStudy seamlessly.

All the objects built with ParameterStudyAction can be shown on console using the "show_study_objects" parameter. Table 1 lists all the objects created.

Table 1: Objects created using the ParameterStudy syntax

Base Type	Type	Name
Action	StochasticToolsAction	`ParameterStudy_stochastic_tools_action`
Distribution	See Random Sampling	`study_distribution_<i>` $^a$
Sampler	See Sampling Method	`study_sampler`
MultiApp	SamplerFullSolveMultiApp	`study_app`
Control	MultiAppSamplerControl	`study_multiapp_control` $^b$
Transfer	SamplerParameterTransfer	`study_parameter_transfer` $^c$
Transfer	SamplerReporterTransfer	`study_qoi_transfer` $^d$
Reporter	StochasticMatrix	`study_results`
Reporter	StatisticsReporter	`study_statistics` $^{d,e}$
Output	CSV	`csv` $^f$
Output	JSON	`json` $^g$

$^a$ <i> corresponds to the index of the distribution in "distributions";
$^b$ If "multiapp_mode" is normal or batch-reset;
$^c$ If "multiapp_mode" is batch-restore, batch-keep-solution, or batch-no-restore;
$^d$ If "quantities_of_interest" is specified;
$^e$ If "compute_statistics" is true;
$^f$ If "output_type" contains csv;
$^g$ If "output_type" contains json or "compute_statistics" is true;

Example Input Syntax

The following two inputs are equivalent, taken from Parameter Study.

[StochasticTools]
[]

[Distributions]
  [gamma]
    type = Uniform
    lower_bound = 0.5
    upper_bound = 2.5
  []
  [q_0]
    type = Weibull
    location = -110
    scale = 20
    shape = 1
  []
  [T_0]
    type = Normal
    mean = 300
    standard_deviation = 45
  []
  [s]
    type = Normal
    mean = 100
    standard_deviation = 25
  []
[]

[Samplers]
  [hypercube]
    type = LatinHypercube
    num_rows = 5000
    distributions = 'gamma q_0 T_0 s'
  []
[]

[MultiApps]
  [runner]
    type = SamplerFullSolveMultiApp
    sampler = hypercube
    input_files = 'diffusion.i'
    mode = batch-restore
  []
[]

[Transfers]
  [parameters]
    type = SamplerParameterTransfer
    to_multi_app = runner
    sampler = hypercube
    parameters = 'Materials/constant/prop_values Kernels/source/value BCs/right/value BCs/left/value'
  []
  [results]
    type = SamplerReporterTransfer
    from_multi_app = runner
    sampler = hypercube
    stochastic_reporter = results
    from_reporter = 'T_avg/value q_left/value'
  []
[]

[Reporters]
  [results]
    type = StochasticReporter
  []
  [stats]
    type = StatisticsReporter
    reporters = 'results/results:T_avg:value results/results:q_left:value'
    compute = 'mean stddev'
    ci_method = 'percentile'
    ci_levels = '0.05 0.95'
  []
[]

[Outputs]
  execute_on = 'FINAL'
  [out]
    type = JSON
  []
[]

[ParameterStudy]
  input = diffusion.i
  parameters = 'Materials/constant/prop_values Kernels/source/value BCs/right/value BCs/left/value'
  quantities_of_interest = 'T_avg/value q_left/value'

  sampling_type = lhs
  num_samples = 5000
  distributions = 'uniform weibull normal normal'
  uniform_lower_bound = 0.5
  uniform_upper_bound = 2.5
  weibull_location = -110
  weibull_scale = 20
  weibull_shape = 1
  normal_mean = '300 100'
  normal_standard_deviation = '45 25'
[]

Input Parameters

inputThe input file containing the physics for the parameter study.
C++ Type:FileName
Unit:(no unit assumed)
Controllable:No
Description:The input file containing the physics for the parameter study.
parametersList of parameters being perturbed for the study.
C++ Type:std::vector<std::string>
Unit:(no unit assumed)
Controllable:No
Description:List of parameters being perturbed for the study.
sampling_typeThe type of sampling to use for the parameter study.
C++ Type:MooseEnum
Unit:(no unit assumed)
Options:monte-carlo, lhs, cartesian-product, csv, input-matrix
Controllable:No
Description:The type of sampling to use for the parameter study.

Required Parameters

active__all__ If specified only the blocks named will be visited and made active
Default:__all__
C++ Type:std::vector<std::string>
Unit:(no unit assumed)
Controllable:No
Description:If specified only the blocks named will be visited and made active
ci_levels0.01 0.05 0.1 0.9 0.95 0.99 A vector of confidence levels to consider for statistics confidence intervals, values must be in (0, 1).
Default:0.01 0.05 0.1 0.9 0.95 0.99
C++ Type:std::vector<double>
Unit:(no unit assumed)
Controllable:No
Description:A vector of confidence levels to consider for statistics confidence intervals, values must be in (0, 1).
ci_replicates1000The number of replicates to use when computing confidence level intervals for statistics.
Default:1000
C++ Type:unsigned int
Unit:(no unit assumed)
Controllable:No
Description:The number of replicates to use when computing confidence level intervals for statistics.
compute_statisticsTrueWhether or not to compute statistics on the 'quantities_of_interest'. The default is to compute mean and standard deviation with 0.01, 0.05, 0.1, 0.9, 0.95, and 0.99 confidence intervals.
Default:True
C++ Type:bool
Unit:(no unit assumed)
Controllable:No
Description:Whether or not to compute statistics on the 'quantities_of_interest'. The default is to compute mean and standard deviation with 0.01, 0.05, 0.1, 0.9, 0.95, and 0.99 confidence intervals.
csv_column_indicesColumn indices in the CSV file to be sampled from for 'csv' sampling. Number of indices here will be the same as the number of columns per matrix.
C++ Type:std::vector<unsigned long>
Unit:(no unit assumed)
Controllable:No
Description:Column indices in the CSV file to be sampled from for 'csv' sampling. Number of indices here will be the same as the number of columns per matrix.
csv_column_namesColumn names in the CSV file to be sampled from for 'csv' sampling. Number of columns names here will be the same as the number of columns per matrix.
C++ Type:std::vector<std::string>
Unit:(no unit assumed)
Controllable:No
Description:Column names in the CSV file to be sampled from for 'csv' sampling. Number of columns names here will be the same as the number of columns per matrix.
csv_samples_fileName of the CSV file that contains the sample matrix for 'csv' sampling.
C++ Type:FileName
Unit:(no unit assumed)
Controllable:No
Description:Name of the CSV file that contains the sample matrix for 'csv' sampling.
distributionsThe types of distribution to use for 'monte-carlo' and 'lhs' sampling. The number of entries defines the number of columns in the matrix.
C++ Type:MultiMooseEnum
Unit:(no unit assumed)
Options:normal, uniform, weibull, lognormal, tnormal
Controllable:No
Description:The types of distribution to use for 'monte-carlo' and 'lhs' sampling. The number of entries defines the number of columns in the matrix.
ignore_solve_not_convergeFalseTrue to continue main app even if a sub app's solve does not converge.
Default:False
C++ Type:bool
Unit:(no unit assumed)
Controllable:No
Description:True to continue main app even if a sub app's solve does not converge.
inactiveIf specified blocks matching these identifiers will be skipped.
C++ Type:std::vector<std::string>
Unit:(no unit assumed)
Controllable:No
Description:If specified blocks matching these identifiers will be skipped.
input_matrixSampling matrix for 'input-matrix' sampling.
C++ Type:Eigen::Matrix<double, -1, -1, 0, -1, -1>
Unit:(no unit assumed)
Controllable:No
Description:Sampling matrix for 'input-matrix' sampling.
linear_space_itemsParameter for defining the 'cartesian-prodcut' sampling scheme. A list of triplets, each item should include the min, step size, and number of steps.
C++ Type:std::vector<double>
Unit:(no unit assumed)
Controllable:No
Description:Parameter for defining the 'cartesian-prodcut' sampling scheme. A list of triplets, each item should include the min, step size, and number of steps.
lognormal_locationThe 'lognormal' distributions' location parameter (m or mu).
C++ Type:std::vector<double>
Unit:(no unit assumed)
Controllable:No
Description:The 'lognormal' distributions' location parameter (m or mu).
lognormal_scaleThe 'lognormal' distributions' scale parameter (s or sigma).
C++ Type:std::vector<double>
Unit:(no unit assumed)
Controllable:No
Description:The 'lognormal' distributions' scale parameter (s or sigma).
min_procs_per_sample1Minimum number of processors to give to each sample. Useful for larger, distributed mesh solves where there are memory constraints.
Default:1
C++ Type:unsigned int
Unit:(no unit assumed)
Controllable:No
Description:Minimum number of processors to give to each sample. Useful for larger, distributed mesh solves where there are memory constraints.
multiapp_modeThe operation mode, 'normal' creates one sub-application for each sample.'batch' creates one sub-app for each processor and re-executes for each local sample. 'reset' re-initializes the sub-app for every sample in the batch. 'restore' does not re-initialize and instead restores to first sample's initialization. 'keep-solution' re-uses the solution obtained from the first sample in the batch. 'no-restore' does not restore the sub-app.The default will be inferred based on the study.
C++ Type:MooseEnum
Unit:(no unit assumed)
Options:normal, batch-reset, batch-restore, batch-keep-solution, batch-no-restore
Controllable:No
Description:The operation mode, 'normal' creates one sub-application for each sample.'batch' creates one sub-app for each processor and re-executes for each local sample. 'reset' re-initializes the sub-app for every sample in the batch. 'restore' does not re-initialize and instead restores to first sample's initialization. 'keep-solution' re-uses the solution obtained from the first sample in the batch. 'no-restore' does not restore the sub-app.The default will be inferred based on the study.
normal_meanMeans (or expectations) of the 'normal' distributions.
C++ Type:std::vector<double>
Unit:(no unit assumed)
Controllable:No
Description:Means (or expectations) of the 'normal' distributions.
normal_standard_deviationStandard deviations of the 'normal' distributions.
C++ Type:std::vector<double>
Unit:(no unit assumed)
Controllable:No
Description:Standard deviations of the 'normal' distributions.
num_samplesThe number of samples to generate for 'monte-carlo' and 'lhs' sampling.
C++ Type:unsigned long
Unit:(no unit assumed)
Controllable:No
Description:The number of samples to generate for 'monte-carlo' and 'lhs' sampling.
output_typejsonMethod in which to output sampler matrix and quantities of interest. Warning: 'csv' output will not include vector-type quantities.
Default:json
C++ Type:MultiMooseEnum
Unit:(no unit assumed)
Options:none, csv, json
Controllable:No
Description:Method in which to output sampler matrix and quantities of interest. Warning: 'csv' output will not include vector-type quantities.
quantities_of_interestList of the reporter names (object_name/value_name) that represent the quantities of interest for the study.
C++ Type:std::vector<ReporterName>
Unit:(no unit assumed)
Controllable:No
Description:List of the reporter names (object_name/value_name) that represent the quantities of interest for the study.
sampler_column_namesNames of the sampler columns for outputting the sampling matrix. If 'parameters' are not bracketed, the default is based on these values. Otherwise, the default is based on the sampler name.
C++ Type:std::vector<ReporterValueName>
Unit:(no unit assumed)
Controllable:No
Description:Names of the sampler columns for outputting the sampling matrix. If 'parameters' are not bracketed, the default is based on these values. Otherwise, the default is based on the sampler name.
seed0Random number generator initial seed
Default:0
C++ Type:unsigned int
Unit:(no unit assumed)
Controllable:No
Description:Random number generator initial seed
show_study_objectsFalseSet to true to show all the objects being built by this action.
Default:False
C++ Type:bool
Unit:(no unit assumed)
Controllable:No
Description:Set to true to show all the objects being built by this action.
statisticsmean stddevThe statistic(s) to compute for the study.
Default:mean stddev
C++ Type:MultiMooseEnum
Unit:(no unit assumed)
Options:min, max, sum, mean, stddev, norm2, ratio, stderr, median, meanabs
Controllable:No
Description:The statistic(s) to compute for the study.
tnormal_lower_bound'tnormal' distributions' lower bound
C++ Type:std::vector<double>
Unit:(no unit assumed)
Controllable:No
Description:'tnormal' distributions' lower bound
tnormal_meanMeans (or expectations) of the 'tnormal' distributions.
C++ Type:std::vector<double>
Unit:(no unit assumed)
Controllable:No
Description:Means (or expectations) of the 'tnormal' distributions.
tnormal_standard_deviationStandard deviations of the 'tnormal' distributions.
C++ Type:std::vector<double>
Unit:(no unit assumed)
Controllable:No
Description:Standard deviations of the 'tnormal' distributions.
tnormal_upper_bound'tnormal' distributions' upper bound
C++ Type:std::vector<double>
Unit:(no unit assumed)
Controllable:No
Description:'tnormal' distributions' upper bound
uniform_lower_boundLower bounds for 'uniform' distributions.
C++ Type:std::vector<double>
Unit:(no unit assumed)
Controllable:No
Description:Lower bounds for 'uniform' distributions.
uniform_upper_boundUpper bounds 'uniform' distributions.
C++ Type:std::vector<double>
Unit:(no unit assumed)
Controllable:No
Description:Upper bounds 'uniform' distributions.
weibull_locationLocation parameter (a or low) for 'weibull' distributions.
C++ Type:std::vector<double>
Unit:(no unit assumed)
Controllable:No
Description:Location parameter (a or low) for 'weibull' distributions.
weibull_scaleScale parameter (b or lambda) for 'weibull' distributions.
C++ Type:std::vector<double>
Unit:(no unit assumed)
Controllable:No
Description:Scale parameter (b or lambda) for 'weibull' distributions.
weibull_shapeShape parameter (c or k) for 'weibull' distributions.
C++ Type:std::vector<double>
Unit:(no unit assumed)
Controllable:No
Description:Shape parameter (c or k) for 'weibull' distributions.

Optional Parameters

control_tagsAdds user-defined labels for accessing object parameters via control logic.
C++ Type:std::vector<std::string>
Unit:(no unit assumed)
Controllable:No
Description:Adds user-defined labels for accessing object parameters via control logic.

Advanced Parameters

Available Actions

Stochastic Tools App
ParameterStudyActionBuilds objects to set up a basic parameter study.

(contrib/moose/modules/stochastic_tools/examples/parameter_study/main.i)

[StochasticTools]
[]

[Distributions]
  [gamma]
    type = Uniform
    lower_bound = 0.5
    upper_bound = 2.5
  []
  [q_0]
    type = Weibull
    location = -110
    scale = 20
    shape = 1
  []
  [T_0]
    type = Normal
    mean = 300
    standard_deviation = 45
  []
  [s]
    type = Normal
    mean = 100
    standard_deviation = 25
  []
[]

[Samplers]
  [hypercube]
    type = LatinHypercube
    num_rows = 5000
    distributions = 'gamma q_0 T_0 s'
  []
[]

[MultiApps]
  [runner]
    type = SamplerFullSolveMultiApp
    sampler = hypercube
    input_files = 'diffusion.i'
    mode = batch-restore
  []
[]

[Transfers]
  [parameters]
    type = SamplerParameterTransfer
    to_multi_app = runner
    sampler = hypercube
    parameters = 'Materials/constant/prop_values Kernels/source/value BCs/right/value BCs/left/value'
  []
  [results]
    type = SamplerReporterTransfer
    from_multi_app = runner
    sampler = hypercube
    stochastic_reporter = results
    from_reporter = 'T_avg/value q_left/value'
  []
[]

[Reporters]
  [results]
    type = StochasticReporter
  []
  [stats]
    type = StatisticsReporter
    reporters = 'results/results:T_avg:value results/results:q_left:value'
    compute = 'mean stddev'
    ci_method = 'percentile'
    ci_levels = '0.05 0.95'
  []
[]

[Outputs]
  execute_on = 'FINAL'
  [out]
    type = JSON
  []
[]

(contrib/moose/modules/stochastic_tools/examples/parameter_study/main_parameter_study.i)

[ParameterStudy]
  input = diffusion.i
  parameters = 'Materials/constant/prop_values Kernels/source/value BCs/right/value BCs/left/value'
  quantities_of_interest = 'T_avg/value q_left/value'

  sampling_type = lhs
  num_samples = 5000
  distributions = 'uniform weibull normal normal'
  uniform_lower_bound = 0.5
  uniform_upper_bound = 2.5
  weibull_location = -110
  weibull_scale = 20
  weibull_shape = 1
  normal_mean = '300 100'
  normal_standard_deviation = '45 25'
[]

Overview
Defining the Study
Sampling Method
Execution Mode
Outputs
List of Objects
Example Input Syntax
Input Parameters
Available Actions