PolynomialRegressionTrainer

Computes coefficients for polynomial regession model.

Overview

This class is responsible for determining the coefficients of a multi-dimensional polynomial which approximates the behavior of the Quantity of Interest (QoI) in the parameter space. For this, the object needs NN training points in the parameters space. These training points can be characterized using their coordinates in the parameter space:

X=[x1,1x1,2x1,Dx2,1x2,2x2,DxN,1xN,2xN,D]=[x1x2xN].\textbf{X} = \begin{bmatrix} x_{1,1} & x_{1,2} & \dots & x_{1,D} \\ x_{2,1} & x_{2,2} & \dots & x_{2,D} \\ \vdots & \vdots & \ddots & \vdots \\ x_{N,1} & x_{N,2} & \dots & x_{N,D} \end{bmatrix} = \begin{bmatrix} \textbf{x}_{1} \\ \textbf{x}_{2} \\ \vdots \\ \textbf{x}_{N} \end{bmatrix}.

where DD denotes the dimension of the parameter space and x\textbf{x} is a D-dimensional vector containing the coordinates in each dimension. Similarly to other Trainer classes, PolynomialRegressionTrainer accesses this matrix from a Sampler object. Of course, the trainer has to know the values of the QoI at these coordinates as well:

y=[y1y2yN].\textbf{y} = \begin{bmatrix} y_{1} \\ y_{2} \\ \vdots\\ y_{N} \end{bmatrix} .

This data is accessed through a VectorPostprocessor. Now that all data is available, y=f(x)\textbf{y}=f(\textbf{x}) unknown function is approximated using a polynomial expression of the following form:

yy^=k=1NpP(x,ik)ck=P(x)c,\textbf{y} \approx \hat{\textbf{y}} = \sum \limits_{k=1}^{N_p}P(\textbf{x}, \textbf{i}_{k})c_k=\textbf{P}(\textbf{x})\textbf{c},(1)

where NpN_p is the number of polynomial terms in the approximation, c=[c1,...,cNp]T\textbf{c}=[c_1,...,c_{N_p}]^T are the unknown coefficients and P(x)=[P(x,i1),...,P(x,iNp)]\textbf{P}(\textbf{x})=[P(\textbf{x}, \textbf{i}_{1}),...,P(\textbf{x}, \textbf{i}_{N_p})]. The used polynomials in this case can be defined as

P(x,i)=xi=j=1Dxjij,P(\textbf{x}, \textbf{i}) = \textbf{x}^\textbf{i} = \prod \limits_{j=1}^D x_j^{i_j},

where xjx_j denotes the jj-th coordinate of parameter vector x\textbf{x}, while i=(i1,...,iD)\textbf{i}=(i_1,...,i_D) is a DD-dimensional tuple containing the powers for each coordinate. This tuple is the same as described in PolynomialChaos. To determine these tuples, the trainer needs an additional input parameter, namely the maximum degree of the polynomial. This limits the number of polynomial terms in Eq. (1). If this number is fixed, the only unknown parameters are the elements of c\textbf{c}.

To determine these, a regression matrix can be defined as:

R=[P(x1)P(x2)P(xN)].\textbf{R} = \begin{bmatrix} \textbf{P}(\textbf{x}_1) \\ \textbf{P}(\textbf{x}_2) \\ \vdots \\ \textbf{P}(\textbf{x}_{N}) \end{bmatrix}.

Ordinary Least Squares (OLS) regression

Using regression matrix R\textbf{R} and Ordinary Least Squares (OLS) approach described on Wikipedia in detail, the unknown coefficients can be determined as follows:

c=(RTR)1RTy.\textbf{c}=\left(\textbf{R}^T\textbf{R}\right)^{-1}\textbf{R}^T\textbf{y}.

Finally, it must be mentioned that this method is only applicable if NpNN_p \leq N and keeping Np<<NN_p << N is recommended.

Ridge regression

Unfortunately, the OLS approach is known to have some issues like: - It is prone to overfit the data, - It yields inaccurate results if the input variables are correlated, - It is sensitive to outliers.

To tackle the problem, an L2L^2 regularization (or Tikhonov regularization is adopted to make sure that the coefficients of the expansion do have uncontrollably high values. This extended least squares regression is often referred to as Ridge Regression. In this scenario the coefficients can be determined by solving:

c=(RTR+λI)1RTy,\textbf{c}=\left(\textbf{R}^T\textbf{R}+\lambda I\right)^{-1}\textbf{R}^T\textbf{y},

where λ\lambda is a penalty parameter which penalizes coefficients with large magnitudes. As λ0\lambda \rightarrow 0, Ridge regression converges to OLS.

Example Input File Syntax

To get the necessary data, two essential blocks have to be included in the master input file. The first, the sampler defined in Samplers, creates the coordinates in matrix X\textbf{X}, while the objects in VectorPostprocessors create, fill and store the result vector y\textbf{y}.

[Samplers]
  [sample]
    type = CartesianProduct
    linear_space_items = '0 1 10
                          0 1 10
                          0 1 10'
  []
[]
(contrib/moose/modules/stochastic_tools/test/tests/surrogates/polynomial_regression/train.i)
[VectorPostprocessors]
  [values]
    type = GFunction
    sampler = sample
    q_vector = '0 0 0'
    execute_on = INITIAL
    outputs = none
  []
[]
(contrib/moose/modules/stochastic_tools/test/tests/surrogates/polynomial_regression/train.i)

Similarly to NearestPointTrainer, a GFunction vector postprocessor from SobolStatistics is used to emulate a full-order model. This simply evaluates a function at sample points.

Using this data and the maximum degree setting (max_degree in the input file), the trainer computes the model coefficients c\textbf{c}. To control the type of regression, the user has to set regression_type to either 'ols' or 'ridge' in the input file:

[Trainers]
  [train]
    type = PolynomialRegressionTrainer
    regression_type = "ols"
    sampler = sample
    response = values/g_values
    max_degree = 3
  []
[]
(contrib/moose/modules/stochastic_tools/test/tests/surrogates/polynomial_regression/train.i)

Input Parameters

  • max_degreeMaximum polynomial degree to use for the regression.

    C++ Type:unsigned int

    Unit:(no unit assumed)

    Controllable:No

    Description:Maximum polynomial degree to use for the regression.

  • regression_typeThe type of regression to perform.

    C++ Type:MooseEnum

    Unit:(no unit assumed)

    Options:ols, ridge

    Controllable:No

    Description:The type of regression to perform.

  • responseReporter value of response results, can be vpp with / or sampler column with 'sampler/col_'.

    C++ Type:ReporterName

    Unit:(no unit assumed)

    Controllable:No

    Description:Reporter value of response results, can be vpp with / or sampler column with 'sampler/col_'.

  • samplerSampler used to create predictor and response data.

    C++ Type:SamplerName

    Unit:(no unit assumed)

    Controllable:No

    Description:Sampler used to create predictor and response data.

Required Parameters

  • converged_reporterReporter value used to determine if a sample's multiapp solve converged.

    C++ Type:ReporterName

    Unit:(no unit assumed)

    Controllable:No

    Description:Reporter value used to determine if a sample's multiapp solve converged.

  • cv_n_trials1Number of repeated trials of cross-validation to perform.

    Default:1

    C++ Type:unsigned int

    Unit:(no unit assumed)

    Controllable:No

    Description:Number of repeated trials of cross-validation to perform.

  • cv_seed4294967295Seed used to initialize random number generator for data splitting during cross validation.

    Default:4294967295

    C++ Type:unsigned int

    Unit:(no unit assumed)

    Controllable:No

    Description:Seed used to initialize random number generator for data splitting during cross validation.

  • cv_splits10Number of splits (k) to use in k-fold cross-validation.

    Default:10

    C++ Type:unsigned int

    Unit:(no unit assumed)

    Controllable:No

    Description:Number of splits (k) to use in k-fold cross-validation.

  • cv_surrogateName of Surrogate object used for model cross-validation.

    C++ Type:UserObjectName

    Unit:(no unit assumed)

    Controllable:No

    Description:Name of Surrogate object used for model cross-validation.

  • cv_typenoneCross-validation method to use for dataset. Options are 'none' or 'k_fold'.

    Default:none

    C++ Type:MooseEnum

    Unit:(no unit assumed)

    Options:none, k_fold

    Controllable:No

    Description:Cross-validation method to use for dataset. Options are 'none' or 'k_fold'.

  • execute_onTIMESTEP_ENDThe list of flag(s) indicating when this object should be executed. For a description of each flag, see https://mooseframework.inl.gov/source/interfaces/SetupInterface.html.

    Default:TIMESTEP_END

    C++ Type:ExecFlagEnum

    Unit:(no unit assumed)

    Options:NONE, INITIAL, LINEAR, NONLINEAR_CONVERGENCE, NONLINEAR, POSTCHECK, TIMESTEP_END, TIMESTEP_BEGIN, MULTIAPP_FIXED_POINT_END, MULTIAPP_FIXED_POINT_BEGIN, FINAL, CUSTOM

    Controllable:No

    Description:The list of flag(s) indicating when this object should be executed. For a description of each flag, see https://mooseframework.inl.gov/source/interfaces/SetupInterface.html.

  • filenameThe name of the file which will be associated with the saved/loaded data.

    C++ Type:FileName

    Unit:(no unit assumed)

    Controllable:No

    Description:The name of the file which will be associated with the saved/loaded data.

  • penalty0Penalty for Ridge regularization.

    Default:0

    C++ Type:double

    Unit:(no unit assumed)

    Controllable:No

    Description:Penalty for Ridge regularization.

  • predictor_colsSampler columns used as the independent random variables, If 'predictors' and 'predictor_cols' are both empty, all sampler columns are used.

    C++ Type:std::vector<unsigned int>

    Unit:(no unit assumed)

    Controllable:No

    Description:Sampler columns used as the independent random variables, If 'predictors' and 'predictor_cols' are both empty, all sampler columns are used.

  • predictorsReporter values used as the independent random variables, If 'predictors' and 'predictor_cols' are both empty, all sampler columns are used.

    C++ Type:std::vector<ReporterName>

    Unit:(no unit assumed)

    Controllable:No

    Description:Reporter values used as the independent random variables, If 'predictors' and 'predictor_cols' are both empty, all sampler columns are used.

  • prop_getter_suffixAn optional suffix parameter that can be appended to any attempt to retrieve/get material properties. The suffix will be prepended with a '_' character.

    C++ Type:MaterialPropertyName

    Unit:(no unit assumed)

    Controllable:No

    Description:An optional suffix parameter that can be appended to any attempt to retrieve/get material properties. The suffix will be prepended with a '_' character.

  • response_typerealResponse data type.

    Default:real

    C++ Type:MooseEnum

    Unit:(no unit assumed)

    Options:real, vector_real

    Controllable:No

    Description:Response data type.

  • skip_unconverged_samplesFalseTrue to skip samples where the multiapp did not converge, 'stochastic_reporter' is required to do this.

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:True to skip samples where the multiapp did not converge, 'stochastic_reporter' is required to do this.

  • use_interpolated_stateFalseFor the old and older state use projected material properties interpolated at the quadrature points. To set up projection use the ProjectedStatefulMaterialStorageAction.

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:For the old and older state use projected material properties interpolated at the quadrature points. To set up projection use the ProjectedStatefulMaterialStorageAction.

Optional Parameters

  • allow_duplicate_execution_on_initialFalseIn the case where this UserObject is depended upon by an initial condition, allow it to be executed twice during the initial setup (once before the IC and again after mesh adaptivity (if applicable).

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:In the case where this UserObject is depended upon by an initial condition, allow it to be executed twice during the initial setup (once before the IC and again after mesh adaptivity (if applicable).

  • control_tagsAdds user-defined labels for accessing object parameters via control logic.

    C++ Type:std::vector<std::string>

    Unit:(no unit assumed)

    Controllable:No

    Description:Adds user-defined labels for accessing object parameters via control logic.

  • enableTrueSet the enabled status of the MooseObject.

    Default:True

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:Yes

    Description:Set the enabled status of the MooseObject.

  • execution_order_group0Execution order groups are executed in increasing order (e.g., the lowest number is executed first). Note that negative group numbers may be used to execute groups before the default (0) group. Please refer to the user object documentation for ordering of user object execution within a group.

    Default:0

    C++ Type:int

    Unit:(no unit assumed)

    Controllable:No

    Description:Execution order groups are executed in increasing order (e.g., the lowest number is executed first). Note that negative group numbers may be used to execute groups before the default (0) group. Please refer to the user object documentation for ordering of user object execution within a group.

  • force_postauxFalseForces the UserObject to be executed in POSTAUX

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:Forces the UserObject to be executed in POSTAUX

  • force_preauxFalseForces the UserObject to be executed in PREAUX

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:Forces the UserObject to be executed in PREAUX

  • force_preicFalseForces the UserObject to be executed in PREIC during initial setup

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:Forces the UserObject to be executed in PREIC during initial setup

  • use_displaced_meshFalseWhether or not this object should use the displaced mesh for computation. Note that in the case this is true but no displacements are provided in the Mesh block the undisplaced mesh will still be used.

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:Whether or not this object should use the displaced mesh for computation. Note that in the case this is true but no displacements are provided in the Mesh block the undisplaced mesh will still be used.

Advanced Parameters