NekRSStandaloneProblem

This class performs all activities related to solving NekRS as a MOOSE application. This class differs from the NekRSProblem class because this class does not couple NekRS to another MOOSE application. Instead, this class simply allows standalone NekRS simulations to be run using the Cardinal executable, which allows uncoupled NekRS simulations to take advantage of a number of features available in Cardinal, such as:

  • Postprocessors to query the solution, evaluate heat balances and pressure drops, or evaluate solution convergence

  • UserObjects to project the NekRS solution onto other discretization schemes, such as a subchannel discretization, or onto other MOOSE applications, such as for providing closures

  • Automatic conversion of nondimensional NekRS solutions into dimensional form

In other words, this class allows NekRS input files to be run through Cardinal without any modifications.

This class must be used in conjunction with two other classes in Cardinal:

  1. NekRSMesh, which builds a mirror of the NekRS mesh in a MOOSE format so that all the usual Transfers understand how to send data into/out of NekRS. The settings on NekRSMesh also determine which coupling type (listed above) is used.

  2. NekTimeStepper, which allows NekRS to control its own time stepping.

Therefore, we recommend first reading the documentation for the above classes before proceeding here.

The smallest possible MOOSE-wrapped input file that can be used to run NekRS standalone cases is shown below. This input file in particular runs the channel example that ships with the NekRS repository.

In this input file, casename is the prefix describing the NekRS input files, i.e. this parameter would be casename = channel if the NekRS input files are channel.re2, channel.par, channel.udf, and channel.oudf.

Listing 1: Smallest possible input file for running a standalone NekRS simulation within Cardinal

[Mesh]
  type = NekRSMesh
  volume = true
[]

[Problem]
  type = NekRSStandaloneProblem
  casename = 'channel'
  output = 'pressure velocity'
[]

[Executioner]
  type = Transient

  [TimeStepper]
    type = NekTimeStepper
  []
[]
(test/tests/nek_standalone/channel/nek.i)

In other words, you can either run the channel NekRS example with the standalone NekRS executable (which does not bring any advantages of Cardinal),

nrsmpi channel 8

or through the Cardinal executable (which brings many useful postprocessing features),

mpiexec -np 8 cardinal-opt -i nek.i

Both will run the NekRS case in exactly the same fashion using exactly the same NekRS input files. All that is needed to wrap NekRS in MOOSE is a thin input file like that shown in Listing 1. The remainder of this page highlights a few potential use cases for running standalone NekRS cases through Cardinal.

Outputting the NekRS Solution

This class enables extracting a number of fields from the NekRS solution onto the NekRSMesh mesh mirror in addition to any fields that are already interpolated onto the mesh mirror for purposes of multiphysics coupling to another MOOSE application. This feature can be used for:

  • Quick visualization of the NekRS solution without needing to rely on NekRS's custom output format

  • Postprocessing the NekRS solution using MOOSE's systems

  • Providing one-way coupling to other MOOSE applications, such as for transporting scalars based on NekRS's velocity solution or for projecting NekRS turbulent viscosity closure terms onto another MOOSE application's mesh

This output feature is used by specifying the fields to be output with the output parameter. Available output fields include:

  • pressure (which creates a MOOSE variable named P)

  • velocity (which creates MOOSE variables named vel_x, vel_y, and vel_z)

  • temperature (which creates a MOOSE variable named temp)

  • scalar01 (which creates a MOOSE variable named scalar01)

  • scalar02 (which creates a MOOSE variable named scalar02)

  • scalar03 (which creates a MOOSE variable named scalar03)

For NekRS simulations that are coupled to MOOSE, the temperature will already be output because it is used as part of the physics transfers.

When outputting data, the NekRS solution fields are interpolated from the GLL points onto the NekRSMesh, which will be either a first or second order representation of the NekRS mesh. This interpolation is performed using Vandermonde matrices. Therefore, the output solution is generally not an exact representation of NekRS's solution, for which the NekRS field files are still required to visualize fully. Because the MOOSE framework supports many different output formats, this is a convenient manner by which to obtain a representation of the NekRS solution in Exodus, VTK, CSV, and other formats.

If volume = true is specified on the NekRSMesh, the output solutions are represented over a volume mesh mirror. Otherwise, if volume = false, the solution is shown only on the boundaries specified with the boundary parameter.

For example, if you would like to extract the NekRS solution on an Exodus mesh, simply specify the output parameter with the desired fields. The following input sections will interpolate the NekRS pressure, velocity, and temperature for the lowMach NekRS example onto a second order representation of the NekRS mesh.

[Mesh]
  type = NekRSMesh
  volume = true
  order = SECOND
[]

[Problem]
  type = NekRSStandaloneProblem
  casename = 'lowMach'
  output = 'pressure velocity temperature'

  # We omit the non-dimensional settings here in order to just extract the
  # non-dimensional solution as-is, without dimensionalizing it.
[]
(test/tests/nek_standalone/lowMach/nek.i)

Then, the output format specified in the [Output] block will be used. Here, the NekRS solution is projected onto an Exodus mesh.

[Outputs]
  exodus = true
[]

For instance, Figure 1 shows the pressure from the NekRS field files (left) and interpolated onto a second-order mesh mirror (right). Because this particular example runs NekRS in a higher order than can be represented on a second-order mesh mirror, the interpolated pressure is clearly not an exact representation of the NekRS solution - only an interpolated version of the NekRS solution.

Figure 1: Pressure from the NekRS field files (left) and after interpolation onto a second order mesh mirror (right).

Postprocessing the NekRS Solution

An advantage of running NekRS cases through Cardinal is the ability to leverage a wide set of MOOSE postprocessors to both monitor the solution as the solve progresses and to easily perform common postprocessing steps. The example below runs the ktauChannel NekRS example through Cardinal. The max_T postprocessor computes the maximum temperature by directly querying the NekRS solution; many other available postprocessors of this type are available in Cardinal and described on the source documentation page.

[Mesh]
  type = NekRSMesh
  volume = true
[]

[Problem]
  type = NekRSStandaloneProblem
  casename = 'channel'
  output = 'pressure velocity temperature'
[]

[Executioner]
  type = Transient

  [TimeStepper]
    type = NekTimeStepper
  []
[]

[Outputs]
  exodus = true
[]

[Postprocessors]
  [max_T]
    type = NekVolumeExtremeValue
    field = temperature
  []
  [T_at_point]
    type = PointValue
    variable = temp
    point = '4.0 -0.5 0.0'
  []
  [pct_change]
    type = PercentChangePostprocessor
    postprocessor = max_T
  []
[]
(test/tests/nek_standalone/ktauChannel/nek.i)

The max_T postprocessor is a special, Nek-specific postprocessor available in Cardinal that directly queries the underlying data structures in NekRS. However, when specifying fields to output from the NekRS solution with the output parameter (described in more detail in Outputting the NekRS Solution ), any of the much wider set of MOOSE postprocessors can also be used. For instance, a PointValue postprocessor is used to extract the value of temperature at a specific point in space, and a PercentChangePostprocessor is used to evaluate the percent change in the maximum temperature between two successive time steps such as for assessing convergence. A few examples of other postprocessors that may be of use to NekRS simulations include:

  • ElementL2Error, which computes the L2^2 norm of a variable relative to a provided function, useful for solution verification

  • FindValueOnLine, which finds the point at which a specified value of a variable occurs, which might be used for evaluating a boundary layer thickness

  • LinearCombinationPostprocessor, which can be used to combine postprocessors together in a general expression a0p0+a1p1+⋯+ba_0p_0+a_1p_1+\cdots+b, where aia_i are coefficients, pip_i are postprocessors, and bb is a constant additive factor. This can be used to compute the temperature rise in a domain by subtracting a postprocessor that computes the inlet temperature from a postprocessor that computes the outlet temperature.

  • TimeExtremeValue, which provides the maximum/minimum value of a variable over the course of an entire simulation, such as for evaluating peak stress in an oscillating system

Please consult the MOOSE documentation for a full list of available postprocessors.

In addition to the postprocessor system, a wide set of userobjects and auxiliary kernels may be applied to the NekRS solution once it is output to the MOOSE mesh. For instance, a LayeredAverage userobject can be used to compute an average of a MOOSE variable in a number of layers. Here, this userobject is applied to average the pressure in four layers in the xx direction (the direction parallel to the wall). The results of this userobject are then visualized with a SpatialUserObjectAux to place the results in the layered_p auxiliary variable.

[UserObjects]
  [layered_average]
    type = LayeredAverage
    direction = x
    variable = P
    num_layers = 4
  []
[]

[AuxVariables]
  [layered_p]
    family = MONOMIAL
    order = CONSTANT
  []
[]

[AuxKernels]
  [layered_p]
    type = SpatialUserObjectAux
    variable = layered_p
    user_object = layered_average
  []
[]
(test/tests/nek_standalone/ktauChannel/nek.i)

These two examples of postprocessor and userobject usage only scratch the surface of the capabilities of the MOOSE framework - please consult the MOOSE documentation to learn about further capabilities.

Outputting the Scratch Array

This class (and all other NekRS wrappings in Cardinal) allows you to write slots in the nrs->usrwrk scratch space array to NekRS field files. This can be useful for viewing the data sent from MOOSE to NekRS (for problem classes that involve multiphysics), as well as to visualize custom user usage of nrs->usrwrk, such as for fetching a wall distance computation from the Nek5000 backend. To write the scratch space to a field file, set usrwrk_output to an array with each "slot" in the nrs->usrwrk array that you want to write. Then, specify a filename prefix to use to name each field file.

In the example below, the first two "slots" in the nrs->usrwrk array will be written to field files on the same interval that NekRS writes its usual field files. These files will be named aaabrick0.f00001, etc. and cccbrick0.f00001, etc. Based on limitations in how NekRS writes its files, the fields written to these files will all be named temperature when visualized.

[Problem]
  type = NekRSStandaloneProblem
  casename = 'brick'
  usrwrk_output = '0 1'
  usrwrk_output_prefix = 'aaa ccc'
[]
(test/tests/nek_file_output/usrwrk/nek.i)

Input Parameters

  • casenameCase name for the NekRS input files; this is in .par, .udf, .oudf, and .re2.

    C++ Type:std::string

    Unit:(no unit assumed)

    Controllable:No

    Description:Case name for the NekRS input files; this is in .par, .udf, .oudf, and .re2.

Required Parameters

  • constant_interval1Constant interval (in units of number of time steps) with which to synchronize the NekRS solution

    Default:1

    C++ Type:unsigned int

    Unit:(no unit assumed)

    Controllable:No

    Description:Constant interval (in units of number of time steps) with which to synchronize the NekRS solution

  • disable_fld_file_outputFalseWhether to turn off all NekRS field file output writing (for the usual field file output - this does not affect writing the usrwrk with 'usrwrk_output')

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:Whether to turn off all NekRS field file output writing (for the usual field file output - this does not affect writing the usrwrk with 'usrwrk_output')

  • first_reserved_usrwrk_slot0Slice (zero-indexed) in nrs->usrwrk where Cardinal will begin reading/writing data; this can be used to shift the usrwrk slots reserved by Cardinal, so that you can use earlier slices for custom purposes

    Default:0

    C++ Type:unsigned int

    Unit:(no unit assumed)

    Controllable:No

    Description:Slice (zero-indexed) in nrs->usrwrk where Cardinal will begin reading/writing data; this can be used to shift the usrwrk slots reserved by Cardinal, so that you can use earlier slices for custom purposes

  • n_usrwrk_slots7Number of slots to allocate in nrs->usrwrk to hold fields either related to coupling (which will be populated by Cardinal), or other custom usages, such as a distance-to-wall calculation (which will be populated by the user from the case files)

    Default:7

    C++ Type:unsigned int

    Unit:(no unit assumed)

    Controllable:No

    Description:Number of slots to allocate in nrs->usrwrk to hold fields either related to coupling (which will be populated by Cardinal), or other custom usages, such as a distance-to-wall calculation (which will be populated by the user from the case files)

  • nondimensionalFalseWhether NekRS is solved in non-dimensional form

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:Whether NekRS is solved in non-dimensional form

  • outputField(s) to output from NekRS onto the mesh mirror

    C++ Type:MultiMooseEnum

    Unit:(no unit assumed)

    Options:temperature, pressure, velocity, scalar01, scalar02, scalar03

    Controllable:No

    Description:Field(s) to output from NekRS onto the mesh mirror

  • regard_general_exceptions_as_errorsFalseIf we catch an exception during residual/Jacobian evaluaton for which we don't have specific handling, immediately error instead of allowing the time step to be cut

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:If we catch an exception during residual/Jacobian evaluaton for which we don't have specific handling, immediately error instead of allowing the time step to be cut

  • skip_final_field_fileFalseBy default, we write a NekRS field file on the last time step; set this to true to disable

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:By default, we write a NekRS field file on the last time step; set this to true to disable

  • solveTrueWhether or not to actually solve the Nonlinear system. This is handy in the case that all you want to do is execute AuxKernels, Transfers, etc. without actually solving anything

    Default:True

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:Yes

    Description:Whether or not to actually solve the Nonlinear system. This is handy in the case that all you want to do is execute AuxKernels, Transfers, etc. without actually solving anything

  • synchronization_intervalconstantWhen to synchronize the NekRS solution with the mesh mirror. By default, the NekRS solution is mapped to/receives data from the mesh mirror for every time step.

    Default:constant

    C++ Type:MooseEnum

    Unit:(no unit assumed)

    Options:constant, parent_app

    Controllable:No

    Description:When to synchronize the NekRS solution with the mesh mirror. By default, the NekRS solution is mapped to/receives data from the mesh mirror for every time step.

  • usrwrk_outputUsrwrk slot(s) to output to NekRS field files; this can be used for viewing the quantities passed from MOOSE to NekRS after interpolation to the CFD mesh. Can also be used for any slots in usrwrk that are written by the user, but unused for coupling.

    C++ Type:std::vector<unsigned int>

    Unit:(no unit assumed)

    Controllable:No

    Description:Usrwrk slot(s) to output to NekRS field files; this can be used for viewing the quantities passed from MOOSE to NekRS after interpolation to the CFD mesh. Can also be used for any slots in usrwrk that are written by the user, but unused for coupling.

  • usrwrk_output_prefixString prefix to use for naming the field file(s); only the first three characters are used in the name based on limitations in NekRS

    C++ Type:std::vector<std::string>

    Unit:(no unit assumed)

    Controllable:No

    Description:String prefix to use for naming the field file(s); only the first three characters are used in the name based on limitations in NekRS

  • write_fld_filesFalseWhether to write NekRS field file output from Cardinal. If true, this will disable any output writing by NekRS itself, and instead produce output files with names a01...a99pin, b01...b99pin, etc.

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:Whether to write NekRS field file output from Cardinal. If true, this will disable any output writing by NekRS itself, and instead produce output files with names a01...a99pin, b01...b99pin, etc.

Optional Parameters

  • allow_initial_conditions_with_restartFalseTrue to allow the user to specify initial conditions when restarting. Initial conditions can override any restarted field

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:True to allow the user to specify initial conditions when restarting. Initial conditions can override any restarted field

  • restart_file_baseFile base name used for restart (e.g. / or /LATEST to grab the latest file available)

    C++ Type:FileNameNoExtension

    Unit:(no unit assumed)

    Controllable:No

    Description:File base name used for restart (e.g. / or /LATEST to grab the latest file available)

Restart Parameters

  • control_tagsAdds user-defined labels for accessing object parameters via control logic.

    C++ Type:std::vector<std::string>

    Unit:(no unit assumed)

    Controllable:No

    Description:Adds user-defined labels for accessing object parameters via control logic.

  • default_ghostingFalseWhether or not to use libMesh's default amount of algebraic and geometric ghosting

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:Whether or not to use libMesh's default amount of algebraic and geometric ghosting

  • enableTrueSet the enabled status of the MooseObject.

    Default:True

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:Set the enabled status of the MooseObject.

Advanced Parameters

  • kernel_coverage_block_listList of subdomains for kernel coverage check. The meaning of this list is controlled by the parameter 'kernel_coverage_check' (whether this is the list of subdomains to be checked, not to be checked or not taken into account).

    C++ Type:std::vector<SubdomainName>

    Unit:(no unit assumed)

    Controllable:No

    Description:List of subdomains for kernel coverage check. The meaning of this list is controlled by the parameter 'kernel_coverage_check' (whether this is the list of subdomains to be checked, not to be checked or not taken into account).

  • material_coverage_block_listList of subdomains for material coverage check. The meaning of this list is controlled by the parameter 'material_coverage_check' (whether this is the list of subdomains to be checked, not to be checked or not taken into account).

    C++ Type:std::vector<SubdomainName>

    Unit:(no unit assumed)

    Controllable:No

    Description:List of subdomains for material coverage check. The meaning of this list is controlled by the parameter 'material_coverage_check' (whether this is the list of subdomains to be checked, not to be checked or not taken into account).

  • material_coverage_checkTRUEControls, if and how a material subdomain coverage check is performed. With 'TRUE' or 'ON' all subdomains are checked (the default). Setting 'FALSE' or 'OFF' will disable the check for all subdomains. To exclude a predefined set of subdomains 'SKIP_LIST' is to be used, while the subdomains to skip are to be defined in the parameter 'material_coverage_block_list'. To limit the check to a list of subdomains, 'ONLY_LIST' is to be used (again, using the parameter 'material_coverage_block_list').

    Default:TRUE

    C++ Type:MooseEnum

    Unit:(no unit assumed)

    Options:FALSE, TRUE, OFF, ON, SKIP_LIST, ONLY_LIST

    Controllable:No

    Description:Controls, if and how a material subdomain coverage check is performed. With 'TRUE' or 'ON' all subdomains are checked (the default). Setting 'FALSE' or 'OFF' will disable the check for all subdomains. To exclude a predefined set of subdomains 'SKIP_LIST' is to be used, while the subdomains to skip are to be defined in the parameter 'material_coverage_block_list'. To limit the check to a list of subdomains, 'ONLY_LIST' is to be used (again, using the parameter 'material_coverage_block_list').

Simulation Checks Parameters

  • parallel_barrier_messagingFalseDisplays messaging from parallel barrier notifications when executing or transferring to/from Multiapps (default: false)

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:Displays messaging from parallel barrier notifications when executing or transferring to/from Multiapps (default: false)

  • verbose_multiappsFalseSet to True to enable verbose screen printing related to MultiApps

    Default:False

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:Set to True to enable verbose screen printing related to MultiApps

  • verbose_setupfalseSet to 'true' to have the problem report on any object created. Set to 'extra' to also display all parameters.

    Default:false

    C++ Type:MooseEnum

    Unit:(no unit assumed)

    Options:false, true, extra

    Controllable:No

    Description:Set to 'true' to have the problem report on any object created. Set to 'extra' to also display all parameters.

Verbosity Parameters

  • show_invalid_solution_consoleTrueSet to true to show the invalid solution occurance summary in console

    Default:True

    C++ Type:bool

    Unit:(no unit assumed)

    Controllable:No

    Description:Set to true to show the invalid solution occurance summary in console

Solution Validity Control Parameters

Input Files