AddScalarTransferAction

Overview

The AddScalarTransferAction is responsible for performing data transfers between NekRS and MOOSE of scalar values (single numbers). This is done with the [ScalarTransfers] block in a Cardinal input file. Examples of field transfers which can be added include:

NekPostprocessorValue, to exchange a postprocessor value
NekScalarValue, to exchange a scalar value (a number)

Usrwrk Array

When direction = to_nek, data is "sent" into NekRS by writing into the nrs->usrwrk scratch space array and also copying from this array into its corresponding array on the host (nrs->o_usrwrk) so that the data is accessible both on host and device within NekRS. You must indicate which slot in the array to write the scalar value using usrwrk_slot. For memory efficiency, the same slot can be used for many different ScalarTransfers, since each transfer only needs a single entry (one double number) in the array. An offset in the array is automatically computed so that different ScalarTransfers don't overwrite one another. Note that Cardinal automatically allocates the nrs->usrwrk array, but you can control the size of it by setting n_usrwrk_slots in NekRSProblem.

warning

Allocation of nrs->usrwrk and nrs->o_usrwrk is done automatically. If you attempt to run a NekRS input file that accesses bc->usrwrk in the .oudf file without a Cardinal executable (e.g. like nrsmpi case 4), then that scratch space will have to be manually allocated in the .udf file, or else your input will seg fault. This will not be typically encountered by most users, but if you really do want to run the NekRS input files intended for a Cardinal case with the NekRS executable (perhaps for debugging), we recommend simply replacing bc->usrwrk by a dummy value, such as bc->flux = 0.0 for the boundary heat flux use case. This just replaces a value that normally comes from MOOSE by a fixed value. All other aspects of the NekRS case files should not require modification.

Example Input File Syntax

As an example, two NekScalarValues are added in the [Problem/ScalarTransfers] block. These objects will pass the numbers 2.5 and 3.5 into NekRS. Both write into the second slot in the scratch space, but due to an automatic offset they will not overwrite each other.

[Problem<<<{"href": "../../syntax/Problem/index.html"}>>>]
  type = NekRSProblem
  casename<<<{"description": "Case name for the NekRS input files; this is <case> in <case>.par, <case>.udf, <case>.oudf, and <case>.re2."}>>> = 'brick'
  n_usrwrk_slots<<<{"description": "Number of slots to allocate in nrs->usrwrk to hold fields either related to coupling (which will be populated by Cardinal), or other custom usages, such as a distance-to-wall calculation"}>>> = 3

  [ScalarTransfers<<<{"href": "../../syntax/Problem/ScalarTransfers/index.html"}>>>]
    [s2]
      type = NekScalarValue<<<{"description": "Transfers a scalar value into NekRS"}>>>
      direction<<<{"description": "Direction in which to send data"}>>> = to_nek
      value<<<{"description": "Scalar value to pass into NekRS"}>>> = 2.5
      usrwrk_slot<<<{"description": "When 'direction = to_nek', the slot in the usrwrk array to write the incoming data"}>>> = 1
      output_postprocessor<<<{"description": "Name of the postprocessor to output the value sent into NekRS"}>>> = s2
    []
    [s3]
      type = NekScalarValue<<<{"description": "Transfers a scalar value into NekRS"}>>>
      direction<<<{"description": "Direction in which to send data"}>>> = to_nek
      value<<<{"description": "Scalar value to pass into NekRS"}>>> = 3.5
      usrwrk_slot<<<{"description": "When 'direction = to_nek', the slot in the usrwrk array to write the incoming data"}>>> = 1
      output_postprocessor<<<{"description": "Name of the postprocessor to output the value sent into NekRS"}>>> = s3
    []
  []

  [FieldTransfers<<<{"href": "../../syntax/Problem/FieldTransfers/index.html"}>>>]
    [scalar01]
      type = NekFieldVariable<<<{"description": "Reads/writes volumetric field data between NekRS and MOOSE."}>>>
      direction<<<{"description": "Direction in which to send data"}>>> = from_nek
    []
    [scalar02]
      type = NekFieldVariable<<<{"description": "Reads/writes volumetric field data between NekRS and MOOSE."}>>>
      direction<<<{"description": "Direction in which to send data"}>>> = from_nek
    []
    [scalar03]
      type = NekFieldVariable<<<{"description": "Reads/writes volumetric field data between NekRS and MOOSE."}>>>
      direction<<<{"description": "Direction in which to send data"}>>> = from_nek
    []
  []
[]

When running this case, Cardinal will print out a table showing a summary of what data is occupying the various slots in the usrwrk arrays and how they can be accessed. This table for the above example is listed below, which shows how the two numbers can be accessed.


-----------------------------------------------------------------------------------------------------------
| Slot | MOOSE quantity |          How to Access (.oudf)          |         How to Access (.udf)          |
-----------------------------------------------------------------------------------------------------------
|    0 | unused         | bc->usrwrk[0 * bc->fieldOffset+bc->idM] | nrs->usrwrk[0 * nrs->fieldOffset + n] |
|    1 | s2             | bc->usrwrk[1 * bc->fieldOffset + 0]     | nrs->usrwrk[1 * nrs->fieldOffset + 0] |
|    1 | s3             | bc->usrwrk[1 * bc->fieldOffset + 1]     | nrs->usrwrk[1 * nrs->fieldOffset + 1] |
|    2 | unused         | bc->usrwrk[2 * bc->fieldOffset+bc->idM] | nrs->usrwrk[2 * nrs->fieldOffset + n] |
-----------------------------------------------------------------------------------------------------------

For instance, this passed data is then used in the brick.udf file to set the values of various passive scalars in NekRS.

void UDF_ExecuteStep(nrs_t *nrs, double time, int tstep)

Available ScalarTransfer Objects

Cardinal App
NekPostprocessorValueTransfers a postprocessor value into NekRS
NekScalarValueTransfers a scalar value into NekRS

Input Parameters

active__all__ If specified only the blocks named will be visited and made active
Default:__all__
C++ Type:std::vector<std::string>
Controllable:No
Description:If specified only the blocks named will be visited and made active
inactiveIf specified blocks matching these identifiers will be skipped.
C++ Type:std::vector<std::string>
Controllable:No
Description:If specified blocks matching these identifiers will be skipped.

Optional Parameters

control_tagsAdds user-defined labels for accessing object parameters via control logic.
C++ Type:std::vector<std::string>
Controllable:No
Description:Adds user-defined labels for accessing object parameters via control logic.

Advanced Parameters

(test/tests/nek_stochastic/shift/nek.i)

[Mesh]
  type = NekRSMesh
  volume = true
[]

[Problem]
  type = NekRSProblem
  casename = 'brick'
  n_usrwrk_slots = 3

  [ScalarTransfers]
    [s2]
      type = NekScalarValue
      direction = to_nek
      value = 2.5
      usrwrk_slot = 1
      output_postprocessor = s2
    []
    [s3]
      type = NekScalarValue
      direction = to_nek
      value = 3.5
      usrwrk_slot = 1
      output_postprocessor = s3
    []
  []

  [FieldTransfers]
    [scalar01]
      type = NekFieldVariable
      direction = from_nek
    []
    [scalar02]
      type = NekFieldVariable
      direction = from_nek
    []
    [scalar03]
      type = NekFieldVariable
      direction = from_nek
    []
  []
[]

[Executioner]
  type = Transient

  [TimeStepper]
    type = NekTimeStepper
  []
[]

[Postprocessors]
  [s2]
    type = Receiver
  []
  [s3]
    type = Receiver
  []

  [max_scalar1]
    type = NodalExtremeValue
    variable = scalar01
    value_type = max
  []
  [min_scalar1]
    type = NodalExtremeValue
    variable = scalar01
    value_type = min
  []
  [max_scalar2]
    type = NodalExtremeValue
    variable = scalar02
    value_type = max
  []
  [min_scalar2]
    type = NodalExtremeValue
    variable = scalar02
    value_type = min
  []
  [max_scalar3]
    type = NodalExtremeValue
    variable = scalar03
    value_type = max
  []
  [min_scalar3]
    type = NodalExtremeValue
    variable = scalar03
    value_type = min
  []
[]

[Outputs]
  csv = true
[]

(test/tests/nek_stochastic/shift/brick.udf)

#include "udf.hpp"

static occa::kernel cFillKernel;
static double * slice;

void UDF_LoadKernels(occa::properties & kernelInfo)
{
  cFillKernel = oudfBuildKernel(kernelInfo, "cFillKernel");
}

void UDF_Setup(nrs_t *nrs)
{
  slice = (double *) calloc(nrs->fieldOffset, sizeof(double));
  for (int i = 0; i < nrs->fieldOffset; ++i)
    slice[i] = 1.5;
}

void UDF_ExecuteStep(nrs_t *nrs, double time, int tstep)
{
  mesh_t *mesh = nrs->meshV;
  cds_t *cds = nrs->cds;

  // note: when running with Cardinal, Cardinal will allocate the usrwrk
  // array. If running with NekRS standalone (e.g. nrsmpi), you need to
  // replace the usrwrk with some other value or allocate it youself from
  // the udf and populate it with values.

  // set scalar1 value, which should not be affected by the incoming data from MOOSE
  if (tstep == 0)
    nrs->o_usrwrk.copyFrom(slice, nrs->fieldOffset * sizeof(dfloat));

  // set scalar2 and scalar3 values
  cFillKernel(mesh->Nelements, 2 * nrs->fieldOffset, nrs->usrwrk[1 * nrs->fieldOffset + 0], nrs->cds->o_S);
  cFillKernel(mesh->Nelements, 3 * nrs->fieldOffset, nrs->usrwrk[1 * nrs->fieldOffset + 1], nrs->cds->o_S);
}

Overview
Usrwrk Array
Example Input File Syntax
Available ScalarTransfer Objects
Input Parameters