CSVDiff

CSVDiff tests compare CSV output(s) for the test to a reference in the specified gold_dir folder. CSV output(s) contains time steps, scalar variables and postprocessors. CSV output(s) may also be generated by VectorPostprocessors. This tester picks up any difference between the reference file and the simulation output, at any time step included in the output. More documentation for the CSVDiff utility may be found here.

Options

Test configuration options are added to the tests file.

  • abs_zero: Sets an absolute tolerance, defaults to 1e-10. Both absolute and relative tolerances must be met for a test to pass.

  • rel_err: Sets a relative tolerance, defaults to 5.5e-6.

  • override_columns: A list of variable names to customize the CSVDiff tolerances

  • override_rel_err: A list of customized relative error tolerances.

  • override_abs_zero: A list of customized absolute zero tolerances.

  • comparison_file: Use supplied custom comparison config file.

  • ignore_columns: Exclude columns in csv file from csv diff comparison.

Other test commands & restrictions may be found in the TestHarness documentation. An example of a custom comparison file for a CSVDiff test is shown below. The relative tolerance criterion for the z_field global variable (usually a postprocessor or a scalar variable) is modified from the global value of 5.5e-6 to 10. This essentially disables comparison for that value, as it can vary by two orders of magnitude and not trigger a failure.

TIME STEPS relative 5.5e-06 floor 1e-11  # min: 0 @ t0  max: 0 @ t5

GLOBAL VARIABLES relative 5.5e-06 floor 1e-11
    z_field relative 1e+01     # min: 0.000e+00 @ t0          max: 4.025e-01 @ t5
    x_field                    # min: 0.000e+00 @ t0          max: 1.103e-01 @ t5
    y_field                    # min: 0.000e+00 @ t0          max: 2.262e-02 @ t5
    time                       # min: 0.000e+00 @ t0          max: 5.000e-01 @ t5
(contrib/moose/test/tests/test_harness/csvdiff_comparison_relative.cmp)

In this example, the global absolute tolerance for all variables is being modified from its default of 1e-11 to 10. This essentially disables comparison for all global variables, but lower values (1e-8 is common) may be useful to loosen tolerances in a test.

TIME STEPS relative 5.5e-06 floor 1e-11  # min: 0 @ t0  max: 0 @ t5

GLOBAL VARIABLES relative 5.5e-06 floor 1e+01
    z_field                    # min: 0.000e+00 @ t0          max: 4.025e-01 @ t5
    x_field                    # min: 0.000e+00 @ t0          max: 1.103e-01 @ t5
    y_field                    # min: 0.000e+00 @ t0          max: 2.262e-02 @ t5
    time                       # min: 0.000e+00 @ t0          max: 5.000e-01 @ t5
(contrib/moose/test/tests/test_harness/csvdiff_comparison_global_floor.cmp)

Example test configuration in the MOOSE test suite

In this example, three CSVDiff tests are created to test a particular type of preconditioner. The data stored in the CSV output, the L2 error between the numerical and an analytical solution, is being compared before and after the steady-state solve.

[Tests]
  design = 'VariableCondensationPreconditioner.md'
  issues = '#15215'
  [condense_amg_test]
    type = 'CSVDiff'
    input = 'vcp_test.i'
    csvdiff = 'vcp_test_out.csv'
    requirement = "The system shall converge when using the VCP interface with AMG as the "
                  "preconditioner."
    mesh_mode = 'replicated'
  []
  [condense_amg_test_dinv]
    type = 'CSVDiff'
    input = 'vcp_test.i'
    csvdiff = 'vcp_test_out.csv'
    cli_args = 'Preconditioning/vcp/is_lm_coupling_diagonal=false'
    requirement = "The system shall converge while computing the full inverse of the coupling "
                  "matrix."
    mesh_mode = 'replicated'
  []
  [no-condense_amg_test]
    type = 'CSVDiff'
    input = 'no_condense_test.i'
    csvdiff = 'no_condense_test_out.csv'
    requirement = "The system shall converge while using LU as the solver."
  []
[]
(contrib/moose/test/tests/preconditioners/vcp/tests)