Building Cardinal with MOOSE's conda Environment

warning:Incompatible with NekRS

MOOSE's conda environment is currently incompatible with Cardinal's NekRS wrapping. MOOSE's environment exports some header files that point to HYPRE, but NekRS needs to use its own special version of HYPRE. The MOOSE team is not aware of any way to exclude header files attached to MOOSE's conda compiler wrappers.

You MUST set export ENABLE_NEK=false if you want to use the conda environment. Otherwise, if you do want NekRS features, you need to instead follow the instructions for building without the conda environment. We hope to relax this restriction in the future - if this is impacting your work, please let us know so that we can increase its priority.

If you're only interested in OpenMC, then you're in the right place - please follow the instructions on this page. Note that you should NOT be building OpenMC separately, or using its conda environment (i.e., do conda deactivate openmc if you were using OpenMC's conda).

note:tldr

All that you need to compile Cardinal is:


conda activate moose
cd $HOME
git clone https://github.com/neams-th-coe/cardinal.git
cd cardinal
./scripts/get-dependencies.sh
export ENABLE_NEK=false
export HDF5_ROOT=$CONDA_PREFIX
export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH
make -j8

If the above produces a cardinal-opt Cardinal executable, you can jump straight to Running . If you want to customize the build or were not successful with the above, please consult the detailed instructions that follow.

Access

To get access to Cardinal, clone the repository and cd into the directory.


git clone https://github.com/neams-th-coe/cardinal.git
cd cardinal

To get MOOSE's conda environment, follow the instructions here.

Prerequisites

The basic prerequisites for building Cardinal are summarized in Table 1.

Table 1: Summary of prerequisites needed for Cardinal.

	Building with NekRS	Building with OpenMC	Both
CMake	$var element = document.getElementById("moose-equation-62852ad9-40c8-40b2-80eb-b56b89429b6d");katex.render("\\checkmark", element, {displayMode:false,throwOnError:false});$	$var element = document.getElementById("moose-equation-8765071c-9aa9-4251-8257-af79df089db6");katex.render("\\checkmark", element, {displayMode:false,throwOnError:false});$	$var element = document.getElementById("moose-equation-13bfc380-4b5c-4464-8f53-1dc387bf4755");katex.render("\\checkmark", element, {displayMode:false,throwOnError:false});$
GNU fortran >= 9.0 compiler	$var element = document.getElementById("moose-equation-f8745185-f8e9-4d92-816a-0a54c5d3ad0e");katex.render("\\checkmark", element, {displayMode:false,throwOnError:false});$		$var element = document.getElementById("moose-equation-f23eddb5-5d89-40b2-b7f9-1d90eebe6b01");katex.render("\\checkmark", element, {displayMode:false,throwOnError:false});$
HDF5		$var element = document.getElementById("moose-equation-02a66476-4bf6-465b-9341-cce5b7a81ec5");katex.render("\\checkmark", element, {displayMode:false,throwOnError:false});$	$var element = document.getElementById("moose-equation-3c640ca8-8a11-4157-b18e-2f671becc592");katex.render("\\checkmark", element, {displayMode:false,throwOnError:false});$
MPI	$var element = document.getElementById("moose-equation-adf940b4-4758-4c2e-a6c3-f17775b4d55b");katex.render("\\checkmark", element, {displayMode:false,throwOnError:false});$	$var element = document.getElementById("moose-equation-7fb99337-4555-4988-a046-5ff71ac9b08a");katex.render("\\checkmark", element, {displayMode:false,throwOnError:false});$	$var element = document.getElementById("moose-equation-755bc242-f873-48aa-91f0-c5847afa76b4");katex.render("\\checkmark", element, {displayMode:false,throwOnError:false});$

tip:How do I know if I have these dependencies?

You will already have all of these if using MOOSE's conda environment. But if you want to learn more, check out our prerequisite guide.

Then, decide whether you want both NekRS and OpenMC, just one, or neither. Both are enabled by default, but you can build Cardinal with only the dependencies that you want. If you do not want to build the NekRS-part of Cardinal, set the following environment variable:


export ENABLE_NEK=false

Likewise, if you do not want to build the OpenMC-part of Cardinal, set the following environment variable:


export ENABLE_OPENMC=false

We support the optional usage of DAGMC's CAD-based models in OpenMC. This capability is off by default, but to build with DAGMC support, set:


export ENABLE_DAGMC=yes

Building

Fetch Dependencies

Cardinal has MOOSE, OpenMC, and NekRS as dependencies. However, you do not need to separately build/compile any of these dependencies - Cardinal's Makefile handles all steps automatically. To fetch the MOOSE, OpenMC, and NekRS dependencies, run:


./scripts/get-dependencies.sh

note:Optional dependencies

Cardinal supports optional coupling to the following codes:

SAM, a tool for systems analysis of advanced non-light water reactors safety analysis. Follow these instructions to obtain the required dependencies for adding the SAM submodule.
Sockeye, a tool for modeling of heat pipe systems. Follow these instructions to obtain the required dependencies for adding the Sockeye submodule.
BISON, a tool for modeling fuel performance and material behavior of nuclear fuels. Follow these instructions to obtain the required dependencies for adding the BISON submodule.

Set Environment Variables

A number of environment variables are required or recommended when building/running Cardinal. Put these in your ~/.bashrc (don't forget to source ~/.bashrc!):


# [REQUIRED IF USING MOOSE CONDA ENV for HDF5] you must set the location of the
# root HDF5 directory provided by MOOSE for OpenMC to find
export HDF5_ROOT=$CONDA_PREFIX

# [REQUIRED ON SOME SYSTEMS] for some systems, libraries won't be linked properly unless
# you explicitly point this variable. We're working on a more elegant fix.
export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH

# [OPTIONAL] it's a good idea to explicitly note that you are using MPI compiler wrappers
export CC=mpicc
export CXX=mpicxx
export FC=mpif90

# [OPTIONAL] if running with OpenMC, you will need cross section data at runtime;
# you will need to set this variable to point to a 'cross_sections.xml' file.
export OPENMC_CROSS_SECTIONS=${HOME}/cross_sections/endfb-vii.1-hdf5/cross_sections.xml

tip:Additional environment variables

For even further control, you can set other optional environment variables to specify the optimization level, dependency locations, and more.

Compile Cardinal

Finally, run make in the top-level directory,


make -j8 MAKEFLAGS=-j8

which will compile Cardinal in parallel with 8 processes (the MAKEFLAGS part is optional, but will also tell CMake to build in parallel with 8 processes - otherwise, the CMake aspects of Cardinal, i.e. OpenMC, NekRS, and DAGMC, will build serially). This will create the executable cardinal-<mode> in the top-level directory. <mode> is the optimization level used to compile MOOSE set with the METHOD environment variable. If you encounter issues while compiling, check out our compile-time troubleshooting guide.

Running

The command to run Cardinal with an input file input.i, <n> MPI ranks, and <s> OpenMP threads is:


mpiexec -np <n> cardinal-opt -i input.i --n-threads=<s>

This command assumes that cardinal-opt is located on your PATH; otherwise, you need to provide the full path to cardinal-opt in the above command or add the cardinal folder to your path. Note that while MOOSE and OpenMC use hybrid parallelism with both MPI and OpenMP, NekRS does not use shared memory parallelism.

tip:Command line options

Cardinal supports all of MOOSE's command line parameters, as well as a few Cardinal-specific command line options. For a full list:


./cardinal-opt --help

Checking the Install

If you would like to check that Cardinal was built correctly and that you have all the basic requirements in place, we can walk you through a few installation checks and try running a few input files. If you run into any issues with the following commands, you can find an FAQ of common issues here.

If using OpenMC, make sure that you have cross sections downloaded. If the following returns an empty line, you need to download cross sections.


echo $OPENMC_CROSS_SECTIONS

If using OpenMC, try running a multiphysics case.


cd test/tests/neutronics/feedback/lattice
mpiexec -np 2 ../../../../../cardinal-opt -i openmc_master.i --n-threads=2

If using OpenMC, try building the OpenMC XML files using OpenMC's Python API. If you run into any issues, you most likely need to install OpenMC's Python API.


cd tutorials/lwr_solid
python make_openmc_model.py

If using NekRS, try running a conjugate heat transfer case.


cd test/tests/cht/sfr_pincell
mpiexec -np 4 ../../../../cardinal-opt -i nek_master.i

Try leveraging NekRS's tools to make a mesh. If you run into any issues, you most likely need to install the NekRS tools.


cd test/tests/conduction/boundary_and_volume/prism
exo2nek

For Developers

You can run Cardinal's regression test suite with the following:


./run_tests -j8

which will run the tests in parallel with 8 processes. OpenMC's tests require you to use this data set. Depending on the availability of various dependencies, some tests may be skipped. The first time you run the test suite, the runtime will be very long due to the just-in-time compilation of NekRS. Subsequent runs will be much faster due to the use of cached build files. If you run into issues running the test suite, please check out our run_tests troubleshooting page.

You can run the unit tests with the following:


cd unit
make -j8
./run_tests -j8

Access
Prerequisites
Building
Running
Checking the Install