====== Brief guide for WRF usage ======

The WRF V4.4 model has been recently installed and is available for all cluster users. 

===== Running WRF =====

For utilizing WRF, it is recommended to follow the instructions provided in the [[https://www2.mmm.ucar.edu/wrf/OnLineTutorial/Introduction/start.php|UCAR tutorial]]. Please remember to
use the ''tcsh'' shell and set the appropriate environment variables:

  setenv DIR /home/software/atmos/WRFv4/Build_WRF/LIBRARIES
  setenv CC gcc
  setenv CXX g++
  setenv FC gfortran
  setenv FCFLAGS -m64
  setenv F77 gfortran
  setenv FFLAGS -m64
  setenv JASPERLIB $DIR/grib2/lib
  setenv JASPERINC $DIR/grib2/include
  setenv LDFLAGS -L$DIR/grib2/lib
  setenv CPPFLAGS -I$DIR/grib2/include
  setenv PATH $DIR/netcdf/bin:$PATH
  setenv NETCDF $DIR/netcdf
  setenv LD_LIBRARY_PATH /home/software/atmos/WRFv4/Build_WRF/LIBRARIES/grib2/lib
  setenv LD_LIBRARY_PATH $DIR/grib2/include:$LD_LIBRARY_PATH

The OPH cluster is configured with OpenMPI. Therefore, before running WRF, please load the
relative module by using the command ''module load mpi/openmpi/4.1.4'' (or include this
command in your job file).


===== Installation details =====

The installation includes the following libraries:   

  *  jasper-1.900.1 
  *  libpng-1.2.50 
  *  zlib-1.2.11
  *  netcdf-4.1.3
  *  openmpi-4.1.4

The WRF configuration chosen is ''dmpar GNU'' (gfortran/gcc) with basic nesting, while WPS has been configured for Linux x86_64, ''gfortran'' (dmpar).
The complete installation, including the necessary libraries, is located in the folder:

  /home/software/atmos/WRFv4 

<WRAP center round alert 60%>
It is recommended not to modify this folder.
</WRAP>


Users that need to customise or modify the source code are invited to install WRF locally on their own account. To do that they should copy the ''WRFv4'' folder to their home directory (it requires around 1.8 GB) and can then independently run WRF by selecting desired characteristics. Only the folder

  /home/temp/atmos/WRF/WRF_software/Build_WRF/WPS_GEOG/ 
  
needs to be shared among users, as it contains the static data (29 GB) required for running ''geogrid.exe''.


===== Instruction for compiling WRF in the cluster =====

Assuming that the Netcdf library, compilers and OpenMP are already been installed (that is the current case in the cluster), you may need to compile WRF to update the model. To compile WRF use a [[oph:cluster:jobs|interactive job]]. In particular, use the command:

  salloc -N 1 --cpus-per-task=4 --time=00:30:00 --mem=32G --constraint=blade --partition=b1

salloc will pause while waiting for the requested resources, so be prepared. It also tells you the value for $JOBID to be used in the following step. Then you can connect your terminal to the running job via:

   srun --pty --overlap --jobid $JOBID bash

Then copy and paste:

  tcsh
  setenv DIR /home/software/atmos/WRFv4/Build_WRF/LIBRARIES
  setenv CC gcc
  setenv CXX g++
  setenv FC gfortran
  setenv FCFLAGS -m64
  setenv F77 gfortran
  setenv FFLAGS -m64
  setenv JASPERLIB $DIR/grib2/lib
  setenv JASPERINC $DIR/grib2/include
  setenv LDFLAGS -L$DIR/grib2/lib
  setenv CPPFLAGS -I$DIR/grib2/include
  setenv PATH $DIR/netcdf/bin:$PATH
  setenv NETCDF $DIR/netcdf
  setenv LD_LIBRARY_PATH /home/software/atmos/WRFv4/Build_WRF/LIBRARIES/grib2/lib
  setenv LD_LIBRARY_PATH $DIR/grib2/include:$LD_LIBRARY_PATH

  module load mpi/openmpi/4.1.4

Inside the WRF folder, type:

  ./configure

Select as configuration 34 (GNU compiler in parallel mode) with basic nesting (1). For WRF version 4.5.1 (at least) the file configure.wrf must be modified. At line 138:

  FC              =       time $(DM_FC)

must be replaced with:
  
  FC              =       $(DM_FC)

Then you can compile (e.g., em_real) with:

   ./compile em_real >& compile.log

Remember to call at the end:

   scancel $JOBID

To build WPS follow the instructions in the [[https://www2.mmm.ucar.edu/wrf/OnLineTutorial/compilation_tutorial.php|NCAR tutorial]].
===== Contacts =====

The WRF software has been installed by the research group of [[https://physics-astronomy.unibo.it/en/research/research-areas/atmospheric-physics|Atmospheric Physics]]. Please contact the following person for any communications:
  * [[https://www.unibo.it/sitoweb/carlo.cintolesi/en|Carlo Cintolesi]]
  * [[https://www.unibo.it/sitoweb/marco.possega2/en|Marco Possega]]