How to Compile Kratos on the Acuario Cluster
The following tutorial is intended to facilitate the compilation of the Kratos on CIMNE's cluser "Acuario".
In the following we will assume that the new_user is a member of the "kratos" group defined on the server, which will provide him with the access to all of the libraries which are needed for a standard compilation.
Contents |
Customarization needed for the user account
Edit the file ".bashrc" and add at the end of it the following code:
#numpy includes
export BASE_PATH=/home/kratos_common
export PYTHONPATH=${BASE_PATH}/numpy-1.2.1/lib64/python2.4/site-packages:$PYTHONPATH
#BOOST LIBRARY
export LD_LIBRARY_PATH=${BASE_PATH}/boost_1_37_0/lib:$LD_LIBRARY_PATH
export PYTHONPATH=${BASE_PATH}/boost_1_37_0/lib:$PYTHONPATH
#swig path
export PATH=${BASE_PATH}/swig-1.3.36/bin:$PATH
#path to the bjam executable
export PATH=/${BASE_PATH}/Files/boost_1_37_0/tools/jam/src/bin.linuxx86_64:$PATH
export BOOST_BUILD_PATH=${BASE_PATH}/Files/boost_1_37_0/tools/build/v2/
#pytrilinos
export LD_LIBRARY_PATH=${BASE_PATH}/trilinos-9.0.0/lib/:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/intel/itac/7.1/bin/rtlib/:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH
export PATH=/usr/lib64:$PATH
source /opt/intel/Compiler/11.1/064/bin/iccvars.sh intel64
Getting Kratos from repository
You can get the last version of kratos from repository using following command:
svn checkout http://kratos.cimne.upc.es/svn/kratos/kratos
|
|
Warning. Please, note that:
You have to pay attention to the DOT at the end of the path you have to copy!!!!
Jamroot file to be used in the compilation
A standard configuration, which compiles all of the applications in the Kratos can be found in
/home/kratos_common/Jamroot
You should copy this to your local kratos directory and use it for compilation. Changing to the Kratos directory in your account and type the following command:
cp /home/kratos_common/Jamroot .
Compilation Command
The compiler that comes with the system is a rather old version of the gcc. Please compile the Kratos using the intel compiler using the compilation command:
bjam threading=multi intel -j4
Running Examples
Scalar or OpenMP parallel applications can be run "on-line" using the standard compilation commands. For longer jobs they have to be submitted to a queue. Full details on how to do this are given on the Cluster's wiki
The following example shows how to run (again in online mode) the MPI example "cantilever2d", which is contained in the directory
kratos/applications/trilinos_application/test_examples/cantilever2d.gid
running 4 processes, distributed between 2 of the cluster nodes (pez001 and pez002) can be obtained as follows:
mpirun --np 4 --host pez001,pez002 python cantilever_aztec.py
Cluster's Documentation
A temporary wiki for the Cluster can be found on the web at the addess
acuario.rmee.upc.es
updated information will be uploaded when available
Trilinos Compilation
The trilinos has been compiled and installed in
/home/kratos_common/trilinos-9.0.0
in principle the users should NOT recompile it. In any case the script used in configuring it is:
../configure \ --enable-mpi \ --enable-shared \ --with-mpi-compilers=/opt/openmpi/1.2.4/bin \ --with-incdirs="-I/opt/openmpi/1.2.4/include -I/home/kratos_common/SuperLU_DIST_2.3/SRC" \ --with-ldflags="-L/home/kratos_common/SuperLU_DIST_2.3/lib -L/home/kratos_common/ParMetis-3.1" \ --with-libs="-lsuperlu_dist_2.3 -lmetis -lparmetis" \ --prefix=/home/kratos_common/trilinos-9.0.0 \ FFLAGS="-O5 -funroll-all-loops -fPIC" \ CFLAGS="-O3 -fPIC -funroll-loops" \ CXXFLAGS="-O3 -fPIC -funroll-loops -ffast-math" \ --enable-amesos --enable-anasazi --enable-aztecoo \ --enable-epetra --enable-epetraext --enable-ifpack --enable-ml --enable-pytrilinos \ --enable-teuchos --enable-triutils --enable-galeri \ --enable-aztecoo-teuchos --enable-amesos-superludist
