# How to use the PaStiX solver in Kratos

(Difference between revisions)
 Revision as of 10:19, 3 January 2017 (view source)Andreas (Talk | contribs) (→Compile the PaStiX library (shared memory parallelism))← Older edit Revision as of 10:23, 3 January 2017 (view source)Andreas (Talk | contribs) (→Compile the PaStiX library (shared memory parallelism))Newer edit → Line 141: Line 141: #CCPASTIX  := \$(CCPASTIX) -I\$(HWLOC_INC) -DWITH_HWLOC #CCPASTIX  := \$(CCPASTIX) -I\$(HWLOC_INC) -DWITH_HWLOC #EXTRALIB  := \$(EXTRALIB) -L\$(HWLOC_LIB) -lhwloc #EXTRALIB  := \$(EXTRALIB) -L\$(HWLOC_LIB) -lhwloc + + Now you can compile the PaStiX library (we are still in the /src directory): + + make + + You can install it to the directory defined in the config.in file by: + + make install + + In order to be able to compile the Kratos with PaStiX now, you have to fix a little bug in one of the header files in the installation directory. In the header file '''pastix_int32_double_real.h''' you have to add: + + #inlcude

# How to use the PaStiX solver in Kratos

The PaStiX (Parallel Sparse matriX package) is a open-source scientific library that provides a high performance parallel solver for very large sparse linear systems based on direct methods. Numerical algorithms are implemented in single or double precision (real or complex) using LLt, LDLt and LU with static pivoting (for non symmetric matrices having a symmetric pattern). This solver provides also an adaptive blockwise iLU(k) factorization that can be used as a parallel preconditioner using approximated supernodes to build a coarser block structure of the incomplete factors ([1]).

It can be linked to the Kratos by the ExternalSolversApplication. The following steps have to be made in order to use be able to use it in Ubuntu version 14.04 LTS 64 bit or 16.04 LTS 64 bit.

## Compile the Scotch library (shared memory parallelism)

PaStiX needs the Scotch library as a prerequisite to work.

```http://gforge.inria.fr/projects/scotch/
```

Copy it to a folder of your choice. In the following we are using "~/compiled_libraries" as a reference.

Extract the files by the command:

```tar -xf scotch_6.0.4.tar.gz
```

Navigate into the source directory by:

```cd scotch_6.0.4/src
```

Copy the correct MakeFile.inc for the version of your system to the source directory. Ín the case of the Ubuntu versions mentioned above it is the Makefile.inc.x86-64_linux2:

```cp ./Make.inc/Makefile.inc.x86-64_linux2 Makefile.inc
```

Additional libraries are necessary, which can be installed by:

```sudo apt-get install flex bison zlib1g-dev
```

In order to install Scotch locally the install path has to be set. Since the "prefix" command is not working this has to be done manually in the Makefile, which is also located in the /src folder. You can see an example for the part of the file to be modified below:

```...some code...
include Makefile.inc
prefix		?= ~/software/scotch_6.0.4-install
bindir		?= \$(prefix)/bin
includedir	?= \$(prefix)/include
libdir		?= \$(prefix)/lib
datarootdir	?= \$(prefix)/share
mandir		?= \$(datarootdir)/man
...some code...

```

Now Scotch can be compiled by (we are still in the /src directory):

```make scotch
```

And can be installed to the directory specified in the MakeFile by:

```make install
```

## Compile the PaStiX library (shared memory parallelism)

```https://gforge.inria.fr/frs/?group_id=186
```

Copy it to a folder of your choice. In the following we are using "~/compiled_libraries" as a reference.

Extract the files by the command:

```tar -xf pastix_5.2.3.tar.bz2
```

Navigate into the source directory by:

```cd pastix_5.2.3/src
```

The different config files can be found in the confi folder. You have to copy the one fitting to your system to the /src folder. In the case of the systems mentioned above it ist the LINUX-GNU.in file. Copy it to the source directory as config.in by:

```cp ./config/LINUX-GNU.in config.in
```

You have to edit or uncomment the following blocks of the config.in file:

```###################################################################
#                  SETTING INSTALL DIRECTORIES                    #
###################################################################
ROOT          = /your/pastix/installation/path
INCLUDEDIR    = \${ROOT}/include
LIBDIR        = \${ROOT}/lib
BINDIR        = \${ROOT}/bin
PYTHON_PREFIX = \${ROOT}
```
```###################################################################
#                          INTEGER TYPE                           #
###################################################################
# Uncomment the following lines for integer type support (Only 1)
#VERSIONINT  = _long
#CCTYPES     = -DFORCE_LONG -DINTSIZELONG
#---------------------------
VERSIONINT  = _int32
CCTYPES     = -DINTSIZE32
#---------------------------
#VERSIONINT  = _int64
#CCTYPES     = -DINTSSIZE64
```
```###################################################################
###################################################################
# Uncomment the following lines for sequential (NOMPI) version
VERSIONMPI  = _nompi
CCTYPES    := \$(CCTYPES) -DFORCE_NOMPI
MPCCPROG    = \$(CCPROG)
MCFPROG     = \$(CFPROG)
MPCXXPROG   = \$(CXXPROG)
...some code...
#CCPASTIX   := \$(CCPASTIX) -DCUDA_SM_VERSION=20
#NVCCOPT    := \$(NVCCOPT) -arch sm_20
```
```###################################################################
#                      GRAPH PARTITIONING                         #
###################################################################
# Uncomment the following lines for using metis ordering
#VERSIONORD  = _metis
#METIS_HOME  = \${HOME}/metis-4.0
#CCPASTIX   := \$(CCPASTIX) -DMETIS -I\$(METIS_HOME)/Lib
#EXTRALIB   := \$(EXTRALIB) -L\$(METIS_HOME) -lmetis
# Scotch always needed to compile
SCOTCH_HOME = \${HOME}/compiled_libraries/scotch_6.0.4
SCOTCH_INC = \$(SCOTCH_HOME)/include
SCOTCH_LIB = \$(SCOTCH_HOME)/lib
# Uncomment on of this blocks
#scotch
CCPASTIX   := \$(CCPASTIX) -I\$(SCOTCH_INC) -DWITH_SCOTCH
EXTRALIB   := \$(EXTRALIB) -L\$(SCOTCH_LIB) -lscotch -lscotcherrexit
#ptscotch
#CCPASTIX   := \$(CCPASTIX) -I\$(SCOTCH_INC) -DDISTRIBUTED -DWITH_SCOTCH
#if scotch >= 6.0
EXTRALIB   := \$(EXTRALIB) -L\$(SCOTCH_LIB) -lptscotch -lscotch -lptscotcherrexit
#else
#EXTRALIB   := \$(EXTRALIB) -L\$(SCOTCH_LIB) -lptscotch -lptscotcherrexit
```
```###################################################################
#                Portable Hardware Locality                       #
###################################################################
# By default PaStiX uses hwloc to bind threads,
# comment this lines if you don't want it (not recommended)
#HWLOC_HOME = ~/compiled_libraries/hwloc-1.11.5 #/opt/hwloc/
#HWLOC_INC  = \$(HWLOC_HOME)/include
#HWLOC_LIB  = \$(HWLOC_HOME)/lib
#CCPASTIX   := \$(CCPASTIX) -I\$(HWLOC_INC) -DWITH_HWLOC
#EXTRALIB   := \$(EXTRALIB) -L\$(HWLOC_LIB) -lhwloc
```

Now you can compile the PaStiX library (we are still in the /src directory):

```make
```

You can install it to the directory defined in the config.in file by:

```make install
```

In order to be able to compile the Kratos with PaStiX now, you have to fix a little bug in one of the header files in the installation directory. In the header file pastix_int32_double_real.h you have to add:

```#inlcude <stdint.h>
```