How to use parallel Pardiso solver for your example

From KratosWiki
Jump to: navigation, search

The package PARDISO is a thread-safe, high-performance, robust, memory efficient and easy to use software for solving large sparse symmetric and unsymmetric linear systems of equations on shared memory multiprocessors. The solver has has been licensed to thousands of researchers at international scientific laboratories and universities since its first release in 2004 [1]. You can use this parallel solver to solve the system of equations and matrices in Kratos. For this purpose, we present some modifications in your Python interface files which help you to solve your problems faster. The important thing is that you should have the new version of Kratos, which includes the folder of mkl_solvers_application in the applications. This folder contains Pardiso solver interface and if you activate it in the main Jamroot of Kratos, it will be automatically compiled and linked to Kratos. We mention the activation lines in the Jamfile and the command to compile Kratos. It is important that you should install intel MKL library on your computer and compile Kratos with Intel compiler, otherwise you can not use this solver. The first step is to introduce the path of installed intel MKL library on your computer :


Jamroot:

##path to the mkl library (STRICTLY OPTIONAL)
path-constant MKL_INCLUDE_DIR : /opt/intel/mkl/10.0.3.020/include/ ;
path-constant MKL_LIB_DIR : /opt/intel/mkl/10.0.3.020/lib ;

The next step is to activate two lines in Jamroot file to include the mkl_solvers_application in the list of applications for compilation and linking to Kratos. The following bold lines should be activated (removing #):


Jamroot:

## MKL solvers: will not be compiled by default ###################################
## this requires intel MKL library installed properly on your system ##############

use-project /kratos-prj/mkl_solvers_application : $(TOP)/applications/mkl_solvers_application ;


# installation
alias install :
 /kratos-prj/mkl_solvers_application//install


After modifing Jamroot, we should specify the type of LHS matrix which we use it in our solution. This can be done by refering to the following file in mkl_solvers_application\external_includes folder and change the following flag:

in mkl_solvers_application\external_includes\mkl_pardiso_solver.h

Setting a flag in the following line

MKL_INT mtype = 1; /* real and structurally symmetric*/

This flag can be assigned according to different matrix structures:

TYPE Matrix
1  real and structurally symmetric
2  real and symmetric positive definite
-2 real and symmetric indefinite
3  complex and structurally symmetric
4  complex and Hermitian positive definite
-4 complex and Hermitian indefinite
6  complex and symmetric
11 real and nonsymmetric
13 complex and nonsymmetric

In this case, we suppose real and symmetric matrices. The next step is to add Pardiso solver into your Python interface files. The first modification is in the solver file in python_scripts folder of your application. We suppose the name of Element2D for our application and the name of static_Element2D_solver_pardiso.py for pardiso solver python file and we add or modify the following lines in this file:


static_Element2D_solver_pardiso.py :

from KratosR1MKLSolversApplication import *

class StaticElement2DSolver:

#######################################################################
def __init__(self,model_part,domain_size):

  self.model_part = model_part
  self.time_scheme = ParallelResidualBasedIncrementalUpdateStaticScheme()


  self.Element2D_linear_solver =  ParallelMKLPardisoSolver()


#definition of the convergence criteria
  self.conv_criteria = ParallelDisplacementCriteria(1e-4,1e-9)        

#######################################################################
#######################################################################
def Initialize(self):
 #creating the solution strategy
   CalculateReactionFlag = False
   ReformDofSetAtEachStep = False
   MoveMeshFlag = False

   import parallel_strategy_python
   self.solver = parallel_strategy_python.SolvingStrategyPython(self.model_part,self.time_scheme,self.Element2D_linear_solver,self.conv_criteria,CalculateReactionFlag,ReformDofSetAtEachStep,MoveMeshFlag)

#######################################################################

The next step is to add this solver to our run code in test_examples folder or any folder that we put our run codes. We name the run code as Run.py:

Run.py :

applications_interface.Import_KratosR1MKLSolversApplication = True
from KratosR1MKLSolversApplication import *
import static_Element2D_solver_pardiso
static_Element2D_solver_pardiso.AddVariables(model_part)
static_Element2D_solver_pardiso.AddDofs(model_part)
#creating a solver object
solver = static_Element2D_solver_pardiso.StaticElement2DSolver(model_part,domain_size)
solver.time_order = 1
solver.echo_level = 0
solver.Initialize()
solver.Solve()

For the final step we should compile Kratos using intel compiler but before that, two enviroment variables should be set to define the number of CPUs and the location of intel MKL library :

export LD_LIBRARY_PATH="/opt/intel/mkl/10.0.3.020/lib"
export OMP_NUM_THREADS = 2


In these two lines we assigned the path of intel MKL library and the number of processors which in our case is the two cores of Core Duo Processor.

For the last command we compile Kratos:

bjam threading=multi -j2 -intel
Personal tools
Categories