How to run your application in cluster
Running application in cluster requires some modification in python script of the solution. Here a description of these modification will be given assuming that the c++ part of the code is already prepared for runing in parallel. The modifications consist in adding a partitioning process and then using a parallel solver.
Contents |
Importing necessary tools
first of all you need to import the boost mpi library by adding the following code:
#importing MPI ... for this boost 1.35 or superior is needed import mpi
then you need to import the partitioning and parallel solver applications.
Adding Partitioning Application
At this moment the partitioning is done using METIS. For this reason first we have to import metis application adding following line:
applications_interface.Import_KratosMetisApplication = True
to the importing application part of the python script. Then importing the application using following code:
from KratosMetisApplication import *
Adding parallel solver
Trilinos library can be used for solution of linear solvers. For using this library first you need to import its interface in Kratos as follow:
applications_interface.Import_KratosTrilinosApplication = True
and then import it:
from KratosTrilinosApplication import *
now the script header must be in following form:
#importing applications import applications_interface applications_interface.Import_StructuralApplication = True applications_interface.Import_KratosTrilinosApplication = True applications_interface.Import_KratosMetisApplication = True applications_interface.ImportApplications(kernel, kratos_applications_path) from KratosStructuralApplication import * from KratosTrilinosApplication import * from KratosMetisApplication import *
Partitioning Process
After importing the KratosMetisApplication the next step is to create the metis partitioning process and use it for partitioning. The MetisPartitioningProcess takes a modelpart, an IO, number of partitions and domain size. Here is an example of partitioning process creatition:
partitioner = MetisPartitioningProcess(model_part, gid_io, number_of_partitions, domain_size);
and finall step is the execuation of the process:
partitioner.Execute()
The finall code will be something like following sample code:
number_of_partitions = mpi.size #we set it equal to the number of processors print "number_of_partitions", number_of_partitions partitioner = MetisPartitioningProcess(model_part, gid_io, number_of_partitions, domain_size); partitioner.Execute()
NOTE: Don't forget to comment the ReadModelPart statement:
#gid_io.ReadModelPart(model_part)
Solvers
As mentioned before the Trilinos library is used for solution of linear systems. There are different libraries inside Trilinos providing different structures and algorithms. Following are imported to Kratos:
- Epetra Classes for distributed matrices and vectors.
- Amesos A common interface to certain sparse direct linear solvers like SuperLU Dist
- AztecOO Linear solver package based on preconditioned Krylov methods.
- ML The algebraic multilevel and domain decomposition preconditioner package
AztecOO Iterative Solvers
The following example shows how to use the AztecOO iterative solvers and preconditioners in Kratos:
########################################################
#defining the linear solver
aztec_parameters = ParameterList()
aztec_parameters.set("AZ_solver","AZ_gmres");
aztec_parameters.set("AZ_kspace",10); // the Krylov space size for gmres
preconditioner_type = "Amesos" // using Amesos direct preconditioners
preconditioner_parameters = ParameterList()
preconditioner_parameters.set("amesos: solver type", "Amesos_Klu");
overlap_level = 3
nit_max = 300
tol = 1e-6
solver.structure_linear_solver = AztecSolver(aztec_parameters,preconditioner_type,preconditioner_parameters,tol,nit_max,overlap_level);
The parameters to be passed are descibed in Trilinos Manuals
Amesos Direct Solvers
The following example shows how to use the Amesos direct solvers and in Kratos:
########################################################
#defining the linear solver
solver.structure_linear_solver = AmesosSolver("Superludist",solver_parameters);
For more information about the Amesos solvers and their parameters please refer to Trilinos Manuals
Examples
There are a set of solvers for different type of applications (structural, fluid) already implemented in Kratos and can be used not only for solution of different problems over cluster but also as a reference for new extensions. Some examples are:
