How to add automatic benchmarking to your example

From KratosWiki
(Difference between revisions)
Jump to: navigation, search
Line 1: Line 1:
 +
== Automatic benchmarking ==
 +
 
Kratos contains an ''automatic benchmarking'' feature which can be used to automatically check the status of nightly build, for example. To use this feature you need to import <tt>benchmarking</tt> module from <tt>''kratos_root''/benchmarking</tt>. To do this add the lines marked in '''bold''' to your code:
 
Kratos contains an ''automatic benchmarking'' feature which can be used to automatically check the status of nightly build, for example. To use this feature you need to import <tt>benchmarking</tt> module from <tt>''kratos_root''/benchmarking</tt>. To do this add the lines marked in '''bold''' to your code:
  
Line 29: Line 31:
 
  ##################################################################
 
  ##################################################################
 
  ##################################################################
 
  ##################################################################
 +
 +
== Specifying the values to be benchmarked ==
  
 
The principle of the ''benchmarking'' module is to first marking some values in your program which need to be examined, or some part of the code which needs to be checked for amount of the time it takes to run. Then, the program is run (using the ''benchmarking'' module) in a special mode, called ''benchmarking mode'', and the required data are collected and stored in a file. These data are called the ''reference data'', and will be used to verify the functionality of the program later. This also is done using the ''benchmarking'' module.
 
The principle of the ''benchmarking'' module is to first marking some values in your program which need to be examined, or some part of the code which needs to be checked for amount of the time it takes to run. Then, the program is run (using the ''benchmarking'' module) in a special mode, called ''benchmarking mode'', and the required data are collected and stored in a file. These data are called the ''reference data'', and will be used to verify the functionality of the program later. This also is done using the ''benchmarking'' module.
Line 37: Line 41:
  
 
In the above, <tt>Var</tt> is the variable you want to benchmark. This variable can either be a floating number, an integer number, a string, or even a boolean. The label is an string, which you can later use to identify the source of a difference, if something goes wrong. For floating point numbers and integers, you may specify the amount of the absolute or relative errors allowed. These default to <tt>None</tt>, which means that the values should be exactly the same every time the program runs. '''Please note that you cannot specify a relative error tolerance for zero reference data!''' The reason should be clear.
 
In the above, <tt>Var</tt> is the variable you want to benchmark. This variable can either be a floating number, an integer number, a string, or even a boolean. The label is an string, which you can later use to identify the source of a difference, if something goes wrong. For floating point numbers and integers, you may specify the amount of the absolute or relative errors allowed. These default to <tt>None</tt>, which means that the values should be exactly the same every time the program runs. '''Please note that you cannot specify a relative error tolerance for zero reference data!''' The reason should be clear.
 +
 +
== Determining if the program is running in benchmarking mode ==
 +
 +
Another useful function is <tt>benchmarking.InBenchmarkingMode</tt>, which tells if the program is being run in ''benchmarking mode''. This is useful if you need to calculate some results '''only''' for benchmarking use:
 +
 +
if (benchmarking.InBenchmarkingMode()):
 +
    ''calculate some results''
 +
    benchmarking.Output(...)
 +
 +
You may note that the <tt>benchmarking.Output</tt> also uses this function to avoid any outputting of any benchmark data during a normal run.
 +
 +
== Checking time consumed by some part of your code ==

Revision as of 17:08, 27 November 2008

Contents

Automatic benchmarking

Kratos contains an automatic benchmarking feature which can be used to automatically check the status of nightly build, for example. To use this feature you need to import benchmarking module from kratos_root/benchmarking. To do this add the lines marked in bold to your code:

#################################################################
##################################################################
## ATTENTION: here the order is important

#including kratos path
kratos_libs_path = '../../../../libs' ##kratos_root/libs
kratos_applications_path = '../../../../applications' ##kratos_root/applications
kratos_benchmarking_path = '../../../../benchmarking' ##kratos_root/benchmarking

import sys
sys.path.append(kratos_libs_path)
sys.path.append(kratos_applications_path)
sys.path.append(kratos_benchmarking_path)

#importing Kratos main library
from Kratos import *
kernel = Kernel()   #defining kernel

#importing applications
import applications_interface
applications_interface.Import_IncompressibleFluidApplication = True
applications_interface.Import_PFEMApplication = True
applications_interface.ImportApplications(kernel, kratos_applications_path)

import benchmarking
## from now on the order is not anymore crucial
##################################################################
##################################################################

Specifying the values to be benchmarked

The principle of the benchmarking module is to first marking some values in your program which need to be examined, or some part of the code which needs to be checked for amount of the time it takes to run. Then, the program is run (using the benchmarking module) in a special mode, called benchmarking mode, and the required data are collected and stored in a file. These data are called the reference data, and will be used to verify the functionality of the program later. This also is done using the benchmarking module.

To specify a value for benchmarking, pass it to benchmarking.Output. The format of using benchmarking.Output is:

benchmarking.Output(Var, Label = "", AbsErr = None, RelErr = None)

In the above, Var is the variable you want to benchmark. This variable can either be a floating number, an integer number, a string, or even a boolean. The label is an string, which you can later use to identify the source of a difference, if something goes wrong. For floating point numbers and integers, you may specify the amount of the absolute or relative errors allowed. These default to None, which means that the values should be exactly the same every time the program runs. Please note that you cannot specify a relative error tolerance for zero reference data! The reason should be clear.

Determining if the program is running in benchmarking mode

Another useful function is benchmarking.InBenchmarkingMode, which tells if the program is being run in benchmarking mode. This is useful if you need to calculate some results only for benchmarking use:

if (benchmarking.InBenchmarkingMode()):
    calculate some results
    benchmarking.Output(...)

You may note that the benchmarking.Output also uses this function to avoid any outputting of any benchmark data during a normal run.

Checking time consumed by some part of your code

Personal tools
Categories