Mirheo coordinator¶
The coordinator class stitches together data containers, Particle Vectors, and all the handlers, and provides functions to manipulate the system components.
A single instance of this class should be created in the beginning of any simulation setup.
Note
Creating the coordinator will internally call the MPI_Init() function, and its destruction will call MPI_Finalize(). Therefore if using a mpi4py Python module, it should be imported in the following way:
import mpi4py
mpi4py.rc(initialize=False, finalize=False)
from mpi4py import MPI
-
class
Mirheo
¶ Main coordination class, should only be one instance at a time
Methods
-
__init__
(nranks: int3, domain: real3, log_filename: str='log', debug_level: int=-1, checkpoint_every: int=0, checkpoint_folder: str='restart/', checkpoint_mode: str='PingPong', max_obj_half_length: float=0.0, cuda_aware_mpi: bool=False, no_splash: bool=False, comm_ptr: int=0) → None¶ Create the Mirheo coordinator.
Warning
Debug level determines the amount of output produced by each of the simulation processes:
- silent: no log output
- only report fatal errors
- report serious errors
- report important information steps of simulation and warnings (this is the default level)
- report not critical information
- report some debug information
- report more debug
- report all the debug
- force flushing to the file after each message
Debug levels above 4 or 5 may significanlty increase the runtime, they are only recommended to debug errors. Flushing increases the runtime yet more, but it is required in order not to lose any messages in case of abnormal program abort.
The default debug level may be modified by setting the
MIRHEO_DEBUG_LEVEL
environment variable to the desired value. This variable may be useful when Mirheo is linked as part of other codes, in which case thedebug_level
variable affects only parts of the execution.Parameters: - nranks – number of MPI simulation tasks per axis: x,y,z. If postprocess is enabled, the same number of the postprocess tasks will be running
- domain – size of the simulation domain in x,y,z. Periodic boundary conditions are applied at the domain boundaries. The domain will be split in equal chunks between the MPI ranks. The largest chunk size that a single MPI rank can have depends on the total number of particles, handlers and hardware, and is typically about \(120^3 - 200^3\).
- log_filename – prefix of the log files that will be created. Logging is implemented in the form of one file per MPI rank, so in the simulation folder NP files with names log_00000.log, log_00001.log, … will be created, where NP is the total number of MPI ranks. Each MPI task (including postprocess) writes messages about itself into his own log file, and the combined log may be created by merging all the individual ones and sorting with respect to time. If this parameter is set to ‘stdout’ or ‘stderr’ standard output or standard error streams will be used instead of the file, however, there is no guarantee that messages from different ranks are synchronized.
- debug_level – Debug level from 0 to 8, see above.
- checkpoint_every – save state of the simulation components (particle vectors and handlers like integrators, plugins, etc.)
- checkpoint_folder – folder where the checkpoint files will reside (for Checkpoint mechanism), or folder prefix (for Snapshot mechanism)
- checkpoint_mode – set to “PingPong” to keep only the last 2 checkpoint states; set to “Incremental” to keep all checkpoint states.
- max_obj_half_length – Half of the maximum size of all objects. Needs to be set when objects are self interacting with pairwise interactions.
- cuda_aware_mpi – enable CUDA Aware MPI. The MPI library must support that feature, otherwise it may fail.
- no_splash – don’t display the splash screen when at the start-up.
- comm_ptr – pointer to communicator. By default MPI_COMM_WORLD will be used
-
applyObjectBelongingChecker
(checker: mirheo::ObjectBelongingChecker, pv: mirheo::ParticleVector, correct_every: int=0, inside: str='', outside: str='') → mirheo::ParticleVector¶ Apply the checker to the given particle vector. One and only one of the options inside or outside has to be specified.
Parameters: - checker – instance of
BelongingChecker
- pv –
ParticleVector
that will be split (source PV) - inside – if specified and not “none”, a new
ParticleVector
with name inside will be returned, that will keep the inner particles of the source PV. If set to “none”, None object will be returned. In any case, the source PV will only contain the outer particles - outside – if specified and not “none”, a new
ParticleVector
with name outside will be returned, that will keep the outer particles of the source PV. If set to “none”, None object will be returned. In any case, the source PV will only contain the inner particles - correct_every – If greater than zero, perform correction every this many time-steps. Correction will move e.g. inner particles of outer PV to the :inner PV and viceversa. If one of the PVs was defined as “none”, the ‘wrong’ particles will be just removed.
Returns: New
ParticleVector
or None depending on inside and outside options- checker – instance of
-
computeVolumeInsideWalls
(walls: List[mirheo::Wall], nSamplesPerRank: int=100000) → float¶ Compute the volume inside the given walls in the whole domain (negative values are the ‘inside’ of the simulation). The computation is made via simple Monte-Carlo.
Parameters: - walls – sdf based walls
- nSamplesPerRank – number of Monte-Carlo samples used per rank
-
deregisterIntegrator
(integrator: mirheo::Integrator) → None¶ Deregister a integrator.
-
deregisterPlugins
(arg0: mirheo::SimulationPlugin, arg1: mirheo::PostprocessPlugin) → None¶ Deregister a plugin.
-
dumpWalls2XDMF
(walls: List[mirheo::Wall], h: real3, filename: str='xdmf/wall') → None¶ Write Signed Distance Function for the intersection of the provided walls (negative values are the ‘inside’ of the simulation)
Parameters: - walls – list of walls to dump; the output sdf will be the union of all walls inside
- h – cell-size of the resulting grid
- filename – base filename output, will create to files filename.xmf and filename.h5
-
getState
(self: Mirheo) → MirState¶ Return mirheo state
-
isComputeTask
(self: Mirheo) → bool¶ Returns
True
if the current rank is a simulation task andFalse
if it is a postrprocess task
-
isMasterTask
(self: Mirheo) → bool¶ Returns
True
if the current rank is the root
-
log_compile_options
(self: Mirheo) → None¶ output compile times options in the log
-
makeFrozenRigidParticles
(checker: mirheo::ObjectBelongingChecker, shape: mirheo::ObjectVector, icShape: mirheo::InitialConditions, interactions: List[mirheo::Interaction], integrator: mirheo::Integrator, number_density: float, mass: float=1.0, dt: float, nsteps: int=1000) → mirheo::ParticleVector¶ Create particles frozen inside object.
Note
A separate simulation will be run for every call to this function, which may take certain amount of time. If you want to save time, consider using restarting mechanism instead
Parameters: - checker – object belonging checker
- shape – object vector describing the shape of the rigid object
- icShape – initial conditions for shape
- interactions – list of
Interaction
that will be used to construct the equilibrium particles distribution - integrator – this
Integrator
will be used to construct the equilibrium particles distribution - number_density – target particle number density
- mass – the mass of a single frozen particle
- dt – time step
- nsteps – run this many steps to achieve equilibrium
Returns: New
ParticleVector
that will contain particles that are close to the wall boundary, but still inside the wall.
-
makeFrozenWallParticles
(pvName: str, walls: List[mirheo::Wall], interactions: List[mirheo::Interaction], integrator: mirheo::Integrator, number_density: float, mass: float=1.0, dt: float, nsteps: int=1000) → mirheo::ParticleVector¶ Create particles frozen inside the walls.
Note
A separate simulation will be run for every call to this function, which may take certain amount of time. If you want to save time, consider using restarting mechanism instead
Parameters: - pvName – name of the created particle vector
- walls – array of instances of
Wall
for which the frozen particles will be generated - interactions – list of
Interaction
that will be used to construct the equilibrium particles distribution - integrator – this
Integrator
will be used to construct the equilibrium particles distribution - number_density – target particle number density
- mass – the mass of a single frozen particle
- dt – time step
- nsteps – run this many steps to achieve equilibrium
Returns: New
ParticleVector
that will contain particles that are close to the wall boundary, but still inside the wall.
-
registerBouncer
(bouncer: mirheo::Bouncer) → None¶ Register Object Bouncer
Parameters: bouncer – the Bouncer
to register
-
registerIntegrator
(integrator: mirheo::Integrator) → None¶ Register an
Integrator
to the coordinatorParameters: integrator – the Integrator
to register
-
registerInteraction
(interaction: mirheo::Interaction) → None¶ Register an
Interaction
to the coordinatorParameters: interaction – the Interaction
to register
-
registerObjectBelongingChecker
(checker: mirheo::ObjectBelongingChecker, ov: mirheo::ObjectVector) → None¶ Register Object Belonging Checker
Parameters: - checker – instance of
BelongingChecker
- ov –
ObjectVector
belonging to which the checker will check
- checker – instance of
-
registerParticleVector
(pv: mirheo::ParticleVector, ic: mirheo::InitialConditions=None) → None¶ Register particle vector
Parameters: - pv –
ParticleVector
- ic –
InitialConditions
that will generate the initial distibution of the particles
- pv –
-
registerPlugins
(arg0: mirheo::SimulationPlugin, arg1: mirheo::PostprocessPlugin) → None¶ Register Plugins
-
registerWall
(wall: mirheo::Wall, check_every: int=0) → None¶ Register a
Wall
.Parameters: - wall – the
Wall
to register - check_every – if positive, check every this many time steps if particles penetrate the walls
- wall – the
-
restart
(folder: str='restart/') → None¶ Restart the simulation. This function should typically be called just before running the simulation. It will read the state of all previously registered instances of
ParticleVector
,Interaction
, etc. If the folder contains no checkpoint file required for one of those, an error occur.Warning
Certain
Plugins
may not implement restarting mechanism and will not restart correctly. Please check the documentation for the plugins.Parameters: folder – folder with the checkpoint files
-
run
(niters: int, dt: float) → None¶ Advance the system for a given amount of time steps.
Parameters: - niters – number of time steps to advance
- dt – time step duration
-
save_dependency_graph_graphml
(fname: str, current: bool=True) → None¶ Exports GraphML file with task graph for the current simulation time-step
Parameters: - fname – the output filename (without extension)
- current – if True, save the current non empty tasks; else, save all tasks that can exist in a simulation
Warning
if current is set to True, this must be called after
mmirheo.Mirheo.run()
.
-
setBouncer
(bouncer: mirheo::Bouncer, ov: mirheo::ObjectVector, pv: mirheo::ParticleVector) → None¶ Assign a
Bouncer
between anObjectVector
and aParticleVector
.Parameters: - bouncer –
Bouncer
compatible with the object vector - ov – the
ObjectVector
to be bounced on - pv – the
ParticleVector
to be bounced
- bouncer –
-
setIntegrator
(integrator: mirheo::Integrator, pv: mirheo::ParticleVector) → None¶ Set a specific
Integrator
to a givenParticleVector
Parameters: - integrator – the
Integrator
to assign - pv – the concerned
ParticleVector
- integrator – the
-
setInteraction
(interaction: mirheo::Interaction, pv1: mirheo::ParticleVector, pv2: mirheo::ParticleVector) → None¶ Forces between two instances of
ParticleVector
(they can be the same) will be computed according to the defined interaction.Parameters: - interaction –
Interaction
to apply - pv1 – first
ParticleVector
- pv2 – second
ParticleVector
- interaction –
-
setWall
(wall: mirheo::Wall, pv: mirheo::ParticleVector, maximum_part_travel: float=0.25) → None¶ Assign a
Wall
bouncer to a givenParticleVector
. The current implementation does not supportObjectVector
.Parameters: - wall – the
Wall
surface which will bounce the particles - pv – the
ParticleVector
to be bounced - maximum_part_travel – maximum distance that one particle travels in one time step. this should be as small as possible for performance reasons but large enough for correctness
- wall – the
-
start_profiler
(self: Mirheo) → None¶ Tells nvprof to start recording timeline
-
stop_profiler
(self: Mirheo) → None¶ Tells nvprof to stop recording timeline
-
Unit system¶
Mirheo assumes all values are dimensionless.
However, users may use Mirheo in combination with the pint Python package, by defining Mirheo’s unit system using set_unit_registry
:
import mirheo as mir
import pint
ureg = pint.UnitRegistry()
# Define Mirheo's unit system.
ureg.define('mirL = 1 um')
ureg.define('mirT = 1 us')
ureg.define('mirM = 1e-20 kg')
mir.set_unit_registry(ureg)
# dt automatically converted to 0.01, matching the value of 1e-8 s in the Mirheo unit system.
u = mir.Mirheo(..., dt=ureg('1e-8 s'), ...)
Global Simulation State¶
Some information about the simulation is global to all Mirheo components. They are stored in the following binded objects:
-
class
DomainInfo
¶ Convert between local domain coordinates (specific to each rank) and global domain coordinates.
Methods
-
global_size
¶ Size of the whole simulation domain.
-
global_start
¶ Subdomain lower corner position of the current rank, in global coordinates.
-
global_to_local
(x: real3) → real3¶ Convert from global coordinates to local coordinates.
Parameters: x – Position in global coordinates.
-
global_to_local_shift
¶ shift to transform global coordinates to local coordinates.
-
is_in_subdomain
(x: real3) → bool¶ Returns True if the given position (in global coordinates) is inside the subdomain of the current rank, False otherwise.
Parameters: x – Position in global coordinates.
-
local_size
¶ Subdomain extents of the current rank.
-
local_to_global
(x: real3) → real3¶ Convert local coordinates to global coordinates.
Parameters: x – Position in local coordinates.
-
local_to_global_shift
¶ shift to transform local coordinates to global coordinates.
-
-
class
MirState
¶ state of the simulation shared by all simulation objects.
Methods
-
current_dt
¶ Current simulation step size dt. Note: this property is accessible only while Mirheo::run() is running.
-
current_step
¶ Current simulation step.
-
current_time
¶ Current simulation time.
-
domain_info
¶ The
DomainInfo
of the current rank.
-