Class to store MPI derived datatypes for individual arrays. More...

#include "lib/mpidata.h"

Public Member Functions
	mpidata (blitz::Array< real, 3 > inputArray, const parallel &parallelData)
	Constructor of the mpidata class. More...

void	createSubarrays (const blitz::TinyVector< int, 3 > globSize, const blitz::TinyVector< int, 3 > coreSize, const blitz::TinyVector< int, 3 > padWidth, const bool xStag, const bool yStag)
	Function to create the subarray MPI_Datatypes. More...

void	syncData ()
	Function to send data across all sub-domain faces. More...

Public Attributes
const parallel &	rankData
	A const reference to the global variables stored in the parallel class to access rank data.

Detailed Description

Class to store MPI derived datatypes for individual arrays.

Since the solver uses staggered and collocated grids, the data of its variables are stored in arrays of different limits depending on whether the variable is staggered or not in each direction. As a result, the limits of the sub-arrays to be sent across inter-processor boundaries is different for different arrays. Hence the mpidata class contains MPI_SUBARRAY derived datatypes to be initialized along with different fields in order to store their sub-arrays for inter-processor communication.

Constructor & Destructor Documentation

◆ mpidata()

mpidata::mpidata	(	blitz::Array< real, 3 >	inputArray,
		const parallel &	parallelData
	)

Constructor of the mpidata class.

     The short constructor of mpidata class merely resizes the array of MPI_Status and MPI_Request datatypes.
     The former is used in non-blocking communication of MPI_Irecv, while the later is used in the MPI_Waitall
     function to complete the non-blocking communication call.

Parameters

inputArray	is the blitz array whose sub-arrays have to be created and synchronised across processors
parallelData	is a const reference to the global data contained in the parallel class

Member Function Documentation

◆ createSubarrays()

void mpidata::createSubarrays	(	const blitz::TinyVector< int, 3 >	globSize,
		const blitz::TinyVector< int, 3 >	coreSize,
		const blitz::TinyVector< int, 3 >	padWidth,
		const bool	xStag,
		const bool	yStag
	)

Function to create the subarray MPI_Datatypes.

     Must be called only after the grid class has been initialized.
     The subarray data-types cannot be created within the constructor of the parallel class as it needs the grid parameters for
     setting the limits of the subarrays.
     For this, the grid class will have to be included in the parallel class.

     However, the grid object cannot be passed to the parallel class as the grid class already includes the parallel object
     within itself, and a reverse include will raise cyclic dependency error.
     As a result, the mpidata class offers an additional layer over the parallel class for grid specific data transfer functions.

Parameters

globSize	stores the global size of a sub-domain - including core and pads
coreSize	stores the size of the core of the sub-domain and is similar to the collocCoreSize variable in the grid class
padWidth	contains the widths of pads along the 3 directions, namely padWidths TinyVector from the grid class
xStag	specifies whether the array to which the instance of mpidata class is associated with has its data points staggered in x-direction or not
yStag	specifies whether the array to which the instance of mpidata class is associated with has its data points staggered in y-direction or not

The loclSize variable holds the local size of the sub-array slice to be sent/received within the sub-domain.

The saStarts variable holds the starting coordinates of the sub-array slice.

The globCopy variable holds a copy of the global size of the sub-domain. This keeps the original array safe

◆ syncData()

void mpidata::syncData ( )

Function to send data across all sub-domain faces.

     This is the core function of the mpidata class.
     The end slices of each sub-domain recieves data from their corresponding neighbouring sub-domains,
     while the interior slices of each sub-domain sends data to their corresponding neighbouring sub-domains.

     All the data slices are send as subarray MPI derived data-types created in the \ref createSubarrays function.
     As a result, \ref syncData must be called only after the subarrays have been created.

     The data transfer is implemented here with a mixture of blocking and non-blocking communication calls.
     The receives are non-blocking, while the sends are blocking. This combination prevents inter-processor deadlock.

The documentation for this class was generated from the following files:

lib/parallel/mpidata.h
lib/parallel/mpidata.cc