I would like to develop a code which mixes coarray language features (for their elegant simplicity when it comes to accessing/transferring data across images) and MPI features (in order to use, for instance, the cluster version of the MKL sparse solver which requires MPI - or any other MPI-base analytical library).
My workstation has two processors - 6 physical cores each; and I do have the cluster edition of Intel XE composer 16 Update 2, and MS Visual Studio 2013. This workstation is used for development of a code that, ultimately, will work on a Linux cluster and on Windows computers similar to mine (using Intel compilers and related tools, to minimize portability issues).
On this workstation, I would like to run two coarray images; one image (MPI rank) per node; each benefiting from the 6 physical cores on that node (for OpenMP purposes). The code below is linked with the impi.lib and impicxx.lib libraries.
It is not really clear at this point how the code shown below (a trivially modified version of one of the Intel examples) should be run from Visual studio (should the /QCoarray option be turned on?) - or from mpiexec.exe (or the GUI version wmpiexec.exe). I understand that the coarray implementation in Intel Fortran is based on the Intel MPI library - but beyond this things are a little fuzzy for me in terms of how the two coexist (when the code also needs MPI calls). I would assume that two "MPI_COMM_WORLD" are created, one implicitly (for the coarrays), the other one explicitly (in the code).
When I manage to launch the code I either get only 1 coarray image and multiple MPI ranks; or some kind of Hydra service error. There are probably issues of thread affinity (to prevent thread migration across nodes or cores) that I should look at - but for now it is not a priority.
Any help to get started on this would be immensely appreciated, thanks in advance!
PROGRAM MAIN USE MPI IMPLICIT NONE INTEGER I, SIZE, RANK, NAMELEN, IERR CHARACTER (LEN=MPI_MAX_PROCESSOR_NAME) :: NAME INTEGER STAT(MPI_STATUS_SIZE) WRITE(*,*) 'I AM IMAGE ',THIS_IMAGE(),' OUT OF ',NUM_IMAGES(),' IMAGES.' CALL MPI_INIT (IERR) CALL MPI_COMM_SIZE (MPI_COMM_WORLD, SIZE, IERR) CALL MPI_COMM_RANK (MPI_COMM_WORLD, RANK, IERR) CALL MPI_GET_PROCESSOR_NAME (NAME, NAMELEN, IERR) IF (RANK.EQ.0) THEN PRINT *, 'HELLO WORLD: RANK ', RANK, ' OF ', SIZE, ' RUNNING ON ', NAME DO I = 1, SIZE - 1 CALL MPI_RECV (RANK, 1, MPI_INTEGER, I, 1, MPI_COMM_WORLD, STAT, IERR) CALL MPI_RECV (SIZE, 1, MPI_INTEGER, I, 1, MPI_COMM_WORLD, STAT, IERR) CALL MPI_RECV (NAMELEN, 1, MPI_INTEGER, I, 1, MPI_COMM_WORLD, STAT, IERR) NAME = '' CALL MPI_RECV (NAME, NAMELEN, MPI_CHARACTER, I, 1, MPI_COMM_WORLD, STAT, IERR) PRINT *, 'HELLO WORLD: RANK ', RANK, ' OF ', SIZE, ' RUNNING ON ', NAME ENDDO ELSE CALL MPI_SEND (RANK, 1, MPI_INTEGER, 0, 1, MPI_COMM_WORLD, IERR) CALL MPI_SEND (SIZE, 1, MPI_INTEGER, 0, 1, MPI_COMM_WORLD, IERR) CALL MPI_SEND (NAMELEN, 1, MPI_INTEGER, 0, 1, MPI_COMM_WORLD, IERR) CALL MPI_SEND (NAME, NAMELEN, MPI_CHARACTER, 0, 1, MPI_COMM_WORLD, IERR) ENDIF CALL MPI_FINALIZE (IERR) END