Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Next »

Goals

Implementation of DRMAAv2 Specification

Background and strategic fit

The Distributed Resource Management Application API Version 2 (DRMAA) specification defines an interface for tightly coupled, but still portable access to the majority of DRM systems. PBS is Distributed Resource Management system(DRMS).  Standardized access to PBS product through a DRMAAv2 implementation(API). High-level API designers, meta-scheduler architects and end users can rely on DRMAAv2 implementations for a unified access to execution resources.

DRMAAv2 Specification (Distributed Resource Management Application API 2)

Overview

As we all know open standards are taking major role in the exponential growth in many areas. DRMAA2 is such an open standard. It is the successor of the wide-spread DRMAA. DRMAA is generally used for submitting jobs (or creating job work-flows) into a compute cluster by using a cluster resource management system like PBS Professional. Unlike DRMAA with DRMAA2 you can not only submit jobs, you can also get cluster related information, like getting host names, types and status information or insight about queues configured in the resource management system. You can also monitor jobs not submitted within the application. Overall it covers many more use cases than the old DRMAA standard. 

More details of DRMAA2 specification can be found from the following document.

https://www.ogf.org/documents/GFD.231.pdf

Assumptions


Requirements

The implementation of DRMAA2 specification for PBS Pro is broadly divided into the following requirements/user stories.

#TitleUser StoryImportanceNotes
1Control and monitoring of Jobs 
2Control and monitoring of Job arrays


3Control and monitoring of Reservations


4Control and monitoring of Nodes


5Control and monitoring of Queues


6Asynchronous event notification to DRMAAv2 application


7Re-entrancy support for the DRMAAv2 API


8Support for Autotools build infrastructure


9Support for RPM package


10Support for unit testing(White-box) using cppunit framework


11Support for unit testing(black-box)


Sample Example

Example1: DRMAA2 Simple Application
 	drmaa2_jsession js; /* Job session object*/		
    drmaa2_jtemplate jt; /* Job session template*/
    drmaa2_j j;  /* Job object*/
    ...
    ...
	...
    /* Create and open job session */
    js = drmaa2_create_jsession("testsession", DRMAA2_UNSET_STRING);
    if (js == NULL) {
   		 ...
       return;
    }
    ...
    ...
	...

    /* Create job template */
    jt = drmaa2_jtemplate_create();
    jt->remoteCommand = strdup("/bin/date");
    j = drmaa2_jsession_run_job(js, jt);

    ...
	...
    drmaa2_j_wait_terminated(j, DRMAA2_INFINITE_TIME);
    drmaa2_jtemplate_free(&jt);
    drmaa2_destroy_jsession("testsession");
    ...
    ...
	...
    drmaa2_j_free(&j);
    drmaa2_jsession_free(&js);
    ...
    ...
	...
Example2: DRMAA2 Advance Application
 	...
    ...
	...
    drmaa2_jtemplate        jt = drmaa2_jtemplate_create(); /* job template object */
    drmaa2_rtemplate        rt = drmaa2_rtemplate_create(); /* reservation template object */
    drmaa2_string_list      cl = drmaa2_list_create(DRMAA2_STRINGLIST, NULL); / *create list of strings */
    drmaa2_dict            env = drmaa2_dict_create(NULL); /* create a dictionary */

    drmaa2_jsession js = drmaa2_create_jsession("myjsession", NULL); /* open sessions to DRM system */
    if (js == NULL) {
        ...
        ...
        return;
    }
    drmaa2_rsession rs = drmaa2_create_rsession("myrsession", NULL); /* create a reservation session */
    if (rs == NULL)
    {
        ...
        ...
        return;
    }
    drmaa2_msession ms = drmaa2_open_msession(NULL); 					 /*create and open monitoring session */

    ml = drmaa2_msession_get_all_machines(ms, DRMAA2_UNSET_LIST);        /* determine name of first machine */
    if (drmaa2_list_size(ml) < 1) {
    	...
        return;
    }
    m = (drmaa2_machineinfo)drmaa2_list_get(ml, 0);
    drmaa2_list_add(cl, m->name);

    rt->maxSlots = 4;                                                    /* perform advance reservation */
    if (DRMAA2_TRUE == drmaa2_supports(DRMAA2_RT_MACHINEOS)) 
        rt->machineOS=DRMAA2_LINUX;
    rt->candidateMachines = cl;
    r = drmaa2_rsession_request_reservation(rs, rt);

    jt->remoteCommand = strdup("/bin/date");                             /* submit job */
    jt->reservationId = drmaa2_r_get_id(r);
    drmaa2_dict_set(env, "PBS_SCP", "/usr/bin/scp");
    jt->jobEnvironment = env;
    j = drmaa2_jsession_run_job(js, jt);

    drmaa2_j_wait_terminated(j, DRMAA2_INFINITE_TIME);                  /* Wait for termination and print exit status */
    ji = drmaa2_j_get_info(j);
    ...
	...
	...

    /* close sessions, cleanup */
    drmaa2_jtemplate_free(&jt);
    drmaa2_rtemplate_free(&rt);
    drmaa2_jinfo_free(&ji);
    drmaa2_j_free(&j);
    drmaa2_r_free(&r);

    drmaa2_close_msession(ms);
    drmaa2_close_rsession(rs);
    drmaa2_close_jsession(js);
    drmaa2_msession_free(&ms);
    drmaa2_rsession_free(&rs);
    drmaa2_jsession_free(&js);

    ...
    ...

Architecture and design

Application stack

<<Image>

Internal architecture

<<Image>


Directory structure

/drmaa2                            The entire DRMAA2 module.
  | -- configure.ac                This file dictates the behavior of the final configure script that is generated by Autoconf.
  | -- INSTALL                     Instructions for successful compilation of this library.
  | -- autogen.sh                  This file provides automatic build system preparation and is useful for projects that use the GNU autotools.  

| -- AUTHORS Altair authors information
  | -- ChangeLog                   Delta change information   
  | -- COPYING                     Altair Copyright information  
  | -- NEWS                        Latest news and updates 
  | -- README                      README
 
  | -- /api                        This directory contains all the function prototypes and definitions included in this API.
  |     | -- drmaa2.cpp
  |     | -- drmaa2.h
  | -- /src                        The C++ interface definition            
  | -- /inc                        The header files required by the source code.
  | -- /m4                         The GNU m4 macros that are called by configure.ac.
  | -- /doc                        The documentation generated by the Doxygen tool.
  | -- /man                        Man pages
  | -- /libdrmaa2                  Autoconf and libtool specification to create library.
 
  | -- /unittesting                The code for the unit testing library of C++, called CppUnit.
  |     | -- /src
  |     | -- /inc

Dependent packages

cppunit, doxygen, PBSPro, 

Questions

Question

Outcome


Not Doing


References

  • No labels