Owners: Annah Ngaruro and Jennifer E Shivers
This is an overview of the DATIM4U site exchange process for transferring site changes from node to global. When this process is employed, the DHIS 4U Node is considered to be the source system for site management. Sites are not expected to be managed or changed in the PEPFAR Global DATIM system for DATIM4U instance country. The objective of the process is to maintain site data synchronization by moving any site metadata updates or changes from the 4U Node system where they are being curated to the global PEPFAR system. This will enable both systems to have the same sites and enable both systems to have the same understanding of ADX (Aggregate Data Exchange) messages that contain the PEPFAR reporting data.
The following system components are involved in the transaction:
- The Global PEPFAR DATIM system (shown in the large blue box below)
- PEPFAR's DHIS2 DATIM Global system containing site data
- A CSD document that contains a cache copy of the 4U Node's site data
- The 4U Node system (shown in the large green box below)
- The 4U Node's DHIS2 system containing site data
- A CSD document that contains the extract of the 4U Node's site data
Data Currently Transferred
The data fields which are mapped in the DXF->CSD transform are below. Some are part of the CSD data model, some are added as a an extension point under an extension.
- parent org unit id
- created timestamp
- last updated timestamp
- geo data (either as lat/long or under a
<csd:extension urn="urn:http://www.opengis.net/gml" type="MultiPolygon"> extension point for polygons)
- level code
- group codes
- org code
- entityID (an attribute value)
- dataSets (if dataElements as a service is checked, which is not for DATIM)
The job wil move site data from the 4U node's DHIS2 system to the Global DATIM DHIS2 system. On the 4U side, there is a sync mediator that manages the routing process.
The steps performed by the mediiator are:
- Extract from 4U Node DHIS2 - This step extracts new or updated site information from the 4u node's DHIS2 system and publish that in the 4U Node's Interlinked registry in a CSD document called XXOU-Managed. The first time the job runs, it creates a time stamp called https://xx.datim4u.org/dhis2/api/dataStore/CSD-Loader-Last-Export/XXOU-Managed and sets that time stamp to the current date/time. ( xx and XX stand for the node's two digit ISO country code. For example (ug / UG for Uganda.) ) Subsequent runs only select site records that have been updated or changed since the time stamp and their parent records.
- Refresh global cache - This step refreshes the global system's cache of the node's CSD document. A request is triggered for the global CSD document titled XXOU-Managed (XX stand for the node's two digit country code. For example (UG for Uganda.)) There is also a time stamp that is used to denote the last time that the Global document cache was refreshed. The document and cache time stamps can be accessed by an administrator in the CSD application.
- Import into Global DHIS2 The final step is to import the additions and/or updates into Global DHIS2. There is also a time stamp in the DHIS2 system that tracks the last date of import and the process selects only the CSD records that have been updated and changed since the last import. The import time stamp is called https://www.datim.org/api/dataStore/CSD-Loader-Last-Import/XXOU-Managed. This date is initially set to the date that the OU has extracted data from datim-global. Therefore any changes or updates since this date will be imported back into global through the two steps noted below:
- Create the DXF from the Global CSD document, XXOU-Managed, for import into Global DHIS2.
- Import the with the updated site information into Global DHIS2.
How to run the process
The job can be triggered in three ways:
- Scheduled job - The scheduled job can be configured to run as often as possible. An OpenHIM administrator can set the time frames for the exchange schedule in the 4U Node's OpenHIM. For more information about scheduled jobs, see the OpenHIM documentation on polling: http://openhim.readthedocs.io/en/latest/user-guide/polling-channels.html?highlight=poll
- Manual trigger - To manually trigger a scheduled job, a user with "Super User" or "Admin" capabilities will need to log into the 4U Node's OpenHIM and perform the following actions:
- Log into the the 4U Node OpenHIM
- Select Channels from the OpenHIM Menu
- Select the Channel name for the job you want to run. For this job, the channel name is "AUTO Trigger update of locally managed sites to global".
- Click the “Manually Trigger” button to start the job.
- Select the “X” in the upper right corner to exit the form.
- During the ADX exchange process - During the ADX exchange process, the job will automatically be triggered to ensure that all sites included in the ADX message have been transferred to the Global DHIS2 DATIM system.
Viewing and checking the process from the Node's OpenHIM perspective
When the job is triggered, you will see a trigger transaction show up in the Node OpenHim Transaction Log. This is the master transaction and will track all steps of the transaction.
After the transaction is completed, one can see the results and details for each step, by clicking on the node's "AUTO Trigger update of locally managed sites to global" transaction and scrolling to the bottom of the page. This will show the details and status codes for each step (Orchestration) in the process:
You may also click on any of the steps to get additional details about the transactions.