This module will contain a Complex Observation Handler used to store unstructured data in the form of a complex observation to any storage systems suited for the purpose. The handler will be able to select a storage DAO according to the MIME type of the data that it's passed. So, for instance, a serialized object can be saved to a NoSQL database and large binary data can be saved to a distributed file/storage system.
It should be noted that there is no such thing in OpenMRS-core as a Complex Note as yet. We propose to get a ComplexData field included in the Note object and for the core to provide a ComplexNoteHandler interface. Any note with a non-null ComplexData property will be handled by a ComplexNoteHandler implementation.
Design
Implementing the ComplexObsHandler (and ComplexNoteHandler in the future)
package org.openmrs.module.shr.unstructureddata.obs.handler; import java.io.IOException; import org.openmrs.Obs; import org.openmrs.api.APIException; import org.openmrs.api.context.Context; import org.openmrs.module.shr.unstructureddata.dao.handler.UnstructuredDAOHandler; import org.openmrs.module.shr.unstructureddata.dao.handler.UnstructuredDAOService; import org.openmrs.obs.ComplexData; import org.openmrs.obs.ComplexObsHandler; import org.openmrs.obs.handler.AbstractHandler; public class UnstructuredDataHandler extends AbstractHandler implements ComplexObsHandler { @Override public Obs saveObs(Obs obs) throws APIException { String contentType = getContentType(obs.getComplexData().getTitle()); String key = obs.getUuid(); if (Context.getService(UnstructuredDAOService.class).getUnstructuredDAO(contentType).saveObject(key, obs.getComplexData().getData())){ obs.setComplexData(null); obs.setValueComplex(obs.getComplexData().getTitle()); } else { throw new IOException(); } return obs; } @Override public boolean purgeComplexData(Obs obs) { String contentType = getContentType(obs.getComplexData().getTitle()); String key = obs.getUuid(); return Context.getService(UnstructuredDAOService.class).getUnstructuredDAO(contentType).purgeObject(key); } @Override public Obs getObs(Obs obs, String view) { String contentType = getContentType(obs.getComplexData().getTitle()); String key = obs.getUuid(); ComplexData complexData = new ComplexData(obs.getComplexData().getTitle(), Context.getService(UnstructuredDAOService.class).getUnstructuredDAO(contentType).getObject(key)); obs.setComplexData(complexData); return obs; } String getContentType(String title){ return null; //do parsing here } }
Service
package org.openmrs.module.shr.unstructureddata.api; import org.openmrs.api.OpenmrsService; import org.openmrs.module.shr.unstructureddata.dao.UnstructuredDAO; import org.openmrs.module.shr.unstructureddata.exception.AlreadyRegisteredException; public interface UnstructuredDAOService extends OpenmrsService { /** To be called by the ComplexObsHandler/ComplexNoteHandler */ UnstructuredDAO getUnstructuredDAO(String contentType); /** For each DAO a call to this method will be added in the ModuleActivator willStart() method */ void RegisterUnstructuredDAO (String contentType, UnstructuredDAO prototype) throws AlreadyRegisteredException; /** For each DAO a call to this method will be added in the ModuleActivator willStop() method */ void DeregisterUnstructuredDAO(String contentType); }
Interfaces
package org.openmrs.module.shr.unstructureddata.dao; public interface UnstructuredDAO { Boolean saveObject(String key, Object value); Object getObject(String key); Boolean purgeObject(String key); }
Possible Database / File Storage Options
As we suspect we'll be storing a lot of XML documents as blobs, a native XML database that supports XPath and XQuery is something we could be looking at.
BaseX seems to be the open-source XML database with the most freely available features.
For binary data storage a distributed file system seems like it will provide the best redundancy and scalabiltiy. RiakCS is a cloud storage platform that uses the Amazon S3 API and the Riak cluster module to provide high-availability storage. OpenStack Swift and Hadoop HDFS are other options to look at.