Monday, July 30, 2018

Week 11

Hi all,

This the end of week 11, and we a heading week 12 which basically the last week of work.
If you missed last week's blog, you probably wanna see it here.

Work done this week.

By the end of this week, the module could ably merge Patient, Location, Encounter and Obs resource.
However, I had to tested it on larger dataset of Patient data. I discovered that with large data I faced java.lang.OutOfMemoryError issues. So I had to make sure my code optimizes memory which I think was solved.

I also done more work on the Audition and done a few changes on the UI.

Challenges Encountered

When you download data from a server with many patient, the file is amazingly big. I of recent downloaded a file of 100mb. Now uploading such a file to another server, I ended in such an error

org.springframework.web.multipart.MaxUploadSizeExceededException: Maximum upload size of
75000000 bytes exceeded; nested exception is org.apache.commons.fileupload.FileUploadBase$SizeLimitExceededException

This calls for overriding the 'maxUploadSize ' property of 'CommonsMultipartResolver'

But I have failed to do this in Openmrs module dev and I posted this on talk here.

I'm also facing another Hibernate issue.

Context of the Hibernate Issue

Like for instance you are merging Patients from serverA to B which a new server with Patients. However, data comes with primary keys and foreign keys set yet we use key Generator class of hibernate. Ok I tried as possible to set all these IDs like patientId to null to tell hibernate that this is a new Object. However, there some culprit id values that I could be missing to set to null which make Hibernate think that we are updating it ending up with StaleStateException

Proposed solution

Check the Hibernate logs to see which row Hibernate is trying to updated or retrieve but I don’t know how to turn the Hibernate logging so I started this talk thread but got no help as yet.

How to Contribute?

If your reading this blog, you could want help me on the following critical blockers that are blocking my progress
  • https://talk.openmrs.org/t/enabling-hibernate-logging/19140/9
  • https://talk.openmrs.org/t/overriding-the-default-multipart-file-size/19193/6
Regards,

Samuel Male

Sunday, July 22, 2018

Week 10

Hi all,

Its now the end of week 10 since the program started.  Incase you missed out last week's feed, you could want to check it out here .

Work done this week.

For this week, I have been working on a few Tickets. These are all about Mapping Patient metadata in a MergeAble format. I finished work on  Mapping Concepts OBS and Encounters  now am assembling everything to get sense out my work. In a nutshell, am now using the Mapping to add support for Encounters.

Challenges Encountered.

In every development stage, there are always challenges and its those challenges that make one learn.

Handling database Table Ids for Merged Data 😕 

For instance we have two servers ie:- Server A and B. A has 50 patients while B has 5 Patients. Regardless of the DBMS, Every row needs an Id. Well while merging data, we could want to merge a Patient from the last row of server A to B. Let me assume that the Patient has an Id of 50. When this Patient is merged, the Patient in Server B may look something like below
Idpatient_name
5 Male Samuel
6 Kenny
7 Kaddu Ronnie
8 Another One
9 Some Patient
50   From Server A
Like you can see, it looks dirty within the database.

Proposed Solution

We could first make first make a check to the database to see whether the Patient we are trying to Merge already exists or not, if it exists, the update it otherwise set the id to null and register it as a new Patient. For a few metadata, this is simple but with other metadata like Encounters, Obs, Locations and Concepts that reference the original Patient id it becomes hard to handle this. You could want to advise me on this modal. Please leave a comment of advice.

Regards,

Samuel Male

Tuesday, July 17, 2018

Week 9

Hi all,

This is the 9th week since the game started. If you missed out about last week's update, you could want to check it out here .

Work Done this week

This was the week were I had my second evaluation. I was making ends meet to seeing myself pass the evaluation. And do you what, I passed the evaluation and thanks to my mentors 😊 

Currently, the module codebase ain't found in the Openmrs repo. Its still within my repo but Steven told me we will just transit it later to the Openmrs repo. So I couldn't use JIRA for PM. But just as a good practice, I came up with a Trello dashboard(MPD) to help me keep track of progress.

So currently I have been working on Mapping Concepts and 85% of the work is complete.
After that I will proceed to Mapping OBS. Note that these are all subtasks for Adding Support for Encounters as a Resource.

Challenges Encountered

I have faced a challenge of Mapping Concepts since they are soo wide and have a complex hierarchy.
I need your tech support in this. But what I believe is that my current implementation(pending) will be able to give the expected performance. What confuses me is "I wonder how to distinguish a Concept from the dictionary from a customized one! "

Otherwise, I expect significant progress due this week.

Regards,

Samuel Male

Monday, July 9, 2018

Week 8

Hi all,

This is the 8th week since I started work on  Merge Patient Data project. I'm very excited to inform you dear reader that we currently have a working module in place, unless otherwise.

Work done this week

Generally, I capitalized from the front end.

  • Audition

  • I managed to put in place a page that lists down all audits that are available. Basically an audit is a brief accountability of what really happened in a given MPD Operation(Export/Import).


    I also put came up with a details page for a give audit.


Sunday, July 1, 2018

week 7

Hi folks,

Its now seven weeks down the road since I begun working on the Merge Patient Data Module.
I think I have been slow with progress. This has been cause of some inevitable blockers. However, lots of work has been done this week.

Work Done

  • I set up my dev environment on a new Platform(mac OSX). Though my performance decreased, I spent sleepless nights configuring stuff and getting used to the platform.

  • More development. Last week I had a bug that blocked me for a while. I'm serializing the data using the  gson library. However I was facing a challenge dealing with generic types. I realized I invested more time trying to fix the bug than moving forward.

Suggested Solution

Just for the use case of serialization of data, I just implemented a dataStore
public class MPDStore {
 public MPDStore() {}
 List<Patient> patients;
 List<Location> locations;
 List types = new ArrayList<>(); 
 public List getTypes() { 
 return types; 
 } 
 public void setTypes(List types) { 
 this.types = types; 
 } 
 public List getPatients() { 
 return patients; 
 } 
 public void setPatients(List patients) { 
 this.patients = patients; 
 } 
 public List getLocations() { 
 return locations; 
 } 
 public void setLocations(List locations) { 
 this.locations = locations; 
 } 
 public void addType(MergeAbleDataCategory type) { 
 this.types.add(type); 
 }
 }



With this model, the module perfectly exports and imports Patient data from one Openmrs instance to another.
Currently I'm working on the audition of the module Operations. I anticipate that this won't take me long then I proceed to the Configuration Services.

I'm looking forward to finishing with both the Audit and Configuration Service implementation by 9th this month.

By the way I'm excited to inform you that the module currently offers the basic functionality inline with the project objectives though more work is still needed at the UI to improve the end User Experience.