Open Everything NYC Recap

I attended Open Everything NYC on Saturday. For those who do not know, Open Everything is an "un-conference" which is a conference where the participants run the show. There are several open sessions, and a few keynotes.

Some of the open session topics were on education, Open Source Software licensing, and even relationships! The Open Relationships talk was really interesting. I find it amusing how many people wind up creating alter egos to prevent employers from finding out things about them.

(Some of) the keynote speakers were amazing:

Schuyler Erle, a guy who wrote some code for UNICEF, utilized text messaging as a means to track distribution of Bed Nets to prevent malaria via a web application, among other things. If you want to research this it's all on the wiki.

Robert Steele, a former spy (yes a spy!) whose talk in my opinion was pretty much utterly useless! I saw diagrams which were well.. horrid. His talk had ZERO flow planning, it seemed to be a bunch of conspiracies, one after the other. Let alone the fact the guy is abrasive. After his talk, I went up to him to let him know that his site (which he linked in his talk), was down; his response was condescending and RUDE. I turned to some people behind me as I was leaving, and said that there was no way I could have summarized anything that guy said (I was writing notes for the wiki); they agreed. The consensus on Robert Steele was pretty universal: Conspiracy nutjob.

The final speaker, Leslie Hawthorne, a woman I have a lot of respect for gave a talk that was short, sweet, and to the point. She disussed how open source software development can be used in a common sense way to solve complex problems we face. OpenMRS, solving the problem of the HIV/AIDS crisis in the developing world; Sahana for the disaster relief management in areas devastated by floods and such. Amazing talk! Best part, she did not use any slides!

John Britton, the conference organizer gave the closing talk, and he took notes as to what should change next time. Longer open sessions etc came up. Additionally, we found out: it's possible to do this ongoing, I'd REALLY enjoy that. I met some freaking awesome people.

We were to screen RIP: "A REMIX MANIFESTO" at McFaddens Bar and Pub by the UN, but the owner didn't allow us to do it; which was stupid. We wound up sitting around half price drinks/appetizers and had some good conversations and fun.

Overall, I had fun and would do it again. I went into this not knowing what to expect.

Accepted into Google Summer of Code 2009

For the second year in a row, I was accepted into Google Summer of Code!!! I will be working with OpenMRS on the Facility Data Module.

My project will focus on developing tools for collecting and ultimately generating reports for aggregate data from external sources.

Groovy Forms Module is NOT dead

Hey folks!

I'm still alive and not dead. The Groovy Forms Module is not a dead project, and WILL be finished. I plan on getting work started back up on it soonish. School is back in session for me. I'm sorry for the long delay.

That is all.

GSoC wrap-up

The program officially ended on August 18^th. I'm awaiting the arrival of my t-shirt!!

Now, I didn't quite have time to finish my project prior to that date since my uncle suddenly passed away on August 10^th.

The project itself is about 80 percent completed. All that is left is to get the rendering working and finish up the management page to edit the form metadata as it appears in the system.

My time is now limited since school started up again for me, so as much time as I can I'm going to devote to finishing up this project.

I'd like to thank the following:

• Burke Mamlin for helping me and answering all the questions I had and guiding me when I was lost.



• Ben Wolfe for putting up with me and helping me when Burke was MIA.



• OpenMRS and its wonderful worldwide network of developers for making this project into what it has become.



• Paul Biondich for encouraging me (along with Burke) to apply for summer of code.



• Last, but certainly not least, Leslie Hawthorn for managing this program and every problem that popped up. She must have super human powers or something.

The list can only be so long, I feel like I'm accepting an academy award here. It was a great experience, and I will definately continue to maintain my project in whatever free time I can find.

GSoC 2008: We're in the home stretch

Well folks, I can't believe it's almost over. What a journey. I never imagined I'd be able to do this. I have an (almost) working project. I have a few more things that need to be done.

Let's highlight what IS done:

1. GroovyForm and its related metadata is stored
2. Model class is interrogated for its properties and those are stored in a container class.
3. From the data collected in the container class, I generate markup checking for a predefined set of data types.
   
4. The view and its related controller is generated -- sneaking in some groovy magic of course!
5. GroovyForms are successfully saved into the system, along with their respective, model, view and controller.
6. Forms are persisted when the module is shutdown and reloaded when it is restarted

What's left:

1. Editing metadata related to the forms currently in the system
2. Writing the servlet which handles submissions
3. Create some sample forms
   
4. Documentation

I did plan to add a lot of other stuff such as the ability to create custom form widgets, however there were a few problems which caused me to redirect my focus. These will come in after I finish with Summer of Code.

GSOC 2008: Week 5

So, it's time for an update. This week got off to a sketchy start, but it's gained momentum. Let me enumerate what I have done thus far with the project. First, I've added AJAX using jquery. I figured I would use jquery mainly because it provided painless AJAX goodness. Prior to even thinking of using jquery, I wrote my own AJAX code using a tutorial that I found while googling. I had to tweak it a bit, but for the most part, it seemed very standard. However, after I wrote the AJAX equivalent code, it didn't feel too elegant. So, first I'm going to show the AJAX code I wrote; then I'll show you the jquery version; finally, I'm going to show you the servlet which handles the AJAX on the server-side.

First, the AJAX I wrote:

function AjaxValidation(url, callback) {
    var req = init()
    req.onreadystatechange = processRequest

    function init() {
        if (window.XMLHttpRequest) {
            return new XMLHttpRequest()
        } else if (window.ActiveXObject) {
            return new ActiveXObject("Mircosoft.XMLHTTP")
        }
    }

    function processRequest() {
        if (req.readyState == 4) {
            if (req.status == 200) {
                if (callback) {
                    chkSyntaxCallBack(req.responseXML)
                }
            }
        }
    }

    this.doGet = function() {
        req.open("GET", url, true)
        req.send(null)
    }
}
function chkSyntaxCallBack(responseXML) {
    var res = responseXML.getElementsByTagName("result")[0].firstChild.nodeValue
    document.getElementById("out").innerHTML = res
}

function checkSyntax() {
    var target = document.getElementById("groovyModel")
    var url = "${pageContext.request.contextPath}/moduleServlet/groovyforms/createGroovyForm?groovyModel=" + escape(target.value)
    var ajax = new AjaxValidation(url, chkSyntaxCallBack)
    ajax.doGet()
}

I warned you that it wasn't too elegant. Understanding this isn't too hard. Here is the call sequence: init() -> doGet() -> processRequest() -> chkSyntaxCallBack(). Not that bad. Now let's see the jquery version.

$(window).ready(function () {
    $("#groovyModel").bind("blur", function () {
        $.ajax({
            type: 'POST',
            data: { groovyModel: $("#groovyModel").val() } ,
            url: "${pageContext.request.contextPath}/moduleServlet/groovyforms/createGroovyForm" ,
            cache: false ,
            success: function(data) {
                var res = $(data).find("result").text()
                $("#out").html(res)

            }
        })
    });
})

Okay, that's much better. A few things are still happening here. When the window is finished loading, I bind my textarea element which has the CSS id of "groovyModel" to the blur event (lost focus). Then you see the AJAX. Now this is very straight forward. We're using the POST method, we're sending whatever value is inside of the textarea at the time the event is fired, we're posting to a servlet, not going to cache, and when we're done, it's printed to the screen. Very straight forward.

Now, like I said, this is all backed on the server-side by a servlet, which is written in Groovy. So here we go:

/**
 * The contents of this file are subject to the OpenMRS Public License
 * Version 1.0 (the "License") you may not use this file except in
 * compliance with the License. You may obtain a copy of the License at
 * http://license.openmrs.org
 *
 * Software distributed under the License is distributed on an "AS IS"
 * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
 * License for the specific language governing rights and limitations
 * under the License.
 *
 * Copyright (C) OpenMRS, LLC.  All Rights Reserved.
 */
package org.openmrs.module.groovyforms.web

import javax.servlet.ServletException
import javax.servlet.http.HttpServlet
import javax.servlet.http.HttpServletRequest
import javax.servlet.http.HttpServletResponse
import org.apache.commons.logging.LogFactory
import org.codehaus.groovy.control.CompilationFailedException

class CreateGroovyFormServlet extends HttpServlet {
  def classLoader
  static final def log = LogFactory.getLog(CreateGroovyFormServlet.class)
  private static final long serialVersionUID = 066373513262051L

  @Override
  protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
    def generateTemplate = request.getParameter("template")
    def generateController = request.getParameter("controller")
    def finalMarkup = request.getParameter("markup")
    def clazz = URLDecoder.decode(request.getParameter("groovyModel"))
    def name = request.getParameter("formName")
    def version = request.getParameter("version")
    def res = this.checkSyntax(clazz)
    if (clazz) {
      if (checkSyntax(clazz)) {
        response.contentType = "text/xml"
        response.setHeader "Cache-Conrol", "no-cache"
        response.writer.write "\n\t$res\n"
      } else {
        response.contentType = "text/xml"
        response.setHeader "Cache-Control", "no-cache"
        response.writer.write "true"
      }
    } else {
      response.contentType = "text/xml"
      response.setHeader "Cache-Control", "no-cache"
      response.writer.write "\n\tPlease fill in the Form Model\n"
    }
  }

  @Override
  protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
    doGet(request, response)
  }

  @Override
  void init() throws ServletException {
    if (log.infoEnabled)
      log.info("Initializing...")
    classLoader = getClassLoader()
  }

  def getClassLoader() {
    def gcl = new GroovyClassLoader(this.getClass().getClassLoader())
    gcl
  }

/**
 * This method is used to relay errors to the user
 * @param clazz the class
 * @return the exception message or null if it was successful
 */
  def checkSyntax(clazz) {
    def sb = new StringBuilder()
    sb << "import org.openmrs.*\n\n\n"
    sb << clazz
    def res = null
    try {
      getClassLoader().parseClass(sb.toString())

    } catch (CompilationFailedException e) {
      res = "Exception: ${e.message}"
    }
    res

  }

/**
 * Check if it is result groovy code.
 * @param clazz the class
 * @return whether or not it is result groovy code
 */
  def isValidGroovy(clazz) {
    def sb = new StringBuilder()
    sb << "import org.openmrs.*\n\n\n"
    sb << clazz
    try {
      getClassLoader().parseClass(sb.toString())
    } catch (CompilationFailedException e) {
      return false
    }
    return true
  }
}

This servlet contains a lot of utility methods. One compiles, one initializes/returns the GroovyClassLoader, and of course doGet(), doPost() and init(). doGet() and doPost() both do the same thing, with doPost() simply delegating to doGet(). The code should be reasonably easy to understand. checkSyntax() returns null if it was parsed cleanly, otherwise it returns the exception message, the stack track wouldn't be useful in my case. It returns an XML tag <result> with "true" if it was successful, the exception message if it was not, and a message stating that the field must be filled in if it's empty or just not passed in. I implicitly import org.openmrs.* to allow for easier access to the OpenMRS domain model classes. The parent classloader for the GroovyClassLoader is set the servlet container's classloader. This gives me access to the classpath when loading Groovy classes.

I still have a bit to do, templating needs to be written in, for the most part it's done. Just have a few problems I'm facing, but I'll get through it. Too many people are relying on me succeeding. I feel that this project could help a lot of people, so i feel pressure to succeed. I still need to write in the "Edit" functionality of the "Manage Groovy Forms" page.

I'm definately making progress. More updates to come, that's for sure.

GSoC 2008: Groovy Forms getting some AJAX!

No, not the cleaning product! The "Web 2.0" kind of AJAX! Now where will this goodness be placed? The pages for creating the forms and managing the forms.

First off, I used Direct Web Remoting (DWR) to provide a link between my backend java codebase and the front-end javascript! It's quite amazing. You can choose to publish/unpublish the forms with the tick of a checkbox and DWR will magically set the published attribute on the form to the proper state. I then decided to go one step furthur, and provide a mechanism to backup the forms via xstream to XML. Now, with the XML files, two files are maintained, one individual XML file for each form, and one global one, which makes up the model for the entire system. The rationale for having the XML file for each form is that you can just zip up the form folder, and unzip it into another OpenMRS install and you're good to go. Adding the ability to re-generate the XML files via the management interface is especially handle so that if you make a quick change on one system, then want to install it onto another you can.

Now, DWR will be used for creating the forms as well. This will be done mostly to easily relay errors, such as syntax errors in the Controller and the Model of the groovy form. Don't fear, I'm going to go overboard with AJAX! It'll only be used up until the point of where templates are generated. After that point, a Servlet will take over the bulk of the work (I am undecided on the Servlet bit.).

Now what's next? Several things need to happen.

First, creating the form:

1. We need to validate that form fields are not empty; also the input must be checked for validity.
   
2. The model's syntax needs to be compiled and syntax errors need to be relayed to the user.
   
3. We need to get the fields of the class and store them.
4. We need to generate templates like: <%= textArea(...); %> -- They will not be run it through the template engine immediately until the user has confirmed what they want,
5. After the view has been tweaked -- it is run through the template engine; at this point we will also run the controller through and generate that and present it at the same time the final view is presented.
6. The user will be presented with the HTML of the view and Groovy controller and given a final opportunity to tweak it before it is saved to the system.
   
7. The form is added to the system -- by default the form is not published.
   
8. Finally, to publish the form so that it can be used to enter data, they go to the management screen and tick the checkbox.
   
9. The form is now ready to be used for data entry.

Pictures are worth a thousand words so here we go: One and Two.

First one contains one form which is published, the rest are not. The second, no forms are published.

The user interface is intuitive and easy to use. It's a great design, of course I'm biased since It was was me who designed it.