Dave Crane is a genius

The other day I picked up a copy of Ajax in Practice, by Dave Crane and several others.

I don’t know why I waited this long.  His Ajax in Action is one of my all-time favorite technical books.  I also have a copy of Prototype & Scriptaculous in Action, though I’ve only read a few sections of it.  I think the fact I haven’t made as much progress as I would have liked in P&SiA is what made me hesitate about Ajax in Practice.

I’m only a few chapters into it now, but let me just say this:  If you ever see a technical book written by Dave Crane, just buy it.  It doesn’t really matter whether you care about the topic or not.  The book’s first few chapters are packed with more wisdom and great discussions than ten other books on my bookshelf.  He also has a free, easy writing style that makes reading his text a joy.  And believe me, it ain’t easy to make JavaScript a joy.

Dave Crane rocks.  If I ever manage to write a technical book myself, I only hope to come anywhere near his level of excellence.

(Of course, Scott Davis isn’t bad, either. :))

Library quirk in the new Eclipse, and random framework musings

I’ve been working with Struts 2 recently, and like most frameworks it relies on a set of jar files.  I like to work with Eclipse as my primary IDE.  In that framework, my normal mode of operating is to create a User Library containing the jar files I need and then add it to my Dynamic Web Projects through the Java Build Path configuration.

It seems that the system has changed in Eclipse 3.3, aka Europa.  I’m working with the Web Tools Platform plugin, so at least I have dynamic web project capabilies.  I can also set up a Tomcat server easily.

What’s changed is that now, when I add a User Library to the build path, Eclipse warns me that it won’t be deployed to the server.  The warning is:

“Classpath entry org.eclipse.jdt.USER_LIBRARY/libname will not be exported or published.  Runtime ClassNotFoundExceptions may result.”

Yer darn tootin’ they’ll result.  I suppose this is a feature, not a bug, because I have had situations where the libraries were already added to my server’s classpath so I didn’t want to duplicate them in the war file.  Also, some war files become enormous because of all the libraries used.  (The last time I wrote a simple app that used JPA, Spring, and Hibernate the resulting war file was huge.  And the size of the war file that comes with a trivial Grails application is easily over 10 megs.)  Now Eclipse leaves the libraries out of the war by default.

I’m reminded of when the Collections framework for Java was created, and ArrayList turned out to be exactly like Vector but without the synchronization.  The theory seemed to be that you can always add synchronization later (Collections.synchronizedList(myList)), but that you couldn’t remove it from Vector when you didn’t need it.  This change has the same feel to it — you can always figure out a way to deploy the libraries, but you never had a way to get rid of them short of editing the resulting war file.

That’s not necessarily a good thing for me, though.  I’m always tempted to add libraries to my server’s classpath, partly to keep the size of the individual war files down, and partly because I tend to use the same libraries over and over and I hate packing them into every single application.  All those repeated files feel like a waste.

Still, it’s dangerous not to include the jar files in the web application itself.  Version problems can easily arise, especially because I tend to upgrade to the latest version of a given library whenever I can.  For that reason, I’ve resisted the temptation to, say, put Hibernate and Spring into my server’s lib directory.

So what do I do in Eclipse?  It turns out there are two solutions:

1. Declare the libraries to be module dependencies for the war.  That’s done by going into the project properties, then Properties –> J2EE Module Dependencies –> Select All.  Then Eclipse treats the libraries the same way it would any dependent project.

Eclipse Properties J2EE Module Dependencies

2. The other approach is to go on the file system and copy the jar files directly into the WEB-INF/lib directory of the project, then do a refresh.  Apparently Eclipse is smart enough to then add the jars to the classpath for my source code.

I’m not sure which way I’ll prefer in the future.  Eclipse does have a Navigator view which supports drag-and-drop copying, so that’s easy enough.  Still, I hate to leave the IDE when I’m in the middle of using it.

This is also one of the few situations where using MyEclipse wouldn’t make any difference.  Version 6 truly rocks, but it doesn’t have a predefined library for Struts 2.  Maybe Struts 2 was still too new when MyEclipse 6 was being developed.

As an aside, it should be interesting to see which way the framework wars will turn in 2008.  There’s still a huge installed base of Struts 1.* clients out there, but many of them are looking at other options.  The other page-based framework I’ve used is Spring Web MVC, which has its own learning curve.

JSF is part of the Java EE spec, but if you’re used to a page-based framework the switch to a component-based framework is not trivial.  Tapestry is still out there, and it has a few very fervent advocates, but I haven’t heard much growth in it.  I also haven’t gotten any calls about it in quite some time.  Seam is growing, too, but I expect that’ll be popular among the same crowd that likes JSF, since it’s based on JSF anyway.   Of course, there’s also Google Web Toolkit if you want to build a desktop application inside a browser.

Struts 2 is very interesting, and has the modern feel of interceptors and filters and POJOs (oh my!).  It has Spring-like dependency injection, though I think they use Guice by default rather than Spring, not that it really matters.  I never learned WebWork before the merger, so the learning curve has been pretty substantial for me.  Personally I feel that Struts 2 is so fundamentally different than Struts 1 that I’m surprised they kept the Struts name.  That marketing decision may make all the difference, however.

I’ll keep an eye on all of them, of course, but for my own applications, I like Grails. 😉

(Yes, I know that’s the same as saying Spring Web MVC.  We’ll see how that plays out, too.)

Moving My Google Maps Mashup to Grails, part two

I made a few improvements to my Google Maps mashup, and though they’re not as significant as the changes made in my previous posts, I still wanted to make a record of them.

I discussed in my last post the problem I had exposing my domain objects as JSON strings. To summarize the problem:

  1. In order to use the Google Maps API, I need to be able to pass latitude and longitude information from my Location objects (written in Groovy) into JavaScript. Grails pulls the data from a database and populates all the Locations, which are passed from the controller to the view. Somehow the JavaScript in the view needs to extract the coordinates in order to draw the markers on the map.
  2. The page where I want to do the rendering actually lists courses, not just locations. My Course class has an attribute of type Location, which is fine, but means I need to traverse a relationship in order to access the data I need. Normally that wouldn’t be an issue, but JavaScript doesn’t know how to do that.
  3. I can use the convenient Grails “converters” plugin to translate my Course objects into JSON, but that leads to difficulties as well. It seems that the converter doesn’t follow relationships, or, rather that it just inserts the foreign key value where the relationship goes. In other words, the JSON version of a particular Course only lists an integer in the location field.
  4. As a result of (3), I decided to pass both the collection of Course objects and the collection of Location objects to the view. That in itself was awkward, because it meant I had to “dereference” the location array myself for each course, which felt highly brittle.
  5. Also, when the converter operated, it dumped all the information about my objects directly into the web page, where it could be read by anyone who knows how to do a “view source” command. Not good.

The way I solved the problem was to create a special class in my Grails project called CourseMapData:

class CourseMapData {
    String title
    Date startDate
    Date endDate
    String city
    String state
    double latitude
    double longitude

That class encapsulates all the data I need out of both the Course object and its associated location. It also leaves out any attributes that might possibly be sensitive. Since it doesn’t have any associations to any other classes, I can transform it into JSON without losing any information.

Since it’s not actually a domain class, I put the source file in the src/groovy folder of my Grails project. In my CourseController, I then added the necessary translation:

def list = {
    if(!params.max)params.max = 10
    def courseList = Course.list(params)
    def cmList = []
    courseList.each { c ->
        def cm = new CourseMapData()
        cm.title = c.title
        cm.startDate = c.startDate
        cm.endDate = c.endDate
        cm.city = c.location.city
        cm.state = c.location.state
        cm.latitude = c.location.latitude
        cm.longitude = c.location.longitude
        cmList << cm

    [ courseList:courseList, cmList:cmList]

I was hoping their might be a more elegant way to populate the object (something similar to how request parameters are passed into objects using “obj.properties = params“), but it was easy enough to just write them out. That’s especially true because the values come from both Course and Location.

By passing the collection of CourseDataObjects, my JavaScript code in my list.gsp page reduces to:

function initialize() {
    map = new google.maps.Map2($("map"));
    var homeJSON = ${Location.findByCityAndState('Marlborough','CT').encodeAsJSON()};
    home = new google.maps.LatLng(homeJSON.latitude,homeJSON.longitude);
    map.setCenter(home, 4);

    var cmsJSON = ${[cms:cmList].encodeAsJSON()};
    var courseDataArray = cmsJSON.cms;


    map.addControl(new google.maps.LargeMapControl());
    map.addControl(new google.maps.MapTypeControl());
    map.addControl(new google.maps.OverviewMapControl());

Now I’m only converting one collection (cmList), and although it dumps a lot of code in the page source, nothing in it is particularly sensitive. The rest is essentially the same as before.

I’m still not quite ready to put my application online, because it hasn’t been secured yet. I plan to use a beforeInterceptor to separate the admin pages from the public pages. It’s a little awkward, though, because this isn’t a site that I actually want anyone but myself to modify. Still, I’d like to be able to access it myself when I’m traveling in order to add in new courses and/or locations.

Hopefully I’ll get a chance to add that functionality soon.

As an aside, I fixed another issue that occurred to me. Obviously I don’t want to have to enter a latitude and a longitude when I create a new Location. I have an earlier post here that talks about using the Google geocoder service to do the translation for me. I finally got around to adding that functionality to my Grails application.

Now the save method in my LocationController looks like:

def save = {
    def location = new Location()
    location.properties = params
    if (location.latitude == 0 && location.longitude == 0) {
        // Use the Google Maps geocoder to fill in latitude and longitude
        def key = 'longUglyGoogleMapsKey'
        def base = 'http://maps.google.com/maps/geo'
        def query = "q=${location.city},+${location.state}&output=csv&key=${key}"
        def url_string = base + '?q=' +
            ',+' + URLEncoder.encode(location.city,"UTF-8") +
            ',+' + URLEncoder.encode(location.state,"UTF-8") + 
        def results = new URL(url_string).text.split(',')
        location.latitude = results[-2].toDouble();
        location.longitude = results[-1].toDouble();
    if(location.save()) {
        flash.message = "Location ${location.city}, ${location.state} created."
    } else {

To use it, I modified my create.gsp page for Locations to include the phrase:

“Note: leave latitude and longitude empty or 0.0 to use the embedded geocoder.”

Then when I enter the data, I can see the latitude and longitude fill in automatically. Sweet.

Moving My Google Maps Mashup to Grails, part one

The Google Maps API is easy to use. The basic idea is to create a Map2 object (the former GMap2, now known as google.maps.Map2) and add Marker objects to it. The Marker objects then use InfoWindow objects to display all the information for that marker, which can either appear on load, or you can set up an event (like a click) to make the info display. I also like the Polyline objects, especially because you can make them follow geodesics.

While there are many good tutorials about Google Maps on the web, my favorite reference is Scott Davis’s book Google Maps API, v2. I purchased the eBook version at Pragmatic Programmer and really like it.

Since the API is in JavaScript, the key issue for me is how to get my location data into the code. It’s all in a database, but it’s also considered poor design to talk directly to a database from the view. When I wrote my app originally in Java, I did the normal process of building a data access layer in front of the database and then using a servlet to access it, turn the locations into Java objects, and then pass the whole lot to the view as request attributes. That’s all well and good, but of course JavaScript code can’t read Java objects directly.

Dave Crane has a good discussion of this issue in his book Ajax in Action. (Of course it’s a good discussion — everything in that book is good. :)) I recently also picked up his Prototype and Scriptaculous in Action book, and it too talked about the mismatch problem.

Crane lists the available options as

  • Content-centric, where the back-end code returns already-formatted HTML which a web page can render,
  • Script-centric, where actual JavaScript code is returned and executed by the browser, and
  • Data-centric, where the information is returned in some format that both sides can understand.

For my situation, the data-centric approach is really the only option. I’m not returning information that I want to render directly. My hope is for Google Maps to do all the rendering. For the same reason, I don’t want to return actual code, because that wouldn’t simplify anything.

I’m going with the data approach. Since JavaScript doesn’t understand either Java or Groovy objects, I need to select an intermediate format for the data. That also means I need to be able to generate the format from the Java or Groovy objects and then parse it in JavaScript.

As Crane points out, the major options are either XML or JSON.

I do remember back when XML was popular. I thought it was pretty cool, too, during that late 90’s and the first few years of this century. But XML gets parsed into DOM trees, and the DOM access API’s in JavaScript are bulky and cumbersome.

(But not as ugly as those in Java. I’ve always been amazed how badly Java interacts with XML, especially now that seemingly every business must, by law, build a service oriented architecture implementing web services.)

Groovy’s approach to XML is refreshingly easy, but again it doesn’t really help me here. I need to get my latitudes and longitudes into the constructor of Google’s LatLng class, which is what Marker uses to draw at a particular location.

The other intermediate format possibility is JSON. I really like JSON. It’s easy, short, and quite flexible. I wonder if JSON had been around when XML first became available if we’d all be using it instead.

Before I get to that, though, I have to mention another possibility that I actually used in my original implementation. There I relied on JSP tags, which generated JavaScript as though it was any other set of strings.

Assuming my courses all contain locations, and that each location has a latitude and a longitude, what I did was:

<c:forEach var="course" items="${courses}">
    loc = new google.maps.LatLng(
    // etc...

and went on from there. The benefit to this approach is that I could access the courses directly as Java objects, using the normal dot notation to get where I needed to go.

The downside to this approach is that it makes for some truly ugly JavaScript. Note that I don’t have the word “var” in front of my (JavaScript) loc variable above. I can’t, because my <c:forEach> loop is going to generate a new copy of that line — and every other line in the loop — for every single course. I also didn’t try to store the values as I went along, which I suppose I could have done by declaring an array ahead of time, assigning each LatLng to an array at the “i”th index location, and manually incrementing the index. The effect is that I’m writing out each iteration of every loop.

I have to say, though, that the approach did work. I saved myself the difficulty of translating from Java objects to anything else, at the expense of repeating about a dozen lines of code over and over again for every course.

That can’t be the “right” way to go. Since I was porting to Grails anyway, I investigated what tools it (and, of course, Groovy) brought to the task.

Chapter 9 of Graeme Rocher’s book The Definitive Guide to Grails discusses the Grails tags available for Ajax applications. He goes through tags <g:remoteField>, <g:remoteLink>, and <g:formRemote> in some detail. Unfortunately for me, all three of those tags have an attribute called “update” which indicates which <div> element will contain the resulting data.

I don’t want to update a <div> element. I want numbers back.

Incidentally, in order to get a better sense of what was going on, I used those tags and looked at the resulting generated HTML. Not surprisingly, the tags all created Ajax.Updater calls in Prototype, my library of choice. What I wanted was an Ajax.Request, from whose results I could extract the text data and parse away.

Since the DGG book is getting a bit dated (already! I hope a good revision when Grails goes to 1.0 in October is already in the works), I checked the tag reference at the Grails web site. In addition to the above three tags, it also listed <g:remoteFunction> and <g:submitToRemote>, but neither of those were necessarily appropriate either.

The fact is, however, the data I need is already in the web page. It’s just in the form of a collection of instances of the Groovy classes called Course and Location, which are part of my domain model. All I need to do is to convert them to JSON and I’m off to the races.

Enter the Converters plug-in project for Grails, which is even built into Grails 0.6. The Converters plugin gives a convenient (according to the web site, YMMV) way to convert your “domain objects, maps, and lists to JSON or XML very quickly.”

The web page has examples on it, but I must admit I found them rather confusing. It’s entirely possible I’m missing something obvious. For example, I’m still not sure how it would help me to do a “render Course.get(0) as JSON” in a controller. I guess in my page I could set up a normal Ajax.Request myself to call an action implemented that way, but I’m not sure that’s they way the example is intended to be used.

I also got messed up when I tried to convert my collection to JSON, rather than a single object. The line above,

render Course.get(0) as JSON

works just fine, but

render courseList as JSON

throws exceptions. Eventually I wound up going to

render [crs:courseList] as JSON

which did the job, at the price of introducing a variable I don’t need. Today, though, I did some poking around on the Grails User mailing list archive at Nabble and found that

def data = courseList as Course[]
render data as JSON

might work instead, but I haven’t yet tried that out.

What I ultimately did was to go into my GSP page and use the converter’s encodeAsJSON() function. My page contains:

function initialize() {
    map = new google.maps.Map2($("map"));
    var homeJSON = ${Location.findByCityAndState('Marlborough','CT')
    home = new google.maps.LatLng(homeJSON.latitude,homeJSON.longitude);
    map.setCenter(home, 4);

    // Transform the course list to a JSON object
    //  whose 'crs' property is the list of individual courses
    var coursesJSON = ${[crs:courseList].encodeAsJSON()};

    // Do the same with the locations
    var locsJSON = ${[locs:locationList].encodeAsJSON()};
    var courseArray = coursesJSON.crs;
    var locsArray = locsJSON.locs;


    map.addControl(new google.maps.LargeMapControl());
    map.addControl(new google.maps.MapTypeControl());
    map.addControl(new google.maps.OverviewMapControl());

That brings me to my other difficulty with using the JSON builder. My location objects are not normally embedded in my course GSP pages. I expected to get to the locations by traversing from the course to its location. The problem is, if you look at the generated JavaScript, the first Course object becomes

      "startDate": "2005-04-04 00:00:00.0",
      "title": "Intro Java with WSAD",
      "class": "Course",
      "endDate": "2005-04-08 00:00:00.0",
      "client": 1,
      "location": 1,
      "id": 1,

which means its location is just a foreign key value, not an actual object. I can’t traverse from Course to Location in JSON form. Since I want information from both objects (i.e., the Course title and dates, and the Location’s latitude and longitude), I had to make both sets of data available to my view.

(Also, note that the date is now firmly a string. I tried parsing it using JavaScript’s Date class, but no luck.)

That, at least, was easy. My CourseController.groovy class originally had

def list = {
    if(!params.max)params.max = 10
    [ courseList: Course.list( params )]

in it, and now it has

def list = {
    if(!params.max)params.max = 10
    [ courseList: Course.list( params ), locationList: Location.list() ]

I had to be sure not to pass the params argument to Location.list(), because while I want the course listing table to have pagination, I don’t know which locations I’m going to need for the courses so I have to send them all. It also forces me to traverse the relationship myself. In my processCourses function, I have

function processCourses(courseArray, locsArray) {
    // === Plot the markers ===
    for (var i = 0; i < courseArray.length; i++) {
        var c = courseArray[i];
        var loc = locsArray[c.location - 1];
        var latlng = new google.maps.LatLng(loc.latitude,loc.longitude);
        var label = "<b>" + c.title + "</b><br />" +
            "<em>" + loc.city + ", " + loc.state + "</em><br />" +
            c.startDate.split(' ')[0] + " -- " +
            c.endDate.split(' ')[0];

        var marker = createMarker(latlng, label);

        var polylineOptions = {geodesic:true};
        var polyline = new google.maps.Polyline(new Array(home,latlng),

and you can see that my location comes from selecting the proper element out of the locations array, based on the index value that comes from course.location. It’s not pretty, but it does seem to work.

This leaves me with two issues that have kept me from putting my application on the web yet.

  1. By default, Grails applications let the user add, edit, and remove elements as well as list them. I have to add some login mechanism before I expose that functionality. 🙂 Jason Rudolph in his Getting Started with Grails book used an interceptor for that. I’ll no doubt start there.
  2. Rendering the course as a JSON object embeds all of its details inside the JavaScript.

That’s a much bigger problem. My course objects have references to clients and even to rates in them, and I’d just as soon not expose that to anyone capable of doing a “view source” on the page (security by obscurity, indeed). I’m not exactly sure how I’m going to handle that.

My first thought is to create some kind of “narrow” interface to course objects that would expose only the fields I need to show. I’m going to try creating a class called, say, CourseMapData, which will consist of

class CourseMapData {
    String title
    Date startDate
    Date endDate
    String city
    String state
    double longitude
    double latitude

which will be populated from a Course object and its associated Location, send that to the view and convert it to JSON. I think that will work, but it’ll mean adding a new Groovy class that isn’t technically a domain object from the Grails point of view. I haven’t done that before, so that’s a good learning opportunity. Of course, I’ll also have to put in a method to do the conversion, too, and that method isn’t exactly a controller action, so I’m not sure where it goes, either.

I guess that’s part of the problem with eliminating formal DAO classes. Grails supplies finders automatically, which is great. I loved being able to use Location.findByCityAndState() without having to write anything. But I would normally put my conversion method in the Course DAO, and now I’m not so sure what to do.

When I get those issues worked out, I’ll no doubt post a “part two” entry here.

Moving My Google Maps Mashup to Grails, part zero

I’m not quite ready to make it live on the web yet, but I’ve just about finished porting my Google Maps mashup to Grails. In earlier posts I’ve discussed how I built a Google Maps mashup showing the names and locations of all the courses I’ve taught over the last three years.

In its earlier incarnation (still on the web here, though eventually that link is going to point to the Grails version), everything was written in straight Java. I stored the data in three database tables, which I accessed by using a service layer supported by Spring and a data access layer that used Hibernate via JPA. The page itself is just straight JSP, with the necessary JavaScript thrown in. The page is just read only, too; there’s no way to add new courses or modify existing ones.

The whole application is a natural for Grails, though. It’s essentially a web interface on some database tables, with a small domain model, some simple searching capabilities, and only a bit of business logic thrown in here and there. That makes it ideal for me to do as a learning activity. Even better, when I’m finished, I’ll have a nice web-based interface to a system where I can store my course information as I go along.

Pardon me while I digress into an interesting story. Years ago, my father decided he really wanted to learn how to sail. As was his wont, he threw himself into the subject, digging up every book he could find, talking to people he knew were sailors, and basically trying to learn the entire subject in zero time.

My father and I are generally quite different people, but we definitely share that characteristic. I’m not sure what drives him to obsess so deeply about whatever subject he gets interest in, but for me, I have a few motivations:

  1. I just plain want to know, as fast as humanly possible.
  2. I’m terrified I’ll make a horrible mistake out of ignorance.
  3. I really want to do whatever the activity is, right away, but I really, really don’t want to look foolish when I make normal beginner mistakes.
  4. Being able to do cool stuff is so much fun I want to show everybody else how to do it, too.

Number 4 is a big reason why I’m an instructor. Number 1 makes me horribly impatient, but highly motivated. Numbers 2 and 3 tend to get in the way a lot, but I’m working on that.

Anyway, when my father was deep in his learning phase about sailing, he met an experienced sailor who offered to take him on a trip on the Chesapeake Bay. They experienced a lot of very high winds and waves — certainly more than my father was expecting. Still, he asked tons of questions and learned a lot.

What his friend told him, though, was that at the time my father was a great “book” sailor. He knew a lot of theory, but didn’t yet have the practical experience necessary to make sense of it all. He certainly knew enough to get into trouble, but probably not enough to know how to get back out again.

I totally get that, too. When I get wrapped up in a new subject, I dig through what feels like tons of material in a very short time. I rapidly get to the point of being a good “book” sailor. The problem is that I always carry this deep-seated fear that I don’t really know what I’m doing. That feeling doesn’t go away until I actually do the things I’m learning about.

That’s something I often think of as the “instructor trap”. Instructors spend so much time teaching that they often don’t have the time or energy left for doing, and it’s the doing that makes all the difference. You can certainly take that too far — I personally believe that industry experience can be vastly overrated. I’ve known plenty of developers who may have spent twenty years developing, but as far as learning anything new goes, it might as well have been the same single year repeated twenty times. Still, if all you know is the book, you’re missing a lot.

The term I apply to that situation is the Ten Canonical Errors. I’ve used that term here, but not in a long time.

(Quick aside: what level of geek do you have to be to use the term “canonical” in a sentence? Worse, what level of geek do you have to be to come up with a term that actually includes the word “canonical”? Yikes. It’s a good think I’m basically in touch with my inner geek.)

The idea behind the Ten Canonical Errors is that every time I learn a technology, when I start to use it there will be at least ten mistakes I’ll make that will cost me hours if not days. It doesn’t matter how many books I’ve read or people I’ve talked to, I’m still going to get stuck over and over. It’s inevitable. No matter how much I try to avoid them, the mistakes are going to happen. It’s just part of the process, just an extremely aggravating part.

Here’s a JavaScript example. HTML is simple, right? There’s nothing to it. I used to thing JavaScript was easy, too, until I started digging into Ajax, but that’s not the error I made.

One day I was trying to do a simple “Google suggest” type of Ajax application, where the server provided suggestions each time I typed in a character. I wasn’t doing anything fancy. As I recall, I as just doing a simple variation on a book example. But no matter what I tried, the JavaScript wouldn’t work.

My error was that I put all the JavaScript in a separate file and called it from the web page. Not a problem, right? In my page, I had:

<script type="text/javascript" src="myscript.js" />

That nice, clean XML, because I put in the “/>” at the end. Unfortunately, even though the contained JavaScript was so simple it HAD to work, it wasn’t working.

Eventually (and it took a while), I discovered this little paragraph in the XHTML specification:

“Given an empty instance of an element whose content model is not EMPTY (for example, an empty title or paragraph) do not use the minimized form (e.g. use <p> </p> and not <p />).”

So guess what that means? I have to write

<script type="text/javascript" src="myscript.js"></script>

instead, because even though I don’t have anything between the open and close tags, the fact that it’s possible to have something between them means I’m not allowed to minimize the tag, even though that’s perfectly valid XML.

Grr. I can’t believe somebody decided that. I have no idea what they were thinking. Still, it didn’t matter how many books I read ahead of time, it was probably inevitable that I was going to miss that. The only good thing about making it was that I could then check one off in the Ten Canonical Errors column.

By this point in my career, though, I generally have a vague sense of how far along I am along the errors path. I’ll describe myself as about two errors in, or even as many as eight. I don’t think I ever consider myself past all ten, no matter how long I’ve been in the field. Still, if I can get past six or seven, I generally feel pretty comfortable using the technology, and I know that if a problem comes up I’ll eventually be able to figure it out.

These days I’d say I was around a 7 on Hibernate and JPA, maybe a 6 or 7 on Spring (it’s so big there are still so many things I haven’t done yet), maybe only a 4 or 5 on JSF, and certainly a 9 on basic Java and probably close to that on server side and advanced Java topics. I’ve been using this stuff for quite some time now.

On Groovy, though, I wouldn’t put myself beyond a 4 yet, though I’m making progress. I didn’t think I was much beyond a 2 on Grails, but the fact that it’s based on Groovy, Spring, and Hibernate means I’m learning fast.

In terms of time invested, I’ve been playing with Groovy since about January, so that’s most a year at this point. I’ve read the bulk of the Grails book, too, so I’m not completely unaware of how it works. Still, when I compare it to what I can do in Java, it’s easy to get intimidated. But Groovy is so much more fun, that the sheer enjoyment of using it keeps me going.

(The whole transition from Java to Groovy really helps me identify with my poor COBOL programmers in my Intro Java classes. That transition is brutal. Moving from Fortran to Java was very hard for me, too, but I made that transition way back in the mid 90’s.)

In the end, building my Grails version of my course mapping application took me less than a week, even though I could only work on it for a few hours after class each night. The bulk of that time has been on topics that had very little to do with Grails, too. I’ve been struggling the last couple of days with how to transform Groovy collections into JavaScript arrays (including the associations — it if was just a single class it would be easy) which I could then process for Google Maps. I think I have it working now, but it’s been quite a battle.

I was going to post that code here today, but it’s been a long day and this post has already gotten out of hand. Look for some good coding info starting tomorrow. 🙂

It’s hard to increment null

After watching Jason Rudolph build a Grails application around an existing, legacy database, I decided to see if I could do the same. My Google Maps application (see my earlier posts on the subject) gave me a reason to finally put my courses taught data into a database. I still need a way to insert new data into that database, in a more user-friendly way than typing raw SQL at the MySQL command line.

Of course, that’s an ideal application for Grails. The Grails scaffolding builds controllers and views that can display domain objects that map to database tables, update existing objects, and even insert new ones. At my stage of learning Grails, it’s very useful for me to let the automatic generators build the basic application and then gradually replace the scaffolding with code I like.

This is also a nice app to experiment with, because at the moment it’s got a whopping three tables in it. There are three obvious domain objects, too: Course, Client, and Location. I’ve also got two one-to-many relationships between them, in that a client has many courses, and a location also has many courses. Nice and simple, but at least more than just a Grails version of Hello, World.

In Grails, it’s easiest to develop if you don’t already have a database. In a way, it’s ideal for domain-driven design, because you start with your domain objects and their relationships, and everything flows from there.

If I was starting from scratch, my domain classes (in Groovy) would look like:

class Client {
    int id
    String name

    static hasMany = [courses:Course]

    String toString() {
        return name

for the Client, where the toString() override is so the relationship drop-down lists will be populated with just the name. Also, the hasMany closure establishes the relationship between Client and Course, at least on this end.

The Location class is

class Location {
    int id
    String city
    String state
    double latitude
    double longtitude

    static hasMany = [courses:Course]

    static constraints = {

    String toString() {
        return "${city}, ${state}"

The latitude and longitude fields are for the Google Maps application. The hasMany closure establishes the relationship between Location and Course. The constraints closure puts the various fields in the order listed, and ensures that neither city nor state can ever be empty. The resulting validation and highlighted error messages are part of the coolness Grails brings to the table.

Finally, the Course class is:

class Course {
    int id
    Date startDate
    Date endDate
    String title

    Location location
    Client client

    static belongsTo = [Client, Location]

    String toString() {
        return title

    static constraints = {

The Course uses the belongsTo closure to establish the relationships in the other direction. The constraints closure is still mostly just for ordering; I imagine I’ll expand it later.

Now, if I didn’t have an existing database, I’d start with this, generate all the scaffolding, and let Grails create the database schema. The thing is, though, I already have the database schema, and some data to go in it.

As Jason showed, I could just let the Hibernate Tools reverse engineer mapping files from my schema. Then I could edit the mapping files to make the existing schema fit my domain classes.

Still, I thought I’d just edit the database to fit the Grails conventions (cue ominous music in the background). I’m doing this application as much to learn Grails as to solve my problem. Even better, I already have a Groovy script that can repopulate the tables if necessary using my existing DAO layer, currently written in Java with Spring and JPA.

The first change I made from my earlier schema was to make the table names singular instead of plural. The rest was fine (or so I thought). Each table had a primary key called id of type int. The course table has two foreign keys, each of which follows the naming convention (location_id and client_id). All of the column names map to the field names in the classes above, with camel case replaced by snake case (i.e., the startDate field maps to the start_date column).

Everything looked good, so I called the various Grails generators (grails create-app, grails create-domain-class, grails create-controller, and finally grails generate-scaffold) to see what happened.

As an aside, I also edited my DataSource.groovy file to point to my MySQL installation and to get rid of the dangerous “create-drop” capability. I didn’t want to overwrite the data by accident.

Unfortunately, I missed a detail. Grails doesn’t just add in a primary key field called id. It also expects a column called “version” in the database, of some integer type. This column is used for optimistic locking, so that if two people are editing a column at the same time, the one to commit first gets their changes written. The person who commits afterwards gets an error telling them that their data is stale, so they can’t commit until they refresh their data to the current values.

I don’t have version columns in my tables. Or, rather, I didn’t. I figured rather than drop everything, regenerate the tables and repopulate them from scratch, I could just execute “alter table” statements to add the new column to each of the three tables. I went ahead and did that.

The application then came up just as it should. I could see all the data, insert new rows, and delete the ones I inserted. Kewl. I spent some time editing the displays to order the data, make the views more visually appealing, and add functionality to search the courses by title and/or year (thanks to Jason’s great Getting Started With Grails book for all that).

What I couldn’t do, no matter what I tried, was to edit an existing object and have it successfully commit the changes. The web pages said that the changes were successful, so the save() method was supposedly returning true. The console didn’t show any errors, either. But nothing was changing in the database.

I really had to dig to figure that one out, but the Red Sox were losing and the Yankees were winning, so I kind of needed something to distract me anyway.

As it turned out the “grails shell” was the key. The Grails shell is a command-line script that allows you to write Groovy code that works with the existing domain classes. I loaded a course, modified its title, and tried to save it.

The resulting several hundred thousands of lines of the exception stack trace (Grails really can generate reams of lines of exceptions) somewhere had the word “version” in it.

I opened up a MySQL shell and looked at my data. Lo and behold, the version columns I added were there, and the data type was int, but of course all of the entries were null. Not zero, but null.

The optimistic lock tries to increment the value in that column. As it turns out, it’s hard to increment null.

The answer was to put zeros in all the version values. Everything worked just fine after that.

There’s a lot more I can and will do with this, but I have to admit, it was an excellent learning experience. It’s a bit daunting to know, however, that not having a version column meant that I almost had to switch to mapping files.

Go to see Grails, learn about Hibernate

Today I finally got to see the two Grails presentations by Jason Rudolph at No Fluff, Just Stuff. It’s not really an exaggeration to say I decided to attend the conference largely because I knew he’d be there. I like the NFJS conferences, but since I’m a one-person company, the “budget” for the conference comes right out of my pocket. I have to really want to go. Not only that, but for the past two years I’ve attended during the March session. It feels like a bit of a luxury to go twice in a given year, especially when many of the presentations in the fall overlap those from the spring.

But Jason was going to be there, and not only is he a real expert, he’s also a big part of the reason I’m involved with Groovy and Grails in the first place. I blogged about it a long time ago, but about a year ago I happened to be teaching a Spring class in Philadelphia. I happened to notice that the local Spring users group was hosting Jason, talking about this Grails thing I’d only heard about.

At the time I’d gone through my burst of enthusiasm and subsequent disappointment about Ruby on Rails. By that summer I’d come to the conclusion that Ruby was just too much and too different for me to make for a practical transition, not to mention the fact that I knew I wasn’t going to be able to follow their conventions and that was going to be a problem. Still, spending all that time digging into both Ruby and Rails taught me a ton. That was the first I’d heard about closures, for example. I also finally saw the power of a dynamic language, which ironically made me appreciate JavaScript for the first time, just in time for the Ajax revolution.

I guess you could say I was the perfect audience for Jason’s presentation. The fact that I could see all the Rails-like productivity gains and still be able to leverage all my Java experience was worth diamonds to me.

Today, he gave that introductory talk again, and followed up with a more advanced discussion of domain modeling with GORM, Grail’s Object-Relational Mapping framework. The introductory part was a nice refresher, but I was familiar with most of that material.

The advanced talk was very interesting, however. I needed to see how to work with an existing database schema, because that’s real life (despite what the Rails people seem to think). I knew in principle that since Grails just uses Hibernate under the hood, you could just fall back to Hibernate mappings to do whatever you needed. I’d never actually seen that in action, though.

But here was the surprising part. At one point Jason asked the audience if anyone had used the Hibernate Tools project to reverse engineer a database schema. I said I had, and I thought I had, but apparently I really missed the boat on that one.

I’ve tried the reverse engineering process inside an IDE. I originally tried it with MyEclipse, then I tried JBoss IDE. In both cases I felt like the wizard generated as much work for me as just writing the mapping files myself. I never seemed to get the hang of them. I hated seeing Java classes show up that had attributes like “userses” due to some naming convention. Since MyEclipse had content assist on the XML mapping files, it felt easier for me to just write the raw XML.

What Jason showed, however, was an intermediate step that hadn’t occurred to me. He didn’t go from the database schema all the way to the Java classes, which the wizards wanted me to do. Instead he used an Ant build to generate just the Hibernate mapping files and then edited them to suit his domain classes.

Imagine the sound of a light finally dawning. Here’s clue: Star Trek: the Original Series, first season episode “What Are Little Girls Made Of?”, quote by Ruk, the android left by the Old Ones:

“THAT was the equation! Existence! Survival … must … outweigh programming!”

In other words, I get it now.

I tried it with one of my own database schemas this evening, and of course it worked like a charm. I certainly don’t mind editing the resulting XML mapping files, and, even better, I can recommend the practice for clients who have dozens or hundreds of tables.

Thanks, Jason. 🙂 Now I can get back to really learning the framework.

%d bloggers like this: