Categories
Groovy

Upcoming Events, and “The Streak”

I’m really not a workaholic. I prefer days off as much as anybody. The problem is that there are all these things I want to do, so I volunteer to do them, and suddenly I’m overbooked so much I don’t have time for a break.

I think part of it comes down to an acronym that’s making the rounds these days. It’s called FOMO, and stands for Fear Of Missing Out. I suffer from it as much as anybody, I guess. I’ll need to work on that.

At any rate, I’m beginning a particularly busy part of my calendar, and I thought I’d mention it here in case I have the opportunity to meet any of you along the way.

On Tuesday, May 26, I’m giving my Groovy Vampires talk at the Austin Java Users Group. I really have a lot of fun with that talk. I plan to update it a bit, especially because a recording of the version I gave at last year’s SpringOne 2GX conference is currently available on InfoQ.

To give you a clue what it’s about, one day I was wandering in a Barnes and Noble and noticed that there was only one book shelf labeled “Computer”:

ComputerBooks

while there were three (yes, three) labeled “Teen Paranormal Romance”:

TeenParanormalRomance

Instead of lamenting the decline of western civilization, I decided that what this meant was that I needed to add some Groovy vampires to my book. The talk is based on that, and involves accessing RESTful web services, parsing JSON data, using a MongoDB database, creating a RESTful service using ratpack and Grails, and more.

The next day, May 27, I’ll be speaking at the Austin Groovy and Grails Users Group, this time on Testing Grails. While the bulk of the talk was created using Grails 2.3 and works under 2.4, I do plan to say a few words about Grails 3.0 as well. The testing mechanisms haven’t changed much in going from 2.3 to 2.4 to 3.0, though now they’ve added Geb as an included dependency, so you can make functional tests easily.

If you’re in the Austin, TX area this week, please drop by and say hi.

Also this week, starting tomorrow I’ll be hanging out at JavaRanch, because my Groovy Fundamentals video course will be featured in the Groovy forum this week. I’ve always been a fan of the ranch. I first joined way back in the late 90’s, when I was getting Java certified for the first time.

Speaking of the Groovy Fundamentals video (did I really need to link to that again? Yes. Yes I did), that’s the first of three Groovy video courses I recorded for O’Reilly recently. It covers the basics, from POGOs to collections to closures, in almost exactly four hours.

The second video course in the series is called “Practical Groovy”, and covers topics like integrating Java and Groovy, runtime metaprogramming, using AST transformations, the Groovy SQL class, and more. That one ought to go live within a week.

The third video in the series is called “Mastering Groovy”. It covers Spock testing (though I use Spock tests in the earlier videos), working with Spring, JPA, and NoSQL databases, functional programming, traits, and RESTful web services, among other topics. That one will go live when I finally finish reviewing all the video segments. It’s rather dismaying to discover that reviewing six hours of video takes at least six hours.

Though I must admit I’m tired of watching and listening to myself, I’m very proud of these videos and hope they’ll be helpful. I used to joke about selling the movie rights to Making Java Groovy, and speculate about who would I would recommend to play me.

(Best suggestion: Vincent D’Onofrio from his Men In Black days; most definitely not his Kingpin days.)

In essence, these video courses are that movie. They capture pretty much everything I’ve learned about Groovy over the years, condensed to an easily digestible form. If you’ve ever seen me give Groovy talks at No Fluff, Just Stuff conferences, you’ve seen a lot of what I’ve included, though with fewer jokes.

That brings me to next weekend, which is the Dallas NFJS event. I’ll be giving a Spock talk, and my “Managing Your Manager” talk, and a series on Android development. Again, if you’re in the neighborhood, please drop by and say hi.

Incidentally, some time this week (tentatively Wednesday, 5/27), Peter Ledbrook, Baruch Sadogursky, and I are planning to do another Groovy Podcast. I love doing those, so assuming that happens I’ll definitely tweet about it.

From Dallas I’m off to San Diego, where I’ll be teaching a (private) Groovy and Grails class. Teaching technical training classes is What I Do, practically my raison d’être, so feel free to contact me for details.

After the Grails class I’m heading to Sebastopol, CA, home of O’Reilly, to get back into the recording studio again. This time I’m working on a couple of Android videos, and if I manage to finish those I’ll also try to record something on Gradle for Android. That will all be the same week that culminates in the Gradle Summit in Santa Clara, where I’m doing an updated talk on the Gradle plugin for Android. I’m really looking forward to that conference, though I may miss the first day if we’re still recording.

(Yes, I’m making progress on the Gradle for Android book. Yes, it would be a lot easier if the Android plugin for Gradle, Android Studio, and even Android itself stopped changing so much so frequently, causing me to have to constantly rewrite chapters. Yes, the video course will be related, and will help me write the book. Yes, I’ll probably scowl if you ask me for an actual release date, but don’t let that stop you.)

When the Gradle Summit ends, I finally get to go home again, at least for a few hours, before I’m headed to South Carolina for another training class. I might have another one after that, too, but I haven’t decided.

Eventually I’m going to need a break, so it’s a good thing I scheduled one. Next year (!) my wife and I decided to go on the JoCo Cruise in the Caribbean, which is a sweet nerd cruise featuring Jonathan Coulton, Wil Wheaton, Paul and Storm, and many others. That really ought to be fun.

Finally, I need to say something about “The Streak”. Like many people in the I.T. field, I was a career changer. I came from engineering. More accurately, I should say that I was a research scientist, specializing in the field of turbomachinery aeroacoustics. What that really meant was I did lots and lots of math and wrote lots and lots of Fortran (the nightmares have ended, but it took a while). Ultimately I joined an AI group, went back to school at night, got my MS in CS and decided to leave my job.

My new job as a technical trainer started May 31, 2000. That day I helped teach a training course in Object Oriented Concepts (remember when did stuff like that?). I spent five years teaching classes (including roughly one Intro Java course a month back then) before I went out on my own in March of 2005.

We’re coming up on May 31, 2015, and in all that time, I have never missed a day of training. Not one. I call that The Streak, and since it looks like I’m going to make it to that date I figured it was okay to announce it here.

That journey, and the life changes that accompanied and preceded it, deserve their own blog post. I didn’t want the date to pass, though, without mentioning it. I’m rather inordinately proud of The Streak. Some of it is certainly luck, and it can’t last forever, but it means a lot to me. Right now my job and my career are everything I ever dreamed they could be, and I think The Streak is a side-effect. At the very least, it always gets me up in the morning. 🙂

Categories
Groovy

Groovy posts in other places

Recently I’ve been writing about Groovy and Grails for my friends at Accelebrate. I do that because:

  1. They’re a great client
  2. They support what I do
  3. They pay me CASH MONEY

The only problem is, whenever I post there, I don’t post here. I thought, therefore, that I’d add some links here to let everyone know what I’ve been writing about on their blog.

  • Calling RESTful services with Groovy is all about using Groovy to invoke Flikr’s API for the reason the Internet was invented: Cat Pictures.
  • That Which We Call A POGO, By Any Other Name, is a historical review of POJOs and POGOs, with Groovy AST transformations, and lots and lots of links to wildly unlikely resources that form the word POGO.
  • Building a Geolocation Web Application with Groovy and Grails was originally entitled, “Where in the World is Steve Heckler?”, but they changed it. In that post, I describe writing a Grails app to display locations on a map. (It helps to know that Steve Heckler is the owner of Accelebrate.) The best part is that I even managed to use pictures of Steve as custom markers on the resulting Google Map. 🙂
  • Kicking AST and Taking Names is all about using Groovy’s abstract syntax tree (AST) transformations to dramatically simplify coding. That sounds dry, but it does have figures that reference CaddyShack, South Park, and even Gladiator, so it’s got that going for it. Which is nice.

So if you’re enjoying any of my posts and you’re wondering why I haven’t been more active here, there you go. And wherever you go — (waits a beat) — there you are.

Categories
Groovy

If a method arg is a closure, for crying out loud pass it a closure

This is a (mildly) embarrassing post, because it demonstrates how in my transition to functional programming I missed something important. I get it now, though, and I might as well help others avoid the same mistake I made.

I teach a lot of Grails training classes, and one question that always comes up is how to map to existing databases rather than generate the schema from the Grails application. Hibernate (and therefore Grails) would love to generate the database schema for you, for a variety of reasons, but mostly because going the other way isn’t unique. You have to make a lot of decisions about a domain model when you reverse-engineer an existing schema, ranging from multiplicities in relationships to how to map many-to-manys to directionality and more.

In my standard Grails course, rather than create a database we can map to, I go with an existing, freely available sample. For years I’ve chosen the Sakila sample database from MySQL.

(As an aside, it took me years to realize that the word Sakila is actually intended to be a subtle joke. It’s the letters SQL with some vowels thrown in to make it possible to pronounce the word, with the “Q” replaced by a “k”. Go figure. As it turns out, it’s also the name of their dolphin emblem, but that’s neither here nor there.)

Over the years I’ve tried to map the Sakila schema to a Grails domain model, with varying degrees of success. I think I have it all worked out now, but that’s the subject for another blog post. Here, instead, I want to talk about stored procedures.

Many of my clients want to use Grails with a database that has lots of stored procedures in it. Hibernate doesn’t like that. Hibernate wants to map all the domain classes directly to tables, without going through any possible intermediaries. While it is possible to invoke a stored procedure from Hibernate, it’s a bit awkward, mostly because Hibernate doesn’t want to do it.

(Length of Java Persistence Using Hibernate: about 850 pages. Number of pages that discuss stored procedures: about 5.)

That’s where Groovy (as opposed to Grails) comes in. The Groovy API includes the wonderful groovy.sql.Sql class, which is a simple, but effective, façade over JDBC. That class will open and close connections, eliminate all the annoying boilerplate code associated with JDBC, and more. If I ever have to write raw SQL code, I always switch to groovy.sql.Sql rather than use JDBC.

(That’s not quite true. The JdbcTemplate class from Spring is nearly as helpful. If I don’t have Groovy available and I have to do JDBC, I go with that.)

Let me get specific. The Sakila database represents a video store (which shows how old that is). It has tables representing stores, and inventory, and actors, and films, and more. But it also contains a handful of stored procedures, one of which is called film_in_stock. According to the documentation, the film_in_stock procedure takes two input parameters (the id of the film and the id of a store) and one output parameter (the number of copies of that film currently at that store). The example they show is:

[sourcecode language=”sql”]
mysql> call film_in_stock(1, 1, @count);
+————–+
| inventory_id |
+————–+
| 1 |
| 2 |
| 3 |
| 4 |
+————–+
4 rows in set (0.06 sec)

Query OK, 0 rows affected (0.06 sec)

mysql> select @count;
+——–+
| @count |
+——–+
| 4 |
+——–+
1 row in set (0.00 sec)
[/sourcecode]

There are four copies of film 1 (“Academy Dinosaur”) at store 1 (a store at “47 MySakila Drive” in Alberta). This provides me with a test case:

[sourcecode language=”groovy”]
import spock.lang.*

class StoreProcServiceIntegrationSpec extends Specification {
def storedProcService // inject the service

void "call film_in_stock stored procedure"() {
expect:
storedProcService.callFilmInStock(1, 1) == 4
}
}
[/sourcecode]
This assumes my call to the stored procedure happens in a class called StoredProcService. I have to use an integration test here, because I want to use the real database, but that means I can rely on Spring dependency injection to get the service into the test.

To access the Sakila database from a Grails app, I configured my conf/DataSource.groovy file to include the dataSource properties:
[sourcecode language=”groovy”]
import org.hibernate.dialect.MySQL5InnoDBDialect

dataSource {
pooled = true
jmxExport = true
driverClassName = "com.mysql.jdbc.Driver"
username = "root"
password = ""
dialect = MySQL5InnoDBDialect
}
// …
environments {
development {
dataSource {
dbCreate = "validate"
url = "jdbc:mysql://localhost:3306/sakila"
}
}
test {
dataSource {
dbCreate = "validate"
url = "jdbc:mysql://localhost:3306/sakila"
}
}
// …
}
[/sourcecode]
(Yeah, I have super high security on this database. Don’t even think about breaking in. You’d never guess that the root user has no password whatsoever. I mean, who would do such a ridiculous thing?)

So the default data source in my Grails app is the Sakila database, using a MySQL driver that I have listed in the conf/BuildConfig.groovy file:

[sourcecode language=”groovy”]
dependencies {
runtime ‘mysql:mysql-connector-java:5.1.29’
test "org.grails:grails-datastore-test-support:1.0-grails-2.4"
}
[/sourcecode]
Again, nothing terribly unusual here.

The key observation is that the dataSource reference in the config file refers to a Spring bean, which can be injected in the usual manner. So I created a Grails service called StoredProcService and specified the dataSource bean as a dependency.

[sourcecode language=”groovy”]
@Transactional
class StoredProcService {
def dataSource
// …
}
[/sourcecode]

The GroovyDocs for the Groovy API show that the groovy.sql.Sql class has a constructor that takes a data source, so I’m all set. Then, to call the stored procedure, I need to invoke the call method from that class.

This is where life gets interesting, and ultimately tricky. The procedure I want to call has both input and output parameters, so my call has to take that into account. It turns out that the way to call a stored procedure with both inputs and outputs is to invoke:

[sourcecode language=”groovy”]
void call(String sql, List<Object> params, Closure closure)
[/sourcecode]

The SQL string uses the JDBC escape syntax for stored procedure calls. The params list take the input parameters and placeholders for the output parameters, and the closure is then invoked with the output parameters.

Wait, what? That’s a little fast. Here’s the actual call, or at least this is the version I had before last weekend:

[sourcecode language=”groovy”]
import groovy.sql.Sql

@Transactional
class StoredProcService {
def dataSource

int callFilmInStock(filmId, storeId) {
String sqlTxt = ‘{call film_in_stock(?,?,?)}’ // JDBC escape syntax
Sql db = new Sql(dataSource) // use the injected datasource
db.call(sqlTxt, [filmId, storeId, Sql.INTEGER]) { count ->
// use the count here
}
}
}
[/sourcecode]

The Sql class provides constants like “static OutParameter INTEGER” to represent output variable in the stored procedure call. Then the last argument is a closure that has as many arguments as there are output parameters (in this case, one, which I called count).

Now, though, I have a problem. I’d like my callFilmInStock method to return the result of invoking the film_in_stock stored procedure, i.e., the count of films at that store. The problem is, I can’t just say return count inside that closure, because returning from a closure returns only from the closure, not the calling method.

I’ve blogged about that before, and even submitted a variation to Baruch Sadogursky as a Groovy Puzzler, but this is where I first encountered that problem.

My solution was always to create a local variable outside the call, assign the count to that variable, and return the variable:

[sourcecode language=”groovy”]
int callFilmInStock(filmId, storeId) {
String sqlTxt = ‘{call film_in_stock(?,?,?)}’ // JDBC escape syntax
Sql db = new Sql(dataSource) // use the injected datasource
int result = 0
db.call(sqlTxt, [filmId, storeId, Sql.INTEGER]) { count ->
result = count
}
return result
}
[/sourcecode]

This works, and (sigh) I’ve been teaching this approach for some time now. I’ve always been uncomfortable with it, though, and when I started digging into Java 8 lambdas I realized why. I’m using a closure here and relying on it having a side effect, which is to modify a variable defined outside the closure. In fact, in Java 8, lambdas are definitely not supposed to do that. The only variables lambdas are supposed to access from outside the lambda are those that are “effectively final”, meaning they’re never modified. Groovy closures don’t have that limitation, but I still had this nagging feeling that I was, as they say, doing it wrong.

Last weekend, the new NFJS season started for me in Minneapolis, and I got a chance to talk about this to the inimitable Venkat Subramaniam, who writes books on this stuff (among other things). After I explained the problem, he thought about it for about five whole entire minutes and said that I should be passing the method a closure, not returning an integer.

The right way to write my service function is to add a closure to the method signature and use it in the sql call:
[sourcecode language=”groovy”]
void callFilmInStock(filmId, storeId, closure) {
String sqlTxt = ‘{call film_in_stock(?,?,?)}’ // JDBC escape syntax
Sql db = new Sql(dataSource) // use the injected datasource
db.call(sqlTxt, [filmId, storeId, Sql.INTEGER], closure)
}
[/sourcecode]

Wow, that’s so much simpler. The client now passes in a closure that uses the returned value, rather than trying to kludge returning the value itself. That means the test case becomes:

[sourcecode language=”groovy”]
void "call film_in_stock stored procedure"() {
expect:
storedProcService.callFilmInStock(1, 1) { count ->
assert count == 4
}
}
[/sourcecode]
(This uses the normal Groovy idiom where a method taking a closure as its last argument can put the closure after the parentheses.)

This test passes just as the previous one did, even though the method return type is now void.

That’s the part I never got before. The groovy.sql.Sql.call(String, List, Closure) method takes a closure as its last argument, which means I can supply the closure myself and the method will use it with the returned count.

When I learned about functional programming, I got the idea of transforming collections rather than iterating over them, and using filters to select only the values I want, and reducing the results down to single values. I got that I’m supposed to favor immutability wherever possible. What I apparently hadn’t grokked was sending closures as method arguments and how that simplified everything. I’m still not sure I’m all the way there, but I’m certainly a lot closer now.

Over the weekend I tweeted that Venkat had fundamentally changed my understanding of functional programming, which is both cool and seriously annoying. Frankly, those are also words I would use to describe Venkat. If he wasn’t such a nice guy, he would be insufferable, mostly due to sheer jealousy. But we rapidly learn on the NFJS tour never to compare ourselves to Venkat in any way or you’ll just drive yourself crazy. Just don’t tell him I said that.

Now I have to go update my Grails materials…

Categories
Grails Groovy

Groovy/Grails – Pivotal == Opportunity

The news broke this morning that Pivotal plans to withdraw its financial support from the Groovy and Grails projects by the end of March, 2015. The heads of both projects, Guillaume Laforge and Graeme Rocher, have each blogged about it, with their typical grace, thanking Pivotal for the opportunity and assuring everybody that both projects have a long, bright future ahead of them.

Since I.T. is a field that frequently has very little memory, let me remind people about a couple of items:

  1. Groovy existed as a successful open source project for at least four years before SpringSource started supporting it. Grails started as an offshoot of Groovy and was just as popular.
  2. Several of the core teams members of both projects formed the G2One company, which was sufficiently successful in its first year that SpringSource acquired it in the first place

Neither Groovy nor Grails are radical departures from their underlying technologies. That makes them quiet projects — they’re popular, but they aren’t showy. They never have been popular among the hipster IT community, but they get the job done. It’s so easy to add Groovy to existing Java projects that most developers who do it don’t feel obligated to crow about it. Groovy is being used throughout the Java industry, and in most places it’s just acknowledged without comment. Grails, for all its power, feels like an easy way to build traditional Spring/Hibernate web applications, which work really well but are hardly sexy.

It’s therefore not surprising that the potential of Groovy and Grails is often underestimated. I have no idea what Pivotal was thinking, and the public statements about them have been uniformly positive (again, not a big surprise given the people involved), but I find it easy to believe Groovy and Grails were underrated yet again.

Many people will now write many posts demonstrating, yet again, how powerful Groovy is and how easy it is for Java developers to learn. The Grails web site already has dozens of success stories. I’m not going to try to add to that, other than to say I love working with both technologies and they’ve completely changed my life.

I want to mention something else. I’ve been in the IT industry for over 20 years. I was in the engineering community for a dozen years before that. I would hold up my academic background against anybody’s. From all those experiences, I’ve known geniuses and incredibly hard workers, and communities who are friendly, hostile, and everything in between.

I’m here to tell you that the Groovy and Grails core teams members are among the brightest people I’ve ever met, but they’re also successful because they’re wonderful individuals to be around. In an industry often marred by what I like to call “boys behaving badly,” I’ve never heard a negative word about anyone involved in Groovy and Grails. That attitude starts at the top, with great people like Graeme and Guillaume, and I feel privileged to know and work with them, however tangentially.

Look, community matters. It makes the difference between struggling to accomplish anything and enjoying your job. I teach technical training courses for a living, and you should see the joy in Java developers eyes when they learn both how much they can now do, and — this is important — how newcomers are treated with respect and how their questions are answered without patronizing or humiliating attitudes.

They say it’s hard for companies to find good developers these days. Well, here’s your opportunity. For the price of supporting technologies that will inevitably help your bottom line, you can acquire a team of coders that are among the most accomplished, most dedicated, and most easy to work with that you’ll ever meet.

Pivotal is now out of the picture. This is a great opportunity for a smart company to step in, acquire two fantastic teams of developers, and be a hero to the open source community. Heck, I’d do it myself if I could afford it. Don’t miss this chance. They don’t come along very often.

Categories
Groovy

SpringOne2GX, Day Minus 3

Yes, that’s a minus sign, because the event in question hasn’t happened yet. This is my first report from the upcoming SpringOne2GX conference, which starts in Dallas, TX next Monday, 9/8/14 (or 8/9/14 for my non-US-based friends). Since there are no talks on the opening day (just an opening keynote and reception), I’m arbitrarily designating that as Day Zero. I’m giving several talks at the conference and I thought I’d comment here on my preparations. I also have a kind of “insider” status, being a member of the No Fluff, Just Stuff conference tour (one of the organizers of SpringOne2GX*) and, as the author of Making Java Groovy, I know a lot of people in the community. That will hopefully give me opportunities to share amusing anecdotes and other non-presentation-based tidbits of information here.

*What is the proper hashtag for the conference? In past years people have recommended #springone2gx, but that’s way too many characters. The value #s12gx seems reasonable, but didn’t seem to catch on. Most people I know used #s2gx, which is probably what I’ll adopt.

Speaking of embarrassing anecdotes, let me give you one right away. Two years ago, after one of my talks at that year’s s2gx conference, Guillaume Laforge and Cedric Champeau came up to me to say hi. Guillaume, who lives near Paris, has an outrageous French accent. Every time I talk to him I keep thinking of this character from the classic movie. So I finally got up the nerve to ask him, “Just once, would you say to me, ‘your mother was a hamster, and your father smelt of elderberries!'”.

He and Cedric both just stared at me like I’d lost my mind. They had no idea what I was talking about. It wasn’t until later I realized that even if they knew Monty Python and the Holy Grail (a minimum requirement for any decent developer), they knew it in French.

Oh well.

This week I’m still polishing (yeah, right, polishing — not writing from scratch, at least as far as you know) my slides. I’m scheduled to give the following talks:

This is an introduction to Grails, combining domain classes (of course, Quest, Task, Knight, and Castle), controllers, services (using the Google geocoder), and more. Experienced Grails developers can comfortably skip it, but newbies will no doubt appreciate the introduction. I plan to add in some architectural discussions, as well as some Hibernate and/or Spring stuff if time allows.

I gave a similar talk a couple years ago, but things have changed significantly since then. As a minimal example, now the default unit testing framework is Spock, rather than the JUnit subclasses we used to use. Everything is annotation based, too. I’ll definitely talk about testing controllers, services, and domain classes (constraints), and plan to do a couple of integration tests as well, and then talk about various plugins, like the Build Test Data plugin and some other functional ones.

As I travel from company to company*, I rarely see the new REST capabilities in Grails 2.3+ being used. Last year Graeme Rocher did a wonderful presentation on them, as well as a brief discussion of the asynchronous capabilities in Grails. I hope to cover much the same (REST) ground, but rather than get into async I’m going to talk about customizing responses, using HAL for hypermedia, and so on.

*My day job is teaching technical training courses in pretty much all areas related to Java, specializing in open source topics like Spring, Hibernate/JPA, Android, Groovy, and Grails. That means I visit somewhere between 30 to 50 companies a year, and while I’m mostly talking to people who don’t know much about Grails yet, I’m always trying to find out what they are using. Between that and the NFJS conferences, I get a decent sense of what is becoming popular in the industry and what isn’t catching on.

As a sample, both Groovy and Grails are still growing at their normal slow, steady pace. Spring Boot is generating a lot of excitement among the more advanced developers, but most Spring users in the field haven’t even heard of it. I imagine this year’s conference will start to change that.

It turns out that Jeff Brown (co-author of DGG2, the Definitive Guide to Grails 2) is giving a talk entitled RESTful Grails 2 at 12:45pm on Tuesday. This is extraordinarily fortunate for me, because that means I can steal (er, reuse) whatever he says. Sweet. 🙂

I gave a similar talk last year, and was just getting ready when Paul King walked into the room. Paul King is one of the Greatest Groovy Experts Ever, and I experienced a flood of Imposter Syndrome as soon as he entered. Fortunately, he didn’t violently disagree with anything I said and was kind enough to share some additional information after it was over. Is it any wonder he is one of my personal heroes?

Another interesting Paul King tidbit: last time I checked, he had more commits to the Groovy code base than anyone, ever. Heck, I just checked and he did six more today.

I’ve updated the talk with new information, which should be fun. The toughest problem I’ll have is not giving away spoilers for the Groovy Puzzlers talk coming later in the conference.

(Go to the puzzlers talk. It’s fun.)

This is a big one for me. The fact that Android applications are now built using Gradle is a big deal for every Android developer, and the Android plugin for Gradle is still evolving at a rapid rate. The new IDE, Android Studio, is also changing all the time. My hope is to give the most up-to-date information on it possible. I’ll show the multi-project build they use by default, add some dependencies, run some tests (using the Robotium plugin), talk about Spring Android, and show some flavors and variants.

Best of all, I expect to try to update Android Studio right before the talk, so I have the latest possible version in the so-called Canary channel. What could possibly go wrong?

This is a revised and extended version of the same talk I gave at the Gr8 conf in Minneapolis in July. Those talks were limited to an hour, and given that lots of speakers were talking about Ratpack*, I decided to focus on Grails for the REST part. Now I should have time for both.

*Dan “Tiger” Woods is giving his Ratpack Web Framework talk at 2:30pm on Tuesday.

Wait, let’s try to improve that:

[code language=”groovy”]
[‘Tiger’,’James’,’Ickey’,’Mirk’,’Sher’].each { nickname ->
println "Dan ‘${nickname}’ Woods"
}
[/code]

Let me know if you have some better suggestions. Also, Lari Hotari is giving a talk on Ratpack and Grails 3.

Since there are no images in this post, here’s a small spoiler from the Groovy Vampires talk. Note the nice subtitle.

Making Java Groovy: Still a better love story than Twilight

If you’re coming to the conference, please drop by and say hi. I hope to write a short blog post every day of the conference, commenting on what talks I attended and which people I spoke to, but that’ll depend on how much last minute “updating” I’ll need to do on my slides.

Categories
Grails

Using a codec in a Grails unit test

This is a small issue, but I encountered it and found a solution on the mailing lists, so I thought I’d document it here.

I was demonstrating a trivial Grails application in class today and decided to unit test it. The app has a single controller, called WelcomeController:

[sourcecode language=”groovy”]
class WelcomeController {
def index = {
def name = params.name ?: "Grails"
render "Hello, $name"
}
}
[/sourcecode]

When I deploy the application and access the Welcome controller (via http://localhost:8080/hellograils/welcome/index), it displays “Hello, Grails!”. If I append “?name=Dolly” to the URL, the result is “Hello, Dolly!”. All nice and simple.

I decided I wanted to write a test case for this, and lately I’ve been learning how to favor unit tests over integration tests as much as possible, mostly for speed. I therefore wrote the following tests:

[sourcecode language=”groovy”]
import grails.test.*

class WelcomeControllerTests extends ControllerUnitTestCase {
void testWelcomeWithoutParameter() {
def wc = new WelcomeController()
wc.index()
assertEquals "Hello, Grails!", wc.response.contentAsString
}

void testWelcomeWithParameter() {
def wc = new WelcomeController()
wc.params.name = "Dolly"
wc.index()
assertEquals "Hello, Dolly!", wc.response.contentAsString
}
}
[/sourcecode]

When I run the unit tests (i.e., grails test-app unit:), everything runs correctly.

One of the students pointed out that though this is a trivial example, it’s open to XSS (cross-site scripting) attacks. In the URL, replace “name=Dolly” with “name=alert('dude, you've been hacked')” and the embedded JavaScript code executes and pops up an alert box.

I knew that an easy solution to this would be to modify the index action in the controller to look like:
[sourcecode language=”groovy”]
class WelcomeController {
def index = {
def name = params.name ?: "Grails"
render "Hello, $name".encodeAsHTML()
}
}
[/sourcecode]

The “encodeAsHTML” method escapes all the HTML, so the output of the hack is just “Hello, alert(…” (i.e., the script is shown as a string, rather than executed) and the problem goes away.

The issue I encountered, though, is that my unit tests started failing, with a missing method exception that claimed that the String class doesn’t have a method called encodeAsHTML. That’s correct, of course, because that method is dynamically injected by Grails based on the org.codehaus.groovy.grails.plugin.codecs.HTMLCodec class. In a unit test, though, the injection doesn’t happen, and I get the exception.

One solution to this, as pointed out on the very valuable grails-users email list, is to add the method to the String class via its metaclass. In other words, in my test, I can add

[sourcecode language=”groovy”]
void setUp() {
super.setUp()
String.metaclass.encodeAsHTML = {
org.codehaus.groovy.grails.plugins.codecs.HTMLCodec.encode(delegate)
}
}
[/sourcecode]

Now the String class has the encodeAsHTML method, and everything works again.

Then I started browsing the Grails API, and found that in ControllerUnitTestCase there’s a method called loadCodec. The GroovyDocs weren’t very informative, but I found in the jira for Grails that issue GRAILS-3816 recommends the addition of the loadCodec method for just this sort of purpose.

That means that I can actually write
[sourcecode language=”groovy”]
void setUp() {
super.setUp()
loadCode(org.codehaus.groovy.grails.plugins.codecs.HTMLCodec)
}
[/sourcecode]
and everything works as it should. Since this isn’t terribly well documented, I thought I’d say something here. Hopefully this will save somebody some looking around.

Categories
Grails

Minor bug in Grails selenium plugin

There is a minor bug in the selenium plugin for Grails. It has been discussed on the mailing list, but I thought I would also document it here. I’m using Grails 1.0.3 with version 0.4 of the selenium plugin, which wraps selenium core version 0.8.3.

If you install the plugin and try to run the new Grails tasks (create-selenium-test, create-selenium-domain-test, or run-selenium), the system fails with the error

Could not execute method appContext
No such property: appContext for class: ...

It turns out that the way the appContext variable is handled in Grails has changed, and the plugin hasn’t yet updated to accommodate it.

The recommended fix is to go into the “plugins/selenium-0.4/scripts” directory in your Grails project, and replace ${appContext} with ${grailsAppName} in all the Groovy scripts.

If you use the Selenium IDE in Firefox to write your tests, then the fix doesn’t matter until you try to use the run-selenium target. Still, might as well fix it everywhere. This came up on the mailing list back in July, so I assume an updated plugin will be available as soon as the author gets around to it.

I also find that adding the path to firefox.exe to my system path (default is “c:\Program Files\Mozilla Firefox” — don’t you hate Windows paths with spaces??) makes it easier to run the test suite.

Incidentally, unlike Canoo Web Test, you have to make sure your web app is running before executing the test suite. Selenium fires up the browser and puts it through the proper sequence, so the app has to be running first.

I’ve decided that rather than choosing between Selenium and WebTest, I actually like using both. The Selenium IDE in Firefox is really slick and easy to use, while the output reports in WebTest are gorgeous.

Categories
Grails Groovy

Getting a list of Grails plugins programmatically

In September, I’m very happy to be giving a couple of presentations at the No Fluff, Just Stuff conference in the Boston area.  One of my presentations is a review of the various available Grails plugins.  To prepare for that, I thought I’d create a Grails application that acted as a survey, so people could rate the plugins they like.

One task is to get a list of available Grails plugins.  I wanted to do that programmatically, too, because I’d like to update the list automatically using the Quartz plugin (of course).

How do you get a list of available plugins?  My first thought was to do the HTML equivalent of screen scraping at the main plugin site, http://grails.org/Plugins .  At that site everything is nicely divided into categories, along with links to descriptions and more.

Screen scraping HTML is not fun, though.  I’ve done it before, when necessary, but it’s not very robust and tends to run into problems.  Many of those problems have to do with the fact that HTML is a mess.  Most web sites are filled with HTML that isn’t even well-formed, making processing it programmatically a real pain.

GinA, however, mentioned HTTPUnit as an easy way to access a web page.  Since it’s a regular old Java library, that meant I could use it with Groovy.  Therefore, my first attempt was:


import com.meterware.httpunit.WebConversation

def baseUrl = 'http://grails.org/Plugins'

def wc = new WebConversation()
def resp = wc.getResponse(baseUrl)

Unfortunately, I’m already in trouble even at that point.  If I run that, I get a massive exception stack trace with shows that the included Neko DOM parser choked on the embedded prototype JavaScript library.

While I was debating what to do about that (I really didn’t want to just open the URL, get the text, and start having Fun With Regular Expressions), I noticed a blog posting here, from someone named Isak Rickyanto, from Jakarta, Indonesia.

(A Java developer from Java.  How cool is that?  Or should I say, “how Groovy?” :))

Isak points out that there is a list of Grails plugins at http://svn.codehaus.org/grails-plugins/ .  As a Subversion repository listing, it’s not full of JavaScript.  Even better, every plugin is listed as a simple link in an unordered list.

I therefore modified my script to look like this:


def baseUrl = 'http://svn.codehaus.org/grails-plugins/'

def wc = new WebConversation()
def resp = wc.getResponse(baseUrl)
def pluginNames = []
resp.links.each { link ->
    if (link.text =~ /^grails/) {
        def name = link.text - 'grails-' - '/'
        pluginNames << name
    }
}
println pluginNames

Here I’m taking advantage of the fact that the WebResponse class (returned from getResponse(url)) has a method called getLinks().  Since there was one link that had the name “.plugin-meta“, I decided to use a trivial regular expression to filter down to the links definitely associated with plugins.  The WebLink.getText() method then returned the text of the link, with gave values of the form

grails-XXX/

for each plugin.  One of the things I love about Groovy is that I can then just subtract out the characters I don’t want, which is how I added the actual plugin names to an array.

Unfortunately, while that’s part of what I want, that isn’t everything I want.  I’d like the version numbers and the descriptions, too, if possible.  I could go digging into the various directories and look for patterns, but a different idea occurred to me.

I finally remembered that the way I normally find out what plugins are available is to run the

grails list-plugins

command and look at the output.  You’ve probably seen it.  It gives an output like

Welcome to Grails 1.0.3 - http://grails.org/
Licensed under Apache Standard License 2.0
Grails home is set to: c:\grails-1.0.3

Base Directory: c:\
Note: No plugin scripts found
Running script c:\grails-1.0.3\scripts\ListPlugins.groovy
Environment set to development

Plug-ins available in the Grails repository are listed below:
-------------------------------------------------------------

acegi               <0.3>            --  Grails Spring Security 2.0 Plugin
aop                 <no releases>    --  No description available
audit-logging       <0.4>            --  adds hibernate audit logging and onChange event handlers ...
authentication      <1.0>            --  Simple, extensible authentication services with signup ....
autorest            <no releases>    --  No description available

etc.  So if I could get this output, I could break each line into the pieces I want with simple String processing.

How can I do that?  In the spirit of reducing it to a problem already solved, I realized I just wanted to execute that command programmatically and capture the output.  One way to do that is to take advantage of Groovy’s ability to run command line scripts (GinA covers this, of course, but so does Scott Davis’s most excellent Groovy Recipes book).  Here’s the result:


def names = []
def out = "cmd /c grails list-plugins".execute().text
out.split("\n").each { line ->
    if (line =~ /<.*>/) {
        def spaceSplit = line.split()
        def tokenSplit = line.split('--')
        def name = spaceSplit[0]
        def version = spaceSplit[1] - '<' - '>'
        def description = tokenSplit[-1].trim()
        names << name
    }
}

Basically I’m executing the list-plugins command at a command prompt under Windows (sorry, but that’s still my life), splitting the output at the carriage returns (for some odd reason, using eachLine directly kept giving me errors), and processing each line individually.  The lines listing plugins are the ones with version numbers in angle brackets (like <0.3>), and the descriptions came after two dashes.  It seemed easiest to just split the lines both ways in order to get the data I wanted.

I ran this script and the other script together to see if I got the same output.  Here’s the result:


println "From 'grails list-plugins': " + names
println "From svn repo: " + pluginNames
println "Difference: " + (pluginNames - names)

From 'grails list-plugins': ["acegi", "aop", "audit-logging", ..., "yui"]
From svn repo: ["acegi", "aop", "audit-logging", ..., "yui"]
Difference: ["extended-data-binding"]

Why the difference? From the list-plugins output, here’s the line for “extended-data-binding“:


ext-ui              <no releases>    --  No description available
extended-data-binding<0.2>            --  This plugin extends Grails' data binding ...

Yup, the name ran into the version number format.  Sigh. Of course, the other problem with this is that at the moment it’s dependent on my own system configuration (Windows, with the grails command in the path), which can’t be a good thing.

Finally, after all this work, I suddenly realized that I already have the script used to list the plugins.  As with all the other Grails commands, it’s a Gant script in the <GRAILS_HOME>\scripts directory called, obviously enough, ListPlugins.groovy.  According to the documentation at the top, it was written by Sergey Nebolsin for version 0.5.5.

What Sergey does is to go to a slightly different URL and then parse the results as XML.  His script accesses

DEFAULT_PLUGIN_DIST = "http://plugins.grails.org"

instead of the SVN repo location listed above, but if you go there, they look remarkably alike.  I wouldn’t be surprised if http://plugins.grails.org is simply an alias for the SVN repository.

Note that the script also creates a cached version of the plugin list, called plugins-list.xml, which is kept in the

"${userHome}/.grails/${grailsVersion}/plugins/"

directory.  That’s completely understandable, but a lousy location on a Windows box.  I never go to my so-called “user home” directory, so I would never occur to me to look there for information.

His script checks to see if that file is missing or out of date.  If it’s necessary to update it, he opens a URL and starts processing:


def remoteRevision = 0
new URL(DEFAULT_PLUGIN_DIST).withReader { Reader reader ->
    def line = reader.readLine()

...

    // for each plugin directory under Grails Plugins SVN in form of 'grails-*'
    while(line=reader.readLine()) {
        line.eachMatch(/<li><a href="grails-(.+?)">/) {
            // extract plugin name
           def pluginName = it[1][0..-2]

           // collect information about plugin
           buildPluginInfo(pluginsList, pluginName)
        }

etc.

So, in effect, he’s screen scraping the SVN page; he’s just doing a better job of it than I was.

Incidentally, the line in his script that lead to my parsing problems is on line 86:

plugins << "${pluginLine.padRight(20, " ")}${versionLine.padRight(16, " ")} --  ${title}"

I could bump up the padding by one, or learn to parse the output better. 🙂 I expect the “right” answer, though, is to do what Sergey did, pretty much. Still, if all I have to do is add a little padding, it’s awfully tempting to just “reuse” Sergey’s existing script.

In an upcoming post, I’ll talk about how I used the RichUI plugin to apply a “star rating” to each entry so that people could vote. I don’t have the site ready yet, though. I’ll be sure to mention it when I do.

Categories
Grails

Silly GORM tricks, part III: SQL keywords as attributes

I was writing a very simple Grails application and ran into a problem when I accidentally used a SQL keyword as a property name. This post documents what happened, and how I (pretty easily) fixed it.

To illustrate the issue, consider a trivial Grails application called “messaging” with a single class called Message.


class Message {
  String from
  String to
  String text
}

This is supposed to represent a simple email message, with fields to represent the sender, the receiver, and the text of the message itself. It seemed quite logical at the time to use the words from, to, and text for the fields, but that leads to problems.

I added a MessageController with dynamic scaffolding (i.e., def scaffold = Message) and started the server. When I accessed the MessageController link, however, I got

org.hibernate.exception.SQLGrammarException: could not execute query

In order to see the actual problem, I modified my DataSource.groovy file to add “loggingSql = true” in the dataSource section. After restarting the server, in the console window I saw that the exception was caused by a SQL command generated by the scaffolded list method in MessageController:


Hibernate: 
    select
        this_.id as id0_0_,
        this_.version as version0_0_,
        this_.from as from0_0_,
        this_.text as text0_0_,
        this_.to as to0_0_ 
    from
        message this_ limit ?

This statement looks fine, and in fact there’s nothing wrong with it. I couldn’t see the problem until I switched to MySQL so that I could browse the database independently.

(Switching to another database is covered many places in the Grails literature. In short, it means adding the JDBC driver to the messaging/lib directory, creating the messaging database in MySQL, and changing the driverClassName, url, username, and password settings in DataSource.groovy.)

When I did that and checked the database with the MySQL client, I found the problem (or at least a symptom of it):

mysql> show tables;
Empty set (0.00 sec)

In other words, the problem was that the message table didn’t exist. Somehow the SQL used to generate the table in the first place didn’t work.

Logging the SQL as I did wasn’t sufficient to show me the CREATE TABLE statement. If, however, I go into Config.groovy and change the value of log4j.logger.org.hibernate to debug, I see in the resulting console:


[1125] hbm2ddl.SchemaExport 
    create table message (
        id bigint not null auto_increment,
        version bigint not null,
        from varchar(255) not null,
        text varchar(255) not null,
        to varchar(255) not null,
        primary key (id)
    )

followed immediately by

[1125] hbm2ddl.SchemaExport Unsuccessful: create table message ...
[1125] hbm2ddl.SchemaExport You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'from varchar(255) not null, text varchar(255) not null, to varchar(255) not null' at line 1

The problem is that the word “from” is a SQL keyword. MySQL got upset when I tried to use it in the create table statement, as shown above.

How can I fix this? I could change the name of the from property, to, say, sender. Since Grails 1.0, however, GORM now has the ability to do custom ORM mappings, which feels like a cleaner way to solve the problem. Therefore, I added the following closure to my class:


static mapping = {
  from column: 'sender'
}

Now the generated create statement is:


[1157] hbm2ddl.SchemaExport 
    create table message (
        id bigint not null auto_increment,
        version bigint not null,
        sender varchar(255) not null,
        text varchar(255) not null,
        to varchar(255) not null,
        primary key (id)
    )

and the new error is

[1157] hbm2ddl.SchemaExport You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'to varchar(255) not null, primary key (id))' at line 1

So apparently the word “to” is also a problem. I therefore modified the mapping closure to include it as well:


static mapping = {
  from column: 'sender'
  to column: 'receiver'
}

Now it all works as it should. The lesson appears to be either that I should keep a list of SQL keywords handy, or simply that the custom ORM DSL is a Good Thing(TM), and so are the logging settings. 🙂

Categories
Grails

Silly GORM tricks, part II: dependent variables

This post discusses a relatively simple topic in GORM: how to use dependent variables in a domain class. It’s simple in the sense that it’s been discussed on the mailing list, but I haven’t seen it documented anywhere so I thought I’d do so here.

I started with a simple two-class domain model that I discussed in my last GORM post.

class Quest {
    String name
    static hasMany = [tasks:Task]
    String toString() { name }
}
class Task {
    String name
    static belongsTo = [quest:Quest]
    String toString() { name }
}

As before, there is a one-to-many relationship between quests and tasks. A quest has many tasks, and the belongsTo setting implies a cascade-all relationship, so inserting, updating, or deleting a quest does the same for all of its associated tasks.

In Bootstrap.groovy, I also have:

def init = { servletContext ->
         new Quest(name:'Seek the grail')
            .addToTasks(name:'Join King Arthur')
            .addToTasks(name:'Defeat Knights Who Say Ni')
            .addToTasks(name:'Fight Killer Rabbit')
            .save()
}

which shows how the classes are intended to work together.

The first change I want to make is to give tasks a start date and end date. My first attempt is to just add properties with those names, of type java.util.Date.

class Task {
  String name
  Date start
  Date end
  // ... rest as before ...
}

This leads to a minor problem. If I start up the server, I don’t see any quests or tasks. The reason is that my bootstrap code tries to create tasks without start and end dates, which violates the database schema restriction. My generated schema marks both start and end columns as “not null”.

There are many ways to fix that. I can either assign both start and end properties for each task in my bootstrap code, or add a constraint in Task that both can be nullable, or do what I did here, which is to give them default values.

class Task {
  String name
  Date start = new Date()
  Date end = new Date() + 1
  // ... rest as before ...
}

I do have a constraint in mind, actually. I’d like to ensure that the end date is after the start date. That requires a custom validator, which is also pretty easy to implement:

class Task {
  // ...
  static constraints = {
    name(blank:false)
    start()
    end(validator: {  value, task ->
       value >= task.start
    })
  }
}

That works fine.

Now for the dependent variable. My tasks all have a start and an end, so implicitly they have a duration. I could add the duration variable to my Task class, but I don’t want to save it in the database. It’s dependent on the values of start and end. I also don’t want to be able to set it from the gui.

Here’s the result:

class Task {
  String name
  Date start
  Date end
  
  int getDuration() { (start..end).size() }
  void setDuration(int value) {}

  static transients = ['duration']

  // ... rest as before ...
}

This computes the duration from the start and end dates by returning the number of days between them. It relies on the fact that Groovy modifies java.util.Date to have the methods next() and previous(), and since Date implements Comparable, it can then be used in a range, as shown.

(As an aside, this implementation is probably pretty inefficient. If the number of days between start and end was substantial, I think this implementation executes the next() method over and over until it reaches the end. I thought about trying to subtract the two dates, but interestingly enough the Date class only has plus() and minus() methods that take int values, not other Dates. I considered adding a category that implemented those methods, but haven’t tried it yet. I’d like to look in the Groovy source code for the plus() and minus() implementations, but I couldn’t find it. I did find something similar in org.codehaus.groovy.runtime.DefaultGroovyMethods, but I’m not sure that’s the same thing. Sigh. Still a lot to learn…)

By putting 'duration' in the transients closure, I ensure that it isn’t saved in the database.

The getDuration method is pretty intuitive, but adding set method as a no-op is somewhat annoying. If I leave it out, then Groovy will generate a setter that can modify the duration. As an alternative, according to GinA I can also supply my own backing field and mark it as final:

class Task {
  // ...
  final int duration

  int getDuration() { (start..end).size() }
  // ...
}

Just to be sure, I added the following test to my TaskTests:

void testSetDuration() {
    Task t = new Task(name:'Join King Arthur')
    shouldFail(ReadOnlyPropertyException) {
        t.duration = 10
    }
   q.addToTasks(t).save()
}

That passed without a problem.

Interestingly, the dynamic scaffold still generates a modifiable input text field for duration, both in the create and edit views. I can put my own value in it and submit the form without a problem. The result does not get saved, which is correct, but I don’t see an exception thrown anywhere in the console. If I generate the static scaffolding, I know that in Task.save there is a line like

t.properties = params

which is how the form parameters are transfered to the object. Presumably the internal logic knows enough to avoid trying to invoke a setter on a final field. Of course, as soon as I generate the static scaffolding, I usually just delete that row in the GSP form.

There’s one final (no pun intended) issue with the dynamic scaffolding. The generated list view puts its properties in <g:sortableColumn> tags. This holds true for the duration, as well. Normally, when I click on the column header, the result is sorted, ascending or descending, by that property. If I click on the duration column header, however, I get an “org.hibernate.QueryException: could not resolve property: duration of: Task“.

It turns out that the User Guide has a “Show Source” link for every tag. When I clicked on that link for the sortableColumn tag, I saw near the top:

if(!attrs.property)
  throwTagError("Tag [sortableColumn] is missing required attribute [property]")

The error I got in the console is “could not resolve property”, but it’s possible this is the source of that issue. I’m not sure. The only other source (again, no pun intended) of the problem I could see was the execution of the list action at the bottom. That would imply that Grails is generating the Hibernate query and we’re failing at that point, which would be consistent with the error reported above.

At any rate, the duration property now works in the domain class. I can always modify the views to ensure I don’t try to set it.