Java 11 HttpClient, Gson, Gradle, and Modularization

This post describes a simple system that uses the new HttpClient class that comes with Java 11 to access a RESTful web service. After demonstrating basic functionality, changes are made to parse the results using Gson, and even modularize the code and have it work with Gradle. The goal is to provide an example for anyone who wants to do similar tasks with their own systems.

Java 11 is a Long Term Support (LTS) release for the Java standard edition, meaning bug fixes and security patches will be available for at least the next three years (see the extensive Java Is Still Free post at for details about the future licensing arrangements). Java 11 contains very few new features, however — many more were added in Java 9 and 10. Most of the industry did not upgrade during those releases, however. Now that another LTS release is available, that will likely change.

One of the few real additions to Java 11 is the HttpClient API, as described in JEP 321. This takes a related incubator project from Java 9 and standardizes it for Java 11. This blog post will not be an exhaustive demonstration of its capabilities, but it will show both a synchronous and asynchronous example of how to use it, along with JUnit 5 test cases and a Gradle build file, and even modularize the code.

The web service is provided by Open Notify, which is an open source project that provides a few links to data from NASA. In this case, the example uses the “number of people in space” link at http://api.open-notify.org/astros.json. The current response to that link is:

{"message": "success", 
 "people": [
    {"craft": "ISS", "name": "Oleg Artemyev"}, 
    {"craft": "ISS", "name": "Andrew Feustel"}, 
    {"craft": "ISS", "name": "Richard Arnold"}, 
    {"craft": "ISS", "name": "Sergey Prokopyev"}, 
    {"craft": "ISS", "name": "Alexander Gerst"}, 
    {"craft": "ISS", "name": "Serena Aunon-Chancellor"}], 
 "number": 6}

The root object contains attributes message, number, and people, the last of which is a collection of objects, each of which contains a name and a craft. As the response shows, at the moment there are six people aboard the International Space Station.

Even though the plan is to map that response to Java classes, at first the example will simply download the JSON data and write it as a string to the console.

The HttpClient class is in the java.net.http package. You create one using a builder, as in:

HttpClient client = HttpClient.newBuilder()
                              .version(HttpClient.Version.HTTP_2)
                              .connectTimeout(Duration.ofSeconds(2))
                              .build();

Two of the many configuration options are shown: using HTTP/2 (which is actually the default) and setting a two-second connection timeout period. Other options, like setting a proxy, following redirects, or using an authenticator are shown in the (early access) JavaDocs.

Requests can be sent synchronously or asynchronously as desired, using the send or sendAsync methods. Either way, the first argument to those methods is an instance of HttpRequest, which is also created using a builder:

String astroUrl = "http://api.open-notify.org/astros.json";
HttpRequest.newBuilder()
           .uri(URI.create(astroUrl))
           .GET()
           .build();

Sending a synchronous request also requires a “body handler”, which comes from the HttpResponse.BodyHandlers inner class factory methods. For this case, it’s easiest just to use the ofString method:

HttpResponse response = client.send(
    buildRequest(),
    HttpResponse.BodyHandlers.ofString());
String output = response.body();

From the HttpResponse you can get the headers, the body (as shown), the status code, or several other properties.

An asynchronous request is sent using the sendAsync method, which returns a CompletableFuture. Extracting the result can be done with code like:

String output = client.sendAsync(buildRequest(),
                                 HttpResponse.BodyHandlers.ofString())
                      .thenApply(HttpResponse::body)
                      .get();

The thenApply method from CompletableFuture takes a java.util.function.Function argument that in this case maps the response into a String containing the body. The subsequent get method blocks until the request and conversion has completed.

To summarize, here is the initial version of the class called AstroClient:

package astro;

import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
import java.util.concurrent.ExecutionException;

public class AstroClient {
    private HttpClient client = HttpClient.newBuilder()
                                          .version(HttpClient.Version.HTTP_2)
                                          .connectTimeout(Duration.ofSeconds(2))
                                          .build();

    private HttpRequest buildRequest() {
        String astroUrl = "http://api.open-notify.org/astros.json";
        return HttpRequest.newBuilder()
                          .uri(URI.create(astroUrl))
                          .GET()
                          .build();
    }

    public String getSync() throws IOException, InterruptedException {
        HttpResponse response = client.send(
                buildRequest(),
                HttpResponse.BodyHandlers.ofString());
        return response.body();
    }


    public String getAsync() throws ExecutionException, InterruptedException {
        return client.sendAsync(buildRequest(),
                                HttpResponse.BodyHandlers.ofString())
                     .thenApply(HttpResponse::body)
                     .get();
    }

}

Ultimately, the tests will use some cool new methods from JUnit 5, but for the moment if you just invoke the getSync and getAsync methods and log the output using a java.util.logging.Logger you get something like:


Sep 22, 2018 12:45:54 PM astro.AstroClientTest getSync
INFO: {"message": "success", "people": [{"craft": "ISS", "name": "Oleg Artemyev"}, {"craft": "ISS", "name": "Andrew Feustel"}, {"craft": "ISS", "name": "Richard Arnold"}, {"craft": "ISS", "name": "Sergey Prokopyev"}, {"craft": "ISS", "name": "Alexander Gerst"}, {"craft": "ISS", "name": "Serena Aunon-Chancellor"}], "number": 6}

Sep 22, 2018 12:45:54 PM astro.AstroClientTest getAsync
INFO: {"message": "success", "people": [{"craft": "ISS", "name": "Oleg Artemyev"}, {"craft": "ISS", "name": "Andrew Feustel"}, {"craft": "ISS", "name": "Richard Arnold"}, {"craft": "ISS", "name": "Sergey Prokopyev"}, {"craft": "ISS", "name": "Alexander Gerst"}, {"craft": "ISS", "name": "Serena Aunon-Chancellor"}], "number": 6}

Obviously we can do better if we parse the response. To do that, add the Google Gson dependency and use it to convert the JSON response to a pair of classes. The classes that map to the JSON structure are Assignment:

package astro.json;

public class Assignment {
    private String name;
    private String craft;

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getCraft() {
        return craft;
    }

    public void setCraft(String craft) {
        this.craft = craft;
    }
}

which includes fields for the name and craft, and AstroResponse:

package astro.json;

import java.util.List;

public class AstroResponse {
    private String message;
    private Integer number;
    private List people;

    public String getMessage() {
        return message;
    }

    public void setMessage(String message) {
        this.message = message;
    }

    public Integer getNumber() {
        return number;
    }

    public void setNumber(Integer number) {
        this.number = number;
    }

    public List getPeople() {
        return people;
    }

    public void setPeople(List people) {
        this.people = people;
    }
}

These POJOs could be simplified by using

  • Lombok annotations
  • Groovy POGOs, preferably with the @Canonical annotation
  • Kotlin data classes

among other ways. Any of those would cut the POJOs down to just their attributes, but these are small enough to be readable even without that.

The changes to the AstroClient class are to add a method that converts the String response to an instance of AstroResponse:

package astro;

import astro.json.AstroResponse;
import com.google.gson.Gson;

import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
import java.util.concurrent.ExecutionException;

public class AstroClient {
    private HttpClient client = ...

    private Gson gson = new Gson();

    private HttpRequest buildRequest() {
        // ... as before ...
    }

    public AstroResponse getSync() throws IOException, InterruptedException {
        HttpResponse response = client.send(
                buildRequest(),
                HttpResponse.BodyHandlers.ofString());
        return getResponse(response.body());
    }


    public AstroResponse getAsync() throws ExecutionException, InterruptedException {
        String json = client.sendAsync(buildRequest(),
                                HttpResponse.BodyHandlers.ofString())
                     .thenApply(HttpResponse::body)
                     .get();
        return getResponse(json);
    }


    private AstroResponse getResponse(String json) {
        return gson.fromJson(json, AstroResponse.class);
    }
}

Now that the methods return an AstroResponse, the tests can get more interesting. Here is the complete AstroClientTest class:

package astro;

import astro.json.Assignment;
import astro.json.AstroResponse;
import org.junit.jupiter.api.Test;

import java.time.Duration;
import java.util.List;
import java.util.logging.Logger;

import static org.junit.jupiter.api.Assertions.*;

class AstroClientTest {
    private Logger logger = Logger.getLogger(AstroClientTest.class.getName());

    private AstroClient client = new AstroClient();

    @Test
    void getSync() {
        AstroResponse response = assertTimeoutPreemptively(
                Duration.ofSeconds(2),
                () -> client.getSync());

        int num = response.getNumber();
        List assignments = response.getPeople();

        assertEquals("success", response.getMessage());
        assertEquals(num, assignments.size());
        assignments.forEach(assignment ->
           assertAll(() -> assertTrue(assignment.getName().length() > 0),
                     () -> assertTrue(assignment.getCraft().length() > 0)));

        logResponse(num, assignments);
    }

    @Test
    void getAsync() {
        AstroResponse response = assertTimeoutPreemptively(
                Duration.ofSeconds(2),
                () -> client.getAsync());

        int num = response.getNumber();
        List assignments = response.getPeople();

        assertEquals("success", response.getMessage());
        assertEquals(num, assignments.size());

        logResponse(num, assignments);
    }

    private void logResponse(int num, List assignments) {
        logger.info(String.format("There are %d people in space", num));
        assignments.forEach(person -> logger.info(
                () -> String.format("%s aboard %s",
                                    person.getName(),
                                    person.getCraft())));
    }
}

The first test uses the neat assertTimeoutPreemtively method. That causes JUnit to spawn a separate thread for the test, which it interrupts if the test does not complete within the specified Duration. The second argument is a ThrowingSupplier, which is like a regular java.util.function.Supplier except that its single abstract method, get, is declared to throw Throwable.

After checking that the message field is “success” and that the size of the Assignment collection (called people to match the JSON attribute) has the right size, each individual Assignment is checked to ensure its name and craft fields are not just empty strings. The assertAll method in JUnit 5 takes a variable argument list of Executable instances, which are like Runnable instances but can throw Exception. The nice thing about assertAll is that all of its Executable arguments will be evaluated, even if some of them fail. The asynchronous test does the same thing.

The private logResponse method just uses the logger to write out how many people are in space and their names and crafts:


Sep 22, 2018 4:58:27 PM astro.AstroClientTest logResponse
INFO: There are 6 people in space
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Oleg Artemyev aboard ISS
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Andrew Feustel aboard ISS
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Richard Arnold aboard ISS
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Sergey Prokopyev aboard ISS
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Alexander Gerst aboard ISS
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Serena Aunon-Chancellor aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest logResponse

INFO: There are 6 people in space
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Oleg Artemyev aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Andrew Feustel aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Richard Arnold aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Sergey Prokopyev aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Alexander Gerst aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Serena Aunon-Chancellor aboard ISS

To cap it all off, here is the Gradle build file (using the current version of Gradle, 4.10.2):

plugins {
    id 'java'
}

group 'kousenit.com'
version '1.0'

sourceCompatibility = 11

repositories {
    jcenter()
}

dependencies {
    implementation 'com.google.code.gson:gson:2.8.5'
    
    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.3.1'
    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.3.1'
}

test {
    useJUnitPlatform()
}

There’s nothing terribly surprising in that file to experienced Gradle users, though as a reminder the useJUnitPlatform method was added to Gradle to make it easier to run JUnit 5 tests.

Honestly, this is good enough. There’s no particular reason to go on and modularize this code; it all works fine on Java 11. However, just to show the kinds of changes needed if you’re foolish enough to add a module-info.java file to the class path, here’s what happens:

First, you need to add the necessary requires statements to the module-info.java file, which resides in the src/main/java directory:

module http.client.demo.main {
    requires java.net.http;
    requires gson;
}

The module name can be anything, but the same “reverse dns” pattern favored for packages is recommended. The name here is the one auto-generated by IntelliJ IDEA if you ask for a new module-info descriptor.

The HttpClient API was in an incubator module in Java 9, which was called jdk.incubator.httpclient. Now that it is standardized, it’s now in the java.net.http module, so that’s the module needed.

The hard part was figuring out how to refer to Gson. Since that API has not yet been modularized, the associated jar files wind up in the “unnamed” module and you need to refer to them by name rather than some official module name. Enough googling (no pun intended) and eventually you’ll find that the right name to use is “gson”. Of course, that’s fragile and will no doubt change at some point in the future.

If you’ve played with modules in Java 9 or 10, you’ll notice something conspicuous by its absence. Prior to Java 11, the Java util logger (the one in the java.util.logging package) was part of a separate module in the JDK called java.logging. In Java 11 that was moved back into java.base, so no additional requires statement is necessary.

The other necessary change is inside the Gradle build file. You have to tell Gradle to use the module path rather than the class path for compilation. Based on the Gradle Guide called Building Java 9 Modules, the following additional block was necessary:

compileJava {
    doFirst {
        options.compilerArgs = [
                '--module-path', classpath.asPath,
        ]
        classpath = files()
    }
}

This code adds a doFirst block to the compileJava task, which is a pre-processing step. It adds the --module-path compiler argument and assigns it to the jars on the classpath. Then the block sets the classpath variable to an empty collection of files.

The Gradle Guide also suggests that the compileTestJava and test tasks also need to be modified, but those turned out not to be necessary.

(Those are the only needed changes, which, looking back on them, was a bit disappointing given how much time it took to figure them out. But that’s no doubt a good thing.:))

The complete code is hosted in GitHub at https://github.com/kousen/http-client-demo . The master branch has everything, including the modularization.

As a reminder, I have a book that discusses the new features added to Java 8 (and much of Java 9) called Modern Java Recipes. The GitHub repositories for the book are https://github.com/kousen/java_8_recipes and https://github.com/kousen/java_9_recipes, both of which now build correctly under Java 11.

Rough cut of Gradle Recipes for Android now available

My latest book, Gradle Recipes for Android, is now available as a “Rough Cut” at O’Reilly. You can get it at http://shop.oreilly.com/product/0636920032656.do.

gradle-recipes-for-android

Rough cuts are preliminary versions of O’Reilly books, which are released while a book is still in progress, without special effort taken for formatting or anything else.

In this case, however, the rough cut is pretty close to the final version. Recipe books at O’Reilly are like their cookbooks, only shorter. My book has about 27 recipes, which are short discussions of how to do specific tasks, in this case involving the Gradle build tool with the Android plugin, used to build Android applications. The book also contains information about how to use the only officially supported IDE, Android Studio.

The O’Reilly authoring system supports Asciidoc, which is wonderful. I found it much, much easier to write my book using Asciidoc, commit it to their git repository, and then generate the resulting pdf afterwards. The book is available in Safari Books Online, too, which show it formatted for HTML. The results are very nice, and allow you to copy and paste code from the browser to your editor of choice. For the record, I wrote the book using the Atom editor from GitHub, which has a nice Asciidoc preview mode.

The book is basically finished. I’m doing some editing and considering adding a recipe or two. If you have any suggestions, however, please feel free to send them along. I hope to complete the book in the next week or so, so it will be available in print form just in time for the holidays. 🙂

I’ve also been doing a lot of video course recording for O’Reilly over the past few months. So far the courses available are:

All the courses are available for sale, or on Safari. The last one (Gradle for Android) is similar to the book, but the book has more depth, a different approach, and lots of reference information. The book has appendices on Groovy and Gradle, while the video summarizes both topics.

I should mention in the midst of all this shameless self-promotion that there is also a Packt book called Gradle for Android by Kevin Pelgrims. Packt books vary wildly in quality, but Kevin’s book is excellent. In fact, it was so good, that (combined with the fact I’d already coincidently recorded a video course of the same name) I decided I needed to switch my book to the recipe style. His book is much more the traditional exposition, with chapters and depth. Mine is more the “here’s a problem, now here’s a solution” style. I think both books are complementary. At least I hope so.

On an unrelated note, I now have a revised home page, http://www.kousenit.com. It’s about time.

I hope you enjoy the book and/or video courses, or even just my new home page. As always, any errors or omissions are entirely my responsibility. I’m just happy to be making all this content available. Now I have to finish this weekend’s No Fluff, Just Stuff conference in Boston, travel to Raleigh for a Groovy course, and then get in line to see The Martian.

%d bloggers like this: