Stuff I've learned recently…

Java 11 HttpClient, Gson, Gradle, and Modularization

This post describes a simple system that uses the new HttpClient class that comes with Java 11 to access a RESTful web service. After demonstrating basic functionality, changes are made to parse the results using Gson, and even modularize the code and have it work with Gradle. The goal is to provide an example for anyone who wants to do similar tasks with their own systems.

Java 11 is a Long Term Support (LTS) release for the Java standard edition, meaning bug fixes and security patches will be available for at least the next three years (see the extensive Java Is Still Free post at for details about the future licensing arrangements). Java 11 contains very few new features, however — many more were added in Java 9 and 10. Most of the industry did not upgrade during those releases, however. Now that another LTS release is available, that will likely change.

One of the few real additions to Java 11 is the HttpClient API, as described in JEP 321. This takes a related incubator project from Java 9 and standardizes it for Java 11. This blog post will not be an exhaustive demonstration of its capabilities, but it will show both a synchronous and asynchronous example of how to use it, along with JUnit 5 test cases and a Gradle build file, and even modularize the code.

The web service is provided by Open Notify, which is an open source project that provides a few links to data from NASA. In this case, the example uses the “number of people in space” link at http://api.open-notify.org/astros.json. The current response to that link is:

{"message": "success", 
 "people": [
    {"craft": "ISS", "name": "Oleg Artemyev"}, 
    {"craft": "ISS", "name": "Andrew Feustel"}, 
    {"craft": "ISS", "name": "Richard Arnold"}, 
    {"craft": "ISS", "name": "Sergey Prokopyev"}, 
    {"craft": "ISS", "name": "Alexander Gerst"}, 
    {"craft": "ISS", "name": "Serena Aunon-Chancellor"}], 
 "number": 6}

The root object contains attributes message, number, and people, the last of which is a collection of objects, each of which contains a name and a craft. As the response shows, at the moment there are six people aboard the International Space Station.

Even though the plan is to map that response to Java classes, at first the example will simply download the JSON data and write it as a string to the console.

The HttpClient class is in the java.net.http package. You create one using a builder, as in:

HttpClient client = HttpClient.newBuilder()
                              .version(HttpClient.Version.HTTP_2)
                              .connectTimeout(Duration.ofSeconds(2))
                              .build();

Two of the many configuration options are shown: using HTTP/2 (which is actually the default) and setting a two-second connection timeout period. Other options, like setting a proxy, following redirects, or using an authenticator are shown in the (early access) JavaDocs.

Requests can be sent synchronously or asynchronously as desired, using the send or sendAsync methods. Either way, the first argument to those methods is an instance of HttpRequest, which is also created using a builder:

String astroUrl = "http://api.open-notify.org/astros.json";
HttpRequest.newBuilder()
           .uri(URI.create(astroUrl))
           .GET()
           .build();

Sending a synchronous request also requires a “body handler”, which comes from the HttpResponse.BodyHandlers inner class factory methods. For this case, it’s easiest just to use the ofString method:

HttpResponse response = client.send(
    buildRequest(),
    HttpResponse.BodyHandlers.ofString());
String output = response.body();

From the HttpResponse you can get the headers, the body (as shown), the status code, or several other properties.

An asynchronous request is sent using the sendAsync method, which returns a CompletableFuture. Extracting the result can be done with code like:

String output = client.sendAsync(buildRequest(),
                                 HttpResponse.BodyHandlers.ofString())
                      .thenApply(HttpResponse::body)
                      .get();

The thenApply method from CompletableFuture takes a java.util.function.Function argument that in this case maps the response into a String containing the body. The subsequent get method blocks until the request and conversion has completed.

To summarize, here is the initial version of the class called AstroClient:

package astro;

import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
import java.util.concurrent.ExecutionException;

public class AstroClient {
    private HttpClient client = HttpClient.newBuilder()
                                          .version(HttpClient.Version.HTTP_2)
                                          .connectTimeout(Duration.ofSeconds(2))
                                          .build();

    private HttpRequest buildRequest() {
        String astroUrl = "http://api.open-notify.org/astros.json";
        return HttpRequest.newBuilder()
                          .uri(URI.create(astroUrl))
                          .GET()
                          .build();
    }

    public String getSync() throws IOException, InterruptedException {
        HttpResponse response = client.send(
                buildRequest(),
                HttpResponse.BodyHandlers.ofString());
        return response.body();
    }


    public String getAsync() throws ExecutionException, InterruptedException {
        return client.sendAsync(buildRequest(),
                                HttpResponse.BodyHandlers.ofString())
                     .thenApply(HttpResponse::body)
                     .get();
    }

}

Ultimately, the tests will use some cool new methods from JUnit 5, but for the moment if you just invoke the getSync and getAsync methods and log the output using a java.util.logging.Logger you get something like:


Sep 22, 2018 12:45:54 PM astro.AstroClientTest getSync
INFO: {"message": "success", "people": [{"craft": "ISS", "name": "Oleg Artemyev"}, {"craft": "ISS", "name": "Andrew Feustel"}, {"craft": "ISS", "name": "Richard Arnold"}, {"craft": "ISS", "name": "Sergey Prokopyev"}, {"craft": "ISS", "name": "Alexander Gerst"}, {"craft": "ISS", "name": "Serena Aunon-Chancellor"}], "number": 6}

Sep 22, 2018 12:45:54 PM astro.AstroClientTest getAsync
INFO: {"message": "success", "people": [{"craft": "ISS", "name": "Oleg Artemyev"}, {"craft": "ISS", "name": "Andrew Feustel"}, {"craft": "ISS", "name": "Richard Arnold"}, {"craft": "ISS", "name": "Sergey Prokopyev"}, {"craft": "ISS", "name": "Alexander Gerst"}, {"craft": "ISS", "name": "Serena Aunon-Chancellor"}], "number": 6}


Obviously we can do better if we parse the response. To do that, add the Google Gson dependency and use it to convert the JSON response to a pair of classes. The classes that map to the JSON structure are Assignment:

package astro.json;

public class Assignment {
    private String name;
    private String craft;

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getCraft() {
        return craft;
    }

    public void setCraft(String craft) {
        this.craft = craft;
    }
}

which includes fields for the name and craft, and AstroResponse:

package astro.json;

import java.util.List;

public class AstroResponse {
    private String message;
    private Integer number;
    private List people;

    public String getMessage() {
        return message;
    }

    public void setMessage(String message) {
        this.message = message;
    }

    public Integer getNumber() {
        return number;
    }

    public void setNumber(Integer number) {
        this.number = number;
    }

    public List getPeople() {
        return people;
    }

    public void setPeople(List people) {
        this.people = people;
    }
}

These POJOs could be simplified by using

• Lombok annotations
• Groovy POGOs, preferably with the @Canonical annotation
• Kotlin data classes

among other ways. Any of those would cut the POJOs down to just their attributes, but these are small enough to be readable even without that.

The changes to the AstroClient class are to add a method that converts the String response to an instance of AstroResponse:

package astro;

import astro.json.AstroResponse;
import com.google.gson.Gson;

import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
import java.util.concurrent.ExecutionException;

public class AstroClient {
    private HttpClient client = ...

    private Gson gson = new Gson();

    private HttpRequest buildRequest() {
        // ... as before ...
    }

    public AstroResponse getSync() throws IOException, InterruptedException {
        HttpResponse response = client.send(
                buildRequest(),
                HttpResponse.BodyHandlers.ofString());
        return getResponse(response.body());
    }


    public AstroResponse getAsync() throws ExecutionException, InterruptedException {
        String json = client.sendAsync(buildRequest(),
                                HttpResponse.BodyHandlers.ofString())
                     .thenApply(HttpResponse::body)
                     .get();
        return getResponse(json);
    }


    private AstroResponse getResponse(String json) {
        return gson.fromJson(json, AstroResponse.class);
    }
}

Now that the methods return an AstroResponse, the tests can get more interesting. Here is the complete AstroClientTest class:

package astro;

import astro.json.Assignment;
import astro.json.AstroResponse;
import org.junit.jupiter.api.Test;

import java.time.Duration;
import java.util.List;
import java.util.logging.Logger;

import static org.junit.jupiter.api.Assertions.*;

class AstroClientTest {
    private Logger logger = Logger.getLogger(AstroClientTest.class.getName());

    private AstroClient client = new AstroClient();

    @Test
    void getSync() {
        AstroResponse response = assertTimeoutPreemptively(
                Duration.ofSeconds(2),
                () -> client.getSync());

        int num = response.getNumber();
        List assignments = response.getPeople();

        assertEquals("success", response.getMessage());
        assertEquals(num, assignments.size());
        assignments.forEach(assignment ->
           assertAll(() -> assertTrue(assignment.getName().length() > 0),
                     () -> assertTrue(assignment.getCraft().length() > 0)));

        logResponse(num, assignments);
    }

    @Test
    void getAsync() {
        AstroResponse response = assertTimeoutPreemptively(
                Duration.ofSeconds(2),
                () -> client.getAsync());

        int num = response.getNumber();
        List assignments = response.getPeople();

        assertEquals("success", response.getMessage());
        assertEquals(num, assignments.size());

        logResponse(num, assignments);
    }

    private void logResponse(int num, List assignments) {
        logger.info(String.format("There are %d people in space", num));
        assignments.forEach(person -> logger.info(
                () -> String.format("%s aboard %s",
                                    person.getName(),
                                    person.getCraft())));
    }
}

The first test uses the neat assertTimeoutPreemtively method. That causes JUnit to spawn a separate thread for the test, which it interrupts if the test does not complete within the specified Duration. The second argument is a ThrowingSupplier, which is like a regular java.util.function.Supplier except that its single abstract method, get, is declared to throw Throwable.

After checking that the message field is “success” and that the size of the Assignment collection (called people to match the JSON attribute) has the right size, each individual Assignment is checked to ensure its name and craft fields are not just empty strings. The assertAll method in JUnit 5 takes a variable argument list of Executable instances, which are like Runnable instances but can throw Exception. The nice thing about assertAll is that all of its Executable arguments will be evaluated, even if some of them fail. The asynchronous test does the same thing.

The private logResponse method just uses the logger to write out how many people are in space and their names and crafts:


Sep 22, 2018 4:58:27 PM astro.AstroClientTest logResponse
INFO: There are 6 people in space
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Oleg Artemyev aboard ISS
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Andrew Feustel aboard ISS
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Richard Arnold aboard ISS
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Sergey Prokopyev aboard ISS
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Alexander Gerst aboard ISS
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Serena Aunon-Chancellor aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest logResponse

INFO: There are 6 people in space
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Oleg Artemyev aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Andrew Feustel aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Richard Arnold aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Sergey Prokopyev aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Alexander Gerst aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Serena Aunon-Chancellor aboard ISS


To cap it all off, here is the Gradle build file (using the current version of Gradle, 4.10.2):

plugins {
    id 'java'
}

group 'kousenit.com'
version '1.0'

sourceCompatibility = 11

repositories {
    jcenter()
}

dependencies {
    implementation 'com.google.code.gson:gson:2.8.5'
    
    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.3.1'
    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.3.1'
}

test {
    useJUnitPlatform()
}

There’s nothing terribly surprising in that file to experienced Gradle users, though as a reminder the useJUnitPlatform method was added to Gradle to make it easier to run JUnit 5 tests.

Honestly, this is good enough. There’s no particular reason to go on and modularize this code; it all works fine on Java 11. However, just to show the kinds of changes needed if you’re foolish enough to add a module-info.java file to the class path, here’s what happens:

First, you need to add the necessary requires statements to the module-info.java file, which resides in the src/main/java directory:

module http.client.demo.main {
    requires java.net.http;
    requires gson;
}

The module name can be anything, but the same “reverse dns” pattern favored for packages is recommended. The name here is the one auto-generated by IntelliJ IDEA if you ask for a new module-info descriptor.

The HttpClient API was in an incubator module in Java 9, which was called jdk.incubator.httpclient. Now that it is standardized, it’s now in the java.net.http module, so that’s the module needed.

The hard part was figuring out how to refer to Gson. Since that API has not yet been modularized, the associated jar files wind up in the “unnamed” module and you need to refer to them by name rather than some official module name. Enough googling (no pun intended) and eventually you’ll find that the right name to use is “gson”. Of course, that’s fragile and will no doubt change at some point in the future.

If you’ve played with modules in Java 9 or 10, you’ll notice something conspicuous by its absence. Prior to Java 11, the Java util logger (the one in the java.util.logging package) was part of a separate module in the JDK called java.logging. In Java 11 that was moved back into java.base, so no additional requires statement is necessary.

The other necessary change is inside the Gradle build file. You have to tell Gradle to use the module path rather than the class path for compilation. Based on the Gradle Guide called Building Java 9 Modules, the following additional block was necessary:

compileJava {
    doFirst {
        options.compilerArgs = [
                '--module-path', classpath.asPath,
        ]
        classpath = files()
    }
}

This code adds a doFirst block to the compileJava task, which is a pre-processing step. It adds the --module-path compiler argument and assigns it to the jars on the classpath. Then the block sets the classpath variable to an empty collection of files.

The Gradle Guide also suggests that the compileTestJava and test tasks also need to be modified, but those turned out not to be necessary.

(Those are the only needed changes, which, looking back on them, was a bit disappointing given how much time it took to figure them out. But that’s no doubt a good thing.:))

The complete code is hosted in GitHub at https://github.com/kousen/http-client-demo . The master branch has everything, including the modularization.

As a reminder, I have a book that discusses the new features added to Java 8 (and much of Java 9) called Modern Java Recipes. The GitHub repositories for the book are https://github.com/kousen/java_8_recipes and https://github.com/kousen/java_9_recipes, both of which now build correctly under Java 11.