Categories
Java Mockito

Why Use Mocks?

Testing A Simple Publisher/Subscriber System With Mockito

One of the challenges I find when teaching Java testing with Mockito is that the docs, while complete, don’t motivate why you want to use mocks in the first place. This post includes a simple example to show why mocking frameworks are important and where they are useful.

As a quick summary, Mockito is a Java-based framework for creating mocks, stubs, and spies. As described in the famous blog post by Martin Fowler, Mocks Aren’t Stubs, the basic ideas for stubs and mocks are:

  • A stub is a class that stands in for another and returns required outputs given specific inputs.
  • A mock is used to verify the interaction between the class you are testing and a stub. It tests the protocol, meaning you check that the methods on the stub were invoked the right number of times in the right order.

The example is based on one used by the Spock testing framework, shown in the Spock Example project on GitHub. I’ve adapted it for Mockito as an illustration and slightly changed the method names. It does a great job motivating why you need mocks in the first place.

Consider a simple Publisher, which maintains a list of Subscriber instances and sends each them a String message.

public class Publisher {
    private List<Subscriber> subscribers = new ArrayList<>();

    public void subscribe(Subscriber sub) {
        if (!subscribers.contains(sub)) {
            subscribers.add(sub);
        }
    }

    public void send(String message) {
        for (Subscriber sub : subscribers) {
            try {
                sub.onNext(message);
            } catch (Exception ignored) {
                // evil, but what can you do?
            }
        }
    }
}

The Subscriber interface in this simplified model contains only the onNext method:

public interface Subscriber {
    void onNext(String message);
}

(The Publisher class and Subscriber interface are simplified forms of the corresponding types defined by the Reactive Streams specification and included in Java SE 9 and above. The implementations here contain only enough details to highlight why we need mocks.)

Here’s the problem: How do you test the send method in Publisher? It returns void, which is always hard to test. The class has no attributes other than the list of subscribers, so you can’t look for a change of state. Subscribers only have an onNext method which also returns void, so there’s nothing coming back. Worst of all, if the Subscriber throws an exception, the Publisher catches it and ignores it.

The only way to check that the send method is working properly is to verify that the onNext method in each subscriber has been invoked exactly once with the proper argument. That’s what mocks do.

(Note that the following tests use JUnit 5.)

The PublisherTest starts by initializing a Publisher and adding two mock Subscriber instances:

public class PublisherTest {

    private final Publisher pub = new Publisher();
    private final Subscriber sub1 = mock(Subscriber.class);
    private final Subscriber sub2 = mock(Subscriber.class);

    @BeforeEach
    public void setUp() {
        pub.subscribe(sub1);
        pub.subscribe(sub2);
    }

    // ... tests to come ...
}

In this particular case, I don’t need to set any expectations on the stubs, because the onNext method doesn’t return anything. I just need to make sure the method is called exactly once on each subscriber with the proper argument. That’s done with a simple verify:

@Test
public void testSend() {
    pub.send("Hello");

    verify(sub1).onNext("Hello");
    verify(sub2).onNext("Hello");
}

The verify method checks that onNext was invoked on each subscriber with the String argument Hello.

If we assume that one of the subscribers fails and throws an exception every time, the awful “catch and ignore” behavior of the send method can be verified as well:

@Test
public void testSendWithBadSubscriber() {
    // sub1 throws an exception every time
    doThrow(RuntimeException.class).when(sub1).onNext(anyString());

    pub.send("message 1");
    pub.send("message 2");

    // sub2 still receives the messages
    verify(sub2, times(2))
            .onNext(argThat(s -> s.matches("message \\d")));
}

In this case, the doThrow method is used to make the first subscriber throw a runtime exception whenever its onNext method is invoked with any string, using the built-in argument matcher anyString.

The verify method this time checks that the onNext method was invoked twice on the second subscriber. The argument employs a custom matcher. Mockito uses the argThat method to let you provide a custom argument matcher, which must implement the ArgumentMatcher interface. That interface has only a single abstract method:

public interface ArgumentMatcher<T> {
    boolean matches(T argument);
}

The ArgumentMatcher is thus a functional interface, so its implementation can be provided with a lambda expression. In the test above, the custom matcher checks that the argument consists of the word "message" followed by a space and a digit.

The testSendWithBadSubscriber method only checks that the second subscriber received the proper message. In principle you could add assertions to verify that the first subscriber threw the exception. This is particularly easy to do with JUnit 5, which includes an assertThrows method (and an assertAll method to ensure both onNext invocations are checked even if one fails):

@Test
public void testSendWithBadSubscriber() {
    // ... after setting expectation and invoking pub.send(...) ...

    assertAll(
            () -> assertThrows(RuntimeException.class,
                    () -> sub1.onNext("message 1")),
            () -> assertThrows(RuntimeException.class,
                    () -> sub1.onNext("message 2")));

    // ... verify from before ...
}

It’s not actually necessary, however, to check that the stubs do what you tell them to do when their expectations were set earlier in the same method. Still, it doesn’t hurt anything either.

The tests check that the publisher’s send method is behaving as designed, through the behavior of the two subscribers. Honestly, it’s hard to think of any other way to do that. Hopefully this gives you a sense of when mocks can be useful.

All the code, along with many more examples I use in my training course on Mockito and the Hamcrest Matchers, is contained this GitHub repository.

Categories
Groovy

Antarctica Time Zones in Kotlin on GraalVM

They say you can tell someone is a developer by whispering the word “timezone” in their ear and watching a shudder go down their spine. Here, from Wikipedia, is a picture of the time zones in Antarctica.

Time zones in Antarctica, from Wikipedia
Time zones in Antarctica, from Wikipedia

Amazing, right? But the funny part isn’t immediately obvious. See how the South Pole is inside that hashed area with alternating red and white stripes? That section is called the Ross Dependency and is claimed by New Zealand, and while most countries don’t recognize territorial claims in Antarctica, that area follows the time zone rules for New Zealand.

That means the South Pole, which experiences six months of daylight followed by six months of darkness, spends part of the year on Daylight Savings Time. Seriously.

Here’s a simple Kotlin script (for the JVM) to check whether or not that’s true at the moment:

import java.time.*

val instant = Instant.now()
val southPole = instant.atZone(ZoneId.of("Antarctica/South_Pole"))
val dst = southPole.zone.rules.isDaylightSavings(instant)
println("It is ${southPole.toLocalTime()} " + 
        "(UTC${southPole.offset}) at the South Pole")
println("The South Pole ${if (dst) "is" else "is not"} " +
        "on Daylight Savings Time")

This is really just Java code using classes from the java.time package, but written using Kotlin syntax.

Running this on my Mac (macOS Catalina, version 10.15.2) running Java 11 gives:

$ kotlinc -script southpole.kts
 It is 05:17:54.524607 (UTC+13:00) at the South Pole
 The South Pole is on Daylight Savings Time

Yeah, go figure. This can be fleshed out a bit to see all the time zones in Antarctica:

import java.time.Instant
import java.time.ZoneId
import java.time.ZonedDateTime

val regex = """.*Antarctica.*""".toRegex()
val instant: Instant = Instant.now()

ZoneId.getAvailableZoneIds()
    .filter { regex.matches(it) }
    .map { instant.atZone(ZoneId.of(it)) }
    .sortedBy { it.offset.totalSeconds }
    .forEach { zdt ->
        println(String.format("%7s: %25s %7s",
            zdt.offset, zdt.zone,
            zdt.zone.rules.isDaylightSavings(instant)))
    }

This produces:

  -03:00:        Antarctica/Rothera   false
  -03:00:         Antarctica/Palmer   false
       Z:          Antarctica/Troll   false
  +03:00:          Antarctica/Syowa   false
  +05:00:         Antarctica/Mawson   false
  +06:00:         Antarctica/Vostok   false
  +07:00:          Antarctica/Davis   false
  +08:00:          Antarctica/Casey   false
  +10:00: Antarctica/DumontDUrville   false
  +11:00:      Antarctica/Macquarie   false
  +13:00:        Antarctica/McMurdo    true
  +13:00:     Antarctica/South_Pole    true

So at the moment, both the South Pole and the scientists at McMurdo Station are on daylight savings time. Keep that in mind next time you plan to set up a conference call with them. And in case you’re wondering, the temperature at McMurdo Station today ranges from a high of 29 F down to a low of 19 F. Not too bad. It is, after all, summer down there.

(Also, if you’re wondering about that “Troll” time zone, that apparently is for Troll Station in Queen Maud Land and it’s practically sitting on the Greenwich meridian. I was kind of hoping someone from the IANA was, well, trolling us.)

The code in the script above is hopefully reasonably clear, with a few explanations:

  • The getAvailableZoneIds method (from Java) returns a Set<String> representing the “region ids” for all the time zones around the world (approximately 600).
  • The filter passes only those in Antarctica.
  • The atZone method returns a ZonedDateTime, so the result of the map operation is a Stream<ZonedDateTime>.
  • These are then sortedBy their offsets in total seconds from GMT, and then printed as the time zone offset, the region ID, and whether or not the time zone is experiencing daylight savings time.

Easy peasy, penguin squeezy, at least until the rest of the Antarctic ice sheet melts or falls into the sea.

Native Image On GraalVM

As it says on the GraalVM web site, Graal is a universal virtual machine for running applications written in multiple languages. Please keep that in mind, because if you talk to one of the Graal developers and you refer only to the native image tool (which I’m about to do), you’ll annoy them unnecessarily. They’re rather sensitive about that, so be careful. I promise I understand that it’s a complete virtual machine, with cross-language capabilities.

Still, what I like about Graal is the native image generator, which allows you to “ahead of time” compile Java code into a stand-alone executable. There are various ways to install the VM and the compiler. For this particular demo, I’m going to use the Gradle plugin.

If you take a look at the source code for my book Kotlin Cookbook, located in this GitHub repository, you’ll find the following sections in the build file build.gradle.kts:

plugins {
    `java-library`
    kotlin("jvm") version "1.3.61"
    id("com.palantir.graal") version "0.6.0-58-gce10e7e"
}

val scriptname: String by project  // from gradle.properties

graal {
    mainClass("scripts.${scriptname.capitalize()}Kt")
    outputName(scriptname) // output is build/graal/${scriptname}
}

The gradle-graal plugin is a simple wrapper around the GraalVM tooling that downloads the necessary tools and caches them locally. It adds the graal task seen in the snippet, which asks for the class with the main method in it. There are actually many properties you can configure — see the plugin documentation for details.

In this case I’m arguably being a bit too clever, but the ideas are:

  • I have a series of Kotlin scripts in a package called scripts under src/main/kotlin.
  • I put the name of the script I want in my local gradle.properties file. The “by project” delegate is used by Gradle to read properties out of that file.
  • When Kotlin compiles a script (meaning code that isn’t wrapped inside a class), by default it generates a class called ScriptKt. I’m using that as the name of the class Graal needs.

The plugin requires me to have a main method in my script, so here’s a mild refactoring of the overall program:

package scripts

import java.time.Instant
import java.time.ZoneId

fun main() {
    val regex = """.*Antarctica.*""".toRegex()
    val instant = Instant.now()

    val zones = ZoneId.getAvailableZoneIds()
        .filter { regex.matches(it) }
        .map { instant.atZone(ZoneId.of(it)) }
        .sortedBy { it.offset.totalSeconds }
        .toList()

    zones.forEach { zdt ->
        println(String.format("%7s: %25s %7s", zdt.offset, zdt.zone,
            zdt.zone.rules.isDaylightSavings(instant)))
    }

    println()
    val southPole = instant.atZone(
        ZoneId.of("Antarctica/South_Pole"))
    val dst = southPole.zone.rules.isDaylightSavings(Instant.now())
    println("It is ${southPole.toLocalTime()} " +
            "(UTC${southPole.offset}) at the South Pole")
    println("The South Pole ${if (dst) "is" else "is not"} " +
            "currently on Daylight Savings Time")
}

The plugin adds a handful of tasks to Gradle. Running gradle tasks gives (among the rest):

Graal tasks
-----------
downloadGraalTooling - Downloads and caches GraalVM binaries.
extractGraalTooling - Extracts GraalVM tooling from downloaded archive using the system's tar command or Gradle's copy method.
nativeImage - Runs GraalVM's native-image command with configured options and parameters.
sharedLibrary - Runs GraalVM's native-image command configured to produce a shared library.

The one I want is the nativeImage task. Running this (the first time, so the download of the toolkit is included):

$ ./gradlew nativeImage

> Task :extractGraalTooling
Downloading: Component catalog from www.graalvm.org
Processing component archive: Native Image
Downloading: Component native-image: Native Image  from github.com
[#####               ]                                                                                                      t: Installing new component: Native Image (org.graalvm.native-image, version 19.2.0)

> Task :nativeImage
Build on Server(pid: 36964, port: 55606)*
[antarctica:36964]    classlist:   2,658.08 ms
[antarctica:36964]        (cap):   2,267.25 ms
[antarctica:36964]        setup:   3,204.49 ms
[antarctica:36964]   (typeflow):   2,251.16 ms
[antarctica:36964]    (objects):   1,932.88 ms
[antarctica:36964]   (features):     248.91 ms
[antarctica:36964]     analysis:   4,527.18 ms
[antarctica:36964]     (clinit):      89.98 ms
[antarctica:36964]     universe:     264.94 ms
[antarctica:36964]      (parse):     319.07 ms
[antarctica:36964]     (inline):   1,054.46 ms
[antarctica:36964]    (compile):   2,818.36 ms
[antarctica:36964]      compile:   4,477.76 ms
[antarctica:36964]        image:     474.30 ms
[antarctica:36964]        write:     175.90 ms
[antarctica:36964]      [total]:  15,920.48 ms
native image available at build/graal/antarctica (5 MB)  

There are now three different ways to execute the original Kotlin script:

  • Compile with kotlinc-jvm; execute it with kotlin
  • Create a jar file by including the runtime; execute it with java
  • Create the native image; execute it directly

So here is using kotlinc-jvm and kotlin:

> kotlinc-jvm antarctica.kt // generates AntarcticaKt.class
> kotlin AntarcticaKt
... output from before ...
> time kotlin AntarcticaKt
kotlin A.. 0.23s user 0.04s system 146% cpu 0.188 total

Now creating the jar first and running java:

> kotlinc-jvm antarctica.kt -include-runtime -d antarctica.jar
> java -jar antarctica.jar
... output from before ...
> time java -jar antarctica.jar
java ...  0.21s user 0.03s system 151% cpu 0.159 total

And finally for the native image:

> sdk install java 19.3.0.r11-grl
> sdk use java 19.3.0.r11-grl
Using java version 19.3.0.r11-grl in this shell
> gu install native-image
Downloading: Component catalog from www.graalvm.org
 Processing Component: Native Image
 Downloading: Component native-image: Native Image  from github.com
 Installing new component: Native Image (org.graalvm.native-image, version 19.3.0)
> native-image -jar antarctica.jar
... lots of output similar to before ...
> ./antarctica
... works ...
> time ./antarctica
./antarctica  0.00s user 0.00s system 68% cpu 0.008 total

Okay, I lied. I didn’t use the Gradle plugin for the terminal demo. I removed the package statement, moved the script to my root, and installed the tools locally. I use the awesome SDKMAN! tool to manage multiple JDKs, so with it I installed the latest GraalVM version of the JDK, and used its gu command to install native-image, and so on.

Summarizing the results from this admittedly trivial experiment:

kotlin: 0.23s user 0.04s system 146% cpu 0.188 total
  java: 0.21s user 0.03s system 151% cpu 0.159 total
native: 0.00s user 0.00s system  68% cpu 0.008 total

Wow, that’s quite a speed improvement (nearly 20x) with the native image tool, which presumably all comes from avoiding the JVM startup time and memory costs. So sure, I had no good reason to do this, but it’s fun to see a speed up like that.

I should note that the native image tool requires the local toolchain to work, so things like glibc-devel, zlib-devel, and gcc need to be available, as they are on a Mac.

I hope you enjoyed this demo. All the steps are included in my Kotlin Cookbook, now available from O’Reilly Media and in print form on Amazon and from other booksellers.

Now if we could just transport some of that Antarctic ice to Australia where they could really use it…

Categories
Groovy

A Few Astronomical Examples in Kotlin

The website Open Notify provides a few URLs that return JSON data from NASA. There are currently three links supported:

Just for fun, this blog post uses Kotlin to access each one, transform the results in Kotlin data classes, and presents the results. In each case, since only HTTP GET requests are supported, the code uses the extension function readText that was added to the java.net.URL class. It also parses the resulting JSON data using Google’s GSON parser.

As usual, the code for these examples resides in the GitHub repository for my new book, Kotlin Cookbook.

People In Space

Consider the “number of people in space” service first. The JSON response takes the form:

{
  "message": "success",
  "number": NUMBER_OF_PEOPLE_IN_SPACE,
  "people": [
    {"name": NAME, "craft": SPACECRAFT_NAME},
    ...
  ]
}

The following Kotlin data classes match up with that JSON structure:

data class AstroResult(
    val message: String,
    val number: Int,
    val people: List<Assignment>
)

data class Assignment(
    val craft: String,
    val name: String
)

An Assignment connects a craft to an astronaut name, and the AstroResult has a String message (which out to be “success“), a number, and a list of assignments.

Going from the base URL to the AstroResult is easy enough. Just instantiate a Gson object and invoke its fromJson method:

Gson().fromJson(
    URL("http://api.open-notify.org/astros.json").readText(),
    AstroResult::class.java
)

GSON doesn’t know anything about Kotlin. It’s a Java library, so it expects to convert JSON strings into Java classes. The AstroResult::class.java syntax uses the double colon to get the KClass (the Kotlin class), whose java property returns the Java class.

This result could be assigned to a local variable for printing, but that’s what the scope functions like also or let are for. For example, to print the results, try this:

Gson().fromJson(
    URL("http://api.open-notify.org/astros.json").readText(),
        AstroResult::class.java
).also { astroResult ->
    println("There are ${astroResult.number} people in space:")
    astroResult.people.forEach { assignment ->
        println("\t${assignment.name} aboard ${assignment.craft}")
    }
}

As of December 2019, this prints:

There are 6 people in space:
    Christina Koch aboard ISS
    Alexander Skvortsov aboard ISS
    Luca Parmitano aboard ISS
    Andrew Morgan aboard ISS
    Oleg Skripochka aboard ISS
    Jessica Meir aboard ISS

Even simpler, though not as nicely structured, is to just drill down to the names and print them:

Gson().fromJson(
    URL("http://api.open-notify.org/astros.json").readText(),
    AstroResult::class.java
).people.map(Assignment::name)
    .also(::println)  // or use let instead of also

As a developer I knew once said, “My debugger is .also(::println)“.

For no good reason, you could create a class for the whole process, and why not make it executable at the same time?

class AstroRequest {
    companion object {
        private const val ASTRO_URL = 
            "http://api.open-notify.org/astros.json"
    }

    operator fun invoke(): AstroResult =
        Gson().fromJson(URL(ASTRO_URL).readText(), 
            AstroResult::class.java)
}

The URL is a constant, so you might as well say so. The operator function invoke means that you can use the “parentheses operator”, (), to execute an instance of this class, as the following test shows:

class AstroRequestTest {
    @Test
    fun `get people in space`() {
        val request = AstroRequest()
        val result = request()  // calls "invoke" function
        assertThat(result.message, `is`("success"))
        assertThat(result.number, `is`(greaterThanOrEqualTo(0)))
        assertThat(result.people.size, `is`(result.number))
    }
}

If you really want to go to extremes (and guarantee you get challenged in a code review), you can actually combine the instantiation and execution into one line:

val result = AstroRequest()()

which works, but please don’t do that. 🙂

Position of the ISS

Moving now to the service that returns the position of the International Space Station, the JSON form of the response is:

{
  "message": "success", 
  "timestamp": UNIX_TIME_STAMP, 
  "iss_position": {
    "latitude": CURRENT_LATITUDE, 
    "longitude": CURRENT_LONGITUDE
  }
}

Ah, the joys of the Unix time stamp. Fortunately, since I’m assuming we’re running on the JVM, there is a straightforward (but overly complicated) way to convert the timestamp into an instance of java.time.ZonedDateTime in your local time zone. Here are the resulting data classes:

data class IssPosition(
    val latitude: Double,
    val longitude: Double
)

data class IssResponse(
    val message: String,
    val iss_position: IssPosition,
    val timestamp: Long
) {
    val zdt: ZonedDateTime
        get() = ZonedDateTime.ofInstant(
            Instant.ofEpochSecond(timestamp),
            TimeZone.getDefault().toZoneId()
        )
}

The IssPosition class just combines the latitude and longitude, so there’s nothing special about that. The IssResponse class includes the message (again, “success”) and the ISS position. The property uses an underscore so no additional annotations are needed to do the JSON mapping. The fun part comes with the timestamp.

The timestamp may be identified as a UNIX timestamp, but as far as Kotlin is concerned it’s just a Long representing the number of seconds elapsed since the current epoch began on midnight, January 1, 1970, UTC. You can certainly use a function to convert the long to a ZonedDateTime, but since the result is just a dependent property, it seems appropriate to make it a property with a custom get method.

Taking the same object-oriented approach as before, here is a class to return either the entire response, or just the position:

class ProcessAstroData {
    companion object {
        const val url = "http://api.open-notify.org/iss-now.json"
    }

    fun getResponse(): IssResponse =
        Gson().fromJson(URL(url).readText(),
            IssResponse::class.java)

    fun getPosition() =
        getResponse().also {
            println("As of " + it.zdt.format(
        DateTimeFormatter.ofLocalizedDateTime(FormatStyle.LONG)))
        }.iss_position
}

The getPosition function could just be written as getPosition().iss_position, but as a side effect it prints the time of the request.

To run this, use the following lines:

val demo = ProcessAstroData()
val (lat, lng) = demo.getPosition()
println("$lat deg N, $lng deg W")

If there was ever a class built for destructuring, it’s a wrapper for latitude and longitude. The current result is:

As of December 19, 2019 at 5:31:47 PM EST
49.0682 deg N, 160.2318 deg W

Overhead Pass Predictions

Moving finally to the overhead pass predictions for the ISS, the JSON output for that service looks like:

{
  "message": "success",
  "request": {
    "latitude": LATITUE,
    "longitude": LONGITUDE, 
    "altitude": ALTITUDE,
    "passes": NUMBER_OF_PASSES,
    "datetime": REQUEST_TIMESTAMP
  },
  "response": [
    {"risetime": TIMESTAMP, "duration": DURATION},
    ...
  ]
}

This time, however, there are input values required. The user has to supply the latitude and longitude, and can optionally supply an altitude and a number of passes (which defaults to five). According to the documentation, the parameters in the service are specified as lat, lon, alt, and n, respectively.

Once again using simple Long variables where the service is expecting a timestamp, the following data classes map to this structure:

data class OverheadResponse(
    val message: String,
    val request: OverheadRequest,
    val response: List<TimeAndDuration>
)

data class OverheadRequest(
    val latitude: Double,
    val longitude: Double,
    val altitude: Double,
    val passes: Int,
    val datetime: Long
)

data class TimeAndDuration(
    val risetime: Long,
    val duration: Long
)

Since there are several timestamp fields, this time use a function to convert the Long values to formatted strings:

fun formatTimestamp(timestamp: Long): String =
    ZonedDateTime.ofInstant(
        Instant.ofEpochSecond(timestamp),
        TimeZone.getDefault().toZoneId()
    ).format(DateTimeFormatter.ofLocalizedDateTime(
        FormatStyle.LONG))

Here is a class to do the work:

class Overhead {
    companion object {
        const val base = "http://api.open-notify.org/iss-pass.json"
    }

    fun getOverheadResponse(lat: Double, 
                            lng: Double,
                            alt: Double = 0.0,
                            num: Int = 5): OverheadResponse {
        val url = "$base?lat=$lat&lon=$lng&alt=$alt&n=$num"
        val json = URL(url).readText()
        return Gson().fromJson(json, OverheadResponse::class.java)
    }
}

Use optional parameters in the Kotlin function to represent the default values for the service. To get the overhead pass times, instantiate the class, get the response, and format them:

// Marlborough, CT
val latitude = 41.6314
val longitude = -72.4596

val output = Overhead().getOverheadResponse(latitude, longitude)
output.response.forEach {
    println(formatTimestamp(it.risetime))
}

The output at the time of this writing is:

December 20, 2019 at 5:42:26 AM EST
December 20, 2019 at 7:17:35 AM EST
December 20, 2019 at 8:55:03 AM EST
December 20, 2019 at 10:32:58 AM EST
December 20, 2019 at 12:10:04 PM EST

These examples used simple GET requests to access the free services and GSON to parse the resulting JSON data. Along the way it used data classes, companion objects, an executable class, optional parameters, and conversion of Unix timestamps to classes from the java.time package. Hopefully you’ll find something interesting in it. Or you can fire it into the sun. 🙂

Categories
Kotlin

Fibonacci in Kotlin

I didn’t realize it until I noticed the trending topic on Twitter, but Saturday, November 23 was Fibonacci Day. That gave me an opportunity to drag out one of my favorite old jokes:

For those unaware or who don’t remember, the Fibonacci series starts with 0 and 1, and then each subsequent number is the sum of the previous two. The Fibonacci numbers are thus 0, 1, 1, 2, 3, 5, 8, 13, …. By the definition, the “zeroth” Fibonacci number F0 = 0, the first F1 = 1, the second F2 = 1, the third F3 = 2, and so on.

(Note the gag only works in the U.S., because we express our dates as month/day/year rather than day/month/year like the rest of the world. For us, November 23 is 11/23, thus Fibonacci day, ha ha. Sorry about the US-centric approach. On the other hand, there’s no 23rd month, so the rest of the world would miss out on Fibonacci day, and that would be sad.)

Naive Recursive Implementation

The Fibonacci algorithm is a classic example of a recursive function, expressed as Fn = Fn-1 + Fn-2:

fib(n) = fib(n - 1) + fib(n - 2)

The problem with a naive implementation of that algorithm is that the amount of duplicated work grows exponentially. For example,

fib(5) = fib(4) + fib(3) 
       = fib(3) + fib(2) + fib(2) + fib(1) 
       = fib(2) + fib(1) + fib(1) + fib(0) 
                + fib(1) + fib(0) + 1
       = fib(1) + fib(0) + 1 + 1 + 0 + 1 + 0 + 1 
       = 1 + 0 + 1 + 1 + 0 + 1 + 0 + 1
       = 5

Yikes, and that’s just for the fifth Fibonacci number. The problem is that intermediate results aren’t being saved, so each number is recalculated over and over again, and the higher you go, the more duplicated work there is.

Tail Call Optimization

In Kotlin, there are several ways to fix this. One is to use tail recursion, which involves replacing recursion with iteration. An implementation in Kotlin can take advantage of the tailrec keyword:

@JvmOverloads
tailrec fun fibonacci(n: Int, a: Int = 0, b: Int = 1): Int =
    when (n) {
        0 -> a
        1 -> b
        else -> fibonacci(n - 1, b, a + b)
    }

There’s a fair amount going on here. First, in order to write a recursive function as a tail call, the recursive call itself has to come last in the implementation. In this example, the last evaluated expression is the else clause in the when loop, which calls the function again.

Also, when you write your function to be called recursively, you need to use an additional parameter to act as the accumulator. Here, n is the nth desired Fibonacci number, the value of b is the previous value, and the value of a is the one before that. In the recursive call, the new value of b is the current value of b plus the value of a, so that is effectively the accumulator.

The next interesting part is that the return type is specified explicitly. Normally in Kotlin when you implement a function as a single expression (using an = sign at the end of the signature) you can leave out the return type. If you do that in this case, however, IntelliJ IDEA complains with “Type checking has run into a recursive problem. Easiest workaround: specify types of your declarations explicitly.” Yeah, okay, I can do that, thus the a : Int at the end of the signature.

The next good part is using default values for the second and third arguments in the function. By initializing a to 0 and b to 1, the function can be invoked as fibonacci(10) without giving values for the other two parameters. That’s the way you want to call the function anyway, so that’s helpful. Unfortunately, Java doesn’t support default parameters, so to make sure the compiler generates overloads that Java can see, the JmvOverloads annotation is added. Now the fibonacci function can be called from Java with only a single argument.

That leads to the best part, which is the tailrec modifier. You could write this tail call algorithm directly in Java, but it wouldn’t save any memory. In Kotlin, however, this page in the reference guide says that compiler will automatically “optimize out the recursion, leaving behind and fast and efficient loop version instead.”

If you play the game in IntelliJ where you “Show Kotlin Bytecodes” and the hit the Decompile button, the resulting Java code looks like:

@JvmOverloads
public static final int fibonacci(int n, int a, int b) {
    while(true) {
        int var10000;
        switch(n) {
        case 0:
            var10000 = a;
            break;
        case 1:
            var10000 = b;
            break;
        default:
            var10000 = n - 1;
            int var10001 = b;
            b += a;
            a = var10001;
            n = var10000;
            continue;
        }

        return var10000;
    }
}

The recursive call has been implemented as a while(true) loop with the accumulator in the default section.

Kotlin fold and reduce

That’s all well and good, and normally I’d go on to discuss ways to test this function, but there are a couple of alternative ways to implement the algorithm. One is this particularly interesting version:

fun fibonacciFold(n: Int) =
    (2 until n).fold(1 to 1) { (prev, curr), _ -> 
        curr to (prev + curr) 
    }.second

Now that’s poetry, in that it’s simple, elegant, and really needs to be explained before it makes any sense. First of all, credit where credit is due. This is from Marcin Moskala (@marcinmoskala on Twitter). He included it in a blog post at Kotlin Academy, based on one of his tweets.

His first example, fib1, is the inefficient recursive algorithm. His second, fib2, is an iterative version of the same. The third one, fib3, is like the tail-call version described above. Finally, the last one, fib4, is the one I want to talk about, repeated in the syntax block prior to the tweet.

First, the expression (2 until n) is a range that runs from 2 to n - 1. An inclusive range would be 2..n.

Next comes the fold function. Kotlin has both fold and reduce functions. The fold function takes two parameters, while reduce only takes one:

inline fun <T, R> Iterable<T>.fold(
    initial: R,
    operation: (acc: R, T) -> R
): R

inline fun <S, T : S> Iterable<T>.reduce(
    operation: (acc: S, T) -> S
): S

Each of them applies the provided operation to each element of the Iterable, and the last evaluated expression from the operation becomes the new value of the acc (accumulator) argument on the next iteration. The difference is that fold takes an initial value for the accumulator, while reduce uses the first element from the sequence to initialize the accumulator.

For those more familiar with Java, the reduce function on Java streams behaves the same way, except that it is overloaded to take either a single BinaryOperator (like Kotlin’s reduce) or two values, where the first is an initial value called identityand the second is a BinaryOperator (like Kotlin’s fold). Note that in Java they refer to the first argument as identity rather than initial, which is more in keeping with the theory of functional programming. In either case the first argument is used as the initial value of the provided operation accumulator, but it’s supposed to be the identity value for the binary operator. The Kotlin implementation works the same way, but doesn’t stress that distinction.

Returning to the Fibonacci calculation, the fold function uses the Pair of 1 to 1 as its initial value, so the operation needs to return a new instance of Pair each time. There are two arguments to the lambda provided as the operation. The first argument is also a Pair, which acts as the accumulator, and the second argument would be each value in the range from 2 to n - 1. The algorithm doesn’t actually need those values, so the implementation just uses an underscore to indicate they’re ignored.

The implementation of the operation creates a new Pair whose first element is the current value and whose second element is the sum of the previous value and the current value. Finally, when the limit of the range is reached, the function returns the second element from the Pair as the desired value.

As usual with a fold function, it’s easiest to understand if you see it in action. Here is a version of it with the previous and current values written out each time:

fun fibonacciFoldDebug(n: Int) =
    (2 until n).fold(1 to 1) { (prev, curr), _ ->
        println("prev=$prev, curr=$curr")
        curr to (prev + curr)
    }.second

Here is the output of F(10):

prev=1, curr=1
prev=1, curr=2
prev=2, curr=3
prev=3, curr=5
prev=5, curr=8
prev=8, curr=13
prev=13, curr=21
prev=21, curr=34
Fib(10) == 55

The initial Pair of 1 to 1 initialized the prev and curr variables, which take new values as described in the fold operation. Once again, a recursive problem has been transformed into an iterative one, but this time in a really beautiful way. You can comfortably compute Fibonacci numbers as far as Int values work, and if you need something higher just reimplement this with BigInteger (and replace + with add) and keep going.

Sequences

There’s one more fundamentally different approach to the same problem, this time using sequences. A sequence in Kotlin is just like a stream in Java, in that values go through a series of intermediate operations followed by a terminal operation, and no data is pulled through the sequence until there is a terminal operation.

There are several ways to create a sequence in Kotlin, from the asSequence function on collections to the generateSequence function that takes a lambda to evaluate for each element. There is also, however, the basic function called sequence:

fun <T> sequence(
    block: suspend SequenceScope<T>.() -> Unit
): Sequence<T>

The sequence function is a suspend function (so it can be used with coroutines) that takes a lambda with a receiver. The lambda takes no arguments and returns nothing (i.e., Unit), and the receiver is an instance of SequenceScope. The documentation says the function returns a sequence by lazily yielding values one by one.

In fact, that documentation page actually contains a Fibonacci example, repeated here:

fun fibonacci() = sequence {
    var terms = Pair(0, 1)

    // this sequence is infinite
    while (true) {
        yield(terms.first)
        terms = Pair(terms.second, terms.first + terms.second)
    }
}

Look familiar? The iteration occurs over Pair instances, as before, in an infinite loop. The yield function “yields a value to the Iterator being built and suspends until the next value is requested.” Note that this returns a sequence, so to invoke it you need to add a terminal operation. The way the code in the documentation example calls this function is:

println(fibonacci().take(10).toList())

The take function is another intermediate operation that requests ten elements, and toList is, at long last, a terminal operation which converts the sequence into a List. The code prints the first 10 Fibonacci numbers in a list, [0, 1, 1, 2, 3, 5, 8, 13, 21, 34].

Wow, right? Lots of ways to solve the same problem. Of course, any functions like this need to be tested. That, too, can be an interesting story, which I’ll leave for the next blog post in this series.

As always, all the code (including extensive JUnit 5 tests) from this post can be found in this GitHub repository for my new book, the Kotlin Cookbook.

Categories
Kotlin

Annotated TOC for Kotlin Cookbook

Last week I received an email asking about a table Of contents for my new Kotlin Cookbook, saying they couldn’t find one either on the O’Reilly Media site or on Amazon. It’s a lot easier to make a decision about buying a recipe book if you have a list of recipes.

When I checked last week, no table of contents was available. Fortunately that’s no longer the case. If you go to the Amazon listing and click on the “Look Inside” link, the table of contents is included, along with the preface, which includes a description of each chapter.

It couldn’t hurt, however, to repeat both of those here as well. That way if anyone asks I can just give them this link, plus you can add whatever comments you like to this post and I’ll see them and potentially be able to respond.

Cover for my Kotlin Cookbook
Cover for my Kotlin Cookbook

Here is the list of chapter descriptions from the Preface.

  • Chapter 1, Installing and Running Kotlin, covers the basic process of installing and running Kotlin, including using the REPL, working with build tools like Maven and Gradle, and employing the native image generator in Graal.
  • Chapter 2, Basic Kotlin, covers some fundamental features of Kotlin—such as nullable types, overloading operators, and converting between types—before examining some more esoteric issues including working with bitwise shift operators or the to extension function on the Pair class.
  • Chapter 3, Object Oriented Programming in Kotlin, focuses on object-oriented features of the language that developers from other languages might find surprising or unusual. It includes how to use the const keyword, how Kotlin handles backing properties, delayed initialization, and the dreaded Nothing class, which is guaranteed to confuse existing Java developers.
  • Chapter 4, Functional Programming, has only a few recipes, which involve functional features that need their own explanations. Functional programming concepts are covered throughout the book, especially when talking about collections, sequences, and coroutines, but there are a handful of techniques included in this chapter that you may find unusual or particularly interesting.
  • Chapter 5, Collections, covers arrays and collections, dealing mostly with non-obvious methods like destructing collections, sorting by multiple properties, building a window on a collection, and creating progressions.
  • Chapter 6, Sequences, shows how Kotlin handles sequences of items lazily, similar to the way Java uses streams. Recipes cover generating sequences, yielding from them, and working with infinite sequences.
  • Chapter 7, Scope Functions, covers another topic unique to Kotlin: functions that execute a block of code in the context of an object. Functions like let, apply, and also are quite useful in Kotlin, and this chapter illustrates why and how to use them.
  • Chapter 8, Kotlin Delegates, discusses a convenient feature of Kotlin: how it implements delegation. Delegation lets you employ composition rather than inheritance, and Kotlin includes several delegates in the standard library, like lazy, observable, and vetoable.
  • Chapter 9, Testing, covers the important topic of testing, with a particular focus on JUnit 5. In its current version, JUnit is designed to work well with Kotlin, and that includes both its regular usage and employing it in Spring Framework applications. This chapter discusses several approaches that make writing and executing tests easier.
  • Chapter 10, Input/Output, includes a couple of recipes specifically for managing resources. File I/O is covered, as is the use function, which has broad applicability in several contexts.
  • Chapter 11, Miscellaneous, covers topics that do not fit easily in any other category. Topics such as how to get the current Kotlin version, how to force the when statement to be exhaustive even when it doesn’t return a value, and how to use the replace function with regular expressions are covered. In addition, the TODO function and the Random class are discussed, as well as how to integrate with Java exception handling.
  • Chapter 12, The Spring Framework, involves the Spring Framework along with Spring Boot, which is very friendly to Kotlin. A few recipes are included to show how to use Kotlin classes as managed beans, how to implement JPA persistence, and how to inject dependencies when needed.
  • Chapter 13, Coroutines and Structured Concurrency, covers the subject of coroutines, one of the most popular features of Kotlin and the basis of concurrent and parallel programming in the language. Recipes cover the basics, like builders and dispatchers, along with how to cancel and debug coroutines, and how to run them on your own custom Java thread pool.

Hopefully those descriptions will be helpful. For the actual recipes, the following list has been (slightly) enhanced by using code formatting for Kotlin functions or keywords.

  1. Installing and Running Kotlin
    1.1 Running Kotlin Without a Local Compiler
    1.2 Installing Kotlin Locally
    1.3 Compiling and Running Kotlin from the Command Line
    1.4 Using the Kotlin REPL
    1.5 Executing a Kotlin Script
    1.6 Building a Standalone Application Using GraalVM
    1.7 Adding the Kotlin Plugin for Gradle (Groovy Syntax)
    1.8 Adding the Kotlin Plugin for Gradle (Kotlin Syntax)
    1.9 Using Gradle to Build Kotlin Projects
    1.10 Using Maven with Kotlin
  2. Basic Kotlin
    2.1 Using Nullable Types in Kotlin
    2.2 Adding Nullability Indicators to Java
    2.3 Adding Overloaded Methods for Java
    2.4 Converting Between Types Explicitly
    2.5 Printing to Different Bases
    2.6 Raising a Number to a Power
    2.7 Using Bitwise Shift Operators
    2.8 Using Bitwise Boolean Operators
    2.9 Creating Pair Instances with to
  3. Object-Oriented Programming in Kotlin
    3.1 Understanding the Difference Between const and val
    3.2 Creating Custom Getters and Setters
    3.3 Defining Data Classes
    3.4 The Backing Property Technique
    3.5 Overloading Operators
    3.6 Using lateinit for Delayed Initialization
    3.7 Using Safe Casting, Reference Equality, and Elvis to Override equals
    3.8 Creating a Singleton
    3.9 Much Ado About Nothing
  4. Functional Programming
    4.1 Using fold in Algorithms
    4.2 Using the reduce Function for Reductions
    4.3 Applying Tail Recursion
  5. Collections
    5.1 Working with Arrays
    5.2 Creating Collections
    5.3 Creating Read-Only Views from Existing Collections
    5.4 Building a Map from a Collection
    5.5 Returning a Default When a Collection Is Empty
    5.6 Restricting a Value to a Given Range
    5.7 Processing a Window on a Collection
    5.8 Destructuring Lists
    5.9 Sorting by Multiple Properties
    5.10 Defining Your Own Iterator
    5.11 Filtering a Collection by Type
    5.12 Making a Range into a Progression
  6. Sequences
    6.1 Using Lazy Sequences
    6.2 Generating Sequences
    6.3 Managing Infinite Sequences
    6.4 Yielding from a Sequence
  7. Scope Functions
    7.1 Initializing Objects After Construction with apply
    7.2 Using also for Side Effects
    7.3 Using the let Function and Elvis
    7.4 Using let with a Temporary Variable
  8. Kotlin Delegates
    8.1 Implementing Composition by Delegation
    8.2 Using the lazy Delegate
    8.3 Ensuring That a Value Is Not Null
    8.4 Using the observable and vetoable Delegates
    8.5 Supplying Maps as Delegates
    8.6 Creating Your Own Delegates
  9. Testing
    9.1 Setting the Test Class Life Cycle
    9.2 Using Data Classes for Tests
    9.3 Using Helper Functions with Default Arguments
    9.4 Repeating JUnit 5 Tests with Different Data
    9.5 Using Data Classes for Parameterized Tests
  10. Input/Output
    10.1 Managing Resources with use
    10.2 Writing to a File
  11. Miscellaneous
    11.1 Working with the Kotlin Version
    11.2 Executing a Lambda Repeatedly
    11.3 Forcing when to Be Exhaustive
    11.4 Using the replace Function with Regular Expressions
    11.5 Converting to Binary String and Back
    11.6 Making a Class Executable
    11.7 Measuring Elapsed Time
    11.8 Starting Threads
    11.9 Forcing Completion with TODO
    11.10 Understanding the Random Behavior of Random
    11.11 Using Special Characters in Function Names
    11.12 Telling Java About Exceptions
  12. The Spring Framework
    12.1 Opening Spring-Managed Bean Classes for Extension
    12.2 Persisting Kotlin Data Classes
    12.3 Injecting Dependencies
  13. Coroutines and Structured Concurrency
    13.1 Choosing Coroutine Builders
    13.2 Replacing async/await with withContext
    13.3 Working with Dispatchers
    13.4 Running Coroutines on a Java Thread Pool
    13.5 Cancelling Coroutines
    13.6 Debugging Coroutines

I believe that’s 87 recipes total. I could have added at least a dozen more, but as Katie Mack (@AstroKatie on Twitter) once said, “the hardest thing about writing a book is that eventually you have to stop writing the book.” My book is done and I believe what’s there is solid. If I ever get to revisit it for a second edition, I’m sure I’ll have much more to say, but this hopefully will give you a good cross-section of what Kotlin can do and how to do it.

Categories
Kotlin

Kotlin Palindrome Checker

This post, based on code from my new Kotlin Cookbook, shows how to write a palindrome checker in Kotlin. Along the way it discusses raw strings and regular expressions, writing functions as single statements, and creating an extension function on String.

First, a quick definition. A palindrome is a string whose characters are the same forwards and backwards, ignoring case and with punctuation removed. Here are a few palindromes, which can serve as test cases later:

  • racecar
  • Madam, in Eden, I’m Adam.
  • Flee to me, remote elf!
  • Sex at noon taxes.
  • A Santa pets rats, as Pat taps a star step at NASA.

Make of them what you will. 🙂

To make a palindrome checker, you need to remove case sensitivity and replace all the punctuation with empty strings. The first task is trivial: just use the String functions toLowerCase or toUpperCase, whichever you prefer.

To handle regular expressions, Kotlin has a few mechanisms, but one is to use a raw string with the regex in it and then invoke the toRegex function on it.

In the case of a palindrome, a decent regex would be """[\W+]""". The square brackets represent a choice. The expression is a predefined character class \W, meaning the opposite of \w, where the lowercase version represents any word character. Word characters are lowercase a-z, uppercase A-Z, the numbers 0-9, and an underscore, _. Therefore \W represents all non-word characters, which can be used to get rid of all punctuation (other than underscores, but if that’s important, that can always be added later). The plus sign means one or more, so the overall effect is to look for one or more non-word characters. Since this post (and the book) is written for the JVM version of Kotlin, you can see the JavaDocs for java.util.regex.Pattern class for details.

The triple quotes surrounding the regex represent a “raw” string in Kotlin, so you don’t have to escape the backslash on the W when writing a regular expression. Raw strings can be written over several lines, too, but that’s not necessary here. To make the raw string into a regular expression, invoke the toRegex function on it.

"""[\W+]""".toRegex()

The tricky “replace” function

To use the regular expression, String has a function called replace, but it’s a bit tricky. There are two overloaded versions of replace, one whose first argument is a string and one whose first argument is a regex. The signatures are:

// First arg is String, last arg has default value "false"
public fun String.replace(
    oldValue: String,
    newValue: String,
    ignoreCase: Boolean
): String

and

// First arg is a regular expression
public inline fun CharSequence.replace(
    regex: Regex,
    replacement: String
): String

For the first version of replace, the library implementation shows that the ignoreCase parameter has a default value of false, so invoking the function without that parameter is legal. That, unfortunately, makes it easy to confuse with the second version, as the following test case shows.

@Test
fun `demonstrate replace with a string vs regex`() {
    assertAll(
        { assertEquals("one*two*", "one.two.".replace(".", "*")) },
        { assertEquals("********", 
              "one.two.".replace(".".toRegex(), "*")) }
    )
}

The first assertion replaces the single dots in “one.two.” with a single star, *, because it’s using the version of replace that takes a string as the first argument. That means it looks for literal dots in the string and replaces them with a star.

The second assertion uses the version of replace that takes a regular expression as the first argument. In regular expressions, a single dot represents any single character, so the result is to replace all the letters in “one.two.” with stars, as shown.

I should mention that I didn’t come up with that example on my own. I saw it in a slide found here, which is one of the presentations, hosted at SpeakerDeck, provided by the Kotlin core team members. The slides are a little dated, but still useful. You can find all of them linked from this GitHub repository.

isPalindrome Function

Returning to the palindrome checker, version 1 is shown as a function called isPalindrome that takes a string argument and returns a boolean:

fun isPalindrome(string: String): Boolean {
    val testString = string.toLowerCase()
        .replace("""[\W+]""".toRegex(), "")
    return testString == testString.reversed()
}

This works by converting the supplied string to lower case, then using the replace function with the needed regex, and then using == to see if it is equivalent to its reverse. It’s notable that in Kotlin String has a reverse function, and that operator overloading means that the == sign invokes the equals function.

While this works, it’s arguably not idiomatic. Kotlin favors single statement functions, where you can just assign a statement implementation to the function signature. The hard part in this particular function is that it is currently two statements — one to create the test string, and one to do the equality check. This is where the convenient scope function called let comes in, as you can see in version 2 of the palindrome function:

fun isPalindrome(string: String) =
    string.toLowerCase()
          .replace("""[\W+]""".toRegex(), "")
          .let { it == it.reversed() }

The replace function returns the new string, so you can chain another function call to it. Then, according to the documentation, the let function “calls the specified function block with this value as its argument and returns its result”. The provided block is a lambda with a single argument, referred to as it, and returns the last evaluated expression, which is the boolean that results from the equivalence check.

To really bring this home, this top-level function seems to be a natural extension function on the String class (or CharSequence interface) , so here is the final version of the function:

fun String.isPalindrome() =
    this.toLowerCase()
        .replace("""[\W+]""".toRegex(), "")
        .let { it == it.reversed() }

Inside the extension function, the current instance is referenced as this, but the rest of the implementation is the same.

Finally, here are a couple of tests to demonstrate that the function works correctly:

@Test
fun `these are palindromes`() {
    assertAll(
        { assertTrue("Madam, in Eden, I'm Adam".isPalindrome()) },
        { assertTrue("Flee to me, remote elf!".isPalindrome()) },
        { assertTrue("Sex at noon taxes".isPalindrome()) },
        { assertTrue("A Santa pets rats, as Pat taps a star step at NASA".isPalindrome()) }
    )
}

@Test
fun `check isPalindrome is not just returning true`() {
    assertFalse("this is not a palindrome.isPalindrome())
}

The result is a simple function that can be used for any string, and the implementation shows how to use replace, reverse, regular expressions, the let function, and how to implement a simple extension function.

As usual, the source code can be found in the GitHub repository for my Kotlin Cookbook. The tests there include one for the massive 543-word palindrome, written as a multi-line string, from the Gigantic List of Palindromes. Just being able to verify that palindrome was entertaining enough to do this entire exercise. 🙂

Categories
Kotlin

A Deep Dive into the KotlinVersion Class

Getting the current Kotlin version is easy, but the actual KotlinVersion class is much more interesting. This post shows how to get the Kotlin version programmatically, but then looks at the details of the KotlinVersion class, including how it demonstrates a great way to write an equals method and more.

Note that this demo is part of my new book, Kotlin Cookbook, from O’Reilly Media.

Book cover for Kotlin Cookbook
The Kotlin Cookbook, coming this Fall from O’Reilly Media

Also, I was lucky enough to be interviewed by Hadi Harriri on his Talking Kotlin podcast, and this class came up. That episode has not yet been released, but should be out soon depending on his current backlog.

To start, here is the trivial one-liner to find out which version of Kotlin is executing your code:

fun main() {
    println("The current Kotlin version is ${KotlinVersion.CURRENT}")
}

At the time of this writing, the current release version of Kotlin is 1.3.50, and when you run this script on that version that’s what you get.

Where life gets interesting is when you look at how the KotlinVersion class is implemented. The next series of snippets will examine that in some detail. The KotlinVersion class in the standard library is in the kotlin package, and begins:

public class KotlinVersion(val major: Int, 
                           val minor: Int, 
                           val patch: Int
) : Comparable<KotlinVersion> {

The class is marked public, which isn’t necessary (public is the default) but is typical of library classes. Then follows the primary constructor, which takes three integer values, labeled major, minor, and patch. After the colon after the signature shows that the class implements the Comparable interface for instances of itself, which is already pretty interesting. The Comparable interface in Kotlin is just like its counterpart in Java. It establishes a “natural ordering” on instances of the class.

The Comparable interface in the standard library consists of:

public interface Comparable<in T> {
    public operator fun compareTo(other: T): Int
}

Again, the word public is not necessary either on the class or the function, but doesn’t hurt anything. The compareTo function is labeled an operator function. In this case, the function is used whenever you use <, >, ==, or one of the combination comparison functions, <=, >=, or !=. The function returns an integer whose value doesn’t matter, but should be negative, zero, or positive if the current object is less than, equal to, or greater than its argument.

Returning to the KotlinVersion class, the implementation of compareTo is:

override fun compareTo(other: KotlinVersion): Int = version - other.version

This assumes that the KotlinVersion class has a version property. The primary constructor didn’t show one, so the implementation provides a private property of that name and the function to compute it:

private val version = versionOf(major, minor, patch)

private fun versionOf(major: Int, minor: Int, patch: Int): Int {
    require(major in 0..MAX_COMPONENT_VALUE && 
            minor in 0..MAX_COMPONENT_VALUE && 
            patch in 0..MAX_COMPONENT_VALUE) {
        "Version components are out of range: $major.$minor.$patch"
     }
     return major.shl(16) + minor.shl(8) + patch
}

So the version property is computed from the major, minor, and patch values. The require statement is a pre-condition that verifies the individual values fall within the required range. For all three, the minimum is zero and the maximum is MAX_COMPONENT_VALUE. Like most constants, MAX_COMPONENT_VALUE is found in the companion object:

companion object {
    public const val MAX_COMPONENT_VALUE = 255

    @kotlin.jvm.JvmField
    public val CURRENT: KotlinVersion = KotlinVersion(1, 3, 50)
}

Note the use of both const and val together. From a Java perspective, the properties inside the companion object are effectively static, and the val keyword indicates they’re both final as well. The const modifier means the value is a compile-time constant, rather than being specified at runtime. The max value is therefore hard-wired to 255, and on this version of Kotlin the CURRENT value is an instance of the KotlinVersion class where major, minor, and patch values are 1, 3, and 50, respectively.

That takes care of the require block in the versionOf function. What about the actual value it returns? That’s computed using the shift-left (or left-shift, but the other way reads better) operator function, shl. That is an infix function that shifts the current value by the specified number of bits. Its signature is given by:

public final infix fun shl(
    bitCount: Int
): Int

This is an infix function, so the idiomatic way to invoke it would be major shl 16, minor shl 8, etc, but the form used here works anyway. Basically, the function shifts by two bytes for major and one byte for minor, which gives them enough of an offset that it is very unlikely a different set of major/minor/patch values will result in the same version. For the record, using 1, 3, and 50 gives 65,536 for 1 shl 16 and 768 for 3 shl 8, and the sum from the versionOf function is 66,354:

@Test
fun `left-shift for major, minor, and patch of 1, 3, and 50`() {
    assertEquals(65536, 1 shl 16)
    assertEquals(768, 3 shl 8)
    assertEquals(66354, (1 shl 16) + (3 shl 8) + 50)
}

The major, minor, and patch values are therefore combined into a single integer, which is used for the ordering. The next interesting part is how the KotlinVersion class implements the standard overrides of toString, equals, and hashCode:

override fun toString(): String = "$major.$minor.$patch"

override fun equals(other: Any?): Boolean {
    if (this === other) return true
    val otherVersion = (other as? KotlinVersion) ?: return false
    return this.version == otherVersion.version
}

override fun hashCode(): Int = version

There’s nothing terribly surprising about the overrides of toString or hashCode. The former is just formatting, and the latter reuses that version calculation just discussed, which is pretty convenient since the left-shifted offset mechanism described above is exactly how Josh Bloch recommended creating a decent hashCode function in his Effective Java book for the last twenty-some-odd years. 🙂

The real fun is in the override of the equals function. The KotlinVersion class, like all Kotlin classes, extends Any. As a reminder, the Any class looks like:

package kotlin

public open class Any {
    public open operator fun equals(other: Any?): Boolean
    public open fun hashCode(): Int
    public open fun toString(): String
}

Returning to the implementation of equals in KotlinVersion, the first check uses the triple equals operator === to see if the current reference and the argument are both pointing to the same instance. If so, the result is equal and no further checking is necessary.

If the two objects are different, the code needs to check the version property. Because the argument to the equals function is nullable, you can’t simply access the version property without checking for null first. The safe cast operator, as?, is used to check that. If the argument is not null, this casts it to an instance of KotlinVersion. If the argument is null, the safe cast operator returns null, so the Elvis operator, ?:, is used to simply return false rather than continue.

That’s really interesting, actually. The “return false” statement aborts the assignment of the otherVersion property. The results is Nothing, a class almost guaranteed to confuse Java developers, but beyond the scope of this discussion. Suffice it to say that because Nothing is a subclass of every other class, the resulting type of otherVersion is KotlinVersion, as desired.

Assuming we make it to the last line in the equals method, now there is a value for otherVersion and a value for the current version. The comparison then checks version == otherVersion.version for equality (since they’re both Int values) and returns the result.

That’s quite a lot of power for three lines of code, and is a great example of how to implement an equivalence check for a nullable property (which happens to be another recipe in the book).

To complete the story, the class has a secondary constructor that can be used when the patch value is unknown.

public constructor(major: Int, minor: Int) : this(major, minor, 0)

Then there are two overloads of the isAtLeast function:

public fun isAtLeast(major: Int, minor: Int): Boolean =
    this.major > major || 
        (this.major == major && this.minor >= minor)

public fun isAtLeast(major: Int, minor: Int, patch: Int): Boolean =
    this.major > major || 
        (this.major == major &&
            (this.minor > minor || 
                 this.minor == minor && this.patch >= patch))

A couple of tests show how the basic comparisons work:

@Test
fun `comparison of KotlinVersion instances work`() {
    val v12 = KotlinVersion(major = 1, minor = 2)
    val v1341 = KotlinVersion(1, 3, 41)
    assertAll(
        { assertTrue(v12 < KotlinVersion.CURRENT) },
        { assertTrue(v1341 <= KotlinVersion.CURRENT) },
        { assertEquals(KotlinVersion(1, 3, 41),
            KotlinVersion(major = 1, minor = 3, patch = 41)) }
    )
}

@Test
fun `versions are Ints less than max`() {
    val max = KotlinVersion.MAX_COMPONENT_VALUE
    assertAll(
        { assertTrue(KotlinVersion.CURRENT.major < max) },
        { assertTrue(KotlinVersion.CURRENT.minor < max) },
        { assertTrue(KotlinVersion.CURRENT.patch < max) }
    )
}

Because the KotlinVersion class implements Comparable, it can be used in a range and has a contains method. In other words, you can write:

@Test
fun `check current version inside range`() {
    assertTrue(KotlinVersion.CURRENT in 
        KotlinVersion(1,2)..KotlinVersion(1,4))
}

What you can not do, however, is to iterate over that range, because ranges are not progressions, but that’s a post for another day.

I hope you enjoyed this deep dive into what is arguably a trivial, but highly instructive, class in the Kotlin library. The GitHub repository for the book is located here and contains this example as well as many, many others.

Categories
Java

Java 11 HttpClient, Gson, Gradle, and Modularization

This post describes a simple system that uses the new HttpClient class that comes with Java 11 to access a RESTful web service. After demonstrating basic functionality, changes are made to parse the results using Gson, and even modularize the code and have it work with Gradle. The goal is to provide an example for anyone who wants to do similar tasks with their own systems.

Java 11 is a Long Term Support (LTS) release for the Java standard edition, meaning bug fixes and security patches will be available for at least the next three years (see the extensive Java Is Still Free post at for details about the future licensing arrangements). Java 11 contains very few new features, however — many more were added in Java 9 and 10. Most of the industry did not upgrade during those releases, however. Now that another LTS release is available, that will likely change.

One of the few real additions to Java 11 is the HttpClient API, as described in JEP 321. This takes a related incubator project from Java 9 and standardizes it for Java 11. This blog post will not be an exhaustive demonstration of its capabilities, but it will show both a synchronous and asynchronous example of how to use it, along with JUnit 5 test cases and a Gradle build file, and even modularize the code.

The web service is provided by Open Notify, which is an open source project that provides a few links to data from NASA. In this case, the example uses the “number of people in space” link at http://api.open-notify.org/astros.json. The current response to that link is:

[code language=”javascript”]
{“message”: “success”,
“people”: [
{“craft”: “ISS”, “name”: “Oleg Artemyev”},
{“craft”: “ISS”, “name”: “Andrew Feustel”},
{“craft”: “ISS”, “name”: “Richard Arnold”},
{“craft”: “ISS”, “name”: “Sergey Prokopyev”},
{“craft”: “ISS”, “name”: “Alexander Gerst”},
{“craft”: “ISS”, “name”: “Serena Aunon-Chancellor”}],
“number”: 6}
[/code]

The root object contains attributes message, number, and people, the last of which is a collection of objects, each of which contains a name and a craft. As the response shows, at the moment there are six people aboard the International Space Station.

Even though the plan is to map that response to Java classes, at first the example will simply download the JSON data and write it as a string to the console.

The HttpClient class is in the java.net.http package. You create one using a builder, as in:
[code language=”java”]
HttpClient client = HttpClient.newBuilder()
.version(HttpClient.Version.HTTP_2)
.connectTimeout(Duration.ofSeconds(2))
.build();
[/code]

Two of the many configuration options are shown: using HTTP/2 (which is actually the default) and setting a two-second connection timeout period. Other options, like setting a proxy, following redirects, or using an authenticator are shown in the (early access) JavaDocs.

Requests can be sent synchronously or asynchronously as desired, using the send or sendAsync methods. Either way, the first argument to those methods is an instance of HttpRequest, which is also created using a builder:

[code language=”java”]
String astroUrl = “http://api.open-notify.org/astros.json&#8221;;
HttpRequest.newBuilder()
.uri(URI.create(astroUrl))
.GET()
.build();
[/code]

Sending a synchronous request also requires a “body handler”, which comes from the HttpResponse.BodyHandlers inner class factory methods. For this case, it’s easiest just to use the ofString method:

[code language=”java”]
HttpResponse response = client.send(
buildRequest(),
HttpResponse.BodyHandlers.ofString());
String output = response.body();
[/code]

From the HttpResponse you can get the headers, the body (as shown), the status code, or several other properties.

An asynchronous request is sent using the sendAsync method, which returns a CompletableFuture. Extracting the result can be done with code like:
[code language=”java”]
String output = client.sendAsync(buildRequest(),
HttpResponse.BodyHandlers.ofString())
.thenApply(HttpResponse::body)
.get();
[/code]

The thenApply method from CompletableFuture takes a java.util.function.Function argument that in this case maps the response into a String containing the body. The subsequent get method blocks until the request and conversion has completed.

To summarize, here is the initial version of the class called AstroClient:
[code language=”java”]
package astro;

import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
import java.util.concurrent.ExecutionException;

public class AstroClient {
private HttpClient client = HttpClient.newBuilder()
.version(HttpClient.Version.HTTP_2)
.connectTimeout(Duration.ofSeconds(2))
.build();

private HttpRequest buildRequest() {
String astroUrl = “http://api.open-notify.org/astros.json&#8221;;
return HttpRequest.newBuilder()
.uri(URI.create(astroUrl))
.GET()
.build();
}

public String getSync() throws IOException, InterruptedException {
HttpResponse response = client.send(
buildRequest(),
HttpResponse.BodyHandlers.ofString());
return response.body();
}

public String getAsync() throws ExecutionException, InterruptedException {
return client.sendAsync(buildRequest(),
HttpResponse.BodyHandlers.ofString())
.thenApply(HttpResponse::body)
.get();
}

}
[/code]

Ultimately, the tests will use some cool new methods from JUnit 5, but for the moment if you just invoke the getSync and getAsync methods and log the output using a java.util.logging.Logger you get something like:


Sep 22, 2018 12:45:54 PM astro.AstroClientTest getSync
INFO: {"message": "success", "people": [{"craft": "ISS", "name": "Oleg Artemyev"}, {"craft": "ISS", "name": "Andrew Feustel"}, {"craft": "ISS", "name": "Richard Arnold"}, {"craft": "ISS", "name": "Sergey Prokopyev"}, {"craft": "ISS", "name": "Alexander Gerst"}, {"craft": "ISS", "name": "Serena Aunon-Chancellor"}], "number": 6}

Sep 22, 2018 12:45:54 PM astro.AstroClientTest getAsync
INFO: {"message": "success", "people": [{"craft": "ISS", "name": "Oleg Artemyev"}, {"craft": "ISS", "name": "Andrew Feustel"}, {"craft": "ISS", "name": "Richard Arnold"}, {"craft": "ISS", "name": "Sergey Prokopyev"}, {"craft": "ISS", "name": "Alexander Gerst"}, {"craft": "ISS", "name": "Serena Aunon-Chancellor"}], "number": 6}

Obviously we can do better if we parse the response. To do that, add the Google Gson dependency and use it to convert the JSON response to a pair of classes. The classes that map to the JSON structure are Assignment:
[code language=”java”]
package astro.json;

public class Assignment {
private String name;
private String craft;

public String getName() {
return name;
}

public void setName(String name) {
this.name = name;
}

public String getCraft() {
return craft;
}

public void setCraft(String craft) {
this.craft = craft;
}
}
[/code]
which includes fields for the name and craft, and AstroResponse:
[code language=”java”]
package astro.json;

import java.util.List;

public class AstroResponse {
private String message;
private Integer number;
private List people;

public String getMessage() {
return message;
}

public void setMessage(String message) {
this.message = message;
}

public Integer getNumber() {
return number;
}

public void setNumber(Integer number) {
this.number = number;
}

public List getPeople() {
return people;
}

public void setPeople(List people) {
this.people = people;
}
}
[/code]

These POJOs could be simplified by using

  • Lombok annotations
  • Groovy POGOs, preferably with the @Canonical annotation
  • Kotlin data classes

among other ways. Any of those would cut the POJOs down to just their attributes, but these are small enough to be readable even without that.

The changes to the AstroClient class are to add a method that converts the String response to an instance of AstroResponse:
[code language=”java”]
package astro;

import astro.json.AstroResponse;
import com.google.gson.Gson;

import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
import java.util.concurrent.ExecutionException;

public class AstroClient {
private HttpClient client = …

private Gson gson = new Gson();

private HttpRequest buildRequest() {
// … as before …
}

public AstroResponse getSync() throws IOException, InterruptedException {
HttpResponse response = client.send(
buildRequest(),
HttpResponse.BodyHandlers.ofString());
return getResponse(response.body());
}

public AstroResponse getAsync() throws ExecutionException, InterruptedException {
String json = client.sendAsync(buildRequest(),
HttpResponse.BodyHandlers.ofString())
.thenApply(HttpResponse::body)
.get();
return getResponse(json);
}

private AstroResponse getResponse(String json) {
return gson.fromJson(json, AstroResponse.class);
}
}
[/code]

Now that the methods return an AstroResponse, the tests can get more interesting. Here is the complete AstroClientTest class:
[code language=”java”]
package astro;

import astro.json.Assignment;
import astro.json.AstroResponse;
import org.junit.jupiter.api.Test;

import java.time.Duration;
import java.util.List;
import java.util.logging.Logger;

import static org.junit.jupiter.api.Assertions.*;

class AstroClientTest {
private Logger logger = Logger.getLogger(AstroClientTest.class.getName());

private AstroClient client = new AstroClient();

@Test
void getSync() {
AstroResponse response = assertTimeoutPreemptively(
Duration.ofSeconds(2),
() -> client.getSync());

int num = response.getNumber();
List assignments = response.getPeople();

assertEquals(“success”, response.getMessage());
assertEquals(num, assignments.size());
assignments.forEach(assignment ->
assertAll(() -> assertTrue(assignment.getName().length() > 0),
() -> assertTrue(assignment.getCraft().length() > 0)));

logResponse(num, assignments);
}

@Test
void getAsync() {
AstroResponse response = assertTimeoutPreemptively(
Duration.ofSeconds(2),
() -> client.getAsync());

int num = response.getNumber();
List assignments = response.getPeople();

assertEquals(“success”, response.getMessage());
assertEquals(num, assignments.size());

logResponse(num, assignments);
}

private void logResponse(int num, List assignments) {
logger.info(String.format(“There are %d people in space”, num));
assignments.forEach(person -> logger.info(
() -> String.format(“%s aboard %s”,
person.getName(),
person.getCraft())));
}
}
[/code]

The first test uses the neat assertTimeoutPreemtively method. That causes JUnit to spawn a separate thread for the test, which it interrupts if the test does not complete within the specified Duration. The second argument is a ThrowingSupplier, which is like a regular java.util.function.Supplier except that its single abstract method, get, is declared to throw Throwable.

After checking that the message field is “success” and that the size of the Assignment collection (called people to match the JSON attribute) has the right size, each individual Assignment is checked to ensure its name and craft fields are not just empty strings. The assertAll method in JUnit 5 takes a variable argument list of Executable instances, which are like Runnable instances but can throw Exception. The nice thing about assertAll is that all of its Executable arguments will be evaluated, even if some of them fail. The asynchronous test does the same thing.

The private logResponse method just uses the logger to write out how many people are in space and their names and crafts:


Sep 22, 2018 4:58:27 PM astro.AstroClientTest logResponse
INFO: There are 6 people in space
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Oleg Artemyev aboard ISS
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Andrew Feustel aboard ISS
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Richard Arnold aboard ISS
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Sergey Prokopyev aboard ISS
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Alexander Gerst aboard ISS
Sep 22, 2018 4:58:27 PM astro.AstroClientTest lambda$logResponse$6
INFO: Serena Aunon-Chancellor aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest logResponse

INFO: There are 6 people in space
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Oleg Artemyev aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Andrew Feustel aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Richard Arnold aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Sergey Prokopyev aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Alexander Gerst aboard ISS
Sep 22, 2018 4:58:28 PM astro.AstroClientTest lambda$logResponse$6
INFO: Serena Aunon-Chancellor aboard ISS

To cap it all off, here is the Gradle build file (using the current version of Gradle, 4.10.2):

[code language=”groovy”]
plugins {
id ‘java’
}

group ‘kousenit.com’
version ‘1.0’

sourceCompatibility = 11

repositories {
jcenter()
}

dependencies {
implementation ‘com.google.code.gson:gson:2.8.5’

testImplementation ‘org.junit.jupiter:junit-jupiter-api:5.3.1’
testRuntimeOnly ‘org.junit.jupiter:junit-jupiter-engine:5.3.1’
}

test {
useJUnitPlatform()
}
[/code]

There’s nothing terribly surprising in that file to experienced Gradle users, though as a reminder the useJUnitPlatform method was added to Gradle to make it easier to run JUnit 5 tests.

Honestly, this is good enough. There’s no particular reason to go on and modularize this code; it all works fine on Java 11. However, just to show the kinds of changes needed if you’re foolish enough to add a module-info.java file to the class path, here’s what happens:

First, you need to add the necessary requires statements to the module-info.java file, which resides in the src/main/java directory:

[code language=”java”]
module http.client.demo.main {
requires java.net.http;
requires gson;
}
[/code]

The module name can be anything, but the same “reverse dns” pattern favored for packages is recommended. The name here is the one auto-generated by IntelliJ IDEA if you ask for a new module-info descriptor.

The HttpClient API was in an incubator module in Java 9, which was called jdk.incubator.httpclient. Now that it is standardized, it’s now in the java.net.http module, so that’s the module needed.

The hard part was figuring out how to refer to Gson. Since that API has not yet been modularized, the associated jar files wind up in the “unnamed” module and you need to refer to them by name rather than some official module name. Enough googling (no pun intended) and eventually you’ll find that the right name to use is “gson”. Of course, that’s fragile and will no doubt change at some point in the future.

If you’ve played with modules in Java 9 or 10, you’ll notice something conspicuous by its absence. Prior to Java 11, the Java util logger (the one in the java.util.logging package) was part of a separate module in the JDK called java.logging. In Java 11 that was moved back into java.base, so no additional requires statement is necessary.

The other necessary change is inside the Gradle build file. You have to tell Gradle to use the module path rather than the class path for compilation. Based on the Gradle Guide called Building Java 9 Modules, the following additional block was necessary:

[code language=”groovy”]
compileJava {
doFirst {
options.compilerArgs = [
‘–module-path’, classpath.asPath,
]
classpath = files()
}
}
[/code]

This code adds a doFirst block to the compileJava task, which is a pre-processing step. It adds the --module-path compiler argument and assigns it to the jars on the classpath. Then the block sets the classpath variable to an empty collection of files.

The Gradle Guide also suggests that the compileTestJava and test tasks also need to be modified, but those turned out not to be necessary.

(Those are the only needed changes, which, looking back on them, was a bit disappointing given how much time it took to figure them out. But that’s no doubt a good thing.:))

The complete code is hosted in GitHub at https://github.com/kousen/http-client-demo . The master branch has everything, including the modularization.

As a reminder, I have a book that discusses the new features added to Java 8 (and much of Java 9) called Modern Java Recipes. The GitHub repositories for the book are https://github.com/kousen/java_8_recipes and https://github.com/kousen/java_9_recipes, both of which now build correctly under Java 11.

Categories
Management

Managing Your Manager: The Video Course

I’m a regular speaker on the No Fluff, Just Stuff tour (I’m now entering my ninth year, which I can hardly believe), and one of the talks I’ve given there for many years is called “Managing Your Manager”. It’s been one of my most successful talks, and I’ve given several versions of it, including as keynotes at UberConf and DevNexus, the latter of which was recorded on a YouTube video.

Over the past couple of years I’ve been recording video courses for Safari Books Online. Until now they’ve all been developer focused, ranging from a Spring Framework Essentials and Reactive Spring to Understanding Java 8 Generics to Gradle Fundamentals to Practical Android. I have about a dozen courses there, all of which involve technical topics. Those courses normally consist of me preparing materials ahead of time, then going to the O’Reilly Media offices in either Boston, MA or Sebastopol, CA, and talking to a camera for a few days. As a frequent conference speaker and someone who teaches training classes for a living, this is quite natural for me.

Late last year I decided it was finally time to turn my Managing Your Manager talk into a video course, which is now available on Safari. This course is an expanded version of the conference talk, based on what I’ve learned in my roughly 30 years in industry, dealing with big companies and small, working with managers ranging from very good to horrible.

The basic concept is that the vast majority of the literature on management is aimed at managers, trying to show them how to be more successful so that they can ultimately rise to what’s referred to as the “C-level suite”, i.e., CEO, CIO, CTO, and so on. The problem is, there’s very little of quality written for the regular worker who has to deal with management all the time but doesn’t necessarily aspire to line management roles.

This comes up a lot in areas related to I.T., though it applies outside that area as well. Many developers are interested in writing code or other software architecture related tasks, and to them success involves accomplishing their goals with well-written, testable, maintainable code that embodies the latest thinking in software development and architectural patterns. They generally care much more about the details of the project than the interpersonal or financial aspects, though those will influence what they do in a concrete way. They may even take the role of a tech lead and have some limited management responsibilities as a result.

Line managers, however, of the type I discuss in the video, have different goals. They are interested in building successful projects as a way for them to gain more responsibility and experience and use that as a way and rise in the organization or prepare for a move to a new company with greater opportunities. This has a significant impact on how they think, and understanding that directly affects how we deal with managers.

A story I use in the video involves a famous actor during his first experience as a movie director. The other actors on the set knew this person as an actor, so they liked having actor-type discussions with him, like the back-story of a given character, deeper motivations that drive the plot, events occurring outside a given scene, and so on. The director, as an actor, also liked those discussions. But as he said in the article:

The whole time I had a little voice in the back of my head saying, “we’re losing the light, we’re going to have the pay the crew overtime, I’ve got to process the dailies for tomorrow, I have to meet with the financiers, and so on.

In other words, despite the director’s years of training and success as an actor, he knew that he was going to be evaluated based on money. The budget was just as important to the definition of success for him as the artistic aspects of the movie.

(BTW, I would link to that interview, but sadly I don’t remember who it was or where I saw it. From what I gather, that’s a pretty common experience though.)

Managers are strongly focused on the money aspects of any project, and the higher you go in management, the more that dominates their thinking. Knowing that is an important part of building a successful relationship with a manager. You can’t change their focus, but you can take that into account when you work with them, and it often helps to explain their actions, especially when they don’t do what you would like.

Other topics that come up in the video course include:

  • The natural reason why so many of our managers so bad
  • The crucial question of loyalty
  • Strategies you can use from the Iterated Prisoner’s Dilemma problem
  • Determining your manager’s communication preferences

That last issue makes use of personality typing techniques, which are controversial at best. The controversy usually arises from applying them incorrectly in industry. When I talk about them, I’m not trying to convince you that they describe reality in any detail, and I’m certainly not trying to get you spend any money. I just use them to help you figure out how to make your arguments in a way that is most likely to be heard by your manager, maximizing the likelihood you will get what you want.

I also have a series of what I call constructive loyalty “How To’s”, like how good enough answers today are much, much better than great answers next week, and understanding that Your Boss Is Not Your Friend, and how to use lessons from Reflective Listening to participate in discussions with your manager when you would rather not, and finally, the Best Way To Tell Your Boss He Or She Is Completely Wrong.

I’m hoping to create a series of blog posts here to describe some of these features in detail, assuming I can free up time from other commitments. Many of the topics I talk about do not appear to be common knowledge, and have come from several bad decisions I made during my career (old saying: “Good judgement comes from experience, but experience comes from bad judgement”), so I want to help others benefit from my thinking and especially from my mistakes.

The inimitable Dan Woods (@danveloper on Twitter and now a manager at Target in Minneapolis) once called a version of this talk the “best talk of the conference” at UberConf, and a friend of mine who is a former Senior Vice President at a Fortune 500 company agreed with many of the recommendations and offered his own advice in several areas. As I say, it’s been one of my most successful talks for years, and now it’s recorded, in expanded form, for anyone to see.

(Okay, anyone with a Safari account, but you can always sign up for the trial, watch the video, and then cancel. I won’t say anything if you won’t. Besides, Safari Books Online has tons of content, from books to videos to training classes. It’s basically the Netflix of technical (and business!) content, so you might like it anyway, especially if you can get your manager to pay for it.)

Since any reputation that I have is based on my work in technical areas, this video might not otherwise be noticed by people outside my direct community. I’m mentioning it here to try to give it a bit more exposure. If you like it, great! Feel free to leave a comment on it. Otherwise, hopefully you’ll find a few ideas to think about in your future career.

Finally, feel free to tell me “that turns out not to be the case” on Twitter (I’m @kenkousen there) if you disagree with everything I said.

Categories
Java

Java 8 Constructor Refs (In All Their Glory)

[Note: my last post announced my new book, Modern Java Recipes, is now available from O’Reilly publishers in Early Release form. As a sample, I included a discussion of the Predicate interface, one of the new functional interfaces in the the java.util.function package. In this post, I highlight constructor references, which are discussed in another recipe in the book.]

Problem

You want to instantiate an object using a method reference as part of a stream pipeline.

Solution

Use the new keyword as part of a method reference.

Discussion

When people talk about the new syntax added to Java 8, they mention lambda expressions, method references, and streams. For example, say you had a list of people and you wanted to convert it to a list of names. One way to do so would be:

[sourcecode language=”java”]
List<String> names = people.stream()
.map(person -> person.getName()) // lambda expression
.collect(Collectors.toList());
[/sourcecode]

In other words, take a list of Person instances, turn it into a stream, map each one to a String by invoking the getName() method on it, and collect them back into a List.

That works, but most developers would use a method reference instead:

[sourcecode language=”java”]
List<String> names = people.stream()
.map(Person::getName) // method reference
.collect(Collectors.toList());
[/sourcecode]

The method reference is slightly shorter, and makes clear that the only thing being done to each person is transforming it using the getName method. Lambda expressions can be far more complicated and versatile. Method references are simple.

What if you want to go the other way? What if you have a list of strings and you want to create a list of Person references from it? In that case you can use a method reference again, but this time using the keyword new. That’s a constructor reference, which I would like to illustrate here.

First, here is the Person class, which is just about the simplest Plain Old Java Object (POJO) imaginable. All it does is wrap a simple string attribute called name.

[sourcecode language=”java”]
public class Person {
private String name;

public Person() {} // default constructor

public Person(String name) {
this.name = name;
}

public String getName() { return name; }
public void setName(String name) { this.name = name; }

// … equals, hashCode, toString methods …
[/sourcecode]

Given a list of string names, I can map them to Person instances using the one-argument constructor.

[sourcecode language=”java”]
List<String> names =
Arrays.asList("Grace Hopper", "Barbara Liskov", "Ada Lovelace",
"Karen Spärck Jones");

List<Person> people =
names.stream()
.map(name -> new Person(name)) // lambda expression
.collect(Collectors.toList());
[/sourcecode]

Now instead of using the lambda expression that invokes the one-argument constructor directly, I can use a constructor reference instead.

[sourcecode language=”java”]
names.stream()
.map(Person::new) // Constructor reference
.collect(Collectors.toList());
[/sourcecode]

Like all lambda expression or method references, context is everything. The map method is invoked on a stream of strings, so the Person::new reference is invoked on each string in the stream. The compiler recognizes that the Person class has a constructor that takes a single string, so it calls it. The default constructor is ignored.

Copy Constructors

To make things more interesting, I’ll add two additional constructors: a “copy constructor” that takes a Person argument, and one that takes a variable argument list of strings.

[sourcecode language=”java”]
public Person(Person p) { // copy constructor
this.name = p.name;
}

public Person(String… names) { // varargs constructor
this.name = Arrays.stream(names)
.collect(Collectors.joining(" "));
}
[/sourcecode]

The copy constructor makes a new Person from an existing Person instance. Say I defined a person, then used that person in a stream without mapping it, and then converted back into a collection. Would I still have the same person?

[sourcecode language=”java”]
Person before = new Person("Grace Hopper");

List<Person> people = Stream.of(before)
.collect(Collectors.toList());
Person after = people.get(0);

assertTrue(before == after); // exact same object

before.setName("Grace Murray Hopper"); // Change name using ‘before’
assertEquals("Grace Murray Hopper", after.getName()); // Same in ‘after’
[/sourcecode]

The point is, if I have a reference to Admiral Hopper before the stream operations and I didn’t map her to another object, I still have the same reference afterwards.

Using a copy constructor I can break that connection.
[sourcecode language=”java”]
people = Stream.of(before)
.map(Person::new) // use copy constructor
.collect(Collectors.toList());
after = people.get(0);
assertFalse(before == after); // different objects
assertEquals(before, after); // but equivalent

before.setName("Rear Admiral Dr. Grace Murray Hopper"); // Change using ‘before’
assertFalse(before.equals(after)); // No longer the same in ‘after’
[/sourcecode]

This time, when invoking the map method, the context is a stream of Person instances. Therefore the Person::new syntax invokes the constructor that takes a Person and returns a new, but equivalent, instance. I’ve broken the connection between the before reference and the after reference.

(Btw, I mean no disrespect by treating Admiral Hopper as an object. I have no doubt she could still kick my a**, and she passed away in 1992.)

Varargs Constructors

The varargs constructor is invoked by the client by passing zero or more string arguments separated by commas. Inside the constructor, the names variable is treated like String[], a string array. The static stream method on the Arrays class is used to convert that into a stream, which is then turned into a single string by calling the collect method, whose argument comes from the convenient joining(String delimiter) method in the Collectors class.

How does that get invoked? Java includes a split method on String that takes a delimiter and returns a String array:
[sourcecode language=”java”]
String[] split(String delimiter)
[/sourcecode]

Since the variable argument list is equivalent to an array, I can use that method to invoke the varargs constructor.

[sourcecode language=”java”]
names.stream() // Stream<String>
.map(name -> name.split(" ")) // Stream<String[]>
.map(Person::new) // Stream<Person> using String… ctor
.collect(Collectors.toList());
[/sourcecode]

This time I map the strings to string arrays before invoking the constructor. Note that this is one of those times where I can’t use a method reference, because there’s no way using a method reference to supply the delimiter argument. I have to use the lambda expression instead.

Since the context after the first map is now a stream of string arrays, the Person::new constructor reference now uses the varargs constructor. If I add a print statement to that constructor:

[sourcecode language=”java”]
System.out.println("Varargs ctor, names=" + Arrays.asList(names));
[/sourcecode]

I can then see it in action:

Varargs ctor, names=[Grace, Hopper]
Varargs ctor, names=[Barbara, Liskov]
Varargs ctor, names=[Ada, Lovelace]
Varargs ctor, names=[Karen, Spärck, Jones]

Arrays

With constructor references, not only can you create instances, you can even create arrays. Say instead of returning a List, I wanted to return an array of person, Person[]. The Stream class has a method called, naturally enough, toArray.

[sourcecode language=”java”]
<A> A[] toArray(IntFunction<A[]> generator)
[/sourcecode]

This method uses A to represent the generic type of the array returned containing the elements of the stream, which is created using the provided generator function. The cool part is that a constructor reference can be used for that, too.

[sourcecode language=”java”]
names.stream()
.map(Person::new) // Person constructor ref
.toArray(Person[]::new); // Person[] constructor ref
[/sourcecode]

The returned value is a Person[] with all the stream elements now included as Person references.

Constructor references are just method references by another name, using the word new to invoke a constructor. Which constructor is determined by the context, as usual. This technique gives a lot of flexibility when processing streams.