Twitter Follower Value in Groovy

Nat Dunn, who runs the training company Webucator, posted an interesting idea on his blog. He was thinking about Twitter, and wondering about how the number of followers and friends (he called them “followees”) affected the likelihood of someone actually reading your tweets. If a person is following too many people, they can’t possibly read them all, and if they have millions of followers they can’t possibly pay attention to all of them.

He proposed a metric he called Twitter Follower Value (TFV), which is simply the ratio of the number of followers to the number of friends a person has. A person with a huge TFV like Tim O’Reilly (1,428,799 followers / 644 friends = 2,218.6) is a valuable person to have following you.

He also proposed a Total Twitter Follower Value (TTFV), which is the sum of the TFV’s of all of your followers. He then finished by saying, “So there it is. Anyone want to build a tool that calculates TTFV? That would be cool?”

Say no more. Twitter has a URL-based API that returns XML, making it ideal for Groovy experimentation.

package com.kousenit.twitter

class TwitterFollowerValue {	

    def countFriendsAndFollowers(id) {
        def url = "http://api.twitter.com/1/users/show.xml?id=$id"
        def response = new XmlSlurper().parse(url)
        [response.friends_count,response.followers_count]*.toInteger()
    }

    def getTFV(id) {
        def (numFriends,numFollowers) = countFriendsAndFollowers(id)
        numFollowers / numFriends
    }
}

The countFriendsAndFollowers method takes an id as an argument (either a Twitter name or an ID). It uses an XmlSlurper to make a GET request and parse the resulting XML response, returning a reference to the root of the tree. If you check the documentation for the show request, you’ll see that the root element is <user>, which has among its direct children elements <friends_count> and <followers_count>. In Groovy, each can be accessed with a simple dot operator. Here I extract both values and convert them to integers before returning them as a list.

The getTFV method invokes the countFriendsAndFollowers method and uses the cool Groovy 1.6+ capability of assigning multiple return values individually to variables. It then divides the number of followers by the number of friends and returns the result.

Here’s a test case to demonstrate how it all works. It’s an integration test because I decided not to mock Twitter (insert your own joke here), so it will fail if I get any more followers or friends, but as a first pass it worked fine.

package com.kousenit.twitter;

import static org.junit.Assert.*;

import org.junit.Before;
import org.junit.Test;

class TwitterValueTest {
    TwitterFollowerValue tv

    @Before
    public void setUp() throws Exception {
        tv = new TwitterFollowerValue()
    }

    @Test
    public void testCountFriendsAndFollowers() {
        def (friendsCount,followersCount) = tv.countFriendsAndFollowers('kenkousen')
        assertEquals 90, friendsCount
        assertEquals 108, followersCount
    }

    @Test
    public void testGetTFV() {
        assertEquals 1.2, tv.getTFV('kenkousen'), 0.0001
    }
}

(Yes, my Twitter id is @kenkousen)

What about TTFV? To do that, you need to find out who is following a person and then compute the TFV for each of them. The Twitter API includes a URL for retrieving the id’s of a given user, which makes that doable.

    // add to TwitterFollowerValue class...

    def getFollowerIds(id) {
        def url = "http://api.twitter.com/1/followers/ids.xml?id=$id&cursor=-1"
        def response = new XmlSlurper().parse(url)
        return response.ids.id
    }

    def getTTFV(id) {
        def totalTTFV = 0.0
        getFollowerIds(id).each { followerId ->
            totalTTFV += getTFV(followerId)
        }
        return totalTTFV
    }
}

The documentation for getting the follower id’s shows that the result is an XML tree with root <id_list>, which has a child element <ids>, and then all the individual id’s are wrapped in <id> grandchildren. Again this is trivial Groovy; you just traverse the tree and it all works.

There is a complication, in that the returned number of id’s is limited to 5000 a page. That’s what the cursor parameter is for. If someone has more than 5000 followers (and most of the big names do), then you’d have to go through the pages one by one to get all the id’s. As it turns out, that wasn’t my biggest problem, as you’ll see.

Computing the Total TFV is simply a case of computing TTV’s for each of the followers.

Now for the test to show that value. Unfortunately, I have no idea what that value is for me. I could have manufactured a test user and tried it out, but instead I just added the following to my test case:

    // add to test case above ...
    @Test
    public void testGetFollowerIds() {
        assertEquals 108, tv.getFollowerIds('kenkousen').size()
    }

// Serious rate-limiter problems guaranteed
//    @Test
//    public void testGetTTFV() {
//        println tv.getTTFV('kenkousen')
//    }

Checking the number of followers was easy. The problem was the commented-out method, which I was going to use just to see what the value was.

It turns out that the Twitter API limits the number of GET requests to 150/hour. If you apply and get on a whitelist, you can get that increased to 20,000/hour. For this calculation, though, that’s going to get used up really, really fast for almost anybody. Even checking my own followers (many of whom are no doubt spammers who follow tons of people) used up my quota almost right away.

Still, it was an interesting experiment, and any excuse to play with Groovy for an hour or so is a good one. 🙂

About Ken Kousen
I teach software development training courses. I specialize in all areas of Java and XML, from EJB3 to web services to open source projects like Spring, Hibernate, Groovy, and Grails. Find me on Google+ I am the author of "Making Java Groovy", a Java / Groovy integration book published by Manning in the Fall of 2013, and "Gradle Recipes for Android", published by O'Reilly in 2015.

4 Responses to Twitter Follower Value in Groovy

  1. Pingback: Twitter Follower Value, revisited « Stuff I’ve learned recently…

  2. This is nice I teach software development training courses.which will be published by Manning some time this year.

  3. Abie says:

    Great! Wow you really impressed me with this. Thank you for sharing. Twitter has a URL-based API that returns XML, making it ideal for Groovy experimentation 🙂

  4. directory says:

    Hi there, i read your blog occasionally and i own a similar
    one and i was just curious if you get a lot of spam feedback?
    If so how do you reduce it, any plugin or anything you can
    suggest? I get so much lately it’s driving me insane so any assistance is very much appreciated.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: