Percy’s Random Ramblings

Fun with Powershell, Part 1 – Intro/Twitter API Monday, 01/09/2012

Posted by Percy in Fun with Powershell, Programming, Technology.
Tags: C#, Development, PowerShell, Programming
trackback

I’ve really been enjoying learning more about Powershell.  I’ve actually gotten to the point that I think about using it first, and then writing a full-fledged application second.  In the past, I’ve listed some of the things I’ve been able to do with Powershell, but I thought I might dig a little deeper and provide some more detailed info about the what and how of those kind of things.  Too, whenever I talk to developers or technology folks about Powershell, they all have the same reaction – “Isn’t that more for system administrators than for developers?”.  While I fervently agree that anyone administering any kind of system needs to know Powershell for their own sanity, it’s a great tool for developers as well.  So, I thought I would highlight how developers can use Powershell by throwing out some real world examples of how I’ve used it.  Also, I’m assuming that you know the basics of Powershell – declaring variables ($), calling methods on objects, piping input into commands, creating new objects, etc.  If you need more clarification, please let me know.

I don’t know how long this series will be, but here we go with part 1.

I was reading through my twitter feed the other day, and I came across this post from The Oatmeal:

“Is there a way to sort my Twitter followers by the number of followers they have? (in descending order)”

My first thought was “I’m sure I can do that in Powershell”.  Now, there are a number of web apps out there that will actually do this for you.  I even think the next tweet points to friendorfollow.com.  However, I thought it would be a neat learning exercise.  Before we begin, I figured I’d throw this out there – I honestly have no idea how this will work.  I don’t have some stock twitter API in my back pocket ready to use.  So, this will be “from scratch”.  So, here’s how I did it.

First off, I’m assuming that there is a twitter API, so I go to twitter’s website, and I click the link at the bottom labeled “Developers” (hey, that’s me!).  That takes me to dev.twitter.com, which has a link for Getting Started with the API.  Bingo!  After looking around there, I find that it’s just HTTP requests of the format “https://api.twitter.com/{Version}/{Controller}/{Action].{Format}?{Parameters}”.  So, in this case, the first thing to do is get the followers for a particular user.  So, the URL I’m going to use is https://api.twitter.com/1/friends/ids.xml?screen_name={your user name}, since I want to use the XML format.  That returns the list of all my followers.  So, now I need to get this into Powershell by firing up my trusted Posh IDE – PowerGUI.

With Powershell, you have access to any .NET library, including the ones in the core.  There’s a class called System.Net.WebClient that I think I can use.  Let me see what options I have:

New-Object System.Net.WebClient | Get-Member 

There’s a method called DownloadString, that takes in a string address as a parameter.  So, lets give this a try and see what happens:

$url = "https://api.twitter.com/1/friends/ids.xml?screen_name={your user name}"
$wc = New-Object System.Net.WebClient
$wc.DownloadString($url) 

That returns the XML response that I’m looking for.  So, now I need to put that string into an xml object, and see what options I have.

[xml] $data = $wc.DownloadString($url)
$data | Get-Member 

Now, one of the properties is called “id_list”.  If I look at the XML returned from the URL I passed in, that’s the root node.   So, by putting it in the XML object, Powershell has effectively serialized my XML into an object tree.  So, now I should be able to get each individual id.

$data.id_list.ids.id |
% {
	$_
} 

From that I can see the list of id’s. Alright, now I can loop through each one of my followers. So, the next step is getting the number of followers for each of those users. Luckily, there is another API call that can give us that information.  So, now, for each one of these users I want to call a URL of the following format - https://api.twitter.com/1/users/lookup.xml?user_id={User ID}&include_entities=true.

Note: The twitter API only allows 150 calls an hour per IP address.  I found this one out the hard way.  So, while you CAN loop through all your friends, I wouldn’t recommend it if you have more than 150 people your are following.

So, now I can do something like this within the loop:

$userUrl = "https://api.twitter.com/1/users/lookup.xml?user_id=$($_)&include_entities=true"
[xml] $userData = $wc.DownloadString($userUrl)

A little more testing tells me that the user name of the current follower can be found at $userData.users.user.screen_name and the follower count can be found at $userData.users.user.followers_count. Now, I want to see this data as a simple data set so I can sort the data. There’s a neat little trick I picked up while reading this article. You can dynamically declare an object structure by using a command similar to this:

$newObj = "" | Select-Object Property1, Property2, Property3

If you then do something like this:

$newObj | Get-Member

You’ll see “NoteProperty” types that correspond to the properties you declared earlier. Too, instead of doing multiple set operations on separate lines, you can do multiple set operations on one line – as long as you get the order correct. So, once I get the data for the user, I can do something like this:

$outputItem = "" | Select-Object Name, ScreenName, FollowersCount
$outputItem.Name, $outputItem.ScreenName, $outputItem.FollowersCount = $userData.users.user.name, $userData.users.user.screen_name, [int] $userData.users.user.followers_count

Now, if I just put $outputItem on a line all by itself, it’ll be returned as a result of this iteration of the loop. So, now my loop returns a data set which contains all the data I want to see. Just to make it a bit nicer, I can pipe it out to Out-GridView. That allows me to see the data and play with it all I want. So, if I put it all together, I’ve got a script that hits the Twitter API and will return a grid view of all of the people that you are following and their follower count in a little over 10 lines of code (and some of that simply for formatting):

$url = "https://api.twitter.com/1/friends/ids.xml?screen_name={your twitter name}"
$wc = New-Object System.Net.WebClient
[xml] $data = $wc.DownloadString($url)
$data.id_list.ids.id |
% {
    $userUrl = "https://api.twitter.com/1/users/lookup.xml?user_id=$($_)&include_entities=true"
    [xml] $userData = $wc.DownloadString($userUrl)
    $outputItem = "" | Select-Object Name, ScreenName, FollowersCount
    $outputItem.Name, $outputItem.ScreenName, $outputItem.FollowersCount = $userData.users.user.name, $userData.users.user.screen_name, [int] $userData.users.user.followers_count
    $outputItem
} | Out-GridView

So, there is is. In the interest of full disclosure, here is my script, with the user limiter code included as well as some commented code I was using for testing purposes:

cls
# New-Object System.Net.WebClient | Get-Member
$url = "https://api.twitter.com/1/friends/ids.xml?screen_name=katman26"
$wc = New-Object System.Net.WebClient
[xml] $data = $wc.DownloadString($url)
$count = 0
$data.id_list.ids.id |
% {
    if($count -lt 5)
    {
        $userUrl = "https://api.twitter.com/1/users/lookup.xml?user_id=$($_)&include_entities=true"
        [xml] $userData = $wc.DownloadString($userUrl)

        $outputItem = "" | Select-Object Name, ScreenName, FollowersCount
        $outputItem.Name, $outputItem.ScreenName, $outputItem.FollowersCount = $userData.users.user.name, $userData.users.user.screen_name, [int] $userData.users.user.followers_count
        $outputItem
    }
    $count++
} | Out-GridView

Let me know if you have any questions about this or any of the examples I’ve shown. I’m not a Powershell expert…yet.