HDFS NameNode Username Hack

I created a userscript to override the username when (when programmatically detectible) to allow you to read HDFS directories and files that aren't world readable.  Nothing fancy here, you could edit the URL yourself, this just makes it easier.  The script is hosted here: http://userscripts.org/scripts/show/179132, and the source is available here: https://gist.github.com/keeganwitt/6810986.

Meld Windows Installer

I'm a huge fan of Meld as a diffing / merging tool.  It's the nicest looking, most powerful tool of it's kind that I've come across.  (Although I'd also like to put a shout out to KDiff3 for being able to handle larger files than anything else I've seen besides GNU Diff, and to WinMerge as another popular choice).  The trouble is, if you run Windows, it's a bit of a pain to set up.  You have to install Python, GTK+ and PyGTK (made easier by the nice all-in-one installer they now have), then Meld itself, and finally write a script to launch the appropriate Python command and create shortcuts to that script for convenience.  This is a shame because I'd really like to get more Windows users using this wonderful tool.
So I made an installer than includes all of these and has no extra dependencies needed.  Just install and go.  The only thing it doesn't currently have is support for syntax highlighting (needs PyGtkSourceView, which is not included in Portable Python, which I used), and VCS browsing (needs GNU Patch via Cygwin).  Here's where you can get it: http://code.google.com/p/meld-installer/.  Lemme know if you find any mistakes, it's my first time using NSIS.

Edit: I've now also created a portable .zip archive.

Edit: Vote here to ask for PyGtkSourceView to be added to Portable Python, so I can use it (edit: it appears PyGTK in Portable Python already includes PyGtkSourceView).  I'll look into doing this without Portable Python (though that'll be easier for me) and possible GNU Patch as well, but am not sure how hard it will be.

Edit: I've made a significant update to this.  My thanks to Angel Ezquerra of TortoiseHG for his suggestions and testing assistance.

A Mercurial Userscript

I've got a minor complaint against the Mercurial web interface, and that is that when you are browsing around using the tip version, links will point to the nodeid tip points to rather than keeping the URLs with the relative tip version.  I think the way it should work is that you stay on the tip, unless you are click into a revision log (so that you can click specific revisions).  This is especially nice for sharing a link to the latest with someone, you would want to keep that link relative.
For the most part, I don't really care what Mercurial does, but since I do use their web interface because it's the SCM of choice for the OpenJDK project, I thought I'd whip up a userscript to remedy the situation.  Check out the script here, then test its effects by browsing the latest JDK sources:
http://hg.openjdk.java.net/jdk6/jdk6-gate/jdk/file/tip/src/share/classes/
http://hg.openjdk.java.net/jdk7/jdk7-gate/jdk/file/tip/src/share/classes/

Let me know if you have any problems or suggestions for improvement.

Running Lego Mindstorms RIS 2.0 on newer windows

While getting my old Lego Mindstorms (an RCX 2.0 setup) set up for my little brother to play with, I learned some thigns about using it with newer versions of Windows (Windows 7 in my case). I found out here that there is a patch needed for the USB IR tower, which resolves the problem of the system locking up when the tower is plugged in. You can download it here: http://cache.lego.com/downloads/education/tower164.zip.
After installing this, things will run fine, unless you decide to use the official program and launch it more than once. At which point you will see a message like "A critical error has occurred. You may be running out of memory, or you may need to reinstall Robotics Invention System 2.0." The solution to this is a bit messy. There is a file left behind here
Vista/Win7: C:\Users\<user>\AppData\Local\Temp\Ris 2.0.mov.#res
XP: C:\Documents and Settings\<user>\Application Data\Temp\Ris 2.0.mov.#res
This file must be deleted each time before the program is launched. What I did was create a batch script to do this, and a Visual Basic script to launch the batch script (so it could happen without launching a command prompt window).
You'll find both scripts here: https://gist.github.com/1531705
I recommend replacing all shortcuts to theLaunchRis2.exe with a shortcut to the Visual Basic script. Note that the VB script assumes you will put put the batch script in %PROGRAMFILES(X86)%\LEGO MINDSTORMS\launchRis.bat (or %PROGRAMFILES%\LEGO MINDSTORMS\launchRis.bat on a 32 bit system), but you can easily edit the script to change the location.
Of course, this second problem is irrelevant if you decide to write the programs youself (I preferred this over the graphical tool Lego provided). There are several languages available.
Let me know if you have suggestions for improvement or run into any issues.

A userscript for Viewvc

I've posted a userscript I've written as a workaround for a request for ViewVC that hasn't been done yet (despite the fact a patch has already been submitted).  The missing feature is a link to the log view for directories.

A few Autohotkey scripts

I decided to post the source for a few Autohotkey scripts a couple of days ago:

• GW Tonic Bot : This one is a bot to drink tonics for you in Guild Wars to help you get your Party Animal title. It maps back and forth between your Guild Hall and drinks 2 tonics (yes, 2 tonics because of a bug in GW) each time it stops.
• GW Drunkard Bot : This one was meant to drink alcohol at specific intervals to achieve optimal points towards your Drunkard title.  This was made obsolete by the March 3, 2011 update.  You can now click them (or have an autoclicker click them) as fast as you want.
• Generic Autoclicker : This one can be for any kind of automated clicking, it just clicks (or double clicks) where you tell it to at an interval you specify.

Note that I haven't tested them in IronAHK, only Autohotkey on Windows.  Also note that it's also easy to convert these to an .exe file if you don't want to install Autohotkey.  And if a non-techie is reading this, I'm happy to provide that for you.  Just leave a comment below.

Also feel free to comment (or fork) either of these scripts if you have improvements.  Some people have found that the delay between mappings in the tonic bot is not long enough for slower internet connections and so you have to tell it to drink more tonics than you'd think (since some clicks will be wasted).

BASH Colors

I was messing with my .dir_colors file (about.com also has a good article on this) and wanted a chart to help me pick out my colors. So I modified the popular script that is floating around on the internet to display every possible combination in a graphical chart.
The way it is read is foreground and effect is on the row titles and the backgrounds are on the column titles. Hopefully someone else will find this useful.

#!/bin/bash
#
#  This file echoes a bunch of color codes to the 
#  terminal to demonstrate what's available.  Each 
#  line is the color code of one forground color,
#  out of 17 (default + 16 escapes), followed by a 
#  test use of that color on all nine background 
#  colors (default + 8 escapes).
# 
#  Attribute codes: 
#   00=none 01=bold 04=underscore 05=blink 07=reverse 08=concealed
#  Text color codes:
#   30=black 31=red 32=green 33=yellow 34=blue 35=magenta 36=cyan 37=white
#  Background color codes:
#   40=black 41=red 42=green 43=yellow 44=blue 45=magenta 46=cyan 47=white
#
#  A quick test can be done by echo -e "\033[<color code><test text>\033[0m"
#   e.g. echo -e "\033[01;32mHello World\033[0m"
#
#  Modified from: http://tldp.org/HOWTO/Bash-Prompt-HOWTO/x329.html

T='gYw'   # The test text

# smaller set without attribute codes commented out below
#echo -e "\n                 40m     41m     42m     43m     44m     45m" \
#        "    46m     47m";
#for FGs in '    m' '   1m' '  30m' '  31m' '  32m' '  33m' '  34m' '  35m' \
#           '  36m' '  37m' ;
echo -e "\n                  40m     41m     42m     43m     44m     45m" \
        "    46m     47m";
for FGs in '     m' '    1m' \
           '   30m' '01;30m' '02;30m' '03;30m' '05;30m' '07;30m' '08;30m' \
           '   31m' '01;31m' '02;31m' '03;31m' '05;31m' '07;31m' '08;31m' \
           '   32m' '01;32m' '02;32m' '03;32m' '05;32m' '07;32m' '08;32m' \
           '   33m' '01;33m' '02;33m' '03;33m' '05;33m' '07;33m' '08;33m' \
           '   34m' '01;34m' '02;34m' '03;34m' '05;34m' '07;34m' '08;34m' \
           '   35m' '01;35m' '02;35m' '03;35m' '05;35m' '07;35m' '08;35m' \
           '   36m' '01;36m' '02;36m' '03;36m' '05;36m' '07;36m' '08;36m' \
           '   37m' '01;37m' '02;37m' '03;37m' '05;37m' '07;37m' '08;37m';
  do FG=${FGs// /}  
  echo -en " $FGs \033[$FG  $T  "
  for BG in '40m' '41m' '42m' '43m' '44m' '45m' '46m' '47m';
    do echo -en "$EINS \033[$FG\033[$BG  $T  \033[0m";
  done
  echo;
done
echo

As a side note, for those who may be like myself and didn't see any logic to the codes used for colors, ls and most terminals use the ANSI escape codes (ECMA-48), which is where the numbers come from.

Snapshots of Wave Robot API Now Available

I've Mavenized the Wave Robot API and the Wave model portion of Wave Protocol, a dependency of Wave Robot API and deployed snapshots to my public repository. Currently, I've only deployed snapshots since the use of this artifact is completely untested. I'm working on creating a Maven archetype for Wave robots (which will use this repository until Google uploads it somewhere), and an example using the archetype. After I've done this and am sure it works (and that I haven't done anything stupid with Maven), then I will tag a release and put it on my release repository.

If anyone would like to try it out in the mean time, your feedback would be most appreciated! Here is what you would need in your pom.xml:

<dependencies>
    <dependency>
        <groupId>com.google.wave</groupId>
        <artifactId>wave-robot-api</artifactId>
        <version>20100408-SNAPSHOT</version>
    </dependency>
</dependencies>
<repositories>
    <repository>
        <id>public-repo_maven2-repository</id>
        <url>http://public-repo.googlecode.com/svn/repository/</url>
        <snapshots>
            <enabled>false</enabled>
        </snapshots>
        <releases>
            <enabled>true</enabled>
        </releases>
    </repository>
    <repository>
        <id>public-repo_maven2-snapshots</id>
        <url>http://public-repo.googlecode.com/svn/snapshots/</url>
        <snapshots>
            <enabled>true</enabled>
        </snapshots>
        <releases>
            <enabled>false</enabled>
        </releases>
    </repository>
</repositories>

Some stuff I picked up along the way:

• When using the WebDAV wagon with Google Code, make sure you use https, otherwise you're liable to get a 405 method not allowed error on a deploy.
• There's an interesting bug with the Robot API's EventSerializerTest. The behavior is different in Javac 1.5 vs 1.6 but compile fine with the Eclipse compiler. There is bug report for Javac here (fixed in 1.7). I opened a ticket for Google to explicitly cast this in the method so the compiler doesn't have to do any inference, so that way it will build fine with Javac even if they're using JDK 1.6. In the mean time, I've opted to use the Eclipse compiler in the pom rather than altering my local copy of the source. My goal is to make this an exact match to the class files in the zip they've distributed.
• Guava, the successor to Google Collections (as a backwards compatible superset) and dependency of the Wave model API is on Maven central, but not on Google's repository. It's a little odd, but shouldn't cause any issues.

A special thanks to Jurgen and commenters for the helpful post that got me started.

Bots updated to v2 API

I've updated WorldCat-Bot and Yodaspeakify to v2 of the Wave API, which boasts some fantastic new features, including an easier way to do annotations, doing away with Cron, no need for a separate profile servlet, and the capabilities.xml is automatically generated for you using annotations.

One thing I voted for as an improvement was to use regular expressions in this method. I'm not sure what the author of the issue was talking about with iterating over Elements since, as I understand, these don't include text. The artifacts are still not on Google's repository and I'm getting pretty tired of a 5 minute commit time when I change some jars around.  I'm working on using Google Code as a repository and adding these artifacts after I mavenize them, then I'll create archetypes for Wave -- watch for an update on this soon, now that my bots are done this is the next project.

Some snags I ran into:
The .replace() of BlipContentRefs doesn't work the way you'd think it would. I thought I could do

BlipContentRefs content = blip.first(originalText).replace(newText);

But the BlipContentRefs will be empty. I got a hint for how to do this from wadrobotframework's BlipUtils:

public static void replaceBlipContent(Blip bleep, String nextContent) {
    BlipContentRefs bcr = bleep.all();
    if (bcr != null) {
        bcr.delete();
        bleep.append(nextContent);
    }
}

Thanks, wadael!

Another issue was when I used DocumentChangedEvent and then did a .getBlip() it would return a null Blip. My understanding of the JavaDoc was that it would return the root blip if no Blip was associated with that event, but this didn't seem to be happening. So, I opted for a BlipSubmittedEvent instead. Which, now that I think about it, is probably the desired behavior anyway.

Another issue was when I was trying to take advantage of the cool new filtering options. WorldCat-Bot used to surround it's searches with angle brackets (<>), but when I tried to filter on this

@Capability(filter = "<.*>")
@Override
public void onDocumentChanged(DocumentChangedEvent event) {
  ...
}

It didn't work because the capabilities.xml that was generated was not well formed. I also tried the escaped version

@Capability(filter = "<.*>")
@Override
public void onDocumentChanged(DocumentChangedEvent event) {
  ...
}

But this didn't work either, it looked for the entire literal not the unescaped form. I don't think this is currently possible to do, so I now use square brackets ([]) to filter on.

While trying to debug this, I set up logging (GAE supports JUL), but I learned it only logs things with INFO, WARN, and SEVERE levels (though I couldn't get INFO to work even with the logging.properties file with .level = ALL. Maybe something else is misconfigured).

But overall, the new API has been a very positive experience.

My Second Wave Bot

Inspired by Piratify, I decided to make a bot that makes everyone talk like Yoda (Yodaspeak as I call it). The bot lives at yodaspeakify.appspot.com, and can be added by adding yodaspeak@appspot.com to your wave. The sourcecode is available here. It works, but needs more work to improve its results.

My First Wave Bot

I finally got my Wave invite, and I immediately got interested in coding with it. I recently finished creating a bot for it that allows you to search WorldCat from inside Wave. The main servlet is in Groovy, the profile is in Java. Their tutorial pretty much works, though one of the methods was renamed:
getRobotProfileUrl() should be called getRobotProfilePageUrl() (there's a request filed to fix this). Google's plugin for Eclipse also works very well. JetBrains also has a plugin for IDEA, though I didn't test this. In their sample project SpringSource has a build script that uses AntBuilder, which I modified to use the folder structure that was already created by the Eclipse plugin. I successfully built with this and deployed the app using appcfg.cmd from the SDK. I used straight html for the bot's profile page, but in their sample SpringSource shows how you could use the MarkupBuilder in groovlets.

Anyway, the app is deployed on the Google App Engine here, it's homepage (where the sourcecode is also available) is here. You can add it to your waves by adding worldcat-bot@appspot.com. For even more bots, check out the list. Mine's listed too. There is another list out there, but it doesn't seem used as much.

Some gotchas:

1. You cannot test robots without deploying them to the Google App Engine, this makes you waste some of the number of deployments you have on the free account (currently you get 1000).
2. Wave caches its bots, so you have to change the version in appengine-web.xml (I don't think you have to change the version in capabilities.xml too unless the capabilities have also changed, but I've been changing both to match). They tell you this, but you have to be careful because even though you may have deleted the old version if you reuse a version descriptor text Wave may still try to call the old version and you will get a ServletUnavailable exception and waste another deployment. You have to wait for the re-caching to occur.
3. If you are using Groovy, you have to upload the groovy-all jar in the war/WEB-INF/lib directory (the war folder can be a different name, but that is the convention used by the Eclipse plugin).

For those who haven't heard about Wave, it's basically a kind of collaborative IM (I once jokingly described it as an MMIM - Massively Multiperson Instant Messaging). But it isn't quite accurate to describe it as a kind of IM. The text is live, you can see it as the person is typing it, but whether its treated as an IM or more like an email is fluid. It depends only on if other people are there at the same time. So, it can be viewed as a kind of mashup between email, IM, and collaborative documents and is more concurrent than traditional email. LifeHacker has some use cases, which make for good propaganda.
Anyone in the conversation (called a Wave) can edit any of the messages (blips). Side conversations can occur in the same stream, these side conversations are called wavelets. APIs exist to have bots in a conversation that automate tasks (such as links for searches, posting the conversation to a blog, bringing in text from a feed, or converting everyone's text to pirate talk) and for gadgets that let you put different kinds content inside the conversation, such as documents, polls, etc. Google plans to open source most of Wave once it's finished, allowing other 'federated' servers to become Wave providers, and they plan to make the protocol they use the predominant protocol on the internet. The Google Wave Federation Protocol is itself built off XMPP (the same protocol Jabber and GoogleTalk use). They have bindings for Python and Java currently (I've heard the Python API is not as polished as the Java one, but I don't know that for sure and I'm sure it will improve), and will probably be adding more languages in the future to support their goal of making their protocol #1. Wikipedia has a pretty good article on Wave as well.

I still have Wave invites left for my friends, if you're interested.

Extending NTFS with Bad Sectors

I wanted to grow an NTFS partition to use unallocated space this weekend on an XP machine. If it were Vista or Win7, I would have used the built in resizing ability in disk management, and I found I could not do so with my usual tool of choice (GParted Live CD). The error message said that it could not complete the operation because of two bad sectors, and to run chkdsk (which I did) and then use ntfsresize with the --bad-sectors option. When I tried this, it said it couldn't grow it unless I make it bigger with fdisk. The only way I know how to do this would be to create an entirely new partition (which would mean all the hassle of reinstalling windows and the needed apps). I was finally able to do it with EASEUS Partition Master Personal. Unfortunately, it is not open source, but it is free and it did what I was trying to do without restarting in minutes.

My First iGoogle Theme

I experimented with making my own iGoogle theme
here
The interface they have is rather limiting (couldn't choose the color black for background) than their API, may experiment with that next.