2008 – The Year in Review

It’s the end of the year, and although I skipped 2007, it is my habit to take a look back here on matters personal and professional. To whit:

I blogged a lot less this year. There are a number of reasons for that. The main one is that I’m just not as interested in blogging as once I was. I think that comes with the increasing “inward focus” my career has taken. I just find that more and more, what matters to me is what matters to me, not what matters to everyone else. That’s not a judgement or a boast, just an observation. It’s quite liberating: for example, I can go look at weird languages (see below) and not worry about it too much. :)

If you really can’t get enough Craig, you can always try me in appetizer size over on my Twitter stream. I seem better able to post a few dozen characters than a few paragraphs these days.

Career-wise, 2008 continued the shift I started in 2007 away from lots of small projects towards one big one. If I remember right, by the end of 2006 I’d worked on about seven different open source and proprietary projects, at a max of something like five simultaneously. That was too much, and I was feeling burned out. These days, pretty much all my professional time goes to a contract at Microsoft, where I’m working on the next version of Visual Studio. That has been very interesting for a variety of reasons, not the least of which is that it’s related to the work I’ve done on the MSDN REST Web Service.

Of course, the most significant event of 2008 had nothing to do with web services: it was the birth of our second child, daughter Susan. She was a cranky newborn, but has since turned into a charming and active baby. Needless to say, having two kids has really changed my life. I’m still coming to terms with giving up my free time – a painful process for me.

Not that I have no free time, of course. If I had none, I wouldn’t have been able to train for and run a marathon. But maybe I shouldn’t have talked so much smack in that post – Engberg whupped me good, and now I have to run another one. I think maybe this time I’ll make qualifying for Boston my stretch goal. Along the way, I’d also like to run a sub-20:00 5K. That’s much more my type of thing – I’m a fast runner, not an endurance runner.

This year saw my fascination with Lisp continue and even deepen, as is obvious from a variety of posts here. My latest interest on the parentheses front is Clojure. I made one (sort-of) provocative post about it, promised to say more, and never did. Well, given how not into blogging I am lately, I’m not sure whether I ever will live up to that promise. So go check it out for yourself: it really is an awesome language. Stu’s book is a good way to get started.

Here at the cusp of the new year, I don’t really know what 2009 will hold for me. I have a pretty good idea that the Microsoft work will continue for a while, but of course as an independent there’s always an increased level of uncertainty. But I like to think that it’s a rare problem that doesn’t yield an opportunity, so I’m optimistic.

On the family front, obviously we’re looking forward to watching both kids continue to grow and flourish. 2009 will be Ellen’s last full year of preschool, and of course Susan will likely start walking, talking, and doing all sorts of things. We hope to take family trips to Taiwan, Hong Kong, and Alaska, too.

On the professional front, I really want to spend more time working with Clojure. I’ve got some ideas for some real apps I can write, since that is, after all, the only true way to learn a language or platform. Plus it gives me a chance to run Ubuntu regularly, which I’ve been enjoying as a departure from the ordinary. (Clojure runs on Windows just fine, BTW.)

In the personal realm there are the running goals I mentioned earlier. I also hope to continue to fly Falcon, and maybe even DCS: Black Shark, which looks really sweet. If I somehow magically find extra time, and the basement finally gets finished, maybe I’ll even start playing bass and/or drums again.

Wishing you all the best in the new year.

A C# REPL (in Clojure)

It’s no secret that I’ve been interested in Lisp for quite a while. Lately, that has meant Clojure. Clojure is a new Lisp dialect that runs on top of the JVM. That means it has the power of Lisp (macros, dynamic typing, etc.) combined with the power of the JVM. The synergy reminds me of an old bumper sticker I saw years ago: “C: combining the speed of assembly with the power of assembly.” We’ve moved on a bit since then, but still. :)

Anyway, you should check out Clojure. It is easily the most exciting technology I’ve seen in years…probably since I first saw .NET in June of 2000. Even beyond just being a Lisp that actually has vast libraries (one of the main complaints against Lisps of the past) is the fact that there are some truly brilliant features of the language above and beyond what other Lisps sport.

One of the cool things I’d heard was that Clojure will run on top of IKVM.NET, which is a .NET implementation of the JVM. I figured that if there were a reasonable story for .NET interop, I’d be able to use Clojure to drive .NET code in a REPL. A REPL is a Read-Eval-Print Loop, which is a fancy way of saying “an interactive programming command line”. It’s like the immediate window in the Visual Studio debugger on steroids, and its absence is one of the increasing number of things that makes C# painful to use as I gain proficiency in more advanced languages.

Yes, I said it: C# is not an advanced language. If you think it is, then in my opinion you don’t know enough programming languages yet. :)

Intentionally provocative statements aside, here’s how you do it:

1. Download and unzip ikvm.net. No installer required.
2. Download Clojure. I recommend grabbing the head of the Subversion tree rather than the release that’s on the website.
3. Build Clojure by running “ant jar” in the clojure directory. You’ll have to install Ant to do this if you haven’t already.
4. Run “ikvmstub mscorlib.dll” to create mscorlib.jar, which will create the Java wrapper classes for the stuff in mscorlib.
5. Launch a Clojure REPL via “ikvm –cp \path\to\clojure.jar;\path\to\mscorlib.jar clojure.lang.Repl”.

Now you have a working REPL, in which all the types in mscorlib are available to be driven by Clojure, which is most definitely an advanced language. Here’s a very simple example (the results of running the commands are indicated by =>):

(import '(cli.System DateTime))
=> nil

(new DateTime)
=> #<DateTime 1/1/0001 12:00:00 AM>

(.get_Now DateTime)
=> #<DateTime 11/19/2008 2:31:02 PM>

(.ToString (new DateTime) "R")
=> "Mon, 01 Jan 0001 00:00:00 GMT"

You get the idea.

Of course this is probably only useful as an exploratory tool: if I were writing Clojure for real, I’d write it against the JVM, not against IKVM.NET. But I use the excellent SnippetCompiler all the time now, but it’s still not the quite same thing as a real REPL, so I’m excited to have a real REPL available.

And really, the reason I’m posting this is that I’m hoping it will be a sort of gateway drug for you, and that what this will really do is kick off your interest in Clojure itself. The website is a pretty good source of information, but you can also check out Stu Halloway’s Programming Clojure book. It’s available in Beta form here. I’ve been tech reviewing it, and even the early beta looks pretty good.

MCM 2008 (Or, “Take That, Engberg”)

I just got done running the 2008 Marine Corps Marathon. My entry in this event was so full of drama, it belongs in a figure skating vignette.

The whole thing started a little over a year ago. I’d taken a fairly long break from running, and had hit the magic weight I’d set for myself that meant I had to start again. I’m a moderately serious runner, having done it since high school. At around the same time I was starting up, my neighbor Steve was talking about starting an exercise program. I convinced him to come out with me, despite the fact that he had never really run regularly before, and certainly not seriously. It was selfish, really: a running partner a few doors down is a precious resource.

Over the next few months, we slowly built up the mileage, doing first six miles, then seven, then longer and longer runs. With each one, Steve would exclaim at how he couldn’t believe he’d just run ten or twelve or whatever miles, and at how the slightly shorter one we’d done last week now seemed like an easy distance. Major credit to Steve: he stayed with it, even when we were running through the January sleet at 6AM.

Cut to six years earlier. I’d just finished running Grandma’s Marathon in a fairly reasonable time of 3:33. My oldest friend, Rob Engberg, had run the race as well, in a very respectable 3:48. I did the marathon “just to do one”, and was feeling beat up enough at the end of it to comment to Rob that I’d run one again if he beat my time. Rob is faster than me all other things being equal, but he is a self-admitted slacker, and I figured he’d never get motivated enough to train to the point where he could take down my time.

As these things do, this one sort of took on a life of its own. The years slid by and Rob ran a few more marathons but never repeated even his old time, let alone mine. And as also so often happens, the years erased the memory of pain and I started to contemplate doing another one. Plus, I figured it wasn’t fair for Rob at 40 to be chasing a time that I ran when I was about 30. So I told him we’d reset the clock every five years. And to make it interesting, we made it more of a bet: the loser has to post a picture of himself on his blog wearing a t-shirt of the winner’s design. In a public place. (Design suggestions welcome, BTW.)

So now that we’re both between 35 and 40, we’re into the second round. Rob ran a 4:07:03 this year, and that became my time to beat. So I convinced neighbor Steve to run the Marine Corps Marathon with me (and more importantly, to train for it with me). We signed up, and started an 18-week training program that culminated with the race today. That’s where the drama comes in.

Steve and I were running well. We had a good program and we followed it faithfully. After working out a few kinks (hint: over about 13 miles, you need to eat during a run), we were even blasting through our long runs in excellent form. Then came the 18-miler.

Running the 18 miles itself was fine. We ran on Sunday morning, and I didn’t even need to take a nap like I did after some of the previous long runs. When I saw Steve across the courtyard the next day, we traded a round of, “How you feeling? Great!” Then that evening I sat down next to my daughter to read her a story, and when I got up five minutes later, I felt like someone had shoved a stack of coins under the ball of my left foot. It happened that suddenly. I immediately went online and ordered new shoes, which I’d been meaning to do anyway, and went to bed. The next day, it was just as bad if not worse, and there was some bruising on my foot. I took Tuesday off of running, but came back at it Wednesday.

And the same damn thing happened – I was fine after the run, and the next day, it suddenly started hurting again. Only this time it didn’t get better.

To make a long story slightly shorter, I wound up having to take about eight weeks off of running. Several (rather expensive) visits to the doctor later, we had a tentative diagnosis of metatarsal capsulitis, which is Latin for “you hurt your foot”. I hit the stationary bike faithfully five times a week for an hour or two at a time, trying to keep up with my running. But as the weeks rolled by and I still couldn’t walk in regular shoes without limping, my hopes for competing dwindled. Meanwhile, Steve was plowing through the training by himself, including three runs of 20 miles or longer.

Finally, finally, my foot started to feel better. Just in time, too: I was able to resume my training with about a month to go before the race. If it had taken any longer, I probably would have given up, as running 26.2 miles takes a certain amount of preparation. I was doubly glad of the timing, because I got better just in time to compete in the Army Ten-Miler, which Steve and I had signed up for as a sort of warm-up race/workout. In fact, it was going to be our first longish run together.

More drama. As I walked out to meet up with Steve at 6AM on race morning, he told me that he couldn’t compete. The reason? “Dude, it feels like someone shoved a stack of coins under the ball of my foot.” Injury! Maybe even the same one!

So now, as it were, the shoe was on the other foot. I was left training by myself, and Steve was left wondering if he was going to recover in time for the marathon. It sucks, by the way, to not know. Both of us experienced the thought that if someone could just tell us we couldn’t run, that would have been better than just not knowing one way or the other.

Fortunately, race day saw us both ready to run. Unfortunately, a major miscommunication before the start meant that we never linked up during the race. So after all that, we both wound up running separately! But even that may have been for the best, as Steve ran with another neighbor of ours, and I was able to run a bit faster than I might have if I’d run with him, important because the only reason I was running the race was to screw over by buddy Rob.

The race itself sort of sucked. The weather was perfect, and it’s cool that the marathon winds through the various monuments in Washington DC, but I have to say that the course was completely inadequate to the number of people running on it. I’m not sure who thought it was a good idea to route 30,000 runners down two-lane roads, but I spent the first ten miles dodging around and between other runners, or getting pissed at being boxed in for the fiftieth time. Fortunately, it opened up after about ten and I was able to run a bit faster, but for the first hour and a half, I was sure I wasn’t even going to beat Rob’s 4:07, let alone run the sub-4:00 marathon I wanted to. The Marine Corps Marathon gets a C-minus – definitely not recommended.

So after all that, what was my time? Three hours, fifty minutes, eight seconds (a pace of 8:47 per mile). Rob, who lives in New Zealand, is running his marathon in about two weeks. It sounds like he’s actually be training for this one, but that time is only two minutes short of his best ever, which he ran when we were barely 30, not nearly 40. So it should be interesting. Me? I’m hoping he runs a 3:50:09. Because it would add to the drama. :)

typing-speed-mode Emacs Minor Mode

I’ve been really into Lisp lately, especially Clojure, which looks really interesting. More on that some other time, though. Anyway, I’m always looking for an excuse to write code in Lisp – any Lisp.

As I was writing some prose in emacs this morning, I got into a real flow. I started to wonder how fast I was typing…and there was my excuse! About an hour later, I had written the code below. It’s an emacs minor mode that displays your typing speed (defined as the number of times self-insert-command has been run in the last five seconds, times twelve) in the mode-line. So you can watch your typing speed go up and down in real time. Note that you can customize a few things about it, like changing the default five-second window.

My emacs-fu (and my Lisp-fu) is not all that it could be, so the code is likely suboptimal in several ways, but it seems to work, and I always hate it when people say, “I’ll post the code once I get a chance to clean it up.” To hell with that: enjoy my code for the garbage it is :). 

;;; typing-speed.el --- Minor mode which displays your typing speed

;; Copyright (C) 2008 Wangdera Corporation

;; Permission is hereby granted, free of charge, to any person
;; obtaining a copy of this software and associated documentation
;; files (the "Software"), to deal in the Software without
;; restriction, including without limitation the rights to use,
;; copy, modify, merge, publish, distribute, sublicense, and/or sell
;; copies of the Software, and to permit persons to whom the
;; Software is furnished to do so, subject to the following
;; conditions:

;; The above copyright notice and this permission notice shall be
;; included in all copies or substantial portions of the Software.

;; THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
;; EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
;; OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
;; NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
;; HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
;; WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
;; FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
;; OTHER DEALINGS IN THE SOFTWARE.

;; Author: Craig Andera <candera@wangdera.com>

;; Commentary: Invoke this minor mode to have your typing speed
;; continuously displayed in the mode line, in the format [75 WPM]
;; To use, just load this file and invoke (typing-speed-mode) or
;; (turn-on-typing-speed-mode)

(define-minor-mode typing-speed-mode
    "Displays your typing speed in the status bar."
  :lighter typing-speed-mode-text
  :group typing-speed
  (if typing-speed-mode
      (progn
    (add-hook 'post-command-hook 'typing-speed-post-command-hook)
    (setq typing-speed-event-queue '())
    (setq typing-speed-update-timer (run-with-timer 0 typing-speed-update-interval 'typing-speed-update)))
      (progn
    (remove-hook 'post-command-hook 'typing-speed-post-command-hook)
    (cancel-timer typing-speed-update-timer))))

(defcustom typing-speed-window 5
  "The window (in seconds) over which typing speed should be evaluated."
  :group 'typing-speed)

(defcustom typing-speed-mode-text-format " [%s WPM]"
  "A format string that controls how the typing speed is displayed in the mode line.
Must contain exactly one %s delimeter where the typing speed will be inserted."
  :group 'typing-speed)

(defcustom typing-speed-update-interval 1
  "How often the typing speed will update in the mode line, in seconds.
It will always also update after every command."
  :group 'typing-speed)

(defvar typing-speed-mode-text (format typing-speed-mode-text-format 0))
(defvar typing-speed-event-queue '())
(defvar typing-speed-update-timer nil)

(defun typing-speed-post-command-hook ()
  "When typing-speed-mode is enabled, fires after every command. If the
command is self-insert-command, log it as a keystroke and update the
typing speed."
  (if (eq this-command 'self-insert-command)
    (let ((current-time (float-time)))
      (push current-time typing-speed-event-queue)
      (typing-speed-update))))

(defun typing-speed-update ()
  "Calculate and display the typing speed."
  (let ((current-time (float-time)))
    (setq typing-speed-event-queue
      (typing-speed-remove-old-events
       (- current-time typing-speed-window)
       typing-speed-event-queue))
    (typing-speed-message-update)))

(defun typing-speed-message-update ()
  "Updates the status bar with the current typing speed"
  (let* ((chars-per-second (/ (length typing-speed-event-queue) (float typing-speed-window)))
     (chars-per-min (* chars-per-second 60))
     (words-per-min (/ chars-per-min 5)))
    (setq typing-speed-mode-text (format " [%s WPM]" (floor words-per-min)))
    (force-mode-line-update)))

(defun typing-speed-remove-old-events (threshold queue)
  "Removes events older than than the threshold (in seconds) from the specified queue"
  (if (or (null queue)
      (> threshold (car queue)))
      nil
      (cons (car queue)
        (typing-speed-remove-old-events threshold (cdr queue)))))

(defun turn-on-typing-speed ()
  "Turns on typing-speed-mode"
  (if (not typing-speed-mode)
      (typing-speed-mode)))

(defun turn-off-typing-speed ()
  "Turns off typing-speed-mode"
  (if typing-speed-mode
      (typing-speed-mode)))

You Think You Like Halloween?

I don’t think you like Halloween like this guy likes Halloween…

I’m glad to have helped in even a very small way.

Announcing Sudo for Windows

If you’ve used a Unix much, I’m sure you’re familiar with sudo, a command-line utility that lets you run things as the superuser. Not only it is very handy, but it is the basis for one of the better XKCD strips.

Sudo is one of those things I find myself wishing for in Windows, especially given the new(ish) UAC features in Vista/Windows 2008. There are lots of times when I just want to run something as administrator, dammit. Typing “sudo notepad2 C:\somewhere\foo.txt” would fit the bill.

I tried Sudo for Windows, but it made me type my password. That seemed silly, given that I don’t need to type my password anywhere else to run things elevated. There are probably other implementations of this out there, but it literally took less time to write my own than it would to crawl through all of them looking for the one I like best. All it does is execute whatever arguments get passed to it, but the program itself has the “require administrator” bit in the manifest, so the target program winds up running elevated as well.

Anyway, it’s in my arsenal. Visit here to get it in yours, too.

Announcing InhibitSS

Short story: I wrote a little tray app that will prevent your screensaver from running. Download it here.

Long story: I’ve been training for a marathon over the last few months, but a couple of weeks ago I hurt my foot and have been unable to run consistently. While I’m recovering (and hoping I’ll be well soon enough for me to be able to race), I’ve got to do something to keep in shape. One of the things I do is to ride my bike on a stationary trainer in front of the computer. While that works well, it’s a pain to unlock the screen if the screensaver kicks in while I’m pedaling, as it sometimes does. So I wrote InhibitSS, which runs in the tray and will prevent the screensaver from running, and therefore the screen from locking. Here’s what it looks like when it starts up:

Then, if you double-click it, it looks like this:

The red X indicates that the screensaver is inhibited.

I’m sure there are a dozen apps just like this. But this one has the feature that I had fun writing it. :)

C# Mixins

Update: Corrected terminology – it’s “generic functions” not “generic methods”.

Aside from macros, another nice feature that Lisp supports is mixins. In fact, this is one of the (very, very) few things I miss about C++. Of course, Lisp does it in a much more powerful way using something called generic functions. See here and here for a great description of generic functions in Common Lisp – it’s one of the cooler parts of Lisp, in my opinion, especially for someone like me who has come from a C++/C#/Java OO background.

Although not a tool for every occasion, and somewhat against the spirit of Inherit to Be Reused, Not to Reuse, mixins are nonetheless handy at times. Since mixins are generally implemented via multiple inheritance, they haven’t really been an option in C#…until extension methods came along. Now you can do something like this:

using System;

public interface TellNameMixin
{
    string Name { get; }
}

public static class MixinImplementation
{
    public static void TellName(this TellNameMixin subject, string prefix)
    {
        Console.WriteLine("My name is: {0}{1}", prefix, subject.Name);
    }
}

What we’ve done is to define an interface type and a corresponding extension method that adds functionality to that interface. Since any number of interfaces can be implemented on a type, we can apply our mixin to classes that have no other type relationship, or that already have a base class. Like this:

public class Craig : MarshalByRefObject, TellNameMixin
{
  public string Name { get { return "Craig"; } }
}

public class Program
{
  public static void Main()
  {
     Craig craig = new Craig();
     craig.TellName("Mr. "); // Prints “Mr. Craig”
  }
}

The best part about this is – as with all reuse mechanisms – there’s only one place where I have to change how TellName works. Obviously, if the Craig class was the only class I was extending, I could skip the mixin class and just use extension methods directly. But if I have additional types that I want to add this functionality to, and if those classes have no other type relationship (a situation I found myself in just today) then this might be handy.

Anyway, it’s not something you’ll use every day, but it’s worth knowing about.

Lisp, Too, Is Mainstream

You’ve seen me talk here before about the fact that I have been lately fascinated with Lisp. Sadly, I haven’t been able to spend the time I’d like with it, but I continue to read and think about it. And I’d really, really like to put in some serious time writing a real app (better: several) in Lisp.

So it was with great interest that I read Lisp, too, is mainstream. I like pretty much everything that Eric has written, and his LispCast screencasts are good, too. This article starts with Greenspun’s Tenth Rule (“Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp”) and extends it to a rather different conclusion:

My point is that it may be too late to start with Lisp so you don't have to reimplement all of its features. Because all of those new languages have already implemented them. At least what most people consider the important ones.

Or put another way:

I guess my point, through all this meandering, is that other languages did borrow a lot from Lisp. About half of it. And now those features are out there, in the world. And in the meantime, while they were borrowing, they got some new features of their own. Features like giant user bases, gazilions of libraries, corporate support, standards bodies. So Lisp has half of the features of Python. Java and Python are far from my ideal language---but so is Common Lisp. The idea that I would have to implement so much of Lisp on my own is a little overblown these days. And speaking of reimplementation: How much of Python's standard library does a complex Lisp program reimplement? How much of Python would you have to reimplement before you regret choosing Common Lisp?

And speaking specifically to macros, which I currently see as the single biggest weakness of C# when compared to Lisp:

Macros let you subsume more code into less code. Macros let you write more functionality with fewer lines. Macros let you abstract away boilerplate into new syntax.

But the corporate manager will say: if everyone writes their own syntax, my programmers can't read each other's code. So instead of having to learn a language once, they will have to learn a new language each time they approach a program for the first time. And the value of macros is lessened.

It’s enough to take the wind out of an aspiring Lisper’s sails. :) However, I haven’t given up yet. I have two questions that I still need to answer for myself before I draw any conclusions of my own.

1. Is the value proposition of macros (i.e. custom syntax/DSLs) different for small teams? And particularly for stuff I read and write only for myself? Because I’ve got a lot of code that falls into those categories.
2. Is the library situation really so dire? I find that I don’t actually wind up using third-party libraries all that much in C#, so either the BCL is extremely complete or the types of problems I’m solving are just naturally self-sufficient. Or I’m doing something wrong. :) Besides, when I cruise the Lisp sites, I actually see lots of libraries I could use. But maybe there’s some critical functionality that I’d have to write myself that would take a long time.

Of course, the answers to these questions are inherently highly subjective. Like pretty much any question touching on programming tools. If I can find a way to go write a fair amount of Lisp, I’ll get my answers. They will, however, be my answers. I’ll share them with you if I ever get there, but don’t expect them to help you much. :)

MSDN Low-Bandwidth Rendering Option

One of the common complaints with the MSDN website is the fact that the pages are pretty fat. The team has done a lot to make this better, but there's still some stuff in there that not everyone needs.

 

If you're interested in an extremely skinny version of the MSDN docs, give the new LOBAND rendering format a try. To use it, stick (loband) - inlcuding the parens - at the end of an MSDN URL, before the .aspx. For example, the documentation for System.Xml.XmlReader is normally at http://msdn.microsoft.com/en-us/library/system.xml.xmlreader.aspx. But if you surf instead to http://msdn.microsoft.com/en-us/library/system.xml.xmlreader(loband).aspx, you'll get the new, low-bandwidth rendering of the page. There's a link at the top of the LOBAND rendering that lets you make it permanent via a cookie (you can turn it off later via a similar link).

 

It makes quite a difference. On my machine, I see the regular version at about 98KB, and the LOBAND version at just over 18KB - and the 98KB doesn't count the TOC tree, which doesn't render in the LOBAND version. Not a huge deal for those on high-bandwidth connections in the US, perhaps, but there are lots of people who don't fit that description.

 

Note that the team already has planned improvements for the next regular update of the site. But don't let that stop you from letting them know if you have any ideas about how to make it better. You can drop a comment here or contact me - I'll make sure it gets sent on.

 

And no, I didn't have anything to do with implementing this - I'm just blogging it. :)

Hobocopy x64 Build Available

I finally got around to hacking together an x64 version of hobocopy, my little utility that copies files even if they are locked. Note that you might need to install the 64-bit C runtime (vcredist_x64) in order to get it to work. Both the updated binaries and vcredist_x64 are available at the SourceForge download page.

 

Enjoy! Of course, if you really like hobocopy, maybe you'd consider becoming the maintainer.

Hobocopy Needs a New Daddy (or Mommy)

I enjoy the process of sharing some of my work as open source. One of my more successful efforts in that area has been Hobocopy.

 

Like most (really, all) software, it's not done. There are any number of things I'd like to add to it, and I still get occasional requests from people for enhancements or bugfixes. But, due to one thing and another, I just haven't been able to get around to doing any work on it for months. So I'm asking you, dear LazyWeb, if you know of anyone who would be interested in adopting this poor, neglected, digital offspring of mine. 

 

I'm open to whatever arrangement that maintains the MIT nature of the license, so if you'd rather move the code somewhere else or transfer the copyright, that's cool with me. Or we can keep it where it is and I can set you up as a contributor.

 

Hobocopy is written in C++. Not very good C++, mind you, but C++ skills are still required. Although with sufficient work, we could probably factor out the bit that needs to be in C++ and rewrite everything else in C#. I'm happy to help with design direction via email, I just don't have time to hack.

 

Anyway, let me know if you're interested. It's a good tool and I'd love to see someone pick it up and move it forward.

Speaking at DC ALT.NET Thursday

Lately I've been enjoying going to the DC ALT.NET meetings. I can't say I've figured out what ALT.NET is supposed to mean, but the DC version appears to equate to smart people talking about interesting things, so that's good enough for me.

 

Anyway, I shot off my mouth (big surprise!) about how cool I think Lisp is at the last couple of meetings, so I figured it might not be a bad idea to offer to do a somewhat more structured talk about it some time. I don't really do much public speaking any more, but I happen to have a little presentation ready about what I've learned about (and from) Lisp, so I offered to give it and they accepted. If you'd like to hear it, come on down on Thursday.

MSDN Updates

Yesterday was a big day for me with respect to MSDN. There were a pair of "the new replaces the old" moments.

 

The first event was the one Larry Jordan amusingly subtitles "There will be only one!" What happened was, the old MSDN finally got turned off - the MTPS-based version of MSDN is now the only version running. http://msdn2.microsoft.com now redirects to http://msdn.microsoft.com. That means no more fun URLs like https://msdn.microsoft.com/library/default.asp?url=/library/en-us/shellcc/platform/shell/programmersguide/shell_basics/shell_basics_extending/fileassociations/fileassoc.asp. Now it's http://msdn2.microsoft.com/en-us/library/bb776847.aspx. Ahh - that's better.

 

Since I helped write the Microsoft/TechNet Publishing System (MTPS), which sits behind the new, one-and-only MSDN website (and the TechNet and Expression websites, for that matter), it was sort of cool to see it "take over".

 

The other thing we did yesterday was to update the MTPS REST API. It's still very much a prototype (i.e. things are broken) but we wanted to push a version out there that has some of our latest thinking in it. Most notably, this release sketches out what we think we want to do for writes. That's right - we intend to support community-authored changes to MSDN (the website supports that now in the form of tags and wiki-like annotations, in case you didn't know).

 

Although I should point out that the writes aren't implemented yet in the REST API because we haven't fleshed out the story for authentication.  But you can see the idea. For example, to add a new tag to an item you'll be able to PUT to http://labs.msdn.microsoft.com/restapi/content/b8a5e1s5/en-us;vs.90/tags/add-tag. And to delete a the foo tag added by user candera you'd DELETE http://labs.msdn.microsoft.com/restapi/content/b8a5e1s5/en-us;vs.90/tags/foo/users/candera. For convenience, we're also planning to support POST to those endpoints with a hidden form parameter of _method, which is set to PUT or DELETE as appropriate. It's just too convenient to be able to drive the service from a browser during development not to do that.

 

Another big change is what lives at ~/content. In this release it's a link to the ~/sites part of the API. In MTPS parlance, a "site" is the host part of a URL (e.g. the "msdn" in http://msdn.microsoft.com). Underneath those are "iroots", which is a lot like a vdir (e.g. the "academic" in http://msdn.microsoft.com/academic). The reason ~/content links to ~/sites is that via ~/sites you can find your way into the TOC tree of any of the content in MTPS…or at least you could if it were hooked up properly everywhere. Still, you can see the idea.

 

That said, we've decided that the link to ~/sites is ugly and we hate it. :) So we'll have to figure something else out there. Maybe a set of virtual TOC nodes that unifies all the TOCs - one node to bring them all and in the API bind them. Or something - we're still noodling on that one.

 

Other changes: we've added ETags and Expires headers to all resources (currently everything is hardcoded to expire 24 hours in the future), fixed some of the encoding problems, added an XHTML DTD where appropriate, and of course there are the obligatory bunch of minor bugfixes. We've also done some slight reorganization of the URLs to fit our evolving understanding of the model, but most things have remained unchanged. I'm curious to hear if anyone has played with the service at all - if you have any questions or feedback, do feel free to contact me.

Friends of the MSDN Library Facebook Group

Larry Jordan has put together a Facebook group ("Friends of the MSDN Library") to aggregate people who have ideas about how MSDN can improve. I have a vested interest in this, as I make part of my living contributing to the code that makes it happen. So join up…think of it as a way to help put my daughters through college. :)

Welcome Susan!

At a little before 11AM on March 15th, Wangdera Corporation welcomed its newest employee when our daughter Susan was born at our home in Virginia. Susan tipped the scales at a hefty 8 pounds, 14 ounces (4.03kg), and she and her mother are both doing extremely well. Big sister Ellen is very happy and proud as well.

More pictures available at our Flickr site.

In what may be a related event, Ellen has announced that she is changing her name to "Princess Ellen Ladybug".

Art Geek Zoo

At our house, the newspaper gets full coverage: I read only the comics and my wife reads everything else. I like comics. So I thought it was pretty cool when I found out my good friend Rob Stenzinger has started to publish Art Geek Zoo, his latest artistic effort, online. Check it out!

 

My First Trademark Violation

A few weeks ago, I posted a pair of little utilities that I wrote to help me play Falcon. They were just one of those spare-time, throwaway things most programmers do from time to time, so I didn't put a ton of thought into them. Well, last week, I got a phone call from a marketing guy at Sassafras Software…it seems I should have put a bit more though into at least the names.

 

See, one of the bits I published, I called KeyServer. I called it that because it's a little TCP listener that receives events and sends keystrokes to the active application. Of course, it's also a registered trademark of Sassafras Software. Oopsie.

 

The marketing guy was pretty nice about it. He never said anything about legal action. But here's the thing: US Trademark law says you have to actively defend a trademark, or you lose it. That means that if you don't sue the people who use your trademark, a competitor can come along and point to that behavior as proof that you don't want your trademark and that it should be taken away.

 

On the one hand, it was somewhat shortsighted of me to name my software "KeyServer". I mean, that was practically guaranteed to be trademarked. On the other hand - subjectively and emotionally - it's never fun to be threatened with legal action, even when it's done politely and when you know that the person delivering the news has no choice about it. Lesson learned, I guess.

 

On the bright side, it was good motivation to add a few features that I've been meaning to get around to. So I'm re-releasing the software under the name "Keylay" (it's a keystroke relayer - get it?). Here are the new features:

 

• A name that won't get me sued!


• Has a less-boring tray icon that changes colors based on whether or not a profile is loaded, whether or not a client is connected, and animates when keystrokes are being relayed.


• Takes a command-line argument that is the path to a profile to be loaded at startup.

 

While I was at it, I also updated F4Panel a bit. In addition to the ICP and the MFDs, I also added an "Aux1" panel that gives access to touchscreen controls of a few miscellaneous things like landing gear, the autopilot, the ECM and RWR, the laser, and the master arm switch. In addition, I made it remember the last address it connected to, since that was driving me nuts.

 

You can download the updated binaries and the source here.

DebuggerStepThroughAttribute

Here's a good one I was reminded of the other day. I was writing some code of the form we've all seen a thousand times before:

 

public Foo Bar {
  get { return _bar; }
  set { _bar = value; }
}

 

Even with automatic properties in the latest version of C#, you still wind up writing code like this a lot. Unless you're fortunate enough to be writing code in a more advanced language…like Lisp. :) (Sorry, had to.)

 

Normally this is no big deal, but there is one time where this construct can get really annoying: when you're debugging and the property is frequently accessed as the means to get somewhere else. In my code that will often look like this:

 

public SomethingProvider SomethingProvider {
  get { return _somethingProvider; }
}

 

when I have some service that I've abstracted, maybe to make testing easier. As a result, my code has lots of calls in it like this:

 

SomethingProvider.PerformOperation(arg1, arg2, …);

 

When I hit that line, I want to debug into it, but of course when I hit "step into" I wind up on the property accessor. It's even worse when there are multiple properties involved, as for example in code like this:

 

SomethingProvider.PerformOperation(OtherProvider.RetrieveValue(), YetAnotherProvider.Calculate());

 

By the time I've finished stepping in and out of that, I'm all, "Aargh! Just take me to the code!" Luckily, there's an attribute that lets you skip all these piddly statements: DebuggerStepThroughAttribute. You use it like any other attribute - apply it to a method, a whole class, or a property accessor. Note that you can't apply it to a property as a whole - it has to go on the individual getters or settors, like so:

 

public SomethingProvider SomethingProvider {
  [DebuggerStepThrough]
  get { return _somethingProvider; }
}

 

Put enough of these in and stepping in to a compound expression becomes much less tedious. Yay!

 

I don't think I'd recommend generally putting this attribute on anything but simple one-line properties of the form I've shown here, and even then you want to be careful: debugging is complicated, and I know I can screw up even simple property access - sometimes stepping into those things is actually useful, as it shows you that you cut and pasted one too many times and wound up returning the wrong value.

 

That said, you can still set breakpoints on code you've applied the attribute to, and the breakpoints will still fire.

Design and Implementation - Episode 3

Update: Part 2 is now available here. I've also updated the links below.

 

I just got done mixing the first part of the third episode of Design and Implementation, the occasional podcast that Tim and I do. This time, we interviewed John Lam, who is a Product Manager at Microsoft on the DLR team. John has been working on IronRuby, an implementation of the Ruby language on the .NET platform.

 

Tim has been big into Ruby lately, so we had a good time talking to John. So good, in fact, that we ran fairly long. So I've split the episode in two - the first part is now ready. When I get done mixing the second part, I'll put that up too. Sorry to be a tease. :)

 

Download episode 3, part 1 (MP3, 30.4 MB).

Download episode 3, part 2 (MP3, 16.6 MB).

 

Previous episodes:

Episode 2 - The MTPS REST Service

Episode 1 - REST Insights

Announcing F4Panel/KeyServer

Update: The term "KeyServer" is a registered trademark, so I had to rename mine. Details and an updated version here. The download link in this entry is no longer active.

 

I've been a fan of the F-16 combat flight simulator Falcon 4 for a long time. As with most flight simulators, situational awareness (SA) can be a bit of a challenge - flying a flight simulator where your only view is the computer screen has been described as like trying to drive while looking through a drinking straw. Well, when I finished FlexWiki, as a reward to myself I bought a TrackIr. It's a head-tracking device that lets me change the view by simply moving my head. It's very intuitive and freakin' awesome - it's hard to imagine flying without it anymore.

 

The problem with using TrackIr is that you start living in the 3D view. That's a problem because in Falcon 4: Allied Force (F4AF), many of the gauges don't work in the 3D view. The biggest problem, though, is that the cockpit isn't clickable. So you wind up bouncing back to the 2D view all the time to toggle this or that with the mouse, which is slightly annoying.

 

I thought about building some sort of USB device (this sort of activity is called "cockpit building", and people spend literally tens of thousands of dollars on it), or maybe buying something, but everything I could find was either a bit on the expensive side or was going to stretch my extremely rusty electronics skills. And was going to take a hell of a lot of time. But wait! I'm a pretty good programmer! I can solve this with software. Hence were born F4Panel and KeyServer.

 

The basic idea I had was to transform my ancient PDA (it runs Pocket PC 2002 to give you some idea) into a touchscreen device that would emulate the major instrument panels in the F-16. It's not an original idea - you can buy a much nicer-looking version of what I wrote for $30 at http://www.pocketmfp.com/default.htm. But it requires Pocket PC 2003, and my device is so old it doesn't support that.

 

To make a long story short, I wrote my own. Here are some pictures of it:

 

 

It works great, and I thought other people might like to make use of it, so I've posted a zipfile here.

 

There are two parts: F4Panel runs on the PDA, and KeyServer runs on the PC. Install F4Panel on the PDA by copying the appropriate F4Panel.cab file to your PDA and clicking it. KeyServer doesn't need to be installed - just run KeyServer.exe. When KeyServer starts, it'll put a very boring icon in the system tray. Right click it and choose "Load Profile". I've included a profile for F4AF, but the syntax is pretty simple and you should be able to figure out how to change it. Once you've got the profile loaded, go to the PDA and run F4Panel. Click Tools->Connect and enter the name of the computer where KeyServer is running. You'll obviously need to the PDA to be connected to the same network as the PC somehow - I just keep mine in its USB docking station.

 

The two programs communicate by means of a very simple HTTP-like protocol: F4Panel sends something like "CLICK ICP1" and KeyServer looks in a profile file for a command called ICP1 to figure out what keys to send to the foreground application. If it sends them okay, it returns "200 OK". I mention this, because I intended KeyServer to be general enough to be used with other games (or whatever) - you'd just need to write the PDA bit that sends "CLICK <commandname>" and the appropriate profile file that maps <commandname> to some sort of key sequence. The socket code is extremely simple - you can look at the F4Panel source to see it.

 

Because this was a fun-time project, and because quite honestly I'm momentarily burned out on full-scale open source projects, the zipfile is just a dump of the source and binaries. I'm sure you can cope. :) Consider it MIT-licensed, so feel free to go forth and improve it.

 

Update: The term "KeyServer" is a registered trademark, so I had to rename mine. Details and an updated version here. The download link in this entry is no longer active.

MTPS REST API Updates

Well, I warned you that we'd be updating the MTPS REST API (located at http://lab.msdn.microsoft.com/restapistubs/). And we did. A bunch of things have changed. Mostly, I've added new functionality. I did, however, change the way one thing works in a fairly significant way, but it just makes for a good opportunity to talk a bit about how the service works.

 

If you visit the content section of the service, you'll be given a list of all the content item IDs in the system. (A content item is an atom of information - for example the documentation for one of the overloads of System.Xml.XmlReader.Read.) Right now that list is very short, because we're still not hooked up to the live database, so the bits that I can't get from the MTPS SOAP service, I just make up. You can tell because they all have "fake" somewhere in the URL. Eventually, we'll list all the content items here. Not because I think that a list of all the content items is inherently useful in and of itself (although you never know), but rather because I believe that every URL in the system ought to be reachable by traversing links from the service root.

 

Clicking on the only non-fake content item ID takes you here. This is what we call the alternates list for the content item, which happens to be the System.Xml.XmlReader content item. "Alternates" because it lists all of the available versions/locales pairs. See, every content item can exist in multiple versions and in multiple languages. So you can click here to get the German, .NET 2.0 version of the System.Xml.XmlReader content item, or you can click here to get the US English, .NET 3.5 version of the same document.

 

Once you click one of those links, you arrive at the content item page. (You can always tell what type of page you're on by examining the class of the <body> tag.) Here, you get links to all the various pieces of information that make up the content item. One of the most interesting is the Mtps.Xhtml primary document. This document contains most (but not all) of the information you're used to seeing on the MSDN2 website when you look up a class or method. And here's where one of the changes came in.

 

In the first version of the service that I posted, if you were to look at an Mtps.Xhtml document (for example, this one), you'd find that clicking in the links it contains would jump you to the other Mtps.Xhtml documents. At first blush, this is a fairly natural thing to do, as it mirrors what happens on the MSDN2 website. However, there was a subtle problem lurking that came up, and now the service no longer behaves that way. Now, if one content item links to another in the Mtps.Xhtml document, that link takes you to the list of alternates for the target document.

 

This has a number of implications. First of all, it means that if you want to use the MTPS REST API as a (really ugly) documentation browser, you're going to do a lot more clicking. To navigate from one Mtps.Xhtml document to another, you're going to have to click once to get to the list of alternates, again to pick a particular version/locale combo, and then again to get to the Mtps.Xhtml primary document. Not optimal from a human standpoint it's worth mentioning that the REST API is not designd to be navigated by a human…except developers who want to get an idea of how it works.

 

No, there's a good reason for this change. You see, it turns out that the underlying MTPS database doesn't actually contain version or locale information. You can see this if you use the SOAP service - that document that comes back is linked solely via something called an asset ID, which is basically like the "b8a5e1s5" (we call that form a "short ID") part of the URL for a content item. In the initial release, I actually wound up just using the locale and version of the linking document to synthesize a link to the target document. Obviously, this would break if the target document happens to be of a different locale and/or version. And it did.

 

But that's not the reason we changed it. The real reason is that it's not clear that the service should be defining what the correct behavior should be. Should we always send links to the latest version? To the same version? If you think you know the answer for your scenario, let me assure you that someone else will have a different one. On the MSDN2 website, we can make assumptions about which way it should go because it's not an API, it's an application. But the REST interface is an API. So we have to leave it to the client, which means linking to the alternates page, so the client can implement whatever policy they want - pick the latest, pop up a dialog, whatever.

 

I mention this because to me, it's really characteristic of the RESTian nature of the interface in two ways. First, we leverage links, rather than expecting clients to synthesize URLs based on knowledge of how the system works. It emphasizes the view of the system as a directed graph of linked resources. That has more of the "of the web, not just on the web" flavor that (IMO) characterizes good REST interfaces.

 

The other thing that it brings home is that the REST API is not a website in the traditional sense. That is, it's not that I've just built a website and decided that all my .aspx pages now have .xml siblings or whatever: the system is fundamentally designed in a different way - to expose as much of the information available in the database as possible, not to present it for human consumption. Which goes part way towards explaining my decision not to include stylesheet links in the head, too. (A post for another day, perhaps.)

 

So this has turned into a somewhat long (and perhaps somewhat rambling) post about a service that - as far as I know - no one is using yet (tell me if you are). But as the first major REST system that I've implemented "for real", I've found the issues that come up to be enormously helpful towards my understanding of the advantages and limitations of REST. Hopefully some of you will, too.

 

I hope to occasionally post more information about the design and implementation of the MTPS REST API, and about the things I'm learning from it, as we move forward.

Design and Implementation - Episode 2

Tim and I sat down and recorded our second episode of "Design and Implementation", our occasional podcast. In this episode, we discuss the newly-launched MTPS REST API.

 

Grab the epsiode here: Design and Implementation Episode 2 (MP3, 33MB).

 

If you missed our first episode, get it here.

Announcing the MTPS REST API

If you've been reading my blog for long, you know that a while back I wrote the MTPS Content Service. And you've probably also noticed that as part of that experience, I learned a lot about SOAP and REST. Well, those lessons weren't just academic; for the last few months I've been working on the next generation of MTPS web services, and this time, they're RESTian!

 

As of today, you can go out to http://lab.msdn.microsoft.com/restapistubs/ and see for yourself. That URL is the root of a prototype REST API that exposes basically everything that the SOAP API does, plus a bunch of other pieces of information. Over the coming months, we'll be adding more functionality to it, too, as we try out some of our ideas for MTPS web services V.next.

 

Before going any further, I'd like to stress that this service is a prototype. You've probably noticed the "lab" part of the domain name in the URL. Well, we're serious: we will be updating and modifying the service over the coming weeks. Don't count on piece of data to stay in the same place, to stay in the same format, or even to work at any given time. That said, we expect a trajectory towards increasing functionality and stability.

 

There are a few notable things about the new service. Aside from the obvious fact that it's REST, not SOAP, perhaps the most significant thing is that the data format is fundamentally different. Specifically, 99% of the returned data is XHTML rather than the XML custom grammars that the SOAP service returns. Do a "view source" on something like the page for System.Xml.XmlReader, and you'll see what I mean. Basically, instead of using custom elements like <locale> or <version>, we've gone with things like <span class="locale"> and <div id="version"> instead.

 

This approach has two major benefits. The first is that you can use the service from a web browser. It won't look pretty  - the format is designed primarily to carry structured data, and only secondarily to be viewable - but you won't need to write any client software to get a pretty good idea of how the service is set up. My experience writing the service is that this is actually a really useful attribute.

 

The other benefit of this approach is that it's, well, XHTML. That is, there are certain things about the grammar that are already understood by any person or automated tool that knows XHTML. So if you see <a href="some/url"> in the data, you know right there that the item you're looking at and the one linked to have a relationship. Not only that, but you know how to retrieve it. You could achieve the same thing with a custom XML grammar, but a) why reinvent the wheel, and b) you'd probably never be able to teach existing web tools (think search engines) what your custom grammar meant.

 

On the downside, XHTML is currently a little harder to work with from a client standpoint than an XML grammar designed to deserialize to an object graph. In my opinion, that has changed somewhat with .NET 3.5, and specifically System.Xml.Linq. I've written a bit about that already here and here and may have a bit more to say about it in a future post.

 

I'd like to call out two features that are unique to the new version of the service: tag and annotation info. If you visit http://lab.msdn.microsoft.com/restapistubs/community, you'll see links to these two sections of the website. This is data reflects a feature that's been on the MSDN website for quite some time, but that seemingly not a lot of people know about - you can tag and/or annotate items in the documentation with your own tags and comments. Check out this as one example of community-added content.

 

Right now, the tag and annotation data available through the REST API is "fake". That is, it's just a bunch of placeholder information, and doesn't reflect the contents of the MTPS database. That's just because we're still running with the version I developed on my laptop as we iron out the last issues with connecting the service to the live database. But it should be hooked up to the real annotations and tags before long, and you'll hear about it here when it is.

 

OK, I think that's enough info for now. I'll post more before long.

Moving On From FlexWiki

It has been something like four years that I’ve been working on FlexWiki. Not very quickly at times, but I do at least try to put in around an hour every day. For quite a while now, that hour has become more a chore than something I look forward to. But I was committed to completing the tasks I said I would take on, so I kept going.

 

Well, now I’m done. I even checked in my experimental output caching code, although it's not in the 2.0 release.  

 

All this is by way of saying: I’m leaving the project. It’s in a good place technically, we’ve got some very competent developers still doing great work on the app, and more and more it’s someone besides me answering user emails. I feel absolutely great about the people that remain being able to maintain and improve the product. There are, of course, any number of things that I’d love to fix or add to FlexWiki, but the truth is that I’m just burned out. Plus, our second child is due to arrive in a few weeks, and I think it would be foolish to assume even my limited free time is going to survive that.

 

So what am I going to do with that hour a day? Not sure yet. Maybe just savor it in the weeks before the birth. :)

FlexWiki 2.0 RTW Available

It's been a little over month since we released FlexWiki 2.0 RC1. Well, unlike certain software companies I could name, when we say "Release Candidate", we mean it: we couldn't find anything majorly wrong with that version, so today I slapped a new label on it and sent it off into the world as FlexWiki 2.0 Released to Web (RTW) version.

 

So if you've been waiting for FlexWiki to shed its beta and RC monikers, go forth and download it from the project website!

 

Thanks again to Derek, John, and Nathan for their contributions, as well as to those others who have contributed since the FlexWiki 1.8 days (sadly, I don't have a list handy).

 

As this concludes something more than two years of work on my part, you can imagine that I'm pretty pleased.