Link to Rory

Apparently, Rory needs links. We lovable nutjobs (my insanity is better hidden…I think) have to stick together, so I'm obliging.

Hello Lisp

I believe it's a good idea to learn at least one new programming language a year. If nothing else, it keeps me from getting bored. You might think I'd choose Ruby, as that's pretty hot right now (in certain circles). And in truth I'm interested. I've even read some of The Pickaxe Book. But the language I've decided to devote my time to is Common Lisp.

 

There are a number of reasons for this. I won't go into all of them, but here are a few:

 

1. I've been meaning to really learn a Lisp for a long time. Probably since I took SICP as a freshman. (One of the more intimidating moments of my life: having Hal Abelson as a student in a class I taught. Really nice guy, fortunately.)


2. I do a lot of work where I turn set-based data (from a database) into hierarchical data (XML). A language based on lists and conses seems worth investigating.


3. Lisp has a pretty interesting reputation as "the most powerful and elegant programming language". That's quite a claim, and begs to be investigated.


4. Every new feature that C# has sprouted has been available in Lisp for decades.


5. I read this and was intrigued.

 

Of course, I'm going to continue to do my paying work in C#, if for no other reason than that no one in their right mind would pay me to write production Lisp code right now. But starting now, I'm going to make an effort to write all the code that I can in Lisp.

 

If you're interested in getting started yourself, it's as easy as going to download an implementation. I'm using CLISP, as it's free and seems to be pretty mature and broadly supported.  

 

Should be an interesting ride!

The Failure of SOAP

It may seem odd to see the title of this post, seeing as it immediately follows one entitled "The MTPS Content Service - A SOAP Success Story". But it was always my intention to write these two back-to-back, because my experience writing the MTPS service was a major part of my most recent revelation: SOAP is no longer my default choice for writing web services.

 

You've seen some of this story already. I said I've been thinking like Tim for a while now. That thinking has really solidified for me over the last few weeks, to the point where I'm ready to essentially walk away from ASMX/WCF/SOAP in all but a few edge cases.

 

There have been three main ingredients in this solidification of my "conversion". First, I managed to finish RESTful Web Services, by Sam Ruby and Leonard Richardson. This is a first-rate book. It is going on my shelf right next to Transactional COM+ as a "must read". Not that I necessarily agree with everything in it - in particular I'm not sure that PUT and DELETE are appropriate choices for most web services - but the way it helped me think about URL structure in particular puts it on the "must read" list for me.

 

The second ingredient was a series of conversations I had with John Elliot, one of the better-known posters on the Win Tech Off Topic list. He's been telling me for years about his ideas about web services, so I took the time to correspond with him to get clarification, and it really resonated. In a nutshell, his position is that web services should be accessible via a web browser, an idea I really like. This implies a read/execute bifurcation along GET/POST, metadata via HTML Forms, and XHTML as a the primary payload. Tasty. (Apologies to John if I've misstated or over-simplified his position.) 

 

The final (and key) ingredient was the realization that SOAP is the COM+ of the modern era. I did a lot of COM+ back in the day, even going so far as to teach classes about it and to write an MSDN article on it. So for some value of "expert", I was an expert on the technology. And yet, my main piece of advice to clients these days is: don't use it unless you have to.

 

Note that I'm not saying that either SOAP technologies or COM+ don't work. They do. If your problem fits the solution (e.g. multi-object transactions in COM+, single-URL RPC interfaces in SOAP, for example), then go ahead. It's just that I get enough benefits and avoid enough overhead from doing it at the lower (and simpler) level in each case that it's not my default choice any more.

 

It's also interesting to note that in both cases it was Microsoft - the vendor with a vested interest in me adopting the technology in question - that was the primary source of the information that led me to go down the SOAP/COM+ path. I'm not questioning anyone's integrity - it's just the nature of information that of course the COM+ message was going to be "use it for everything". And similarly SOAP. The failure to analyze whether the technology in question was the most appropriate for my problems was, of course, entirely mine. But hey, like Tim always says, "I reserve the right to get smarter." :)

 

If this seems like a repeat of things I've said already, well, I apologize. But I just had to share - the realization of the parallels between my personal journey with COM+ and my personal journey with SOAP just struck a real chord with me. Your mileage may vary.

The MTPS Content Service - A SOAP Success Story

When I wrote the MTPS Content Service (was that a year ago already?) I didn't really expect people to use it much. I figured there'd be a few small and interesting apps - for instance, I wrote msdnman mostly for fun - but that it would largely be a useful but little-used service.

 

I was quite wrong.

 

The other day I found out the usage numbers for the service. The conversation went something like this:

 

Kim: "Hey, I wanted to tell you the stats are in for the content service."

Me: "Oh really? I'm curious - what are they like."

Kim: "Here, I'll email you the numbers."

Me: "Wow! It's only been a year and there's been 1.3 million hits - that's a lot!"

Kim: "No, that was for yesterday."

Me: "…"

 

I was quite literally speechless. Which, if you know me, you know is a rare thing. :) That's fifteen requests per second, so perhaps I can be forgiven my momentary surprise. (Traffic has since subsided to a mere 600K requests or so per day - an average of around seven requests per second.)

 

I have to say I'm fairly proud of having written a web service that can sustain those sorts of loads. Of course, the credit is not due all nor even (close to) mostly to me: the service is a fairly thin wrapper around the MTPS system, and I was only one part of the team that did that. Plus, without a crack ops team it doesn't matter how good your software is - you're not scaling. Kudos to those guys as well. All I had to do was not screw up the translation to SOAP. :)

 

We don't really have good data on what applications are creating all this traffic. We do know that the recently-released PackageThis is responsible for somewhere around a third of the traffic. We know that because PackageThis is the only app currently sending an AppID header with their requests. An AppID header is just a short string that identifies the application making the request. This is something we added rather late in the development cycle, never really documented, and made optional. So maybe it isn't surprising that it's not showing up in a lot of the requests. Hopefully we can fix that at some point.

 

Sandcastle also uses the service, and I wouldn't be too surprised if it accounts for the bulk of the remainder of the traffic. They're adding an AppID header soon, so we'll have better data then. It certainly will be interesting to see - personally I'd love it if there were still a whole bunch of unknown apps in the traffic pattern, as it would suggest that lots of people have found good ways to use the service. Which was our hope when we wrote it.

FlexWiki 2.0 Security Bug Alert

I noticed something weird on one of the wikis I’ve upgraded to FlexWiki 2.0. Even though I’d locked down a namespace to only allow authenticated users to edit, I was still seeing new topics getting created. Sure enough, when I tried it myself, I was able to create a new topic even though I wasn’t logged in. Fortunately, the problem only seems to manifest with new topics: editing of existing pages is still correctly prevented by the security provider.

 

After a bit of digging, I figured out that the problem is with the way permissions are handled for nonexistent topics. Basically, users were granted full control over nonexistent topics. The correct behavior is for nonexistent topics to be given the default permissions for the namespace, as once they are created, that’s what they’ll have (absent explicit permission statements). I’ve coded the fix and submitted it – it’s present in build 2.0.0.49 and forward.

 

Note that the fix makes the wiki secure by ensuring that unauthorized writes can't happen, but that the UI is still somewhat wanting: You're not told that the write is going to fail beforehand. I'll make that change soon. I just wanted to get a patch out to solve the underlying problem as quickly as possible.

 

If you’ve deployed FlexWiki 2.0 you should seriously consider upgrading to this latest build. Available here.

Microsoft vs. TestDriven.Net Express

TestDriven.NET is the only Visual Studio extension I ever use. It's fantastic. So I was really bummed to read this, which tells about the legal trouble Jamie has been having with Microsoft over his truly useful product.

 

I'm no legal expert, so I have no idea which side is "right" here (assuming that it's even as black and white as that). All I know is it would suck if Jamie had to stop making TestDriven.Net, or even got so annoyed that he didn't feel like working on it any more, and that he has my sympathy. I've been in situations where legalities influenced technical decisions in distasteful ways many a time, and it always offends my sense of software aesthetics.

 

I wanted to blog this because I think people should be aware of it. It's days like this I seriously consider ditching as much proprietary software as I possibly can and living the free and open software lifestyle for real.

The Secret To Software Estimation: Lie Like Scotty

I see that Scott Hanselman posted about software estimation. Excellent advice, as usual. Here's my somewhat less serious advice, in the form of a quote from "Star Trek III: The Search for Spock":

 

Kirk: "How long to re-fit?"

Scotty: "Eight weeks. But you don't have eight weeks, so I'll do it for you in two."

Kirk: "Do you always multiply your repair estimates by a factor of four?"

Scotty: "How else to maintain my reputation as a miracle worker?"

Kirk: "Your reputation is safe with me."

 

I always multiply my initial gut-reaction estimate by four. So if I think it'll take a day, I tell the client it will take a week. If I think it'll take a week, I tell the client it'll take a month. This works really well for a couple of reasons:

 

1. My initial gut-reaction estimate is almost always wrong by a factor of two or three.


2. When I tell the client it's going to take a week and it takes three days, they're thrilled.

 

The funny thing about all this is that it still works even if I tell the client what I'm doing beforehand. Weird but true: I tell them a week, they know I actually think it'll take a day, but they're still happy when it's done in three.

 

Of course, this really only works after I've established a track record of delivering solutions to hard problems. And my miracle worker reputation gets a bigger jump from not telling them beforehand, so I usually don't. Except for the ones that read this blog.