How Very 1998 of Them

If you've read my blog for a while, you know that I've long since left GotDotNet for SourceForge. Generally, I've been more happy there, particularly because I far prefer CVS to the crazy contraption GDN has, but not all is rosy. To start with, I find that during the middle of the day, SF has lately been slow to the point of unusability. This is particularly painful when I'm trying to use the bug and feature tracking pages, which require several clicks to get what I'm looking for. Each click can take as long as 30 seconds.

At times I've thought that maybe I should just set up my own bug database. But then I'd have to evaluate the options, pick one, and then maintain it. Bleh. Still, it might be better than having to wait half a minute for each page to come back. Then, the other day, I stumbled across this, a feature of SF that lets you get the entire contents of a tracker in “XML” form. (More about the quotes in a second.)

Brilliant! With this, I figured I could suck the bug and feature databases onto my machine, then use Excel or InfoPath (if I ever figure out InfoPath) or whatever to browse the database quickly. I can't do updates, but often times all I want to do is read through a bug description or do a quick search, and a local export is perfect for that. So I sat down to automate download, got done with that in a few seconds (gotta love wget), and then went to load up the FlexWiki bug database in Excel.

XML Parse Error.

WTF? I ran it through one of the XML utilities I wrote, and found out that there was an unescaped “smart quote” in the text. Disturbing that it wound up in the export, but oh well - three seconds with “find and replace” fixed that. Fire it up in Excel again and...

XML Parse Error.

OK, now what? Well, it turns out that the smart quote issue was just the tip of the iceberg. Here's an exerpt from the exported file that shows the problem in all its glaring obviousness:

<detail>Produces this output (first </ol> not needed, second
opening of <ol> not needed).  If that was fixed then the
incorrect double <ul> should also be fixed (single <ul>)
as they would be no longer necessary. BTW:  also seen
here the unnecessary blank lines
</detail>

Note the numerous problems with this. Here it's unescaped HTML tags being produced verbatim, but unescaped & and < characters (not to mention the smart quotes) litter the export. This is a classic sign of XML being built via string concatenation, and it renders this export virtually useless for consumption - all the dangling close and open tags completely destroy the ability of any reasonable XML parser to work with this document. I would have thought this sort of thing went out with go-go boots, but here it is, right in my face, in 2005.

Sigh. It's a fairly simple fix to process the document with custom code to escape the “XML” well enough to be able to work with it, but I shouldn't have to. The moral of the story here is that if you find yourself doing something like this:

xml = “<foo>” + fooContents + “</foo>”;

then you should lose points on your programming license.

Beam Him Up

James Doohan, the actor who played Scotty on the original Star Trek series, died Wednesday of complications related to pneumonia. He was 85. I think it's safe to say that the chief engineer on the Enterprise was an iconic figure to a pretty good portion of our profession. I know for my part I quite often employ his trick of multiplying all my estimates by four. It's funny, but even when you tell people this is what you're doing, you still look good when you finish in half the time you said it would take you.

Hats off to Mr. Doohan for helping to create such an unforgettable character.

Convergence

As part of my undergraduate education, I spent a fair amount of time programming in Scheme, which is a dialect of LISP. At the time, I didn't really appreciate how truly powerful LISP is, but in the intervening years, as I've seen languages like C# and Java grow up, I've made or heard the comment “Yeah, LISP already has that” enough times to make me consider trying to program exclusively in LISP for a while. As was the case with my self-imposed switch to Test Driven Development, forcing myself to use a new technology is guaranteed to be a great way to learn something. The kicker, of course, is that I'd really miss the .NET libraries - at this stage of my career writing your own file IO routines is only fun for about ten minutes, and learning new ones is only slightly more fun.

Not that that's going to stop me from learning Ruby¹. You've probably heard the buzz around it lately - I certainly have. Buzz I can generally ignore, but when smart people like Brad get excited about something, I generally try to get around to looking into it sooner or later. Particularly when Scott raves about Watir, and I need something that looks a lot like that for several projects I'm working on. So yesterday I started to work through a pretty good Ruby tutorial on my way to playing with Watir, and I thought to myself, “Man, this looks a lot like LISP's child by a shell script mother.” Not the syntax - there Ruby and LISP are pretty different - but in a qualitative way that I have a hard time pinning down...they just “feel“ similar in how they deal with programming abstractions at a high level. I don't know: it's hard for me to express given how little Ruby I've done and how little LISP I've done lately.

Anyway, this morning I'm flipping through my aggregator entries, and I see this post from Pinku Surana about how it would be relatively easy for someone to knock out a LISP.NET implementation. Combine this with the fact that I'm basically between gigs right now, and I think the planets are aligning - it seems I'm being directed to learn a higher-level language. I just need some flaming letters on the mountainside to be sure. :)

---

1. Although it looks like there's a Ruby/.NET bridge - have to check that out at some point.

Take a Walk on the Wild Side

I recently found myself in the interesting position of being asked by my client to help them write a web service client...in Java. I know basically nothing about Java, but I'm usually pretty good about picking things up quickly, so I dove in. I don't think I have anything particularly technical to add to the body of knowledge, but I thought I'd relate my experience as a complete Java n00b in the web services space.

My initial involvement in the effort happened after the client's attempts to get Axis working failed. This was probably due to the fact that Axis seems to be primarily a server-side technology, and they need to get it working from within an applet running in a browser (they're providing a rich UI for a complicated part of an ASP.NET web application). I imagine we could have gotten Axis to work had we banged on it for long enough, but this part of the project was rapidly becoming the biggest schedule risk. So I went and asked some of my smart Java friends what they recommended as an alternative.

One of them suggested XFire. It seemed like a reasonable alternative, but my client was unable to get it working either, even after a few days of hacking on it. Meetings were called, and more and more senior managers were called in. This is where I took an active hand.

Not knowing anything about Java or Java web service toolkits, I figured that given all the knobs and buttons we were having trouble figuring out, the most predictable route was one that eschewed web service toolkits entirely. So I asked some smart friends again, and got pointed to JAXB, which is Sun's equivalent of XmlSerializer. Once I located xjc.bat, I was able to generate some simple classes and to serialize/deserialze them to/from XML within a few hours.

At that point, I felt a lot better. The code to do low-level HTTP in Java is pretty straightforward, and with the JAXB-generated classes, I knew we'd be able to do what we needed to do. But since I was now slightly ahead of the three days I told my client it would take me to figure out the problem, I decided to spend a little time working with SAAJ, which is the next layer in Sun's web service stack. It abstracts away the HTTP part of doing XML web services. I figured it was worth a little time to experiment, as it was likely a higher-quality version of the code I was going to wind up writing myself anyway. As it turned out, it was also pretty easy to figure out, and after a few more hours I was able to use my JAXB classes together with the SAAJ stuff to call a simple .NET web service successfully.

The last part of the infrastructure I wrote by hand. It was a fairly simple XSLT that would walk a WSDL document and generate a Java proxy class that would make the SAAJ calls I'd figured out. That took me another few hours. All told I had something workable in around a day and a half. It's not the world's most robust thing right now, but it unblocked the guy who's writing the applet, and that's a Very Good Thing. As we move forward, I'm sure we'll need to add things like better SOAP fault handling, but I don't think that's going to be too terribly difficult either.

The bottom line of all this is that - in the Java world - it appears to have been much easier for me to get up and running with low-level, XML-oriented APIs in Java than with a big OO-oriented toolkit (Tim would be proud). I am, of course, no expert in Java, and the real test will come as we try to use the code I generated in a production application, but I found the experience quite interesting.

Visual Studio Guidelines

Via John Robbins, I came across this little gem today. It's a registry setting that lets you display vertical guidelines in Visual Studio (2002, 2003, or 2005). This is handy as a cue that you might be writing lines that will be too long for your teammates to read without scrolling. Looks a bit like this:

Here's a .reg file for doing it in VS.NET 2003. Note that I've set my guides in columns 80 and 120 - you'll probably want something different.

Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\7.1\Text Editor]
"Guides"="RGB(128,0,0) 80, 120"

PerformanceCounter Slowness Explained

The unparalleled David Gutierrez from Microsoft stopped by and left a comment on my post about horrible PerformanceCounter slowness. His explanation in its entirety:

 I can shed some light on these issues. PerformanceCounters use a machine wide memory mapped file to publish perf counter data. That file contains an object graph and various places we use as spinlocks.

The problem comes in when processes get killed unexpectedly. A process which dies while holding a spinlock won't release it, meaning the next process gets blocked. After spinning 10000 times, we assume some process died and continue on, so we won't block forever - 4 minutes on your machine. My guess is that IIS is terminating your app when it resets it.

In Whidbey we've fixed these problems. If you're interested, I can post a blog entry with more details.

The bug you mention actually isn't related. It has to do with reading perf counters, and that really is horribly slow. It was also specific to services.

Obviously, I still need to track down why this is happening, but I think his guess is likely a good one: ASP.NET worker process resets could be dropping the spinlock on the floor. Many thanks to Mr. Gutierrez!

PerformanceCounter Constructor Horribly Slow

I'm doing some work on FlexWiki, and I'm finding that this call:

PerformanceCounter answer = new PerformanceCounter(s_PerformanceCounterCategoryName, name, false);

is sometimes taking a really, really long time to come back. Like several minutes. I started tracking it down via Reflector and native debugging, and it looks as though PerformanceCounter internally spins in this loop, inside a BCL-internal class called SharedPerformanceCounter:

int num1 = SharedPerformanceCounter.MaxSpinCount;
while (spinLockPointer[0] != 0)
{
  Thread.Sleep(1);
  num1--;
  if (num1 == 0)
  {
    spinLockPointer[0] = 0;
  }
}

Aside from the fact that this looks thread-unsafe to me, I'm not entirely sure what it's waiting for at this point. But given that MaxSpinCount is 10000, on my machine that seems to work out to around four minutes, so that explains where the delay is coming from. But what I don't understand is under what conditions this occurs - the SharedPerformanceCounter code is a bit hard to wade through, and I haven't made much headway with it.

In an RTFM moment, Shawn Van Ness suggested I look over at the Product Feedback Center to see if there were any similar bugs. I'd Googled, but hadn't landed there, although I should have thought of it. At any rate, when I did look I found this rather disturbing bug. Why disturbing? Because it implies that slowness in PerformanceCounter is to be expected. Although one could also read the bug report to merely indicate that the service hanging because of a long-running operation is the expected behavior. Either way, not very helpful.

Anyway, I haven't been able to figure anything out on this, so I'm blogging it. Ideally someone will know what I'm doing wrong. As a second best, perhaps the entry will serve as a sanity check for others with the same problem. Has anyone seen this constructor take a really, really long time to come back before?