Thursday, June 21, 2007

The MTPS Content Service - A SOAP Success Story

When I wrote the MTPS Content Service (was that a year ago already?) I didn't really expect people to use it much. I figured there'd be a few small and interesting apps - for instance, I wrote msdnman mostly for fun - but that it would largely be a useful but little-used service.


I was quite wrong.


The other day I found out the usage numbers for the service. The conversation went something like this:


Kim: "Hey, I wanted to tell you the stats are in for the content service."

Me: "Oh really? I'm curious - what are they like."

Kim: "Here, I'll email you the numbers."

Me: "Wow! It's only been a year and there's been 1.3 million hits - that's a lot!"

Kim: "No, that was for yesterday."

Me: "…"


I was quite literally speechless. Which, if you know me, you know is a rare thing. :) That's fifteen requests per second, so perhaps I can be forgiven my momentary surprise. (Traffic has since subsided to a mere 600K requests or so per day - an average of around seven requests per second.)


I have to say I'm fairly proud of having written a web service that can sustain those sorts of loads. Of course, the credit is not due all nor even (close to) mostly to me: the service is a fairly thin wrapper around the MTPS system, and I was only one part of the team that did that. Plus, without a crack ops team it doesn't matter how good your software is - you're not scaling. Kudos to those guys as well. All I had to do was not screw up the translation to SOAP. :)


We don't really have good data on what applications are creating all this traffic. We do know that the recently-released PackageThis is responsible for somewhere around a third of the traffic. We know that because PackageThis is the only app currently sending an AppID header with their requests. An AppID header is just a short string that identifies the application making the request. This is something we added rather late in the development cycle, never really documented, and made optional. So maybe it isn't surprising that it's not showing up in a lot of the requests. Hopefully we can fix that at some point.


Sandcastle also uses the service, and I wouldn't be too surprised if it accounts for the bulk of the remainder of the traffic. They're adding an AppID header soon, so we'll have better data then. It certainly will be interesting to see - personally I'd love it if there were still a whole bunch of unknown apps in the traffic pattern, as it would suggest that lots of people have found good ways to use the service. Which was our hope when we wrote it.


  1. Really glad to hear that, it is a kind of statistic one does want to have a name behind.

    It was about time we had a SOAP revenge :-)

  2. Stay tuned - the title was chosen very carefully, but maybe not for the reasons you think.

  3. I dropped my reasons as a comment on another entry on your blog but hey always open to suggestions.

    Of course Content twist is always possible :-)

  4. I'll probably post tomorrow with the followup.

  5. Now if only you figured out how to surreptitiously serve an ad for each request. You'd be rich!!! ;)

    Nice work.

  6. That *would* be nice. And I'm sure Microsoft would happily split the money with me. :)

  7. That's cool to hear.

    Congrats on all your hard work paying off in such a manner.

  8. With either

    "msdnman.exe ms310241" or

    "msdnman.exe ms123401" I get "The server committed a protocol violation."

    Is it temporary or has something changed?

  9. Must have been temporary: both of those work for me right now.

  10. It must be my setup then.

    Congrats for coming up with the nice tool nevertheless! (Y)

  11. There's a note about proxy setup on the msdnman homepage [1] - perhaps that's the issue?


  12. Indeed, it was our corporate firewall (not proxy, though). I tried from home and everything worked like a charm!