Friday, April 27, 2007

Thinking Like Tim

I talk to Tim on a regular basis. So when he outed himself as a REST convert lately, it actually wasn't a huge surprise to me - we've been talking a lot lately about what the last few years have taught us about web service programming, and how more and more SOAP/WS-* just doesn't seem to be all that and a bag of chips. Still, the key part of the analysis in his latest post - that you can use HTTP as a way to publish application states and the transitions between them - really struck me as something significant, and brought a lot more clarity to the whole problem.

 

Here's where my thinking is at: Basically every SOAP web service I've written in the last four years could have been done just as well as a straight HTTP interface. And a several of them would have been significantly better. The ones that couldn't are very clearly RPC style, for better or for worse. I've been feeling this especially acutely lately as I try to optimize one of the services I've written. I found myself implementing HTTP over SOAP, which just seems stupid, not to mention clumsy.

 

For me it's really not about whether the thing is "REST" or not. I find the term REST to be too laden to be useful. I think more in terms of "SOAP via toolkits" (SVT?) versus Plain Old XML over HTTP (POX). After all, I'm perfectly willing to use a SOAP Envelope as long as I still get to use HTTP GET. But at the end of the day, ignoring particular terminology, I agree with Tim's analysis: SVT is for stuff I'd use DCOM for in the old days, not for putting up a large-scale, public-facing, web-based, programmatic interfaces.

 

I realize that programmers love their tools and toolkits, but honestly, when you build big giant systems sometimes you have to do things the "hard" way. It always surprises me when my clients complain that complex problems contain complexity in the solution. But in any case, I'm not even convinced that throwing away toolkits like WCF will raise the complexity level - there's a lot to be said for being closer to the wire. Plus, things like Linq for XML look really tasty when you consider how they might help with this style of programming.

 

I do really like the analysis of REST as Representational State *Transition*, as opposed to the T being for "Transfer". To me, viewing the design exercise as figuring out how to expose a connected state graph (i.e.

both transitions and states are explicit) brings a lot of mental order to what I already knew I wanted to do anyway. In particular, it helped me understand how a POX interface is useful for writes...because for reads it's dead obvious: GET kicks ass, especially if you care at all about performance. For a long time I was hung up on trying to figure out how to use SVT on write, and GET on read, because that seemed to make the most sense. But if the thing is just your application state where each state has an address, it's pretty obvious that it's okay to POST to a URL to cause a state transition. Or even for that state transition to significantly alter the URL space.

 

Of course, I have yet to build a large, real system using this new approach. I'd love to, though, because more and more it seems like the right thing. And no battle plan survives first contact with the enemy - witness SVT. The big question is whether the big concepts will prove out for me personally. Hence my eagerness to put my customers' money where my mouth is :).

 

And before you point it out in the comments, I should say that I'm aware of the efforts to make WCF more amenable to straight HTTP-style programming. It's just that I'm dubious about the viability of exposing transport semantics explicitly via an API that's all about abstracting transport semantics. It seems to me that if you're going to remove a layer of abstraction, you should actually remove a layer of abstraction. But again, I'll need to build something real with it before I can draw any supportable conclusions.

28 comments:

  1. When you think things are not possible: WCF duplex callbacks through NATs and firewalls - safe and secure...

    ReplyDelete
  2. I'm a bit flat out at the moment, and I haven't read all the stuff floating around, but this jumped out at me:



    RE: "Representational State *Transition*"



    ...I think that misses the point.



    URIs are pointers to 'entities', which are 'objects' in Plato's sense: they're transcendental. What you can do is get a representation of their 'state' *at a point in time*. Their state can change independently of your actions over time (trivially, in 1953 your URL was throwing 404; more complicatedly (that's a word, now), other users can affect an entity's state). So... it's a faux-enlightenment, I think.



    Once you understand what 'state transfer' is all about then I think that's when you get, um, *HTTP*.



    Certainly URLs point at a state, but indirectly, and time always goes forward. So you don't actually model state transitions, you model "entity representation" transitions, where "entity representations" are your state. It's a relevant difference, I think. (Hint: ever pressed F5 in your browser? Why? Where's the model of the state transition, then? Ah...)



    p.s. What we need is a REST 2.0, to address the whole subject/object thing. (AKA: HTTP with milk and cookies. :)



    p.p.s. PUT and DELETE Considered Harmful, and Google got it wrong. RUHIHF. ;)



    p.p.p.s. Sorry about the spelling mistakes. Lousy web browser.

    ReplyDelete
  3. So, my understanding on this is evolving. I know that you're way ahead of me on thinking about this problem space, so you may have to give me some time to catch up, if indeed I ever do. :)



    I agree with you that URIs are not solely a mechanism for driving state transitions. However, if you come at it from a heritage of SOAP-style interfaces, with an incomplete understanding of what "REST" is (as I have) then the realization that URIs can *also* be a mechanism for driving state transitions is key. Perhaps you hit that quite some time ago.



    Anyway, I don't think I disagree with you. I just found Tim's comments helpful in breaking down a particular barrier to understanding (i.e. how do "writes" fit into the picture) because of my particular background.

    ReplyDelete
  4. > the realization that URIs can *also* be a mechanism for driving state transitions is key <



    Totally. It's just 'state transitions' is kinda the wrong term to be using to point at the concept you're trying to isolate, because in the context of REST 'state' means something a little different to what you're talking about. You might instead consider the terms 'page transitions', or 'workflow', or... something.



    I think we're basically on the same page.



    Regarding terminology to avoid the term 'REST' have you considered "HTTP"? :)

    ReplyDelete
  5. Agreed.



    And I've already started to eschew the term "REST". You can see that a bit in the post above - I've pretty much abandoned it entirely since then as belonging in the same category as "ActiveX" as a useful term.



    I'm not sure if "HTTP" is the replacement I'll settle on, but it does seem to be the leading candidate. :)

    ReplyDelete
  6. I rather fancy POST-modern. :)

    ReplyDelete
  7. I've got it - HAPI. HTTP API. Catchy, yes?

    ReplyDelete
  8. For some reason this [1] is the first thing HAPI conjures in my mind.



    [1] http://www.youtube.com/watch?v=Qo3mnXGbJlg

    ReplyDelete
  9. Cool yet disturbing. :)

    ReplyDelete
  10. Like the web? :P

    ReplyDelete
  11. I wrote a long comment earlier explaining away the sloppy language I used earlier, but I accidentally pressed a few wrong keys in my browser and it closed destroying my draft (go figure). Basically when I said "entity representation" I should have said "resource representation" which is an "entity", and where I said "entity" I should have said "resource". I also explained away my sloppiness, but I don't feel like doing that again. Anyway... that's not the reason I popped back over. I wanted to comment (obliquely, opaquely, obtusely, etc.) again on this:



    > the realization that URIs can *also* be a mechanism for driving state transitions is key <



    So... what of AJAX? Oh, gosh. The difficulty!

    ReplyDelete
  12. Re: entities and resources - the distinction is lost on me, so I guess your imprecision did no harm. :)



    I'm not sure I understand your AJAX comment. What's the problem with an application within an application? Or is it that the URI of the containing page no longer has a clear logical model for what it represents?

    ReplyDelete
  13. > What's the problem with an application within an application? <



    The problem I have is making a value judgement about exactly that. Is AJAX good or bad? I can't say... it's too difficult. :P



    It's difficult because with AJAX a URL doesn't indicate a single application state on the client any more. Suddenly the client's state is a composition of a heap of other states, and that composite state no longer has a URL which identifies it. On the other hand, it's got to have a place, but then you get into that whole "application within an application" space which feels like an infinite loop.



    I know that when I use web-applications I really don't like it when I get into a view where the URL (and the time, and me) doesn't identify what I'm looking at. And I don't like it when I'm not given a 'link' to a view (so that I can open it in a new window (which is just another way of saying "download exactly that data again")).



    And that's really all I know. Thinking about the implications of AJAX and reaching a solid "position" about it is too hard for me. I tend to lean toward the "AJAX is a bad idea" position, but I'm not sure I could make a clear argument about why. Maybe for the same reason that building HTTP on top of SOAP on top of HTTP isn't real crash hot.

    ReplyDelete
  14. > Re: entities and resources - the distinction is lost on me <



    At the risk of being a bit sloppy again, a 'resource' is a thing that's identified by a URI. Fielding describes it as an abstract object. You can't 'touch' and abstract object, but you can get a representation of it's state at a point in time, and that's what you get when you apply GET to a URI for example. An 'entity' in the context of REST/HTTP is just the term used to describe the payload of a HTTP message (i.e. the bit that isn't a header).



    In my own thinking I tend to use the term 'entity' instead of the term 'resource' to isolate the same concept. I'd prefer to use the term 'object', but that's a too heavily loaded term.

    ReplyDelete
  15. I'm not quite sure how to think about AJAX, either. The thing is, I hate browser apps, so anything that punches them up and makes them more functional appeals to the user in me.



    Thanks for the description of entities and resources - that makes sense now.

    ReplyDelete
  16. > The thing is, I hate browser apps, so anything that punches them up and makes them more functional appeals to the user in me. <



    At the price of feeling like your every move is being watched?



    I hate browser applications, but the 'less' they do the happier I am.



    Frankly I think that browsers should just be better. I.e. the 'rich' part of a web-application should be provided by the browser, not by script. That's the direction I'm generally headed in, i.e. to write services exposed by HTTP and then implement my own browser for them. Until I can get the browser part done the HTTP-based services can be useful via HTML.



    <snip content="rant" subject="How much I hate Microsoft" />

    ReplyDelete
  17. > At the price of feeling like your every move is being watched?



    That's an excellent point that I don't think many people of have thought of, but of course it's also a point of personal preference. Look at the popularity of shoppers' club cards (you have those in Oz?) - clearly people are willing to sell their personal information for convenience/price/whatever.



    I'm curious how you'd model an AJAXy application in your build-a-browser world, and how it would provide more anonymity - can you elaborate?

    ReplyDelete
  18. > Look at the popularity of shoppers' club cards (you have those in Oz?) <



    Yeah, they're called "Fly Buys" or something here. I'd never sign up for anything like that, and I don't have a TV, and rarely go to the supermarket... I'm probably the worst person to ask about those. :P



    > I'm curious how you'd model an AJAXy application in your build-a-browser world <



    Well I think that some things that are missing from the browser are client-side persistence, rich text editing, spell checking, and things like that.



    I wouldn't write a general purpose browser-as-a-platform thing. I'd write a very simple report presentation and/or data-entry UI, targeting a simple LoB doctype, similar in many ways to XHTML (or maybe exactly XHTML). Maybe.



    It wouldn't be general purpose, is the key idea. I.e. I'd just write a new app to support a new LoB, and I'd be interested in HTTP as the document transfer layer (and REST as a general guide to my architecture), and possibly (though not necessarily) XHTML (note the X) as the document format (else perhaps CSV, or custom XML, etc.).



    Then again, I'm not trying to solve the same types of problems as a lot of other people (I don't generally write apps for massive public consumption, for example; nor do I want to), and I wouldn't likely stop supporting web standards (e.g. HTML/CSS/JS). Like I said, I'm just sort of 'heading that way', I'm not committed to anything -- just thinking about it, leaving options like that open with what I do today. Who knows what my future will bring..? (Pain, probably. :P)



    I'll dump Microsoft entirely (Internet Explorer and IIS specifically) the very first chance I get. That's kinda high on my list of priorities. :P

    ReplyDelete
  19. Btw, this probably bears mentioning:

    When Windows was on the rise, and shortly thereafter the web, there was a little bit of a cultural/philosophical gulf between the two schools of thought.

    One thing which I think has been almost completely overlooked, and perhaps even abandoned, is the homogeneity that was supposed to be present in the Windows user-interface. I.e. I often heard (and agreed with) the notion that all apps across a user's desktop should be similar. Similar colour schemes, similar UI metaphors, similar layouts, similar shortcut keys, similar accessibility features, etc. I'm a "content is king", "keep it simple", "I have call-centre staff that need to be trained and a whole heap of apps that they need to use" kind of a person. So I liked the general premise of the Windows UI, which was never fully realised and is now in the throes of being entirely abandoned.

    Win32 programming, in any way you want to dress it up (i.e. VB6 for me), is a crap metaphor for programming document-oriented systems used by LoB applications (or as I like to think about it more generally "those systems which facilitate bureaucracy"). The whole event-driven model is broken at that level of abstraction, and I’ve basically come away from thinking that “general-purpose” or “extensible” is a good way to think about software. The web metaphors (a la REST) are perfect for the document-oriented style of system. They have a very specific non-extensible document format (though I could design a better HTML in my sleep -- with the benefit of hindsight, obviously), and a simple and intelligent guiding network architecture (REST, basically -- the inter-machine analogue of the Unix pipe, done right (sans PUT and DELETE, which are mistakes in my view). Microsoft tries to dress this up these days in their confused, they-barely-get-it-themselves, they-are-contemptuous-of-their-customer-who-they-call-Mort, corporate-BS way as saying that "declarative programming" is a better model. ...and they're close, but still don't quite get it, in my view (I’d tell more people, except I’d get shouted down, and I don’t have much interest in that).

    At any rate, I’ve spent most of my career suffering at the hands of Microsoft’s proprietary platforms and marketing spin-doctors, so if I can (and I think I probably can) I’m going to try and jump ship all together to the safe harbours of GNU/Linux (Debian, specifically), where I can own my own technology stacks. I feel like it’s sort of reaching the point where the damage Microsoft does to me isn’t offset by the value they create for me (i.e. value I can find elsewhere, damage I don’t need).

    That might all seem a little tangential, but it’s all relevant to the context which frames my thinking on REST, HTTP, HTML, the web, etc.

    ReplyDelete
  20. All very interesting. And - believe it or not - there are days where I think of punting MSFT tech, too. I suppose if I knew Ruby (or some other language with good libraries, advanced syntax, and a garbage collector) I'd be even more likely to want to.

    ReplyDelete
  21. RE: Ruby



    I've looked at that, and I'm not going to play. I think that a lot of its hype is rooted in innocent naivety. That said, if you can't escape The Ball Of Mud, then Ruby is as good as Javascript (which I recently discovered for IIS/ASP) and probably OK for little things.



    I'd be looking at either C# (via Mono) or Java, I think. I wouldn't rule out C/C++ either, although I'd probably prefer to avoid those (I keep gettin' my ones and zeros mixed up!).



    I haven't been paying a lot of attention yet, but I think Sun opening up Solaris and Java is a big deal. Some people I know who have been a lot closer to Sun over the course of their career than I have say that it's "too little, too late", but my feeling is that they could spring back on the map in a big way. I think a lot of Java's problems are rooted in the fact that people were still coming to terms with OO, and most significantly they didn't heed the advice to "prefer composition over inheritance". Also Java made a few mistakes that were fixed up in C# (Value types being a big deal, I think).



    Some days I flirt with the idea of just writing my own programming language. :P (Spolsky did it!) I guess what I'd like to build is an end-to-end "information technology" platform for business. It'd be consultant-ware, I guess. I'd be pretty happy to deal MS out of the picture entirely. There were a few years there where I thought they were making a genuine attempt to be "better", but now I think they were just making a genuine attempt to "look" better (where's my XHTML, puppy!?). Their platforms are a shoddy and constantly moving target, and it costs me too much to try and keep up with them with insifficient progress and not enough control (and ultimately they turn me in to a commodity, and I don't care to help them do that).



    I'm still surveying the territory, really (and I'm not in a huge hurry either). ...but Ruby's not it for me.



    ReplyDelete
  22. s/insifficient/insufficient



    ...I had to watch that one tease me for a few seconds, because I spotted it immediatley as I clicked the 'Submit' button. :P

    ReplyDelete
  23. I agree that there's a lot of hype around Ruby, but I need to build (several) something(s) real with it before I can make up my mind. Certainly my first impression is "the love child of LISP and BASIC". Which is actually fairly appealing to me if it pans out.



    Of course, C# 3.0 is looking really sweet. Java would just seem like such a step backwards - I'd go C#/Mono before I went Java.



    I think I need to get the wife to support me before I can make the leap, though. :)

    ReplyDelete
  24. > I think I need to get the wife to support me before I can make the leap, though. <



    ...my contingency plan is still a kung-fu monastry in Thailand.



    Either that, or I'm buying a bong and a surf board.



    :)

    ReplyDelete
  25. I was thinking more about my AJAX complaint above, i.e. the concern that a URI doesn't point at a single state, and I was thinking about images in HTML content. If I have a URL,



    http://example.org/index.html



    And it has an <img> tag for,



    http://example.org/image.png



    Then isn't what the user sees a 'composite' state, anyway?



    As an aside, I've heard that it's more 'pure' to use <object> rather than <img>... and I've also heard complaints about <iframe>. I have trouble reconciling that. On the one hand it's considered better to use a more generic <object> tag to embed something specifying the mimetype, and on the other hand it's considered 'wrong' to embed an object of type "text/html", which is basically what <iframe> does... I should really go look this up, because I don't remember, but I wonder if it's actually possible to use <object> instead of <iframe> to embed HTML content in a HTML page..? In any case, I don't quite understand what the problem with <iframe> is supposed to be...

    ReplyDelete
  26. The <object> element is a generic wrapper for <iframe> [1]. Perhaps that's the only reason people complain about iframe? Or perhaps there are some browser implementation bugs with <object> as iframe.



    I was thinking about the difference between embedding linked content and embedding script which loads content, and (tenuously) the only difference seems to be that a program can accept user input before loading content, whereas a link will (sort of) deterministically load content ASAP.



    So HTML itself is kind of like a "program within a program" then.



    [1] http://www.w3.org/TR/html401/struct/objects.html



    ReplyDelete
  27. Few personal thoughts here, the concern on "AJAX" not having a state URI is not worth it. "AJAX" approach wasn't revolutionary either, but heck I have been trying to circumvent HTML, JS and IIS and NT for far too long.



    Sure it works, until you pile some real load at it and see how the entire show scales rather poorly, on UI and on 'buy more boxes' preaching (which again I have not seen justified anywhere to date with Java or NET or big vendors; and I did build for it).



    The state URI is not the answer to state- or transfer management if you see my point.



    Browsers are far too limited and heavy in what they do with existing 'standards', I am with you on that completely.





    All hackery for the browser and HTTP is really expensive for the domain I target, although I will admit any day it was such a pleasure working with it. It is always at the back of my mind but I do not need another prop IIS handling or NT either, or REST vagueness.



    I have change quite a few models and environments for a single application over 10 years and started with COM and Web, and the gradients in-between.



    Coming back to it I believe I see that a common core is possible and one that could even target to become a model on its own. The idea was to keep all of the other environments satisfied.



    Linux is certainly always an option, own stacks are too. It is the beauty of Internet, despite HTTP limitations. I am in complete agreement that being 'tied-in' and 'taken a prisoner of bloatware' is not leading to an advance.



    Ruby is a result of "myMethod-ologists" that brought another lock-in, their agenda.



    While I have no problems with a rise of UriTemplate I fear it will lead to another great mess, of typeinfo, of XML overabuse, and of shoe-horning everything under HTTP-induced breakage.



    As great Jim said, 'we do not need frameworks, we need environments'.



    And a stack can be a part of many environments.



    And stacks can have a context that extends far, far beyond a nice but fragile idea, URI.



    This is why I still believe SOAP might have got a wrong, wrong end of the stick and became a misconception. And I am no XML fan (all the high-level rants at http://sixyears.wordpress.com say more).



    There is nothing wrong with playing by your rules and utilising Internet or even Web for any idea you have. Slapping HTTP on top of TCP is dead easy. That is the type of freedom that Web will be challenged on and finally cracks are appearing.



    Frankly, Web had a very good run, one I never expected to last so long or to twist so much (oh well pardon my Lisp/Ruby/SmallTalk):



    Bollox in Space.



    Especially in frameworks, XML and blogging that I prefer to think of my bad grammar more than anything these days.

    ReplyDelete