Roger Session's latest ObjectWatch newsletter relates planning a SOA implementation to thinking about how pilots win dogfights.  The moral of the story - iterative development is a winning idea:

"Boyd [an Air Force Colonel who thought about winning dog fights] teaches us that we win by iterating faster than our enemy, not by building a more expensive plane than our enemy. We will do better by implementing and deploying a small piece of the SOA, getting it right, showing its value, learning from the result, and then iterating on to the next piece."

The benefits of iterative development is an idea that keeps coming back to me from multiple angles.  It was the key idea I took away from a recent 3-day training course on the Microsoft Solutions Framework.  It's one of the key principles from Lean Development and from the XP's idea of iterations.

Well, he may not be blogging (yet) but there is an excellent interview with Pat Helland on TheServerSide.NET.  Pat proposes a solution to the debates about what SOA is and how it is defined.   Pat says we should focus instead on HST (Hooking Shit Together):

"Hooking Shit Together emphasizes the fact that I've got two things, they're separate, they're independent, and it's about: How the heck do I make them work together. That then takes me to messaging, that then takes me to Schema, that then takes me to the contracts and the flows between them while allowing them to be independent. I think there's far more in common in doing that inside an IT shop, and doing it across IT shops, than there are differences. So I actually like HST because it emphasizes what's in common about tying things together across machines, without regard to whether they're in the same company or not."

Pat will always have a special place in my heart because he taught me about Microsoft Transaction Server using an analogy about banana splits on an AVI movie that was part of a Visual Basic 5(?) multimedia (remember when that meant something?) CD-ROM.  The man's a genius.

Michael Platt, a .NET Architect with Microsoft UK, writes a great story about architectural thinking.  He describes being called by a distraught customer who claimed that their n-tier application was failing to scale past 50 users because 'Windows wasn't scalable'.  Without spoiling the story, the post highlights a couple of points for me:

• The need to avoid panicking and remain clear-headed when thinking about problems (e.g. don't invoke magic, witchcraft or the Windows platform as the primary reason for a problem)
• The evils of premature optimisation (though to be fair, this problem seemed to illustrate the evils of optimising without really understanding what's going on)
• The value of understanding software architecture.  Every now and then I devalue what I know about software architecture because it seems like such a well documented area - surely everyone knows about it?  Stories like this remind me of the value of knowledge and experience.

When I was clarifying the Microsoft Messaging Message I said I'd watch the PDC DAT 420 presentation on BizTalk and Indigo.  As my PDC DVDs arrived this week I didn't have to download the presentation I finally got around to watching it.  Here are the key points I took from the presentation:

• From a BizTalk perspective, Indigo is the vehicle that will provide secure, reliable transacted services (over more than just http).
• Indigo is an API, BizTalk is a set of tools.  Indigo will natively integrated into the next version of BizTalk after BizTalk 2004.  The demos showed a prototype Indigo adapter that worked with BizTalk 2004.
• BizTalk has been achieving reliable services through the BizTalk Framework 2.  In future this will be deprecated and replaced with WS-ReliableMessaging, which will be provided by Indigo.
• BizTalk will use the security model built into Indigo to enable WS-Security support for Username tokens and X509 certificates that can be easily configured through attributes, configuration files and policy.

The presentation helped me understand that Indigo is the plumbing.  A lot of business functionality will require the use of tools that take advantage of that plumbing, such as BizTalk.  So I was interested a few days later to see an announcement that the BizTalk Orchestration will ship as part of Indigo in Longhorn.  As the article points out, BizTalk seems to debut features that eventually become part of the plumbing. 

I really enjoyed the presentation, and not just because Scott Woodgate has such a pleasing accent to listen to (New Zealand, so similar to Australian with stranger vowels).   The BizTalk 2004 health and activity monitor combined with the Orchestration debugger were great features.  I can imagine that these would reduce a lot of the pain with administrating and developing complex service-oriented systems.  I'm hoping that I'll get to benefit from these tools on future projects.

Suresh Kumar has posted a newsgroup exchange from Don Box and Scott Woodgate that cover most of the points from the talk on how BizTalk will take advantage of the features provided by Indigo.  Suresh also has some other good posts on general BizTalk 2004 resources, BizTalk and Publish/Subscribe and XML Web Services in BizTalk 2004 including talk of a WSE adapter for BizTalk later this year.   If I could find his RSS feed I'd subscribe.

Feedster does a great job of turning an OPML file at a URL into a custom RSS feed.  I found this tip on Frank Arrigo's list of Australian .NET Bloggers (yes, I still monitor the Old Country).  If you have an OPML file visible at a URL it's possible to use Feedster to search them and then consume the results as an RSS feed.  Very cool.

Being able to consume a number of blogs in one feed certainly makes it easier to track a number of blogs in a news aggregator.

Here's the =2004}&sort=date&ie=UTF-8&limit=30&inopml=http://www.sneath.org/tim/ukbloggers.opml&type=rss">RSS feed for UK .NET bloggers and here's the =2004}&sort=date&ie=UTF-8&limit=30&inopml=http://radio.weblogs.com/0124955/gems/aus-dotnet.opml&type=rss">RSS feed for Australian .NET Bloggers.

Steve Maine writes about his eureka moment with the WS-Addressing spec.  The basic ideas behind it is that it's a good idea to write the address (e.g. put the address in the SOAP headers) on the front of an envelope rather than only writing the name on the envelope and telling the postman the address (e.g. leaving some address information in the SOAP envelope and some on the transport protocol headers).

The other useful aspect of the WS-Addressing spec is the idea of the <wsa:EndpointReference>.  This is an XML node that can contain any information a client or server require.  It's like a cookie mechanism for SOAP messages.  Omri talks more about it and shows how it's useful in WS-Eventing.  Omri refers to the 'resource model' that WS-Addressing provides:

The idea is that the organization of resources behind a Web service is opaque to the caller.  There can be a single resource that "fronts" for a bunch of other resources - which in turn can be organized hierarchically, scaled out, or employ whatever internal organization is appropriate.  The caller does not (in fact should not) need to know about this internal organization. 

I think this use of a front-controller style pattern is another of the ideas that excited Steve.  Don Box also talks this when he says that WS-Addressing is a solution to URI manipulation.  Instead of having to use querystrings to indicate what should be done at the receiving end, this information can be sent through with the EndpointReference.

Tim Sneath is maintaining a list of UK .NET Bloggers.  Hopefully we'll achieve his aim of having another Bloggers' Dinner after the success of last week's efforts.

Update:  Using the power of Feedster, you can track this list with an =2004}&sort=date&ie=UTF-8&limit=30&inopml=http://www.sneath.org/tim/ukbloggers.opml&type=rss">RSS feed for UK .NET bloggers.

Someone who's coming along to my .NET Exchange User Group presentation in Ipswich tomorrow night contacted me about finding a .NET implementation that handles SOAP with Attachments.  Here's some background and a possible solution (let me know if there's a better one).

The Problem
He needs to write a client to consume a web service provided by a finance group that are using Java to publish their messages in the SOAP Messages with Attachments (SwA) specification.   The frustration for his company is that even though Microsoft was a co-author on this specification they provide no implementation that supports SwA.  Omri Gazitts, an Indigo Program Manager has the definitive explanation about why Microsoft has co-authored four SOAP attachments specifications.  Dave Bettin has also been asking good questions about this issue.   Basically the requirements have changed as time progressed.  The main problems with SwA are that it uses MIME which provides no message header length so that the whole message stream must be parsed to find a text boundary between attachments, which is slow, and it doesn't work well with SOAP Headers which makes it hard to integrate with other WS-* specs such as WS-Security. 

Omri has another post that explains the future, which is MTOM that will be backward compatible with SwA.  Unfortunately as Omri says 'we have no plans to support "vanilla" SwA at this time, given that it neither imposes an infoset data model nor is as efficient as DIME - in other words it's the worst of both worlds'.  Where does this leave people like my colleague who need to consume a web service that uses SwA in the wild?

Possible Solution
Well, there is an MSDN magazine article from two years ago that has a lightweight SoapExtension implementation to convert SwA messages into a single SOAP message with the attachments written out as elements inside the SOAP Body (thanks to fellow RD, Christian Weyer for pointing me to this).  It's not too much trouble to convert this SoapExtension to work on the client.  The other part of the problem is an improved MIME parser.  I found the cpShere Email component project on GotDotNet  provides this.   With these two parts I think it's possible to kludge together something that works while we wait for a .NET implementation of MTOM in future.

If anyone has implemented or knows of a better .NET solution please let me know.

This post is technical content-free.  If you're a serious content type then I suggest moving on to another post.  Some of us moved off to the pub afterwards ... 

• There was a straw poll.  Everyone at the bar read The Scobelizer.  Apparently there was an advert in the mens toilet that talked to you as you entered the room.  A surprisingly large number of people believe that Rory would be interested in it.  I wonder if Robert and Rory realise their impact on global culture?
• Ian Griffiths and his co-author Matthew Adams were there.  I hadn't seen these guys together since the PDC.  They are a brilliantly funny pair: think of them as the .NET equivalent to the old guys from The Muppets who throw in comments from the balcony.
• Two guys from Microsoft US who work in the Policy and Strategy group were there.  I asked them what they were working on.  Basically it turned out they'd have to kill me if they told me.  They mentioned cryptic clues about w3c choreography and a public presentation from an early session by a guy with Meridith as a surname.  Apparently it could change the future of computing, work and life as we know it.  I felt like I was in an X-Files episode.
• Chris Anderson spoke about his wife experiencing his Geek Fame.  There was a lot of 'we're not worthy' stuff going on.  Apparently she was only a little impressed.  I'm just glad I never relied on blogger/geek fame to attract or impress women.
• Someone knew who Scott Hanselman was by asking 'isn't he the guy who has a photo of him in his wedding tux on his blog?'.  I've never admitted this, but my photo at the top of this page is from my wedding photos.  A sort of homage to Scott.
• James Clarke was there with a nifty mobile phone and camera, moblogging the whole evening in real time.  He was pretty impressed to meet his first Microsoft Regional Director (I think it was the beer).  Here's his photo of me.

Here's me with Chris Anderson.  Another one for the photo blog roll:

Tonight was my first bloggers dinner and quite a night.  I was the first one to sign up and the last one to arrive as Chris Anderson worked out.  What was most surprising was that other than Chris, Don and Ian 'I have no blog comments because I hand-coded my blog myself' Griffiths, I only met a few other bloggers (Chris has a better list).  A lot of blow-in blog readers!  But all are welcome and everyone's company was appreciated.

By the time I rolled up the only chair left was next to Don himself.  Luckily I'd boned up on things to talk about (John 'Policy' Bristowe also primed with a couple of last questions via instant messenge) and managed to garner the following:

• I started off with asking how do you communicate semantic intent with policy?  Don's response was: you don't.  You do it out of band with the telephone.  It's what humans are for.  There are a lot of people trying to apply formal methods to determine the structural correctness of the messages, but the intent is something that humans will work out.
• WS-Addressing.  I raised my beef about it being all well an good to have an opaque string as the address, but where was the protocol binding.  Apparently Indigo is going to use the simple URL style scheme, so a tcp address will look like tcp://domainname etc.
• The important of the Endpoint references is to provide a SOAP cookie style mechanism without having to get into the URL hackery such as query strings.
• Someone said "it just felt unusual to send text-based messages around - isn't that slow?".  Don's response was that the first time he had sex it felt weird, the next time better and now he couldn't imagine it any other way.
• What's different between the M4 and M5 releases of Indigo (Clemens seems to know a lot about this)?  Apparently the programming model has been refactored toto make it more unified and simpler (great news here).  The System.MessageBus.Servicess library has changed significantly because what was there was just not as good as it could have been.  I hope this means better terminology than DatagramPortTypeAttribute and DialogPortTypeAttribute, which always seemed a bit complex to me.
• What's the best way to approach Indigo today?  If you want to learn about indigo today in a way that will pay of in the future, don't worry too much about the high level details.  If you work from the wire level back up it will be most useful (this is least likely to change). 
• The focus with V1 of Indigo is proving that they can get great performance with messaging.
• We discussed Clemens' idea that because Indigo uses a streaming reader over the message body, it may be possible that on intermediary/endpoint may start sending onto another intermediary/endpoint before the full message is received.  Don mentioned that there's no reason to think that the message may ever finish.  Queue zen-like moment of poignant reflection and silence.  'Sort of like one giant Congo-line of a message throughout the Internet' I said.  Don didn't say it but I felt him think 'Yes, Grashopper'.
• We spoke about the fact that my pregnant wife who has always been worried about the size of my head, thought that Don was the only person I have on my PDC photowall that had a larger head than me.  Don said his head was larger than mine in more ways than one.  We agreed that his head was impressive, but he was leaning forward in the photo and clearly his head was not awesomely large (at least physically).
• I suggested Don try the Cinnamon Club for dinner.  We had the Axxiant Xmas dinner there and it was very good (though it was late and I had been drinking ... ).

Afterwards some of us moved off to the pub ...

January 26 is Australia Day.  It's a special day for me as well.  On this day three years ago I met my wife on a train to the Big Day Out in Sydney.

I'd moved to Sydney from my home town, Adelaide in South Australia to take advantage of the dot-com boom.  It was a pretty simple decision really: move states and triple my income doing the same work with bigger and better companies.  It had also coincided with the end of a five year relationship.  I'd decided to have some quality solo time (re: I discovered alcohol) and do things I enjoyed doing, with the hope that I'd find someone who shared my interests without too much work.

This strategy worked well.  I did the usual life-enhancing things like drawing classes, voice training, dance classes at the Sydney Dance Company studios, bushwalking.  I also did all the things you see in the classifieds 'movies, walks along the beach, good restaurants'.  But it soon became clear that while I was having a good time I wasn't meeting anyone.  When I went bush walking I had some picture of a bunch of Elle McPhersons who loved to hike in the Australian bush.  It turned out to be a 70 year old woman ('Roma by name .. Roma by nature ...') and a couple of 40 year old guys who still lived with their mums.

The prospects at work weren't any better.  Unfortunately there just aren't many females into IT.  I was working at the Sydney Morning Herald, which did have a large marketing department that had the reverse gender balance to the IT department, but I soon worked out that chances were slim.  Whenever a new female did arrive it was like throwing mince meat in shark infested waters.  Sizing up every female colleague as a potential partner just felt predatory.  It was clear I need to re-think my strategy.

After a lot of soul searching I realised that my chances of meeting someone were pretty small as long as I never spoke to anyone I didn't know.  Using my psychology and social science training I hypothesized there was a correlation between the number of women I spoke to and the likelihood of getting a date.  So, it was clear, I just had to approach strangers and try and strike up a conversation.

My greatest fear was making a fool or myself, or worse, approaching a woman who had  a boyfriend and ending up in a punch up (I could clearly visualize this happening).  I began researching how I might locate someone and make an approach.  If women were in pairs then it was a pretty hard ask, women in groups of three were just impossible (three legged chairs are the most stable).  It was clear: It would be easiest to do this if I found a woman on her own.  The challenge was how to do this without being creepy.

I continued going out and enjoying myself.  Eventually I was out dancing at shows by the Sydney Opera House and saw a woman who was on her own.  I struck up a conversation - 'Good DJ hey!' - and had to confront my worst nightmare: a boyfriend.  Strangely, nothing bad happened.  My confidence levels shot up.

The Big Day Out is a travelling rock concert that goes through most Australian capital cities and even a couple of days in New Zealand.  In Sydney it's hosted at Homebush, the Olympics venue, which is serviced by a train from central station.  I had been to quite a few events at the Olympics and had enjoyed the atmosphere on the train, where everyone had started talking to each other.  While I was walking down the platform I noticed this very attractive woman looking out of the door to see when the train left.  'She's more attractive than anyone I'd normally talk to' I thought.  Then the cognitive-behavioural psychology part of my brain started saying things like 'why not talk to her?' and 'which part of turning around and opening your mouth to make sound are you unable to do?'.

So, feeling my heart in my throat, I turned around.  The first question is the key I thought - it has to communicate interest without looking desperate or creepy.  It can't be a 'line' or too cheesy.  So I thought about what I was generally interested in knowing about someone else.  I walked into the empty carriage and sat in the seat across the aisle from her and opened with 'Is this the first time you've been to the Big Day Out?'

I later realised that this was dangerously close to 'Do you come here often?' but luckily we managed to get over that first hurdle, got talking on the way out and ended up agreeing to meet up later in the day 'in front of the sound desk before Coldplay'.   I was pretty excited from that moment on.  Enough that I even sat through a Rammstein performance (I was the only guy laughing at how funny these right-wing German rockers were).  In fact, I was so excited, and I'm so bad at hiding how I feel, that I ended up showing off like a teenager, performing a cartwheel and losing my sunglasses. Tragic, but true.

Anyway, later that year we ended up moving to London together.  I proposed in Paris under the Eiffel tower (another story), we got married last year outside Sydney and are now expecting our first child.  All this from the second stranger I'd ever spoken to.

In preparation for the London Bloggers Dinner with Don Box and Chris Anderson on Monday (see the report of this dinner, and what went on at the pub afterwards), I thought I’d get up to speed and make some notes of the key points made in Don’s interview on the ServerSide.NET.  To my mind this is the best interview I've seen or read about Indigo.

I'm posting these notes mainly so that I can have them handy in SharpReader when I'm offline and in case others might find them useful.  You can see the interview and download the full transcript on the ServerSide.NET site.

What is Indigo? 

• A connective tissue between programs.  It makes it simple for developers to put a message oriented façade on the edge of your application to allow easy communication with others.

Positioning Remoting and ASMX web services

• The problem with .NET remoting was that even though it supported interfaces, it really worked best when both ends of the pipe shared code such as assemblies or DLLs to integrate.  Sharing types like this turns out hard to manage because you often don’t have control over the deployment versioning and testing at each end of the pipe.
• ASMX Web Services (including WSE) are more flexible because they use WSDL and schema which is more adaptable to change than the OO style type interface used in remoting.  ASMX is proving more popular than remoting in the field because it is actually a simpler model and simplicity is a feature.
• Developers should go down the path of integration using messaging because it is simpler and more flexible, but support for remoting won’t go away.
• You don’t have to use XML Schemas and WSDL to describe structures and contracts, but that’s the practical way that everybody does it.
• The problem with components was that they tried to do the right thing separating interface and implementation, but it was too easy to ‘leak’ object concepts and require the sharing of types.

Do components have a place in the world?  

• Don argues that the world is simpler if you think about objects, which are great for programming software that gets integrated, tested and deployed, more or less as an atomic unit and services, which are the central abstraction for deployed, autonomous pieces of software.  Given this perspective he suggests there may not be a place for components.

What’s the difference between service-oriented programming and service oriented architecture?

• SOA is the buzz word du jour, system message bus will probably be next, whereas service-oriented programming is more focussed on the reality of the code.
• Service oriented programming is about four ideas:

1. Boundaries are explicit.  In Indigo you have to place attributes on your code that you want publicly exposed
2. Services are autonomous.  Indigo doesn’t assume you’ll deploy a whole system as a single unit.  Each service should be secure and reliable as a stand-alone unit.
3. Share schema and contract, not class.  It’s too easy to mix up interfaces and implementation, so it’s a better idea to separate schema and contract into two distinct ideas.  Schemas are defined in XML Schema language, and contracts, which are message exchange patterns, are defined in WSDL.
4. Compatibility based on Policy.  To know that services are compatible you have to care about other things than just the signature of the messages.  WS-Policy is supported in Indigo and provides a way to share this over the wire in an interoperable way.

Does Policy impose versioning restrictions?

• Policy is similar to marker interfaces in Java or C#.  You have to have a stable name that is immutable that is decoupled from the actual signatures.  Both parties have to agree on the name and it has very strict version requirements.
• WS-Policy provides a way to declare compatibility with any number of versions of a given policy assertion.  The logic engine is able to determine if party A’s assertion lines up with party B’s assertion.
• In the PDC bits you can see the first message sent on the wire is a ‘get policy’ message.

Can policy be used to create versioning schemas?

• There are three policy specs:

• WS-Policy which is the XML form for writing a logic expression
• WS-PolicyAttachment, which provides a way of taking those expressions and applying them to a particular domain
• WS-PolicyAssertion which contains a small number of concrete assertions that plug into the expressions defined in WS-Policy.

• WS-PolicyAssertion contains an assertion called Spec Version that can be used to determine compatibility at the naming level.  It’s looser than strict or nominal type equivalence that is used in C# and Java.  It provides some flexibility, not complete flexibility and Policy is used to do the version negotiation.

Are there situations where the Indigo approach doesn’t work well?

• In future the service-oriented programming model may be a practical way of doing things like cross-process, cross-app-domain or even within a single exe.
• The focus is making sure Indigo has a solid model that can be implemented and scale down well.  Objects took a ‘near’ metaphor and tried to stretch it out, Indigo is trying to take a ‘far’ metaphor and shrink it.  Indigo is striving to have very low performance costs within a single exe.

What are the performance optimizations being worked on now?

• The team is very focused on make sure that the I/O implementation is as well tuned as possible.  They want to make sure we can get bytes in, in XML and other forms, in to and out of an app domain as fast as humanly possible. 
• They want to make sure that it’s not tied to a specific protocol, such as HTTP.
• The idea is ‘I create a message here, I need it to appear in your program as fast as possible’.
• Performance is an important goal for the first release.

WSE and Indigo

• WSE is the vehicle to give developers access to implementations of the WS-* protocols as quickly as possible.
• The goal is still to integrate them into the .NET platform, but sometimes the platform has to ship before standards are set, which is why WSE will have a future post-Indigo.
• Post-Indigo WSE may be used to track protocol evolution on top of the Indigo infrastructure rather than just the ASMX infrastructure.

Will Indigo do anything new at the wire level?

• Indigo is about the software productization of ideas (such as the WS-* specs) rather than interesting new wire protocols.  For example, transactions on the wire will use WS-AtomicTransactions but within the service they are doing a lot to make transactions easier to use from managed code with extremely efficient implementations.

What's a Microsoft Regional Director?  Fellow RD Jon Box answers this well.  Clemens also has a good post. Basically Microsoft Regional Directors are independent developers and architects, volunteers chosen for their leadership in their local technology circles. You've probably seen many of them at conferences, read their blogs, or heard them on .NET Rocks.

I'm looking forward to continuing to spread the word about .NET, web services and Indigo in the coming year.  As shown on my presentations page, I'm doing a talk on WSE to the VBUG group in Ipswich on Feb 3, then a talk on Indigo in London to the .NET UK user group on Feb 23.