John Udell has written an article on WS Alphabet Soup based on his discussions with John Schewuck, an architect on the Indigo team.  Here are the interesting points:

• Indigo may ship before Longhorn to enable people to provide developers with access to the functionality provided by the newer WS-* specs.  I know the message from the PDC was Whidbey < Indigo <= Longhorn, but I guess haven't really considered it will come before Longhorn (I thought this was what WSE was for since Indigo is a 'baked in' part of the .NET Framework).  Not that I'd complain if it came earlier.
• There are a lot of Web Service (WS-*) specifications, but this is a good thing because they are composable.  You only need to implement the specifications you need.  The alternative would be a single specification that tries to do everything.
• The WS-* specs are inventing a lot of what has been provided before in DCOM and CORBA.  As Don Box says, 'The entire motivation for defining SOAP was to get the industry out of the "you need my runtime to talk to me" business that DCOM and Java RMI so happily got us into in the 1990's.'
• The advantage of using SOAP as a message description format  is that it provides extensible SOAP headers ("an accordion-like construct").  Previous network communications protocols was their dependence on fixed-offset dependencies that broke if you had to extend or modify the packets.
• Microsoft is wise to spend time worrying about the usability of its APIs.  This was my initial reaction at the PDC - 'Wow, they have done a lot to make this really easy for developers'.
• The key to reducing the cognitive overload caused by the specifications is an easy programming model (this acknowledges the reality that not everyone gets as excited as John Bristowe about a new specification).  Indigo's use of attributes in the Service Model is a major plus here.  How awesome is it to be able to write code like this to add WS-Security (signing, encryption), WS-ReliableMessaging and WS-AtomicTransaction support:

[Service]
[Security(Disabled=false, 
 SecurityProfileName="MyDynamicProfile",        
 ClientAuthentication=UsageMode.Required,
 IntegritySupport=UsageMode.Required,
 ConfidentialitySupport=UsageMode.Required)]
[ContextPropagation(typeof(ITransaction)]
[Connection(DeliveryAssurances.Full)]
public class MySecureReliableTransactedService
{
    [ServiceMethod]
    [Authorization(Role="Clerk")] 
    public void CreditAccount(string acc, decimal amount)
    {
        …
    }
}

The Loosely Coupled blog also has more commentary on the interview with John.

Matthew Adams left a comment on my last post disagree with my characterisation about services being more to do with messages as opposed to remote procedure calls (RPC).   It was a good question that I thought was worthy of a lengthier response.

What I was suggesting in the original post is that overtime there will continue to be a move towards services that use messages, and in particular SOAP messages based on the XML Infoset, rather than RPC style calls over time.  SOAP because it gives a broader reach for interoperability because of it's lowest common denominator approach and associated standards.  Messaging because it gives more flexibility in terms of what is sent and the message exchange patterns that are supported. 

The Great SOA Definition Debate
I've been trying to get away for getting into a debate about what services are, mainly because I haven't found a useful definition of service that says more than 'it's a great idea to have blocks of code that do something' and when designing these blocks of code, encapsulation and loose coupling are good practices.

By Matthew's definition a service is something that can do something, at a particular addressable location on a particular protocol, with a particular message format, in a such a way as the consumer doesn't have to care about how that something is done.

Using this definition of a 'service' it is true that the service doesn't have to be implemented as a 'message' end-point.  It is possible to implement these 'services' in ASMX web services, in-process .NET classes and remotable objects.  The problem is that these choices can impact the amount that the consumer has to know about the way the service is called, if not how it is implemented.

Don's Definition of Services
I like Don Box's definition of a service as a 'program that you communicate with using messages'.  These messages are SOAP messages based on the XML Infoset (a logical view of an XML document - angle brackets and text representations may or may not be involved).

The beauty of using SOAP messaging is that it is a lowest-common denominator of solving the problems involved with providing a service:

• How do we address the service, which protocol should we use to talk to it? (WS-Addressing)
• How do we describe the format of the messages that are sent and received? (SOAP)
• How do we describe the format of the contents of these messages (WSDL, XML Schema)
• How do we describe the pattern of the messages that are exchanged? (WSDL)
• How do we describe the other characteristics we need to successfully talk (WS-Policy)

What's wrong with RPC end points?
My main issue with RPC end points is interoperability and flexibily.  RPC-style calls to RPC-end points in Matthew's terminology calls can do all of these things, but they are more likely to do it in a less-interoperable way that requires greater knowledge of the underlying platform, protocol or programming model (for example you may need to agree on the binary representation of the parameters used in the procedure call which may mean sharing DLL's or worse, an entire development platform).

Of course any remote procedure call can be thought of as just a particular kind of message.  So it's possible to create services that are based around RPC-style calls made within a message (which is how ASMX web services work today).  Within web services there are two types of SOAP message format - RPC and document.  As Yasser Shohoud points out in his article RPC/Literal and Freedom of Choice, RPC formatted SOAP messages are just a subset of the document formatted (I'd argue message-based) SOAP messages .  As he says, ASMX web services show it's possible to use a document style message format but with an RPC programming model.

[Update: See Shawn Smith's post on this topic.  In the last section of this post I wasn't very clear about the distinction between SOAP message formats and message exchange patterns.  My point was that it is a better idea to implement more flexible approaches to both the message format and the message exchange patterns because the more flexible approach can accommodate the more rigid approach.  In this sense the SOAP Document format can accommodate the SOAP RPC message format and the asynchronous message exchange patterns can accommodate RPC message exchange patterns (synchronous request/reply).

Ultimately, it's not worth getting too fussed up about this as both the ASMX and Indigo programming model will support you whichever way you'd like to work.  You can write services based around asynchronous messages or using synchronous request/response (stop-and-wait) RPC-style calls.  As I said at the start of this post, over time I believe there will be a movement away from synchronous RPC calls towards asynchronous messaging.]

As preparation for my Indigo presentation at the London .NET User Group next Monday, I thought I'd write a couple of posts about Indigo, starting with 'The Road To Indigo'.

Indigo is a new technology that represents on the unification of existing distributed application technology.  It takes Remoting, Enterprise Services, Web Services and Microsoft Message Queue and combines them into a single area of platform-level technology.  In fact, Microsoft has literally taken all of the teams responsible for these technologies and put them in the same building.  Instead of thinking of Indigo as a completely new technology, think of it as the next version of all the existing approaches where all of the best features of each are combined into a single managed-code programming model.  Here's a diagram showing the progression of these technologies:

 

Indigo takes the best features of each approach and makes them available in one location.  If you're developing ASMX and Web Services today you can't  take advantage of reliable messaging (as in MSMQ) or transactions and object pooling (as in Enterprise Services) across web services calls.  With Indigo it is possible.  If you're an Enterprise Services developer today it's difficult to take advantage of flexible versioning provided by schema or being able to describe requirements using policy files (as you can in web services with WSE).  Indigo makes it possible to take advantage of all of these features.

Indigo also lets you chose the way you want to build applications without tying you in.  Indigo supports and encourages building service-oriented applications using messages based on SOAP.  However, Indigo still supports you and provides features even if you want to create distributed applications using a distributed component approach using remote procedure calls.  Choosing one approach over the other doesn't mean giving up features such as message-level encryption or transactions.  As Ingo Rammer says, Indigo recognises that you shouldn't be locked into 'a specific programming model, a serialization model, a transport model, or even just a serialization format'.

Combining the features of these four different technology areas into a single programming model is great news for developers.  We get the best features from each previous approach while only having to learn one programming model!  There's no more complexity about which technology or programming model to use for a particular distributed application scenario. It's the same programming model whether it's a peer-to-peer application on a hand-held device, right up to a secure, reliable and transacted message queue behind a massive online retailers.  

For those times when the platform needs to be extended, Indigo also provides an excellent extensibility model.  Even in this case there's only one extensibility model to learn.

Indigo brings us closer to Microsoft's long-term aim of helping developers spend less time on the plumbing and more time on the application code.

Two pieces of news have made this Valentine's Day a geek day of love.

• JetBrains, the creators of IntelliJ have released an ealy build of their C# tool, ReSharper which I have been waiting for.   Luke Hutteman has a good review of the features (and info on how to access the download site, which currently isn't handling new registrations).
• Erik Sink announces the release of Vault 2.0 which includes a free Single User Edition.

Bruce Williams from the Indigo team has a nice walk through of the the XML elements in WS-Eventing.  When you strip away all of the namespaces and XML noise, it's a nice, understandable piece of work.

I like WS-Eventing because it's another tool in the architecture toolbox.  To me it suggests the end of polling in web service applications.  In some of the applications I've worked on today the client has had to poll the webservice to check for updates.  With eventing the client will be able to wait to be told if something has changed, rather than pestering the server like a kid in the back of a car on a long road trip asking 'are we there yet?' all of the time.   It's much more efficient to wait and be notified than constantly having to poll to check if something has changed.

I'm not 100% sure about how large corporates will handle security will work across the Internet and corporate firewalls.  For the client to receive the notification they have to have an open port that the notification service can send to.  Perhaps a solution is to use an approach like the MSDN SoapMail sample, where messages were returned to some central location, like a mailbox, which is polled by another service that picks the message up and forwards it to the client.  Perhaps the answer lies in the Security Considerations area of the spec which suggests the notification should be signed with security tokens.

Some random notes from last night's Tuesday Extreme Club:

• Jamie Cansdale, an all-round very clever guy, came along for the first time and got a chance to receive thanks from many of the people that had used his NUnit Addin. We had a great chat about Mono, Rotor and the strengths of .NET being a formal standard as well as having a share-source implementation.
• Joe Walnes from ThoughtWorks about message-oriented development, the decorator design pattern, how Aspect Oriented Programming could be thought of as generic decorators.  He also spoke of his joy after achieving his life aim of becoming the owner of a blow-up a bouncy castle (apparently only £250 including generator)
• Joe also mentioned Prevayler that basically removes the need for a data store with projects that use less than 2GB.  The simple idea is that the entire memory space can be persisted to disk.  Each action performed on the model uses the Command pattern to write out a transaction log that can be replayed if the system goes down between persistent writes.  This allows for easy replication of systems between machines as well as meaning that programs don't need a database.  A very simple, but potentially powerful idea.  For small applications, do I really need to struggle with a database, or could I manage it with in-memory objects?
• Tim McKinnon told the story of the invention of Mock Objects. The Pragmatic Programmers have a useful list of 7 reasons why you would want to use Mock Objects. For Steve, the most important was being able to test relationships between objects.
• Tim gave a great overview of the problems with Visual Studio .NET.  Basically it takes to long to locate the right piece of code. His question was why can't use hyperlinks to navigate around the code (Reflector does this brilliantly) or even a Find Type dialogue box that allows you to type in the name of the type you are looking for with IntelliSense.
• Chris Stevenson was at his last meeting before doing the 'reverse-outsource' and moving to India to join the ThoughtWorks team there.  Can't wait to read about it in his blog.

I spent this afternoon watching Scott Guthrie present ASP.NET and Visual Studio .NET Whidbey to a packed audience out at Microsoft UK's Campus at Thames Valley Park.   ASP.NET 2.0 does so much out of the box that it's almost threatening to a developer - It's been a long time since I was this scared about a technology!   See my complete notes of the presentation slides and demo here.

When's ASP.NET Whidbey going to be available?
The aim is to ship a Beta in June and for RTM in Q1 2005.   There will be a 'go live license' that will let people put Beta 2 into production, later this year.  The Beta in June will be a public beta and it will be possible to download ASP.NET and Visual Studio Whidbey.  The build that Scott demonstrated with was from last Monday when he said they achieved 'Code Complete.  So they are now in that long stabilisation phase. 

Cooler applications in less time (even for enterprise applications!)
The key message is that ASP.NET allows you to create cooler applications in a fraction of the time it takes today. This sounds very 'marketese', but based on what I saw demoed it's very true. The hardest thing is that there have been plenty of demos like this before - from FrontPage and Visual InterDev. It used to be the case that you'd walk away saying 'good demo, but it will never work on a serious enterprise site'. But this time they've got the architecture right. Databinding to plain objects makes it possible to do proper 3-tier development. The Provider model means that there are interfaces at nearly every level that you can implement or override (apparently they'll ship the source code to the existing providers to make this easier).

It will be interesting to see how ASP.NET 2.0 works in real world situations.  Many of the features demonstrated like the login and authorisation and the paging and sorting of data are things that I've developed in the past so I know that they can literally take weeks to develop.  Now they are simple drop-and-drag and configure scenarios.  Also, since many of these ideas, like the data grid are now on their third or fourth iteration (I remember Visual InterDev 6.0 and it's 1.0 style implementation - urgh!) they have had time to develop to the point where they (look like) they are really going to work in the wild.

What a demo!
Out of the box Scott built a site with full security (login control, password reset and email page), customisable content using Web Parts, master-detail data pages (with paging and sorting enabled with simple checkboxes) using data binding in both 2 tier (direct to the db) or 3 tier scenarios (just plain public methods returning IEnumerable - no need for an interface or attributes!), easy internationalisation, support for navigation files (including treeviews and bread crumb trails) and Master Pages with Skins for easy site-wide look and feel manipulation. It truly does raise the bar about what will be expected as minimum website functionality in future.

The Full Notes
To save clogging up the aggregators and my front page, I've put my 3,500 word 'summary' of the slides and presentations here.

In summary I'd say that Scott is a great presenter.  He did a morning session on the basics, then the session I attended.  He spoke from 2pm to past 5pm and hung around until we were thrown out of the building to ensure that everyone's questions were answered.  It's obvious that he's pumped about the product and the platform that he's been instrumental in creating for developing web applications.

Roger Session's latest ObjectWatch newsletter relates planning a SOA implementation to thinking about how pilots win dogfights.  The moral of the story - iterative development is a winning idea:

"Boyd [an Air Force Colonel who thought about winning dog fights] teaches us that we win by iterating faster than our enemy, not by building a more expensive plane than our enemy. We will do better by implementing and deploying a small piece of the SOA, getting it right, showing its value, learning from the result, and then iterating on to the next piece."

The benefits of iterative development is an idea that keeps coming back to me from multiple angles.  It was the key idea I took away from a recent 3-day training course on the Microsoft Solutions Framework.  It's one of the key principles from Lean Development and from the XP's idea of iterations.

Well, he may not be blogging (yet) but there is an excellent interview with Pat Helland on TheServerSide.NET.  Pat proposes a solution to the debates about what SOA is and how it is defined.   Pat says we should focus instead on HST (Hooking Shit Together):

"Hooking Shit Together emphasizes the fact that I've got two things, they're separate, they're independent, and it's about: How the heck do I make them work together. That then takes me to messaging, that then takes me to Schema, that then takes me to the contracts and the flows between them while allowing them to be independent. I think there's far more in common in doing that inside an IT shop, and doing it across IT shops, than there are differences. So I actually like HST because it emphasizes what's in common about tying things together across machines, without regard to whether they're in the same company or not."

Pat will always have a special place in my heart because he taught me about Microsoft Transaction Server using an analogy about banana splits on an AVI movie that was part of a Visual Basic 5(?) multimedia (remember when that meant something?) CD-ROM.  The man's a genius.

Michael Platt, a .NET Architect with Microsoft UK, writes a great story about architectural thinking.  He describes being called by a distraught customer who claimed that their n-tier application was failing to scale past 50 users because 'Windows wasn't scalable'.  Without spoiling the story, the post highlights a couple of points for me:

• The need to avoid panicking and remain clear-headed when thinking about problems (e.g. don't invoke magic, witchcraft or the Windows platform as the primary reason for a problem)
• The evils of premature optimisation (though to be fair, this problem seemed to illustrate the evils of optimising without really understanding what's going on)
• The value of understanding software architecture.  Every now and then I devalue what I know about software architecture because it seems like such a well documented area - surely everyone knows about it?  Stories like this remind me of the value of knowledge and experience.

When I was clarifying the Microsoft Messaging Message I said I'd watch the PDC DAT 420 presentation on BizTalk and Indigo.  As my PDC DVDs arrived this week I didn't have to download the presentation I finally got around to watching it.  Here are the key points I took from the presentation:

• From a BizTalk perspective, Indigo is the vehicle that will provide secure, reliable transacted services (over more than just http).
• Indigo is an API, BizTalk is a set of tools.  Indigo will natively integrated into the next version of BizTalk after BizTalk 2004.  The demos showed a prototype Indigo adapter that worked with BizTalk 2004.
• BizTalk has been achieving reliable services through the BizTalk Framework 2.  In future this will be deprecated and replaced with WS-ReliableMessaging, which will be provided by Indigo.
• BizTalk will use the security model built into Indigo to enable WS-Security support for Username tokens and X509 certificates that can be easily configured through attributes, configuration files and policy.

The presentation helped me understand that Indigo is the plumbing.  A lot of business functionality will require the use of tools that take advantage of that plumbing, such as BizTalk.  So I was interested a few days later to see an announcement that the BizTalk Orchestration will ship as part of Indigo in Longhorn.  As the article points out, BizTalk seems to debut features that eventually become part of the plumbing. 

I really enjoyed the presentation, and not just because Scott Woodgate has such a pleasing accent to listen to (New Zealand, so similar to Australian with stranger vowels).   The BizTalk 2004 health and activity monitor combined with the Orchestration debugger were great features.  I can imagine that these would reduce a lot of the pain with administrating and developing complex service-oriented systems.  I'm hoping that I'll get to benefit from these tools on future projects.

Suresh Kumar has posted a newsgroup exchange from Don Box and Scott Woodgate that cover most of the points from the talk on how BizTalk will take advantage of the features provided by Indigo.  Suresh also has some other good posts on general BizTalk 2004 resources, BizTalk and Publish/Subscribe and XML Web Services in BizTalk 2004 including talk of a WSE adapter for BizTalk later this year.   If I could find his RSS feed I'd subscribe.

Feedster does a great job of turning an OPML file at a URL into a custom RSS feed.  I found this tip on Frank Arrigo's list of Australian .NET Bloggers (yes, I still monitor the Old Country).  If you have an OPML file visible at a URL it's possible to use Feedster to search them and then consume the results as an RSS feed.  Very cool.

Being able to consume a number of blogs in one feed certainly makes it easier to track a number of blogs in a news aggregator.

Here's the =2004}&sort=date&ie=UTF-8&limit=30&inopml=http://www.sneath.org/tim/ukbloggers.opml&type=rss">RSS feed for UK .NET bloggers and here's the =2004}&sort=date&ie=UTF-8&limit=30&inopml=http://radio.weblogs.com/0124955/gems/aus-dotnet.opml&type=rss">RSS feed for Australian .NET Bloggers.