I'm attending OT2004 conference (Michael Platt is also here and has blogged day 1) and had my first chance to hear Martin Fowler talk.  He did a keynote on MVC Patterns and the role of an Architect in an XP Team.

What is the MVC?

• Model - maintains state
• View - observes state and projects it to the user
• Controller - poked by user events and sends them to the model, which does some domain interpretation

The key idea is to avoid having domain logic inside the UI.  MVC is hardly ever done in the classic way.  There are two ideas inside MVC, one which is popular and one that is unpopular.

Separate the model from the view and the controller
The popular concept is to separate the model from the view and the controller.  The advantage of this is that things are layered, decoupled and separated.  The domain doesn't know anything about the presentation.  This is a Good Habit.

Separate the view and the controller
The less popular concept is to separate the view and the controller.  The original idea was that you could swap our or change the controller.  So if you had a read0only screen you could swap out the controller to ignore user 'pokes'.  The problem is that user controls often combine the view and the control.  A widget can receive input and display the results.  This second concept is often misunderstood and is one of the reasons for tangled descriptions about MVC. 

How MVC was messed up
One reason was the idea of Interface, Entities and Controllers.  The idea is that an interface represented the UI, the Entities represented the domain objects and the Controller was an intermediary that glued the entities to the interface.  The problem with this approach is that the Entities ended up just being structs and all of the behaviour was in the controller which was essentially procedural coding.

A lot of languages are out there but there is a shortage of OO code.  Martin wrote about this in the anaemic domain model - the domain model is just a thin wrapper around data.

Some people have solved this with a service layer.  The idea is that some of the logic associated with talking with other systems that is not wanted in the domain model.  The question is how to mix them.  Martin's view was that it was OK to have a thin service layer, but the key was to avoid having nothing in the domain model.

MVC variations
Based on talking with TW and others, Martin believes that there are two variations to the MVC model.  These are the presentation model and the model view presenter.  The drivers of these variations is testing.  This is another situation where testability has some good effects.

All too often there are two tier architectures where everything is in the view.  Testing in this approach is about driving the application through the UI.  This is too hard to keep updating, the test are fragile and it never works in practice.

So these variations are responding to the need to test without having to rely on the user interface.  One approach is to keep the view so stupid that you don't have to test it.  Another is to put the logic somewhere else.

Presentation Model
One key problem with the MVC model is that the view couldn't be a simple projection of the domain model.  Often you need to alter the data for the user interface.  Sometimes this is trivial, sometimes it isn't.  It's also hard to mix UI widgets with these adaptations.  Also, there is a need to store state that is about the UI and not the model (e.g. is this control disabled, is the text visible?).  Where should you put this in the MVC model?  The change is to have a presenter that is responsible for adapting the domain model for presentation and holds the view state.  It's also responsible for the synchronization with the state of the widget.  In the Presentation Model approach you can ignore the view and test the presentation which works as long as the synchronisation between the presentation and view is so simple.

The MVP Model
The twist in this approach is that the controller can't respond to the user's pokes.  The user pokes the view and the view pokes the controller (named the presenter in this model).  The Presenter tells the widgets to updated and then updates the underlying model   In the MVP model you can use Mock Presenters and test the messages are coming in order.

How do we group as a profession?  How can we improve what we do?
Martin's view is that we should have reflectors who wander around project trying to push our knowledge and practices along.  A lot like Martin really.

Why does XP have no technical lead?
This is where Martin spoke about his observation of David Rice at ThoughWorks and the way he worked as an architect.  He believes there are two views based on how you view the relationship between programming and design.  See his article Who Needs an Architect for more information.

Architects as Controllers
One view is that architects are the ones who come up with what has to be done and the programmers simply implement it.

Architects as 'Guides'
Architects in Martin's view should be someone who is still involved with a project but should be wandering around the team seeing things, making sure that problems are addressed.  Architects should be reactive to problems, and teacher-like helping to mentor the students.

The key for Martin was the architects should stay close to development, lest they become part of a centralized architecture team responsible for process and policy and really haven't developed in a long time.

Relation between XP Architects and User Interaction Architects
Martin believes that web UIs are popular because they are simple to put together and cheap.  Martin's beef with people like Alan Cooper and Interaction Designers are that they don't factor in the cost of designs.  Good Uis are expensive.  Martin handles this in XP by making a simple UI a user story then telling the customer to create another story for a better UI.

What's the link between the MVC and Architect topics?
Reflection.  They key is that Architects must be able to reflect on experience, which is hard.  They should look for what it is being done, decided what is good and bad about it and then communicate that knowledge to a wider audience (sounds a lot like an author doesn't it?)

Martin is a reflector on other reflectors, a sort of meta-reflector.  He mentioned that reflecting on experience is key.  It's the difference between a 9 to 5 job and someone who wants to get the best out of what they do.  Martin left us with the message that 'I have fun, you should look for opportunities to do that too'.

I attended the UK launch of BizTalk 2004 today.  Nearly everyone else there was in suit and tie, highlighting that BizTalk is a business focused product.  I like BizTalk and the 2004 product is a great improvement on previous versions (Darrel Norton has a great list of BizTalk resources).  Messaging and integration are two areas where I think IT can add a lot of business value.  BizTalk is making technology that had previously been very expensive available to greater number of customers and developers.  As Cameron Reilly notes, one CIO he knows says he could buy BizTalk 'out of the stationery budget'.

There were some interesting customer presentations including how Virgin Megastore were using BizTalk to spot employee theft by analyzing real time point of sale information (case study here).  Loss Prevention Agents (I'm thinking Agent Smith from The Matrix) are notified on their mobile devices as business rules are triggered. 

My favourite customer moment was the CTO Scottish and Southern Energy who revealed their unofficial mission statement was "boring but successful".  Brilliant.

Mehran was there with his assortment of Microsoft OS gadgets and even gave me a ride back to London in his very nice new BMW with Windows Automotive.   This is serious geek gadgetry.  There's 'Toy Boys' like Carl Franklin sings about and there's guys like Mehran who go out and buy the car.

Don Box points out the Death of Hypercard.  I loved HyperCard.  I lost my 'programming virginity' to it when I used it to build a custom web browser and hyperlink editor as part of my Psychology Honours Thesis.  It will always have a special place in my heart as my 'first time'. 

Sure, I'd done some great work in primary school with the Logo turtle, and spent a long time learning Basic programming on the Commodore 64 (annoying my brothers who wanted to watch the TV) but I'd never 'released' anything for other people to use.  Here are some things I learnt (more about the thesis at the end).

What I learnt from my 'first time'
Here are some learning experiences that stuck in my mind:

• Estimating development time is difficult.  I was incredibly optimistic and ended up being late and having to cancel the first day of the experiment.
• Coding through the night is a waste of time.  In desperation I had stayed up programming through the night to get it finished.  After a sleep the next day I realised that everything I did post midnight was useless and had to delete it.
• Pair programming is a great way to learn.  After pulling the all-nighter my brain was fried but the program wasn't finished.  Luckily a friendly network admin in the psych department paired with me and helped me solve a complex problem.
• Milestones and frequent, iterative releases are a Good Thing.  I tried to do it all in one go and make it 'perfect' rather than focussing on getting it working end-to-end and gradually adding features.
• Usability testing is important.  Although I showed the program to friends, I didn't actually get anyone to use the software before I started the experiment.  Having to watch 200 people struggle with my poor interface was an important lesson in the importance of doing some discount usability before spending time writing code.  Ironically enough, one of the main authors I referenced in the thesis was Jacob Nielsen (in his academic phase, before the crazy days of the dot-com).
• Books are a great way to learn.  I love books as a way of learning something (I even taught myself to juggle from a book).  I used Danny Goodman's The Complete Hypercard Handbook (here's his view on the death of Hypercard) to teach myself how to implement my solution.  I got everything I needed from there (even implementing my own scrollbar with user-drawn arrow heads and mouse tracking).  Later I got his JavaScript Bible, which was the most expensive book I'd ever bough at AUS$70 but translated into a AUS$2000 contract to build a multiple choice system for the uni.  I couldn't believe the financial leverage!  Recently I've dampened my love of books slightly in favour of learning by 'just writing the code'.

What was my thesis about?
My thesis was looking at whether hyperlinks could be useful in helping people learn (the thesis is now available to the public at my uni library).  I built a set of pages describing key topics in first year psychology that the first year students could use to revise for their exam.  In my study there were three groups of students - those who browsed a set of hypertext pages (we'd say web pages today) which had links authored by the Lecturer, and two other groups that had to link the topics themselves.  One group had to choose from a list of associations to describe the connections between two topics and the other group had to type a description of a relationship themselves.  My hypothesis was that the groups that created the hyperlinks would do better at remembering the material (and get a better exam grade) than those who just passively read the material.  In the end there were no significant relationships (ever the way in honours topics), but it was a great excuse for me to learn how to program.

Another great night at the Extreme Tuesday Club where I met Chris Matts, a Business Analyst (or Agile Business Coach) who works for ThoughtWorks.  It was an interesting discussion about the role of an analyst in a XP/Agile project.  Chris views his role as helping the business customer and the development team get closer together and helping the developers learn more about the customers' domain problem.  This kind of thinking about software development practices is a key reason why I like the XP/Agile movements.

I'd heard about the great work Chris had done on a large project that was going off the rails.  By tracking close to the business issues he was able to influence the project being cancelled and restarted.  We spoke about how to influence projects when it's obvious they aren't going to be successful, the complexity that can be involved when the contract work depends on the project continuing and the personal integrity involved in sticking with a decision you believe in.

I was interested to find out more about how a Business Analyst fits into such an Agile-oriented work place.  To me one of the big gaps in the Extreme Programming methodology is the generic concept of 'The Customer'.  It's a great thing that a lightweight methodology exists that is pushing the importance of customer involvement, but to me XP is thin on the details of what's involved.  In XP Refactored the authors point out that the notion of the onsite customer is a difficult idea to achieve.  Most work places wont allow their best people to leave the workplace in order to sit alongside the development team.  Even if they did they aren't always able to effectively communicate with the team.

Chris' concept was that he understood the technology and the business domain knowledge.  He believed that one of his benefits was to bring the customer and the development team closer together.  He was more effective than just a customer because he was skilled in learning about the business problems and analysing the requirements for a solution. He believed he had a role to play in an Agile team by viewing his role as the Business Coach focussed on transferring his business knowledge through to the development team.

One of the techniques that Chris mentioned as being effective was having the customer draw the business model on a blank sheet of paper at each meeting.  They found this produced much better customer feedback than they received from the customer reviewing their diagrams.  Drawing and talking about the business model produced a greater involvement than someone simply reading and reviewing a document.

Chris had extended this idea further with the concept of producing documents as a way of learning more about the business himself, but not necessarily publishing or sharing those documents with the team.  A very 'Agile' perspective that says the value of the analysis is the impact it can have on the rest of the team.

Quite an interesting discussion developed around the concept of 'How People Learn' or how a Business Analyst could help the team learn about the business.  I take the 'constructivist' perspective which says people learn through constructing their own view of the world.  The key to learning is to engage people in thinking in order to develop more sophisticated connections between concepts.  Any activities that encourage people to actively think about topics and relationships between topics are good.   Drawing a diagram and talking about it are much more likely to engage someone in thinking than passively reading or reviewing a document.

Optimizing is a complex problem.  Deciding what to do about our coming baby's nappies (US translation: diapers) highlighted the need to understand what is being optimised and what the trade-offs are.

Currently my wife and I have been discussing what to do about nappies for our coming baby.  I'm still dealing with this at a numerical rather than the emotional level as I'm still dealing with the sock that there may be over 5000 nappies in the first few years of a child's life.  While disposable nappies have the attraction of convenience it just feels bad to throw that many things out.  There's lots of evidence that disposable nappies represent around 2% of landfill and are generally evil as this clearly biased site  demonstrates.

I spoke about this problem with a good friend of mine who studied Computer Science.  He was a father himself and didn't like the idea of not using disposables because of landfill.  He came back to me last week with the following comment:

"While it is true that nappies do contribute to a small amount of landfill, there are great contributors to landfill such as kitchen waste.  As a Computer Scientist if I was trying to optimise something I'd start with the area that I could have most impact"

I liked my friend's thinking and told it to my wife, who said "yes, but choosing a cloth nappy washing service is a simple decision that is easier to implement than having to think about all of our waste".

I'm not sure which approach is "right" (we'll be using a cloth service for the first month) but to me it highlights that optimization is a complex problem that involves understanding what to optimize and what the trade-offs are.

While it is easy to see the benefits in the idea that services expose explicit boundaries, there is still some distance between this and understanding where service boundaries should be drawn and what it should look like.  A collection or recent posts from three Microsoft Architects highlight that services work best at the business level where they can share the same logical data model and single database engine.  It's nice to see the SOA discussion break through the academic debate to this practical guidance level.

Ramkumar Kothandaraman, a Solutions Architect on the .NET Enterprise Architecture Team, talks about  implementing the service-oriented tenet that services should have a clearly defined boundary when there are issues about searching or updating the shared data.  He proposes that it can be beneficial to share a database between related services and to expose these services as a collection or business services that share the same logical data model, rather than individual services. 

I believe this is similar to Pat Helland's session at the PDC where he spoke about business services encapsulating database-resident data and that each business services' data should live in a single database engine.  If the data can't fit into a single engine then look for ways to split it into two business services but don't have a service span database engines.  If the service needs to scale then consider partitioning the data between individual database servers rather than trying to merge and replicate.

Michael Platt, a UK-based Field Architect from Microsoft, shares the view that services need to be exposed at the business level.  He provides a useful, pragmatic definition of service oriented architecture as a model or abstraction of loosely coupled business-level functionalities or services.  I also enjoyed his reflections on a recent meeting discussing SOA:

There was a lot of discussion about what SOA was: explicit boundary’s, autonomous, etc which reminded me a lot of the COM definitions but with different words. Alas there was no discussion as to what a Service should look like, how big, what level of functionality etc; this was left as an exercise for the designer.

In terms of guidance on how to architect services, he mentions a story of the good old 'component oriented architecture' days where a client had 3000 components in a system, as a warning not to go overboard again.  Michael also invokes his 'First Law' as design guidance for a services which essentially says that that services should be coarse grained because of the huge communication and network stack they have to go through.

It can be difficult to talk to groups that are hostile about Microsoft technology because their beliefs are often based on sweeping generalizations or bigoted opinions.  Although Microsoft isn't perfect and doesn't always get things right it is surprisingly good at improving its products over releases and for engaging with the community to get feedback.

John Bristowe writes about his experiences attending a Linux User Group and talking to people about Microsoft and Windows:

What's funny is when you approach these people to challenge their stereotypes about Windows, they incite some form of moral/religion conviction against Microsoft and/or refuse to discuss the issue any further. I try to tell them, "Look, I'm all for a good debate. Let's discuss this." Nope. Nothing.

I had a similar experience to this on Tuesday night at the Extreme Tuesday Club.  I walked into a discussion about C# and .NET that went roughly like this

Attendee: "Microsoft standardised C# purely as a marketing move because Java isn't administered by a standards body.  Besides, the group that administers the standard is the easiest to manipulate standards body out there."

Me: "So you're saying that even though it's administered by an international standards body you think it may not be a '100% pure' standards body and for that reason you don't see any value in it?"

Attendee: "Well, Microsoft has never been interested in open standards!"

Me: "Well what about Microsoft's work in the WS-* space.  For WSE they have shifted from a pre-OASIS version of WS-Security to the OASIS version of WS-Security to ensure industry support.  They are also holding open workshops with any vendor who wants to attend to make sure that the standards are workable.  How does that work in the context of not being interested in open standards."

Attendee: "Hrmph.  I don't know anything about that."

As John mentions, there's a kind of snigger that goes through a group when they criticise .NET.

Fortunately I managed to have deeper conversations where I found out more about pain points that were creating a negative perception of .NET amongst a corporate group that was switching from Java to .NET.  I agree that some of these things are problems with the existing .NET technologies, but found it interesting that they were seen as such major issue.  What impresses me though is that some are being fixed or improved in Whidbey and that there are many Microsoft employees comment or discussing them in public.

• No refactoring in the current version of Visual Studio.  This is difficult for developers used to a Test Driven approach to coding and the awesome support provided in Java IDEs such as IntelliJ or Eclipse.  One guy said that the lack of this feature meant that ".NET code was unbelievably brittle: it was easy to write the code initially but very difficult to change it later".  Refactoring has been added to the Whidbey version of Visual Studio.  Jay Bazuzi is a developer on the Visual C# IDE team and has an excellent blog where he talks about his work implementing refactoring support for Visual Studio.
• The inability to set different access levels on property set and get statements.  This has been fixed in the Whidbey version of C#.
• Some interesting C# enum behaviour that Chris Stevenson mentions (here and here).  These were mentioned to me as evidence that ".NET is fundamentally flawed" even though the person I was speaking to didn't actually know the issues in detail.  I admit I didn't either, but I was impressed to see that Eric Gunnerson had replied to both of Chris' posts.

I'm not trying to say that .NET is perfect or that Microsoft always gets things right, just that discussions are more interesting when they are based on detailed information rather than sweeping generalisations.  I'm glad that the Java IDEs are currently better at supporting refactoring than Visual Studio.  I love the competition and the increasing number of good ideas that it creates.  I also like that Microsoft, eventually, take notice and improves its products.

Last night I had the pleasure of catching up with fellow RD Ingo Rammer for dinner at the Cinnamon Club here in London.  Ingo is a thoroughly nice guy and I had a great time chatting with him about the usual suspects (Remoting, Enterprise Services, WSE and Indigo) as well as presentation skills. 

In this prized addition to my photo blog roll Ingo is showing me the 'Ingo Rammer Signature Photo' pose (which I suspect may be a trademark, though he was kind enough to allow me to attempt to copy it).

It was also nice to catch up with IanG, Sebastien and Ingo's fiancée Katja.

I've been blog-lite while I prepared and then gave my Indigo presentation to the London .NET User Group.  The audience seemed to enjoy it and I had a good time presenting it.  Some quick points:

• The selling point of the Indigo Service model is that it is simple and easy to use.  The benefits for developers is that they get all the benefits of enterprise-quality features such as transactions, reliable messaging and security without having to write any code.  Letting Microsoft do the plumbing allows developers more time to write code that focuses on solving complex and interesting business problems.
• I took on board Clemens' experience and started talking about the 'why' of distributed technologies before delving into how Indigo works.  Luckily this group was very switched on.  I got some good questions about the performance of Indigo across AppDomains/Processes/Machines.
• Someone asked me to provide some background on why Enterprise Services are a better choice than Remoting for communicating between components on the local machine.  Clemens' has some more information on it here and here. 

Doing a presentation where you don't have the current version of the software to demo is a challenge (the PDC Indigo bits are from an older, M4 build, which has been substantially refactored in the now-being-coded M5 build, so there's not a lot of value in showing demos with the PDC bit).  In order to get some code samples I had to transcribe screen shots from the PDC videos (difficult since Steve Swartz didn't use word-wrap in Visual Studio).  I'm filled with (even more) respect for Don Box and Steve Swartz's presentation skills after realizing they managed to do four Indigo presentations at the PDC without even compiling, let alone running any of the applications.

Given the PowerPoint dependence and Clemens' reports on the complexity of the topic I decide I'd have to resort to audience bribery.  I drew on my experience in TheatreSports and brought several bags of Minties from the Australia Shop at Covent Garden (to my horror I discovered today that they are made in New Zealand!) and threw them to the audience whenever I detected the signs of PowerPoint-induced lethargy.

Pair programming with Jamie Cansdale at ThoughtWork's London Geek Night last Wednesday highlighted how much I like this practice.  Geek Night is basically an open night for developers who want to work on projects (ranging from open source development, to personal hobby projects or just for those who want to experience pair programming).  Jamie and I were working on creating a Visual Studio Add-In to remove unused 'using statements' in C# files (essentially, to replicate the 'Imports Tidy' functionality provided by C# Refactory that Scott Hanselman says is almost worth the price of the add-in alone).

Here were the things I enjoyed about Pair Programming:

• Knowledge sharing.  The knowledge transfer ranges from ways of using Visual Studio ("what did you press to get that?") to how to structure unit tests ("would you split that into a second method?") and features of the object model ("how can I find the assembly that defines this type?").  Even in a short session there's a great amount of knowledge that can be transferred.
• Continuous development speed. When sharing the 'driving' seat it means that it's possible to keep developing at a constant rate.  When I develop on my own sometimes I find it hard to keep going once I've achieved a certain milestone.  In Pair Programming it's possible to sit back for a while and let the other person drive.
• It makes you braver. Pair programming provides courage to start having a go even at complex problems.  It's easier to get started attacking a problem using two heads at the same time.

We started out by discussing how to implement the functionality.  Jamie was arguing for a brute-force but fool-proof mechanism of using a regular expression to search through the code, comment out any using statement that it finds and dynamically compile the code to see if it generates an error.  I was itching to write my own stripped-down C# parse that could scan through the source code and understand the using statements (including the alias keyword and nested statements).  To settle the stale-mate by switching our focus from the implementation to the functionality.  We decided to write the tests first (yes, XP is a simple set of practices but it's easy to get off track).

The beauty of writing the tests first was that the tests were independent of the implementation.  After writing a couple of tests it became clear that Jamie's approach was likely to be 'the simplest thing that worked' in the two hours that we had.  The best part was that we were able to achieve a working implementation inside this time!

I'm still eyeing off the SharpDevelop open-source C# editor, which as C# parsing code which I'm tempted to use (or maybe it's time to get into Mono), but for now I'm sold on the value of pair programming and test first development.