# Saturday, December 06, 2003

Steve Maine writes about the importance of getting the API right when designing software:

Naming the abstractions present in a software system is an important design consideration, especially when designing a consumable API. The names you choose will influence the way consumers of your API will think about your software – you can either choose to make it easy for them, or hard. Convincing ourselves that we've made the right choice is proving difficult

Rather than trying to convince yourselves about the design I'd suggest trying it out or doing some usability testing.

In terms of trying it out, I'd suggest taking the Test Driven Development idea (or the Microsoft idea of Dogfooding) and try writing the sample code first to see how the code looks when you use it yourself. Often design discussions are 'tainted' by people knowing the implementation rather than thinking from about the intention in a users mind.

For the usability testing you could start with the idea of Personas to describe the users of your software (e.g. is it for a 'mort' and 'Einstein' or an 'Elvis'). Think about what goals the users of your they are trying to achieve and then write those down. Get around 5 people who might write your code and ask them how they might achieve these tasks using the code. Give them the object model and ask them to write out the pseudo-code. If it's just the naming that you're concerned with try describing the job is does and asking potential users for what they think it might be called.

What would Jacob Do?  Here I'm trying to find out.You can go all psycho-serious and see what value you can extract from this Microsoft Research presentation "Describing and evaluating API usability at Microsoft" but I'd go with Jacob Nielsen and start with the discount usability approach. Get a small number of potential users, think of open questions you can ask or tasks that they could do then follow the golden rule of sitting back and listening/watching without saying anything (the photo on the right me with Jacob Nielsen.  I'm smiling even though I paid £400 of my own money just to hear Jacob tell me this simple rule).

posted on Saturday, December 06, 2003 9:30:50 PM (GMT Standard Time, UTC+00:00)  #   

Recently I've been reviewing the most efficient way of retrieving values from an XML document in .NET.  As Scott Hansleman mentions, the best way to find this out is to write some code and measure the performance, so I wrote some code that can be downloaded to test the performance of three different approaches:

  • an XmlDocument using XPath queries with SelectSingleNode
  • an XPathDocument with an XPathNavigator
  • using the XmlSerializer to deserialize into a custom class

The results show that the XmlSerializer is fastest once the initial cost of creating temporary assemblies has been overcome.  In situations where the initial performance is most important then an XPathNavigator over an XPathDocument is the fastest.

Approach
Here's more detail about each approach:

Object Type XmlDocument XPathDocument XmlSerializer
Retrieval Method XPath query using XmlDocument.SelectSingleNode XPath queries using an XPathDocument Object properties
Advantages Familiar to many developers. 
XPath queries allow for quick evaluation of complex expressions.
Optimized for XPath and XSLT transformations.
XPath queries allow for quick evaluation of complex expressions.
Likely to become more important in future.
Turns XML into Objects
Disadvantages Slow, requires the whole document to be in memory. Slightly more complex for developers to write than XmlDocument. Requires familiarity with XSD and is more complex to set up. 
Can't match XPath's complex expressions.
Slow due to generation of dynamic assemblies on first use.
Example XmlDocument doc =
   new XmlDocument();
doc.Load(filePath);
XmlNodeList selection =
   doc.SelectNodes(XPath);
result = selection.Item(0).InnerText;
XPathDocument doc =
   new XPathDocument(filePath);
XPathNavigator nav =
   doc.CreateNavigator();
XPathNodeIterator it =
   nav.Select(XPath);
it.MoveNext();
result = it.Current.Value;
XmlTextReader reader =
   new XmlTextReader(filePath);
XmlSerializer ser =
   new XmlSerializer(typeof(message));
message mymsg =
   (message)ser.Deserialize(reader);
result = myMsg.MessageID;

Aaron Skonnard provides some excellent background on the benefits of the XPathDocument over the XmldDocument in his article .NET XML Best Practices: Part I: Choosing an XML API.

For the timing tests I used the code available in the EggHead Cafe article "High-Precision Code Timing in .NET".  Unfortunately the Ticks property of the System.DateTime class is only accurate to around 16ms even though it displays values to 100 nanoseconds.

Results
In this app I take a small XML document with no namespace and retrieve four values from it.  Here are the figures I got when running the console application for the first run and then a repeat run:

Runs XmlDocument XPathDocument/Navigator XML Serialization
1 0.00543 0.00129 0.09020
1 0.00051 0.00035 0.00028

The first time the application is run the XML Serializer has to create temporary dynamic assemblies so it performs the worst, however on subsequent runs within the application it performs the fastest since it can use a cached copy of the temporary assemblies (As Scott found out, these assemblies are not cached in .NET 1.0 if you specify a namespace).  Daniel Cazzulino has some good background on how the XmlSerializer works).  In both the first and the second runs the XPathDocument/XPathNavigator approach is faster than the XmlDocument.

In the sample code I've also presented an ASP.NET web application that can host the same tests.  ASP.NET creates a new AppDomain for each website it hosts and is maintained across requests to the server until it is shut down or recycled. Since the XmlSerializer caches its temporary assemblies based on the AppDomain, using the XmlSerializer in an ASP.NET web application or web service application is actually the fastest technique for retrieving values from an XML document.

In situations where the AppDomain is created per-request then using the XPathDocument/XPathNavigator will be more efficient than an XmlDocument and the XmlSerializier.

Discussion
The XmlSerializer is the fastest way to retrieve values from a small XML file if it is possible to overcome the cost of the creation of the temporary assemblies.  In situations where the first retrieval is the most important, or where more complex XPath queries are used the using an XPathNavigator over an XPathDocument provides better performance than the XmlDocument. 

Thankfully it will be possible with .NET 2.0/Whidbey to use a tool (sgen.exe) to pre-create and compile Serializers.  Doug Purdy covered this in his PDC talk, see Scott's notes.  I believe this will make XmlSerializer the fastest approach to retrieving values from XML, but testing will tell.

Click here to download the sample code.

posted on Saturday, December 06, 2003 7:12:06 PM (GMT Standard Time, UTC+00:00)  #   
# Wednesday, December 03, 2003

A month after the Indigo 'Kimono opening' at the PDC there's still a lack of clarity about what Indigo is, how it relates to other messaging technologies and what's the best way to start developing applications today.  While a lot of this was covered at the PDC my perception is that some of the message hasn't been ack'd successfully from the audience.  [Update: See my more recent post 'More on the Microsoft Messaging message' for some answers to these questions]

The Longhorn DevelopMentor mailing list had an excellent exchange yesterday on Indigo, which has lead me to highlight some areas where I'd like a clearer message:

  • What's the relationship between Indigo, MSMQ, BizTalk and SQL Server Service Broker?  When and where is each technology appropriate? (Thanks to John Cavnar-Johnson for highlighting this)
  • How does MSMQ fit into the Indigo longer term picture? Is MSMQ for within the Enterprise or will it focus more on outside the Enterprise messaging?
  • In terms of EAI technologies, where should we look for this functionality - Indigo or BizTalk?
  • Which parts of Indigo are going to ship in Whidbey and which bits will come in Longhorn?
  • Given that the last version of WSE will be wire-level compatible with Indigo and that a future version of WSE is likely to support WS-ReliableMessaging, what are the benefits of Indigo other than the simplified programming model?
  • Indigo is committed to supporting WS-* standards and interoperability, but what extra functionality will be available if the whole environment is made up of Indigo boxes?

I'm  know some of these were addressed at the PDC.  I'm still digesting some of it (for example, the PDC DAT406 presentation on the "Yukon" Service Broker shows how MSMQ, Indigo and Service Broker are positioned, though not BizTalk).  Other questions I'm still researching.

posted on Wednesday, December 03, 2003 9:29:39 AM (GMT Standard Time, UTC+00:00)  #   
# Monday, December 01, 2003

Steve Maine has a post on using a Gantt Wall to manage projects: it's big, visible, lo-tech and easy to manipulate and change.  I think it's a great idea in line with the XP/Agile spirit of getting away from traditional, rational (time and motion) style project management tools in favour of simpler approaches that are more likely to work.

I think MS Project style Gantt charts are useful, but too often they form part of a projects Death March when they are never updated.  Initial estimates of features become prison sentences for developers since they become set-in-stone.  Weekly meetings are focussed on how far behind developers are from their original estimates.  The whole project becomes burdened with the fact that it's slipped, is late or behind.  This is where lo-tech idea's like the Gantt Wall are so useful.  As Darrel Norton says:

the visible wall-Gantt is more than just a Gantt chart, it builds morale, serves as a social focal point for the team, and allows the celebration of success on a daily basis.  Until we can digitize all that, the low-tech solution will continue to reign supreme.

My Favourite Project Anti-Pattern
All of this makes me think of my favourite project anti-pattern which is to simply avoid dealing with any problems on the project and force everyone to work as many hours as possible each week for the length of the project.  It doesn't make the project any more successful, but it certainly saves having to deal with anxiety that can come from admitting and dealing with the situation where the project is off the rails.  When it comes to the deadline for the project, the work may not be done, but the excuse is difficult to refute: 'could we have worked any harder?'.

posted on Monday, December 01, 2003 10:45:03 PM (GMT Standard Time, UTC+00:00)  #   
# Sunday, November 30, 2003
WS-Addressing is one of the newer WS-* specifications. The specification brings us closer to the reality of sending SOAP messages across multiple transports in an interoperable way. Going down the plumbing level with WS-Addressing helped me highlight the gap between the specification and implementations. Along the way there's a great SOAP Mail sample application using WSE 2.0 that highlights some of the promise of being able to send SOAP messages via HTTP and SMTP ...
posted on Sunday, November 30, 2003 10:45:27 PM (GMT Standard Time, UTC+00:00)  #   
# Tuesday, November 25, 2003

There's been lots of talk about offshore development recently.  Larry O'Brien is chatting with John Montgomery about whether he'll still have a job as an independent developer , Rory reports that Dell have moved their call centre back from India.  The pragmatic programmers have released a t-shirt advertising their 'how to keep your job' with the slogan 'My programming job went offshore, and all I got was this lousy T-shirt'.  The current project I'm working on has bought their offshore development team on shore.

On top of all of this I notice today that Chris Stevenson, who's been the source of many good insights into .NET/Java/XP/Agile development at the Extreme Tuesday Club and who is also from my home town of Adelaide, South Australia, is going the other way and is moving to India for 6 months.  Funny old world.  I can't wait to read about Chris' experiences.

posted on Tuesday, November 25, 2003 11:17:40 PM (GMT Standard Time, UTC+00:00)  #   
 

Many others have noted that the PDC Sessions are available online as streaming audio with synchronized PPT and demos, so who do I think I am posting it again? Well, here's my value-add: it's possible to download the Windows Media streaming audio and the demo videos using a free tool called Net Transport.  I'm using this to max out my broadband connection as well as letting me relive the PDC sessions I was too worn out to make when I was at the conference, on my daily four hour train commute.  Today I spent time on the train with Anders, tomorrow more Don and I'm considering using Doug Purdy's presentation as my new motivational tape to help me get out of bed in the morning.

To start each PDC folder has a file with a 0Media.asx file, which is an xml file containing links to all of the media files in the presentation.  Net Transport expands the xml and starts downloading the individual files.  It's even possible to pause the downloads.  Here's the link for Don's overview of Indigo for example  (you can test the accuracy of my transcription of Don's talk at the same time!).  Once you've downloaded the files, edit the Media0.asx file to use local paths to the individual files and launch it in Windows Media Player.  You'll have to move the slides yourself, but it makes it feel more like a slide night.

If I had more time I'd adapt early & adopter's script to download the slides to download all of the streaming content.

posted on Tuesday, November 25, 2003 10:52:29 PM (GMT Standard Time, UTC+00:00)  #   

I've just listened to Ander's PDC presentation on C# 2.0 (TLS320).  C# is incredibly cool, the new features are great and it's amazingly useful to have Anders explain them.  Languages enhancements are as exciting for a developer as a better quality hammer is to a carpenter.

 

What's interesting about language enhancements is that to take advantage of them you have to teach your brain to think more like a compiler.  The Java language exam is probably the best example of this that I've come across.  Initially I thought that having to answer questions that the compiler would check was a waste of time.  Luckily a Java development mate of mine argued that it was important, especially in becoming a better developer.  Having done the exam, I have to agree with my mate that there are real benefits from being able to look at code and know how the compiler would treat it.

 

So listening to Anders it isn't just exciting to know that they've developed a compiler that knows how to compile impressive statements, but also realising that I'll have to train my head to do the same thing.  Luckily I find learning things like this a real pleasure.   Having new C# syntax to learn and play with is like a bright blue sunny sky for me after the grey clouds of my current project (arguing with a developer who stubbornly insisted that GOTOs were the best way to code a particular method).  Sure this looks a bit scary now, but I can't wait to get familiar with the syntax and to start developing with this extra power:

 

class Dictionary: IDictionary
    where K: IComparable
    where V: IKeyProvider, IPersistable, new()
{
    public void Add(K key, V value)
    {
 }
}

 

Some random points from Anders speech:

  • Generics  These are great in general and I can't wait to be able to use them in anger.  I have a brilliant C++ developer mate who said he wont switch to another language unless he can use collections without having to cast.  Hopefully he'll be swayed.  I also like the use of the T.default and null checking that is thrown away by the JITer if it's a value type.
  • Anonymous methods Anonymous methods seem similar to Java's anonymous classes.  Dare says: "[Anonymous methods] fix the lack of anonymous inner classes in C# by adding anonymous methods for use in the exact same situations most people use anonymous inner classes for".  It was good to understand how the implementation works under the covers to allow anonymous methods to access the parameters of the outer methods (Steve Maine waxed lyrical about this feature and lambda methods on the bus back from Universal Studios, now I finally understand his passion).  I love the idea of passing code as a parameter.
  • Iterators I did once try and implement these in VB6 and it was a horror, so I'm glad the story is that writing iterators that can be consumed by the foreach statement is now easy.  Also, statements like foreach (Item x in Items.Subrange(10, 20)) are just cool.
  • Partial types Anders explained there were two main reasons for this feature.  I was wondering since Juval Lowry couldn't really see much point in them when he did the .NET radio show pre-PDC. Basically they help people who generate code - the generated class can be regenerated without breaking code written in the other part of the partial class, and where there are long classes (though this is a bad idea in general).  It's also a key part of the XAML story. 
  • Other enhancements Finally C# lets us have different accessibility levels on property set's and get's, so the set can be internal and the get is public.  This just feels more right.
posted on Tuesday, November 25, 2003 10:33:19 PM (GMT Standard Time, UTC+00:00)  #   
# Monday, November 17, 2003

Roy's at it again, campaigning for IntelliJ and asking Visual Studio to take notice and compete with the advanced developer experience provided by the IntelliJ IDE:

 "I felt like I was not coding alone ... if you decide you should have a new method ... you just type the method name and provide its parameters if necessary.  IDEA will prompt you with an unobtrusive popup with a suggestion to create a new method ... the method will be created in the corresponding class with the proper method signature (including correct parameters), so that you can implement it later"

I saw the same feature demoed by Erich Gamma, (the G in the GoF Patterns book!) when he came to London to talk about Eclipse.  I was so impressed the hairs on my arm stood up.  Forget clapping at the PDC, you know a feature is good when your body reacts with goose bumps.  As I mentioned in a previous post about refactoring, I think this feature would solve a lot of the frustration that Randy mentioned where Visual Studio's Intellisense clobbers any new methods or properties written in tests before the object exists.  It's the difference between thinking of the IDE as a text editor and thinking of it as a developer tool.

Luckily, IntelliJ are developing an IntelliC# add-in for Visual Studio that will bring its intelligence to .NET.  I can't wait to get it.

As Roy laments, what's Visual Studio doing?  I remember the glory days, back when Jim McCarthy (of Dynamics of Software Development fame) was working with the Visual C++ team making the IDE great.  Where's the passion, the spirit, the sense of competition gone?  For all the talk at the PDC there was noticeably little about Orcas, the Visual Studio version after Whidbey.  Visual Studio is good, but it can be even greater.  Let's hope that good things are in store.

posted on Monday, November 17, 2003 10:30:08 PM (GMT Standard Time, UTC+00:00)  #   

Ted Neward has some great advice about how to review a technical manuscript that I wanted to highlight.  I'm using my 4-hour train commute to review a technical book that I'm pretty excited about.  Ted's article was helpful.  He also has a great way of looking at technical books - positioning books as a deal between a book and a reader:

when I write a book, I'm offering the reader a deal: "You trade me $40 of your hard-earned cash and roughly 40-80 hours of your non-refundable time, and in return, I will give you something you didn't have before, some knowledge that you find useful." If my book doesn't keep my end of the bargain, the reader takes it back to the store, but can never get back the time spent, and so is left feeling cheated and upset. In some books, the classics like "Design Patterns", or K&R, or "Rapid Development", the deal is so obviously lopsided in the reader's favor that these books go on people's shelves right next to the monitor, for frequent perusal. But the thing to realize is that it's all about the deal. Does the book fulfil its end of the bargain?

Searching for and finding books that keep their end of the bargain is one of my key passions.

posted on Monday, November 17, 2003 9:54:45 PM (GMT Standard Time, UTC+00:00)  #   
# Friday, November 14, 2003
 As Tim Sneath spotted, I did another YAWP (Yet Another WSE Presentation) to the .NET Exchange User Group at Microsoft UK campus in Reading last night.  I'm doing a number of presentations around the UK in the next month (though I think John Bristowe's record of 20 YAWPs a year is safe), details on my presentations page.  Here are my slides and some of my demos

Web services are being adopted in the wild
There certainly is a lot of material to cover in web services from the specifications to the implementation, so we only briefly touched on Indigo.  There were 30 or so developers there. Starting with the mandatory 'polling' questions I was surprised that around 60% of them had used web services already.  Of that group about a third had added some form of security (HTTPS, VPN or internal network).  There were some good questions in the group ('Can I use WS-Security in a PHP application?  Will Microsoft compete with Tibco and the EAI vendors with Indigo and/or BizTalk?  Has Don Box stopped using the term GXA?)

Blogs are a great way to learn about Web Services
I mentioned that blogging is a great way of tracking what's happening web services. I showed how:

How to get started reading blogs
First download a news aggregator like SharpReader or RSSBandit then download my modified OPML file.  It contains many Microsoft bloggers as well as other web service related blogs (including those on Matt Powell's list).

I also promised a link to the code that downloads all of the PDC powerpoint presentations.

posted on Friday, November 14, 2003 10:15:47 AM (GMT Standard Time, UTC+00:00)  #   
# Monday, November 10, 2003

As part of a new consulting engagement I started today, I found out that the latest version of the .NET Solution Build & Deployment Process & Tools has a visual designer for MSBuild Files.  The Tools are writtien by Andy Reeves, from Microsoft UK's Solution Development Centre and were the basis for the extensions included with MSBuild.  The download page also has more about why MSBuild was chosen over NAnt.  John Lam has also posted about the NAnt community's reaction to MSBuild.

The new engagement involves 4 hours train travel each day, so I'm going to have to kurb my blogging to get to bed.

posted on Monday, November 10, 2003 10:21:27 PM (GMT Standard Time, UTC+00:00)  #