# Friday, August 06, 2004

I downloaded Picasa tonight – a free photo management tool that Google recent bought.  Having 12,000 photos from the last three years I’m seriously into digital photography.  I really like Picasa, especially the timeline view of the photographs – similar to the Avalon demos from the PDC keynote – only it works today.

 

posted on Friday, August 06, 2004 12:10:58 AM (GMT Daylight Time, UTC+01:00)  #   
# Wednesday, August 04, 2004
I was pairing with someone last night trying to test the Command pattern which lead to a useful example of using NMock to create a dynamic Mock object to help test it.  Here are some notes on how NMock can be used to create DynamicMock objects based on an interface to quickly create loosely-coupled test code.

The command pattern allows you to wrap a command inside an object so that you can pass it to another object to execute that object.  We were using a simplified version of the pattern where we had a command interface that has an Execute method and a Results property to retrieve the results:

public interface ICommand

{

      void Execute();

 

      string Result{ get; }

}

 

Objects of this type are then passed in as an IList to a CommandExecutor that loops through the commands and calls Execute on each one:

 

public class CommandExecutor

{

      public void Execute(IList commands)

      {

            // loop through the commands and

            // call the command.Execute method

      }

}

So we wanted to write a test that ensured that the Execute method was called on each command that we provided to the CommandExecutor.  Initially we thought about sending through a concrete Command (say a ConcatenationCommand) and determining that the result was what we expected.  However, this would have coupled our design to that particular Command - if we made a change to the results that returned we would have to update our tests.  In this case we aren't concerned about the actual result, just that Execute is called on each command.  The question was how can we test this behaviour without having to worry about a concrete Command or its result?

This is where Mock objects come in (see the original Mock paper or the Pragmatic Programmer's overview).  They allow you to test the behaviour of an object rather than just its outcome.  The idea is to create a 'fake' or stub object that validates the object was used in the way we expected.  In our case we could create a MockCommand that derives from ICommand.  We could provide a boolean property - ExecuteWasCalled - on the class that could be set inside the execute command.

public class MockCommand : ICommand

{

      public bool ExecuteWasCalled;

      public void Execute()

      { ExecuteWasCalled = true; }

      public string Result { get { return "result"; }}

}

While this would work fine, since our needs are pretty simple in this case we could use NMock which provides a way of creating a DynamicMock object based on an Interface.  The DynamicMock exposes a property that can set expectations about the behaviour of an object, such as a particular method being called.  The DyanmicMock object also has a MockInstance method that returns an instance of the Interface. Using this technique saves us having to create a concrete Mock command.  Here's the resulting code:

[TestFixture]

public class CommandExecutorTests

{

      [Test]

      public void ExecuteMultipleCommands()

      {

            // Setup our mock Commands

            DynamicMock cmd1 = new DynamicMock(typeof (ICommand));

            DynamicMock cmd2 = new DynamicMock(typeof (ICommand));

 

            // Set our expectation that the execute method will
            // be called on each command

            cmd1.Expect("Execute");

            cmd2.Expect("Execute");

            CommandExecutor executor = new CommandExecutor();

 

            // Call our execute method on our executor

            // passing in an array of mock commands, using the

            // MockInstance property to create the instances.

            executor.Execute( new ICommand[] {(ICommand) md1.MockInstance, 
                                             
(ICommand) cmd2.MockInstance});

 

            // Use the Verify method on the DynamicMock to

            // ensure that all of our expectations have been met.

            cmd1.Verify();

            cmd2.Verify();

      }

}

 

While I still have reservations about the extent to which MockObjects are useful, I think that NMock and its DynamicMocks are a useful technique to quickly produce loosely coupled test code.

posted on Wednesday, August 04, 2004 10:43:33 PM (GMT Daylight Time, UTC+01:00)  #   
# Thursday, July 22, 2004

If you'd like to understand more about how to do messaging in WSE and would like to see the  three levels of messaging within WSE in action then why not register for my first MSDN webcast here?.  I'm going to be presenting it on Mon 9 Aug at 1PM EST (GMT -8). 

I'll be covering some of the demos from the Keith Ballinger presentation from TechEd San Diego that I presented in Amsterdam, along with some new material.  Here's the abstract:

Using messaging systems to support application functionality allows technical solutions to better match business problems.  WSE 2.0 provides messaging implementations that range from low-level explicit messaging through to high-level, more transparent models.  This webcast demonstrates various messaging levels within WSE including a custom WS-Eventing sample and how message, network addresses, intermediaries and queues need to become first-class citizens of your application. Learn how messaging can enable you to create powerful web services applications that cross machine and network boundaries.

I've put these details on my updated presentations page as well.

posted on Thursday, July 22, 2004 11:35:21 PM (GMT Daylight Time, UTC+01:00)  #   
# Saturday, July 17, 2004

Dave Bettin writes about his experiences doing a user group presentation on Indigo.  His main disappointment was that few attendees seemed to know about Indigo or the existing ASMX web services in .NET.  This got me thinking about how to get the message out about distributed apps, service-orientation and Indigo.  Is it the case that most developers need to be educated about the benefits of scalable, distributed, service-oriented apps or is it that they don't need these approaches to solve their business problems?

Dave's feedback is similar to Clemens' experience earlier in the year, where he spoke of the difficulties selling the relevance of Indigo today before he settled on explaining the 'why' over the how.  Both Dave and Clemens' experienced the same difficulty making the connection between Indigo and developers' jobs today.  Clemens' says that one of the barriers is that people think Indigo is only going to be relevant for 'Big Apps', when in fact it will be important to anyone who builds apps that expose functionality to other apps. 

One argument is that in order to get more interest in Indigo we need to educating developers about the benefits of services and distributed apps and get them to stop writing monolithic apps and start separating the logic across tiers and services.  We need to teach them more about the benefits of the services and distributed application approach, such as scalability.  Sam Gentile touched on this sentiment when he wrote that most .NET developers don't grok distributed computing

The other position regarding evangelising Indigo is mentioned by Alex Lowe in a comment on Clemens' post, is that:

Of course, we know that in the real world there are lots of applications that don't expose functionality to other applications. There are plenty of line of business applications for small and medium businesses that don't require ASMX, Remoting, and in the future Indigo.

Alex also has a summary post on his position where he says that the number of applications that need (and by extension, developers) that actually need to know about how to architect scalable applications is low.  He argues that bloggers are a distorted sample because they represent the group that need to build scalable distributed applications and they are generally well up on things.

I'm interested what other people think.  Is the problem that most developers don't understand the benefits of service-oriented distributed applications or is it that most developers are writing apps that wouldn't benefit from service-oriented distributed applications?  What business application scenarios are most likely to benefit from adopting service-orientation and Indigo?

posted on Saturday, July 17, 2004 12:07:17 AM (GMT Daylight Time, UTC+01:00)  #   
# Tuesday, July 13, 2004

To complement Simon Horrell's MSDN article on messaging with WSE 2.0, I came across this CodeProject article by Roman Kiss describing the three levels of messaging within WSE 2.0 as part of his MSMQ custom transport for WSE 2.0.  This is an excellent bit of detective work (reflectoring, as they say) as there has not been much written about  WSE 2.0's SoapTransport and it's in-memory queue that exist at the lowest level of the WSE messaging stack.

The benefit of the channel/queue model is that the message receiver can retrieve messages from the queue as they are ready to process them, rather than having to process them on demand.

Here's a simple example based on a Windows Forms application that has a button that can be clicked to retrieve messages from the in memory queue and display them in ListBox.

private ISoapInputChannel channel = null;
private void Form1_Load(object sender, System.EventArgs e)
{
   // Get our channel that's listening for incoming messages

   channel = SoapTransport.StaticGetInputChannel(
      new Uri("soap.tcp://localhost:8088/admin"),
      SoapChannelCapabilities.ActivelyListening);
}

private void button1_Click(object sender, System.EventArgs e)
{
   // When we're ready, process the messages off the in-memory queue
   SoapEnvelope message = channel.Receive();
   String messageBody =
      message.GetBodyObject(
typeof(String)) as String;
   
this.listBox1.Items.Add( messageBody );
}

In the Form_Load event the channel is retrieved from the SoapTransport.StaticGetInputChannel method, based on the channel listening on a particular network address (in this case using tcp) with particular capabilities (the options are ActivitelyListening, meaning the channel should create a listener at that address, or None, meaning the channel should connect to an existing listener).  As messages are received they are placed in an in-memory queue managed by this channel.  When the Button_Click event is fired the next message can be retrieved from the channel by calling the Receive() method which returns a SoapEnvelope from which the body can be unpacked.

posted on Tuesday, July 13, 2004 10:11:40 PM (GMT Daylight Time, UTC+01:00)  #   
# Monday, July 12, 2004

Here are some photos and conference highlights from TechEd Amsterdam (completing my backlog of blog posts).  Aside from what I've blogged already, the highlights of the conference for me were:

  • Being there when Don Box spoke for the first time about BOA.  See here for a summary of BOA postings. [update: The concept of Business Oriented Agents (BOA) was a joke designed to send up the hype about the 'next new thing' and the lack of clarity in press reporting about concepts such as SOA.  Unfortunately the joke was not clearly understood and some people are understandably upset about it.  This post was my only reference to it and I apologise if any readers felt mislead.]
  • Hanging out with the other Indigo guys and others who I'd met at PDC and TechEd US.
  • Meeting many people from the UK community at the BoF and Chalk and Talk sessions.  Thanks to everyone that came along and those who had the courage to spend some time on the park bench answering questions and making statements.
  • Increasing my list of UK Microsoft bloggers - welcome Johnny Hall (XP afficionado) and Peter Foot (Compact Framework MVP)

Here are some photos from TechEd Amsterdam, completing my backlog of posts.

Some of the 6,000 drums at the start of the keynote. Delegates enjoying the 3D presentation that finished the keynote.

Pat Helland trying out his Wizard costume in the speaker lounge. Heidi, one of the amazing organising team, showing a way to deal with the size of the RAI conference centre.
A "family photo" of all of the Microsoft Regional Directors at the Staff and Speaker Dinner on Friday. The Boom Chicago Team making fun of the competition for Best Speaker evals between Kimberly L Tripp, Rafal Lukawiecki and Steve Riley

 

posted on Monday, July 12, 2004 11:47:27 PM (GMT Daylight Time, UTC+01:00)  #   

I hosted a lively session on what Service Orientated Architecture really means at TechEd Amsterdam.  While it was a Birds of a Feather session, I decided to run it as a park bench format in order to take advantage of having David Chappell, Michele Leroux Bustamante and John Hooper (blogless MCS UK Architect) come to the session.  Here were some of the interesting discussion points that came up:

  • There was some agreement that SOA is a pragmatic marketing term that unites many existing architectural principles around SOAP. 
  • The closest agreement about a definition for SOA was that it was based around common architectural principles of encapsulation, loose coupling and messaging.
  • There was some discussion about whether asynchronous messaging was a necessary part of service orientation. My feeling is that since you can achieve synchronous patterns over asynchronous communications that having asynchronous messaging capabilities is extremely useful.
  • Although it's possible that these principles could be applied without SOAP, it's the fact that Microsoft, IBM, BEA and others have agreed that SOAP will be the lowest common denominator that is the pragmatic reason behind the current push for SOA.
  • David made a point that service orientation will be whatever Indigo supports when it ships.  Shipping software always wins.  I think there's a lot of merit in this argument, but to the extent that SOA is based on generic architectural principles it is worth considering using these principles in systems that are design today (as Clemens demonstrated all conference).  If SOA principles help solve your business problems today then it's definitely worth starting today rather than waiting for Indigo.
  • Some delegates were suspicious that using Indigo would enable them to interoperate with other systems that didn't use Indigo.  There was some confusion about the idea that Indigo will provide an object model that can be used to develop a system that has the capability to send messages between the systems that are based on interoperable WS-* specifications.
  • The four tenets of service orientation are necessary but not sufficient for a system to be considered a service oriented architecture.  Some people thought they were too technologically focussed because they were tied too closely to XML technologies.
  • David mentioned that the best SOA installation he'd seen was using CORBA several years ago - it had support for finding services, common schemas etc.  Michele backed up the need for shared industry schemas based on some of her experienced.
  • John Hooper was interested in Pat Helland's assertion that services should not share transactions, which led into the difference between WS-Transactions (classic two-phase commit) and WS-BusinessActivity (compensating transactions).
  • A delegate who was learning about the topic came up to the chairs and spoke about what he'd learnt at TechEd.  This was great to hear and generated a lot of discussion.  Clemens' presentations clearly had some impact with many in the audience.

me with David ChappellHere's me with David Chappell after the session.  I've been a fan of his since reading Understanding ActiveX and OLE when I started Microsoft programming.

 

 

 

posted on Monday, July 12, 2004 10:51:11 PM (GMT Daylight Time, UTC+01:00)  #   
# Wednesday, July 07, 2004

Clemens' session on his ProseWare application at TechEd Amsterdam last week was one of the best conference sessions I've seen.  Proseware is "an industrial-strength, robust, service-oriented example application that newtelligence has designed and implemented for Microsoft".  The application clearly demonstrates how to go about building services today with currently shipping technology, reinforcing that there's no need to wait for Indigo to start building service oriented apps!  I'm hoping that we see a public release of these bits soon on MSDN.

Points that grabbed me:

  • Guidance on where to use messaging patterns: Use the OneWay pattern where there is no intelligent immediate reply or no reply is needed, use Request Response pattern where a message asks a question that can be answered immediately (in under a second) and use the Duplex pattern when a message asks a question that can be answered later as the service has capacity (e.g. anything that takes over a second).
  • Clemens showed how to achieving 'near enough' reliable web services with HTTP and services that use MSMQ transactional queues behind the service interface.  If there were any problems placing the message onto the queue then an exception would be returned inside a SOAP fault response.  He optimised this further by using a void response type on the web service method, even though it was not marked as OneWay, so that if there were no problems placing the message on the queue then the web service response message would be small.
  • ProseWare is based around a repository of XML schema files which he uses to dynamically generated the 'message' classes in the application using pre-build steps.
  • All of the service projects shared these schemas rather than having any project references linking to the binaries (services are autonomous).
  • The pre-build step inserts ISerializable attributes onto the message classes so that they can work with Remoting.
  • These message classes are used as the only input parameter to all of the public web service methods.  These classes leverage XML Schema's support for allowing any element or any attributes to come through, which is a powerful way of allowing for future extension to the message.  Dare goes through this in one of his previous posts.  The XML Serialization attributes look similar to this:

[System.Xml.Serialization.XmlAnyElementAttribute()]
public System.Xml.XmlElement[] Any;

[System.Xml.Serialization.XmlAnyAttributeAttribute()]
public System.Xml.XmlAttribute[] AnyAttr;

  • Clemens showed how to use the properties in COM+ 1.5 in XP/2003 to set the home directory for an application, meaning it is possible to use a .NET config file to store the config. He's blogged about this previously here and here.
  • He created his own object pool to create something similar to the ADO connection pooling but for COM+ objects.  He simply pops a component out of the pool and pushes it back when he's done, avoiding the excessive overhead when calling new in a COM+ environment (which has to create the pipeline connections).  He has also blogged about this JIT activation pooling here and here.
  • He mentioned how LRPC, which is used under the covers in ES, is the fastest way to go cross-process on a single machine.  In order to get this benefit you need to use the JIT activation pooling in order to avoid having the performance gains wiped out by the cost of creating ServicedComponents (again, something Clemens' previously blogged).
  • The nice part was that Clemens had done the hard yards and built an application installer that handled created the SQL Server and Windows user and group accounts.  He even showed where he'd found bugs in the OS and how he'd had to work around them.  Information like this is priceless (well, worth a lot of contracting dollars) if you're ever working in a situation where you need to achieve these outcomes.
posted on Wednesday, July 07, 2004 7:49:36 AM (GMT Daylight Time, UTC+01:00)  #   
# Wednesday, June 30, 2004
James Newkirk leading the Unit Testing BoFJames Newkirk lead a Birds of a Feather session on Test Driven Development.  The room was packed to the rafters, showing that Unit Testing is starting to reach a critical mass.  Here are my notes on the discussion from the session which covered how to write tests, how to use tests against legacy systems, how to test against the database and many other topics.

Should we write test code against interfaces or something more abstract than the implementation?
James mentioned that MBUnit is a tool that allows you to test against an interface.  The question was whether you should create interfaces that enable tests to be written against them in case further implementations were created in future.  James' attitude was that this might result in wasted work ('you aint gonna need it') since you may not need it, or may not need it now.  Instead, abstract things out when you need them - don't create an interface just to test it.

James also said that an interface is not a good example of the contract of what is being done - it is the name of the method with input and outputs, but does not reflect how the method reacts to the input. James writes tests that show the real interaction between someone that calls the code and what it produces.

How much of the class should we test? Public methods only or protected and private methods as well?
Around a third of the group thought you should test protected and private methods, about another third thought that we should only test public methods.

The arguments for testing internal and protected methods:

  • To ensure that the internals work. One delegate mentioned writing a test before each internal method. Someone else said the start with the public method then refactor and hide the method by turning it private.
  • Another argument was An argument is that if you write a granular class with lots of private methods, then the tests should be just as granular as the thing being tested.
  • The internals are the most important as they contain the biggest areas of logic, so they should be tested directly.

The argument for testing public methods:

  • Private method tests inhibit refactoring - you have to refactor your tests with a change, increasing the burden and making it less likely that the tests are updaed.
  • You may need to test all of the scenarios from the real world against the public method. If your class is written well then the private methods should be tested.
  • If you separate the tests from the production code then you must test the public methods.
  • Using public methods gives you a clear limit to the number of unit tests that you need. Public tests will get most of the issues, there's no payback from more investment by testing private and internals.

James says that there is no winning this argument. He favors testing the public interface because it decouples the test from the implementation which will discourage refactoring. However, there are cases where it doesn't make sense to expose something just to test it. He thinks it is 80/20 or 90/10 favouring testing through the public interface.

Whidbey will have friend assemblies that allow developer's to split two assemblies, this other assembly is akin to being inside the same assembly. You can do this today with a multi-module assembly built through the command line, not through Visual Studio.

How do we introduce Unit Tests to a 'legacy system' without tests?
No one in the session had seen legacy code that is easy to test if the testing hasn't been thought of upfront.  One person mentioned that they created tests for each issue logged through the help desk and then used these as a regression suite.  This also demonstrated the value of unit testing to their organisation.

James suggested drawing a line around a piece of the code that you need to change. Test it's external behaviour, make changes and ensure the tests still run. They aren't unit tests, but who cares? Create a boundary, create tests and then change. The key point was to conserve effort and only write tests on things that you are going to change.

James also mentioned a book to be published in September by Michael Feathers called 'Working Effectively With Legacy Code' that describes how to handle this difficult situation.

How to convince people to do test first? Argue against the concept that it will take too long to write the tests first?
One delegate mentioned that this is hard because the best way to convince people is to show results, which requires a practical example, which means you have to know how to do it (what classes, how to unit testing). It takes time and experience but it will work eventually.

Someone else mentioned the green bar of success (my favourite) on the screen is a big part of demonstrating it.

You have to make sure the person has the right mindset. Not everyone has a zero-defect mindset. They want to write tests more than write code.

James says the story is not about the individual developer who wrote the code and knows it works. It is about the team and the ongoing evolution and maintenance is where the tests matter. How can you be confident of your result without them? With unit tests I know they works, before I relied on something else, but now I know.

Why should we write the tests first?
James says that no one will take my word for it. What convinces people is having to test something that hasn't been created with testing as a priority. If a setup method has 20 lines of code and many objects being created and only contains a single assertion - it wasn't written with testing in mind. Someone will say it is really hard to test it.  So the next question is 'What would you do differently?'  The answer to this question is what people should do in Test First.

James said that he believes that testing has to be seen as a primary part of development. We have to incorporate test inside development. There may be QA, but they start at a different level. You have to incorporate testing as part of development.

In the PAG group they have 2,000 unit tests that get run against the library every time someone modifies the code. When James talked to the testing and QA group who do integration testing -they didn't know what to do - the unit tests did the easy part of the testing. They have to start at a higher, more complex level to start thinking about critiquing what is going on rather than I just pass 32,000 characters in a web app and it breaks. These are necessary tests, but it doesn't take a lot of skill to do these types of tests.

Someone made the point that software development lags behind electronic engineering where hardware must be tested first.

James mentioned that studies have show that Test First is 16% longer, but the quality was much higher.

How do you test a function that is dependant on other objects or libraries?
One delegate mentioned that using Dynamic Mock objects relied on finding a sweet spot.  They are useful with objects that have relationships, but the problem is that if you refactor your code then it is highly likely that a whole slew of tests that will break.  The problem is compounded with dynamic mocks since you only see this at run time rather than compile. It works well at mocking the database, but not the more dynamic

James described mock objects as the situation where object A interacts with objects B and you need some way of 'switching out' Object B to create a 'dummy response' from the calls of Object A.

James believed that if you are interacting with something complex, building the simulation of something that is complex is a worthless activity - you spend more time writing the simulator than the test and it doesn't tell you anything? Just because my mocks work, does my real system function?  James uses a rule that if mock objects have an if statement inside them then that's too far - the behavior is too complicated and there's no value (it says there are multiple situations). They do have value, but we need to separate a simulator and a mock object. For example, writing a mock JDBC implementation - that is way too complicated.  Be skeptical of spending a lot of time spending time building MockObjects.

Someone asked - 'Don't you think that it depends on the situation?' James: "I could answer everything that way"

James mentioned the Inversion of Control pattern - this is the a situation where object A depends on object B - the developer wants to be able to switch object B out as a mock or a stub - how can we do that? Inversion of control says you create the dependent object B, or a mock or stub external to object A and then pass it into object A as a interface parameter in a constructor. That's a lot of complexity to add to the initialization,  James was interested in opinions about whether the complexity is worth it?  He mentioned that some people have pushed back on the idea because of the complexity of the construction - if all you are doing this for testability (it also decouples the design) then it is too much work?  He believes that what has to happen is that we need different language structures and we will never get there if we dismiss these ideas now. In the Java world there are lots of work being done on using containers in another way.

How do you we test the database? Where do you get your data for testing? How do you unit test things that involve the database?
James thought that the problem with the database is similar to the MockObjects. It is a lot of effort to Mock out the database. Sometime it is easier to use the database than mock, you need a database to run the unit tests and you have to fill the database, read in a dump, which takes longer and is more effort.

James' experience is that at some point you write tests against the database because he wants to test that the Data Access Layer works. When he write DAL tests he uses the database. But he don't use the DB for anything that uses the DAL - he mocks them out. A technique he use to ensure he's written good unit testing is to deliberately break something and see the results.  If he finds a cascade of errors, it means he hasn't isolated the tests correctly.  He also mentioned that if he comes back to an a feature afterwards and spend a lot of time in QA it means he didn't do well enough with the tests.

With databases,  it is a good idea to  use a transaction and rollback the transaction at the end of it to ensure that.  There was some discussion about how to construct database tests.  The problem with building the database in scripts the unit tests take too long and they wont be run. How long is too long?

How long should the tests take?
There were a range of experiences including:

  • Four or five machines run it each product, it takes 6 hours across machines.
  • Someone else has developer tests and then smoke and build tests, it can take a long time in the nightly build.
  • Another project had two CruiseControl systems that built at different intervals, running a short and long set of tests.

James thinks an hour is too long - the reason is that ideally you would be able to get very quick work around. This is how it works successfully. Waiting an hour for the feedback means that a developer could only make 8 or 10 changes a day. In that hour a number of things are accumulated. James mentioned his experience on an early project where it took 12 hours to recompile that application.  In this project if someone made a change to a header file in C++  they had to recompile, so what they'd do is create global static variables instead of the header to ensure it works before 'doing it correctly' (which was never done). Sorting out problems was very complex because it was effectively an integration nightmare..

Should all developers run all tests? It depends on the architecture - if there are subsystems you could do that. In XP it says all of the tests all of the time. James' group run 2000 tests in 340 seconds against 3 databases.

How do you manage dependence tests with the database? How to restore state with persistent storage.
It is a good idea to stub out the system so that you can test against something that you depend upon. Then when the implementation is delivered you can run the tests and it becomes clear where there is an integration problem.

One delegate spoke of how he stubs out the system, then write tests and implementation, then the stubs are removed and the real implementation of the rest of the tests are done. The platform should support this - James mentioned a project where he used 'linker polymorphism' - relied on the linker to substitute.  Another delegate said they had written something that as part of the the daily build checks out the project file, it changes the XML structure of the C# project to switch references and changes them to the actual assemblies.

How do you test concurrency? Multithreading?
These are just really hard. Testing timing problems with unit testing are hard. One delegate spoke about how he had written unit tests for a multithreaded apps and said it was one of the best examples to convince others of the value of unit tests. It took 2 months to create, but then it only took 1 day to shift it from Java to C#.

The xUnit tools should support this but don't make it any easier.

Another delegate used a timer library and extended the Nunit assertion.

Roadmap for MS
VS Team System will include unit testing support some time next year. It will have a number of integrated testing tools. The keynote yesterday showed a tool 'very much like' and 'subtly' different tool to Nunit. James will be talking about the difference.

There are load testing tools, web testing tools in there. It is a team system that is extensible by many different partners. Many partners, such as CompuWare (did a functional UI testing), doing stuff inside this 'platform' that can be extended.

Someone will write something that will allow Nunit to execute in this environment.

How do you generate tests?
Team system will create a stub test from the code. These are just boiler plate - it doesn't do analysis of the running code and propose a 'good test'

One delegate had a poor experience with a tool that created tests automatically because the tests didn't take into account the intention of the class which resulted in the tests failing. Writing the test manually is more useful as these can act as a specification for the class.

James thinks looking at the implementation to drive the tests is looking at it the wrong way. If you wrote the implementation wrong and then create tests off these, what's the value? The tests should say 'this is what the implementation should do'

People often propose this if they haven't done unit testing before.

How do you test distributed applications?
It relies on subsystems - just write tests that test the local machine, when you have to integrate it all together you might need to run tests, but these are not unit tests in this situation.

How do you unit test GUIs?
One delegate used Rational Robot - costs a fortune. If you are careful and script it then you can get away for a few builds without it breaking. SendKeys was also used.  Another person just avoided the problem by trying to get all of the functionality out, recognising that it is hard to test the UI.  Someone said that it is hard to test drag-and-drop operations.

James menioned that Robot is not good to drive development because they require the UI to be done, but it is hard to do.  NUnit Forms and NUnit ASP were also mentioned.

What frameworks can you recommend?
Nunit, csUnit, mbUnit, CLRUnit
HarnessIt (commercial)
Xunity (Commercial)

How do you do Web unit testing?
HttpUnit works, but it is Java, these can be used successfully to drive HTTP requests and do some testing on the HTML that is returned.
They suffer from many of the brittleness problems that the robot testing does - you have to use id's on the elements - but it requires a lot of thinking about how you output.

posted on Wednesday, June 30, 2004 2:13:41 PM (GMT Daylight Time, UTC+01:00)  #