After writing about how WSE 2.0 can use policy and config files to secure web services with no lines of code, I was thinking about how 'magic' it seemed and had an aha! moment when I realised that this demonstrated the power of the Pipes and Filters pattern or aspect-oriented style approaches.  I believe that these approaches will play an important role in service-oriented applications in future.  Here are some recent quotes that back this up.

In my post about using Policy with WSE to create secure web services with no lines of code, I mentioned that this seemed like a good practical demonstration of an aspect-like approach, similar to the Pipes and Filters patterns from Gregor's book.  The policy filters are hooked into the incoming and outgoing messages pipelines are able to ensure that messages to and from the service conform to a particular policy, including retrieving tokens and signing and encrypting.  This means that the security of the service can be configured outside the service code, making for cleaner implementations.

Harry Pierson mentions that WS-Policy is aspect-like in his interview on TheServerSide.NET:

We do a lot of thinking around aspects in Web services, we just don’t call them aspects Web services, we call the Policy. All of the work around Policy is very aspect-like. If you’re spinning up a Web services with Web services enhancements, the new 2.0 stuff, there’s a Policy that defines, okay, if you want to talk to me, you have to be encrypted and digitally signed ... now I can communicate that to whomever is sending me a message. ... the WS-I engine can actually now say, okay, I’m expecting messages that [are signed and ecnrypted]  so I can actually enforce to make sure that those things have actually occurred before it ever reaches the business logic. So, that’s very aspect-like, and that’s going to be very critical going forward around service oriented architecture

Ted Neward writes about a recent presentation on Shadowfax and mentions it uses the concept of a pipeline of interceptors. He mentions that Shadowfax uses this approach to deliver functionality such as tracing, authorization, duplicate detection, instrumentation, authentication, authorizations and transactions.  Chris Garty started an interesting thread about this on the Shadowfax message board, where it was revealed that the Shadowfax team spent some time Gregor Kiczales (one of the creators of AspectJ).

This same pipeline/interception approach has been used in WSE, ASP.NET, Remoting and COM+.  Indigo will implement the same kind of approach using Channels and ChannelProviders.  I'm going to keep reading around this area to understand more about this approach and where it how it can be used successfully.

John Bristowe and I are featured on MSDN TV enthusing about the launch of WSE 2.0.  It was filmed in the Cabana areas at TechEd 2.0.   Here's the blurb:

Celebrating the launch of the Web Service Enhancements (WSE) 2.0 at Tech·Ed 2004, Benjamin Mitchell and John Bristowe talk about the advanced Web services specifications that it supports, focusing on WS-Security.

You can also read the transcript.  You can tell that John and I aren't from Microsoft since we don't use 'so' enough when starting our sentences

I'll be on .NET Rocks! tonight talking about WSE 2.0 and building web services with Microsoft technology today.  I'll be joined by fellow Regional Director and Commonwealth Citizen, John Bristowe.  If you can't listen live, it should be available streaming on Monday.

Update: As Carl has mentioned, the streaming/download version of the show has been posted.

my previous post, Mark Naughton asked an excellent question about how he'd apply WSE 2.0 security to a particular scenario.  The answer highlights how to determine which SecurityToken to use in your environment, how to encrypt a UsernameToken with an X509 certificates with code and policy as well as handling authorization with X509 certificates and determining how to distinguish tokens received by a service.

Martin's scenario

An End User uses a Web-based UI Application (ASP.NET 1.1).
The Web Application talks to a Web Service (ASMX) for data storage and other processing.
The Web Service needs to identify the End User and the "direct" calling application (The Web UI App), since there may be more than one "direct" calling application.
We also want to sign and encrypt the Soap messages in both directions.

Since we're talking about adding security to a web service, we'll need WSE 2.0 installed on all of the calling applications. 

What's the best SecurityToken to use?
The first thing to consider is what SecurityTokens are applicable to the scenario.  Aside from custom xml or binary tokens, the three options that WSE supports out of the box are as follows.

Username Name and Password

• For - easy to construct, no distribution problems, WSE handles automatic Windows authentication and the construction of the SecurityToken.Principal for authorization.
• Against - on their own they are only as secure as the passwords.

X509 Certificate

• For - cryptographically strong, can carry extra information (e.g. certificate name). 
• Against - need to manage the distribution to clients.

Kerberos Ticket

• For - powerful cryptographically, inbuilt WSE support for authentication and creating the SecurityToken.Principal for authorization.
• Against - your application and the service you access must be running on computers joined to a Kerberos realm (e.g. not good over the Internet) unless you implement a custom security token service to issue service tickets.

So any of these tokens can identify the end user or the application - it's a matter of working out which one works best for your situation.  If you can handle the distribution and installation of X509 certificates to all of the calling applications, I'd suggest using them to sign and encrypt the message.  In your scenario, the ASP.NET web server could create a UsernameToken to represent the End User of your web application.  For best security, I'd suggest encrypting the UsernameToken with the X509 certificate (hiding the username and password/password digest).

The code would look something like this:

// ... Assume we have an X509SecurityToken and a UsernameToken, and a reference to the web service called proxy.

// Add both tokens to the SOAP envelope

proxy.RequestSoapContext.Security.Tokens.Add( x509token );

proxy.RequestSoapContext.Security.Tokens.Add( usernameToken );

 

/* Encrypt the username token with the X509 token.

 When encrypting, WSE looks for XML elements with an Id attribute from the http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd namespace, which the username token uses.

The "#" indicates the Id with this value is local to this message */

proxy.RequestSoapContext.Security.Elements.Add(new EncryptedData( x509token, "#" + usernameToken.Id ));

 

// Encrypt the message body with the X509 token to ensure no one can read it.

proxy.RequestSoapContext.Security.Elements.Add(new EncryptedData( x509token ));

// Sign the message with the X509 token to ensure its integrity

proxy.RequestSoapContext.Security.Elements.Add(new MessageSignature( x509token ));

You'll also need to decide on what token type to use when sending the signed and encrypted response.  Again, I'd recommend using an X509 certificate for the most cryptographically strong security.  The downside is that you'll need to install the certificate on each of the clients.  If you can't handle this install requirement then you are stuck with UsernameTokens.

Distinguishing the tokens on the service
Using the combination of a UsernameToken and an X509SecurityToken to represent the identity of the end user of the calling application and the identity of the calling application itself makes it easy for the web service to work out which token is which.  The web service has to search through the tokens in the RequestSoapContext.Security.Tokens collection to locate each token.  If you decided to use two username tokens for example, you would need to distinguish them somehow.  For username tokens you could achieve this through the username values, or perhaps by giving them well-know identifiers in their securityToken.Id property.  For X509 certificates you could use the certificate name.

Performing authorization
If you use a UsernameToken encrypted with the X509SecurityToken, and you don't want to send the password as plain text, then you'll need to create your own UsernameTokenManager.  This is responsible for authenticating the user and creating the SecurityToken.Principal object which can be used for authorization.  For the X509SecurityToken you can create a custom X509SecurityTokenManager and in the AuthenticateToken method, after calling the base class's implementation, then create your own generic principal and attach this to the SecurityToken.Principal (Ingo Rammer wrote an excellent article on this last September for MSDN, but it's disappeared.  You can find my notes on it here).  The benefit of doing this is, rather than just testing for the certificate name inside the service code, is that WSE Policy validation input filters inspect the SecurityToken.Principal when verifying the policy assertion.

Wrapping it all up with policy
As I've indicated in previous posts, using policy and configuration to avoid writing security code within your service is an excellent idea.   You could do that in this situation as well, except that signing the username token is a little tricky to indicate in policy (you'd have to hand-craft the policy XML file as this is a little way-off the WSE Security Settings Tool territory).   In a confidentiality assertion you'd need to modify the MessageParts elements in the policy file to indicate the UsernameToken's Id attribute.  I'll leave this as an exercise to the reader (as my daughter is about to wake up again), but Aaron Skonnard shows how you can use XPath 1.0 to achieve this in his excellent article, WS-Policy and WSE 2.0 Assertion Handlers.

Mehran Nikoo mentions the new Thames Valley User Group - looks interesting:

If you live/work near Thames Valley Park then this is for you.

The kick-off meeting is at Microsoft Campus on June 21st. Scott Guthrie (co-founder of ASP.net) and Mike Ormond (Microsoft DPE team) are among the speakers.

SecretGeek announces a sexy new methodology - TODO Driven Development [via Mike Gunderloy].  There's a full description of the process (there's even tool support in Visual Studio 2003!) and this at the end:

Does giving it a name make it legitimate? Can I write a book on this and do a lecture tour now?

Too funny. 

My TechEd conference-buddy John Bristowe has a blow-by-blow account of my CTS302 Securing Web Services with WSE 2.0 session at Teched.  Michael Earls has some notes and a couple of photos from the repeat session (which was a little fast because it turned out to be 45 minutes rather than an hour).  Aaron Skonnard mentions my first session in his TechEd trip report on his new PluralSight blog:

Benjamin Mitchell's session on Web services security using WSE was excellent. His was the clearest presentation I've seen on general security concepts along with concrete code examples.

That's going straight to the pool room. 

After covering so many other peoples' talks it feels strange to read coverage of my own talk.

In the last post I showed how it takes only 1 line of code to ensure that a web service client signs all messages with a UsernameToken by creating a send-side policy with the WSE 2.0 Security Settings Tool.  In this post I show the same feat can be achieved with an X509Token without writing a single line of code.  I also show how this functionality powers WSE's support for automatic secure conversation without having to write any code, something that blew me away the first time I saw it.

X509Tokens can be located through Policy and Config
In the last post I covered how the PolicyEnforcementOutputFilter checks the send-side policy when processing output messages through the Pipeline and attempts to find a matching token to fulfil the policy.  In the case of UsernameTokens, this means searching the SoapContext.Security.Tokens collection or looking in the PolicyEnforcementTokenCache (hence the one line of code).  However, with X509Tokens it is possible for WSE to locate the certificate without a single line of code.  The Security Settings Tool allows you to configure which X509 certificate you would like to use and stores an identifier for this key in the policy file.  This information is combined with the the <x509> element in the Microsoft.Web.Services2 config section handler that specifies which certificate store to find the token in.  So the combination of the policy file and the config file gives WSE enough information to find the correct X509 certificate without writing any security-related code within the service.

Policy saves code on the receive-side as well
Policy files can be used to save writing code on the receive-side as well.  On the receive-side the PolicyValidationInputFilter is used to validate that the incoming message meets the assertions defined in the policy file.  The policy file can perform checks such as whether the message is signed and/or encrypted with a specific token type or token as well as whether particular message parts have been signed.  If an incoming message does not satisfy these assertions then a security fault exception is raised before your service code is even executed.  As with send-side policy, the WSE 2.0 Security Settings Tool can help you author this policy, saving you from paying the XML angle bracket tax. 

The samples provided with WSE 2.0 have examples of solutions that rely on code and the same solutions using policy.  Comparing these solutions side-by-side highlights the many benefits of using policy instead of code to perform receive-side validation.  The first is that it keeps your service code much cleaner.  Second, it saves you having to remember to make the same calls at the start of each service.  Third, you can change your security configuration without having to recompile the code.

Putting it all together: automatic secure conversation
The best example I've seen of the power of no-code security through policy and configuration files is the support in WSE 2.0 for automatic secure conversation.  WSE supports the WS-SecureConversation specification that defines a SecurityContextToken that is a fast, light-weight security token that can provide message-level secure communication across multiple calls between a client and a service.  It's fast because it is based on a shared symmetric key, rather than an asymmetric key (which is over 1,000 times slower to process).  WS-SecureConversation builds upon WS-Trust which defines the notion of a Security Token Service that receives RequestSecurityToken messages and returns the issued SecurityContextToken as part of a RequestSecurityTokenResponse message.  WS-SecureConversation uses these mechanisms to request and retrieve the SecurityContextToken.  While all of this may sound a little complicated, it is possible to achieve all of this in WSE using the Security Settings Tool.  Using the ideas presented above, if you use X509Tokens then all of this can be achieved without writing any code.  This is the first demo I showed in my TechEd presentation.

Here's my take on how it performs this magic under the covers (feel free to chime in any time Hervey).  On the send-side, the PolicyEnforcementOutputFilter loads the policy file which specifies that all sent messages must be signed and encrypted with a SecurityContextToken.  I think that WSE makes an assumption that the web service can act as a SecurityTokenService and issue SecurityContextTokens (This is enabled on the service by adding the automaticSecureConversation element to the config file).  So when a SecurityContextToken assertion is found in the policy file WSE loads the SecurityContextTokenManager class and calls the LoadTokenFromSecurityTokenAssertion() method.  This method retrieves the tokens that will be used to sign the request before calling the RequestTokenFromIssuer() method that sends the RequestSecurityToken message and unpacks the SecurityContextToken from the RequestSecurityTokenResponse message sent back from the token issuer (which is often the same location as the service).  The PolicyEnforcmentOutputFilter then uses this SecurityContextToken to sign and encrypt the outgoing messages.

Phew, that certainly was a lot of digging with Reflector.  But it illustrates how powerful policy can be: you can request tokens from a token issuer and use them to sign and encrypt messages without writing a single line of code.  This blew me away the first time I saw it working (I didn't believe it until I saw the wire-level traces).  I pinged John Bristowe and Christian Weyer asking 'how does this work?  It seems like Magic but I know it can't be'.  When I thought about it more I realised that this was a demonstration of the power of the concepts such as aspect oriented programming or the Pipes and Filters pattern from Gregor Hohpe's Enterprise Integration Patterns.  More on this in a future post.

Making more of a good thing: custom policy assertions
As well as using the built-in WS-SecurityPolicy features that WSE enables with its Security Settings Tool, it is also possible to create your own custom policy assertions as John Bristowe has demonstrated.  Aaron Skonnard also has more about custom policy assertions.  WSE has great extensibility hooks that let you write code that uses your own policy assertions, allowing you to write validation code in one location that can be hooked into your service through the config file without having to reference it in your code.

While playing with the WSE Security Settings Wizard I discovered that the generated policy requires a DerivedKeyToken to be used to sign the messages rather than the original security tokens.  This is a good thing, but isn't obvious from the wizard screens.  I thought I'd provide some background on what derived keys are, why they are useful and how to ensure your WSE services use them through code or policy. 

Derived Keys: what are they and why are they useful?
Using a derived key is a good thing as it means a different key is used to sign and/or encrypt each message.  Changing the key each time makes it more difficult to perform a ciphertext-only attack.  The DerivedKey can use many different algorithms to generate the key.  WSE 2.0 uses the algorithm defined in the WS-SecureConversation specification, which creates the derived key by performing a SHA1 hash over the original key (in the case below, a reference to the key), along with the combination of a label and a nonce value (a unique value that's seen only once).  Here's what a DerivedKeyToken based upon a UsernameToken looks like on the wire:

<wssc:DerivedKeyToken wsu:Id="SecurityToken-d06a92f7-990c-43a4-a996-f8f3a359e450" wssc:Algorithm="http://schemas.xmlsoap.org/ws/2004/04/security/sc/dk/p_sha1" xmlns:wssc="http://schemas.xmlsoap.org/ws/2004/04/sc">
   <wsse:SecurityTokenReference>
   <wsse:Reference URI="#SecurityToken-6c059c7a-c748-44b1-b957-4b927dd2a8f3" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#UsernameToken" />
   </wsse:SecurityTokenReference>
   <wssc:Generation>0</wssc:Generation>
   <wssc:Length>16</wssc:Length>
   <wssc:Label>WS-SecureConversation</wssc:Label>
   <wsse:Nonce>mDRVPaA7343zsrMjD+CatA==</wsse:Nonce>
</wssc:DerivedKeyToken>

See Martin Gudgin's MSDN article on Using WS-Trust and WS-SecureConversation for further information.

Ensuring compliance with the WSE generated policy
To comply with the server-side policy that the WSE Security Settings Wizard generates you can either create a derived token in your code or use a send-side policy.  The advantage of using the send-side policy is that it requires fewer lines of code and provides the potential to change the security configuration in future without having to recompile the code.

Signing a message using a DerivedKeyToken with code
Creating a DerivedKeyToken requires two more lines of code than would be required to sign a message with a standard user name token.  The first extra line creates the DerivedKeyToken and the second line adds it to the RequestSoapContext.Security.Tokens collection so that it will appear on the wire when WSE sends the message.  Here's the code:

// Create the username token and add it to the message headers
UsernameToken usernameToken = new UsernameToken("TechedClient", "TechEd2004!", PasswordOption.SendPlainText );
proxy.RequestSoapContext.Security.Tokens.Add( usernameToken );

// Create a derived token, based on our username token
DerivedKeyToken derivedToken = new DerivedKeyToken( usernameToken );

// Add the derived key token to the message headers
proxy.RequestSoapContext.Security.Tokens.Add( derivedToken );

// Sign the message with the derived key token.
proxy.RequestSoapContext.Security.Elements.Add( new MessageSignature( derivedToken ) );

Signing a message using a DerivedKeyToken with send-side Policy
The simplest way to ensure that a message is sent using a DerivedKeyToken based on a UsernameToken is to use the WSE Security Setting Tool to create a policy file on the client specifying that output messages should be signed with a UsernameToken.  When WSE sends the message through its pipeline the PolicyEnforcementOutputFilter will create a DerivedToken based on the UsernameToken and sign the message with it.  For this to work, WSE needs to know where to find the UsernameToken to base the DerivedKeyToken on.  WSE first searches for a matching token in the SoapContext.Security.Tokens collection and if it doesn't find any there it looks in the PolicyEnforcementSecurityTokenCache. 

So you can either use the first two lines of the sample code above, or the following:

// Create the username token and add it to the message headers
UsernameToken usernameToken = new UsernameToken("TechedClient", "TechEd2004!", PasswordOption.SendPlainText );
PolicyEnforcementSecurityTokenCache.GlobalCache.Add( usernameToken );

The advantage of using the PolicyEnforcementSecurityTokenCache is that the username token can more easily be attached to any message leaving the client, regardless of which service is being used (e.g. it is independent of the webservice the message is being sent to, unlike the SoapContext which is related to a specific webservice address).  The second advantage is that if you later decide to sign with X509SecurityTokens rather than UsernameTokens then you wouldn't need to change any code or recompile (the UsernameToken in the GlobalCache would simply not be used).  I'll write more about this topic in a future post.