IanG on Tap

My apologies to everyone who attended my WPF course in London last month - it looks like I forgot to upload the demos. I'm afraid I confused myself by putting the wrong title on the previous week's demos...until today the Silverlight demos from the previous week had a title claiming to be WPF demos. Oops.

So here, at least are the WPF demos from the WPF course I ran last month.

The BBC has a Silverlight deep zoom app as part of the web coverage of the ‘Radio 1 Big Weekend’ event. (“Europe’s biggest free ticketed event” apparently.) If you live in the UK and have watched BBC television at all this week, you’ve probably already seen the adverts for the event itself.

The deep zoom app provides a flavour of the event for people who can’t turn up in person – photos of the event will be uploaded throughout the weekend. (And I guess people who did go could try and find themselves in the crowd shots next week.)

From a technical perspective, it’s nothing you won’t already have seen if you’ve looked at the Hard Rock Cafe’s memorabilia site – indeed, the BBC app is somewhat simpler. The main difference is that the BBC’s photo collection that will grow as the event unfolds – not quite live of course, but a bit less static than the Hard Rock application. (And more raw for it of course, but that’s probably in keeping with the nature of the event.)

Deep Zoom and Photo Collections

What I find interesting is that I like this way of browsing photos much more than I thought I would. Deep zoom is definitely one of the more conspicuously fun features of Silverlight 2, but as it’s ultimately a bit of a one-trick pony, I thought I’d be pretty much done with it after playing with the Hard Rock demo.

But it turns out still to be growing on me long after the initial ‘ooh’ factor. I think it’s a surprisingly natural way to present photographs.

I just went to flickr, and coincidentally enough the first picture it showed me on the front page was a shot from a concert. It’s a good picture, but wow the user interface feels flat! Having got used to the deep zoom style of photo browsing, flickr now feels positively broken.

So I want deep zoom for my online photo collections please!

Nulls can complicate code. One of the lesser known C# language features, the ?? operator, can sometimes help. This operator lets you provide an alternate value to be used in the event that an expression evaluates to null. For example:

string displayName = elem.Name ?? "[Unknown]";

If elem.Name evaluates to anything other than null, then displayName will be elem.Name. But if elem.Name is null, displayName will become "[Unknown]". To get the same result in C# 1 requires the cryptic but useful ternary operator (aka conditional operator) (‘Confusing C family language newbies since 1978’). The result is longer, and requires you to write out the expression twice:

string displayName = elem.Name == null ? "[Unknown]" : elem.Name;

In my experience, most C# developers don’t seem to know about ??. In my Applied WPF course, one of the labs uses it in passing, and I invariably get people either asking me what it does, or occasionally telling me it’s a typo!

People usually like it once they get it, because it’s a clear improvement for this sort of example. So why isn’t it better known? I have a theory: I think it’s less useful than it seems. It only solves one specific example of the ‘nulls are a pain’ class of problems. Most of the time, I want something slightly more powerful than ??.

What I’d Like

While I frequently have to deal with values that might be null, I’ve found that in the non-null case I usually want to work with a member of the object, rather than the object itself. For example, consider this very simple singly-linked list node:

public class Node
{
    public Node Next;
    public string Value;
}

Suppose you have a variable of type Node which may be null, and that you want to return the Value if the variable’s non-null, and return null otherwise. You can’t use the ?? operator here – you have to revert to the good old ternary operator:

Node n = GetNode();
string value = n == null ? null : n.Value;

This is tolerable, but it’s a case that comes up often enough that I can see the value in having a special syntax to handle it. Sadly, the ?? operator is not that syntax – it only seems to cover a small minority of the cases.

A different solution suggests itself when you look at another feature added in C# 2: lifting for nullable types.

Lifting

Lifting in C# is a convenient way of working with values that might be null: rather than crashing when you try and use a null value, the nullness just ends up propagating. For example, C# supports lifting when adding numbers together:

public static int? Add(int? x, int? y)
{
    return x + y;
}

If you call this function with integers, it will add them as expected. But you can pass in null for either argument, in which case the result will be null. So we say that the + operator is lifted. (See Eric Lippert’s description of lifting in C# for more information.)

Not all operators are lifted in C#. The arithmetic ones are, so you can perform numeric calculations with nullable types. But many are not. Member access is not lifted, for example. If it were, I’d be able to replace this:

string value = n == null ? null : n.Value;

with this:

string value = n.Value;

If ‘.’ were lifted, value would end up as null if either n itself were null, or the Value member of the object referred to by n were null.

This seems like a much more succinct way to deal with the problem, but of course that’s not what ‘.’ does in practice. Morever, we cannot overload ‘.’ in C#, so on the face of it, it seems that only the C# compiler team would be able to make this work. However, we can achieve something close by getting the relevant code into expression tree form, and then modifying the tree.

Expression Tree Rewriting

C# can convert an expression into tree of objects. This means we can change how an expression works by simply changing the tree. We could use that to modify the behaviour of ‘.’ to add lifting.

(This idea is pretty similar to LISP’s macro system – code rewriting expressions or generating new ones. There’s a significant difference though: LISP macros run at compile time. As far as I can tell, we don’t have that option with expression trees in C# 3. However, even though we are obliged to wait until runtime to perform the tree rewriting, we do get to compile the result into IL. So after the initial overhead, the performance should be identical to normally compiled code.)

One slight snag is that expression trees are immutable. So you can’t just change them – you end up having to build a new tree. Fortunately, Microsoft provides an example class in the documentation that helps you do this: ExpressionVisitor. This walks every node in an expression tree, and lets you build an edited copy. And it’s smart about unchanged subtrees – it just reuses those rather than copying them. (That’s one of the advantages of immutability – it’s safe to reuse objects like that because you know they’re not going to change.)

The following class derives from the ExpressionVisitor example, and overrides VisitMemberAccess to change the way member access works:

public class NullLiftModifier : ExpressionVisitor
{
  // The method that kicks off the tree walk is
  // protected, so we need to define a public
  // method that provides access to it.
  public Expression Modify(Expression expression)
  {
    return Visit(expression);
  }


  // Change how '.' performs member access.
  protected override Expression VisitMemberAccess(
               MemberExpression originalExpression)
  {
    MemberExpression memberAccessExpression =
     (MemberExpression) base.VisitMemberAccess(originalExpression);


    Expression nullTest = Expression.Equal(
        memberAccessExpression.Expression,
        Expression.Constant(null,
          memberAccessExpression.Expression.Type));
    return Expression.Condition(
        nullTest,
        Expression.Constant(null, memberAccessExpression.Type),
        memberAccessExpression);
  }
}

The call to the base class builds the expression we’re about to rewrite (possibly recursively rewriting some of its sub expressions.) We then generate some extra expressions to test the target object reference for null, and then to return either a null constant or the original expression, depending on whether the target reference is null. Or to put it in code, it changes this:

obj.Member

to this:

obj == null ? null : obj.Member

For ease of use, we can wrap this rewriter in a helper function that takes an expression, rewrites it, and then compiles the result:

static Func<T, TResult> LiftMemberAccessToNull<T, TResult>(
    Expression<Func<T, TResult>> expr)
{
    NullLiftModifier modifier = new NullLiftModifier();
    var x = (Expression<Func<T, TResult>>) modifier.Modify(expr);
    return x.Compile();
}

The compilation here will happen at runtime – Compile turns the expression tree into IL in a dynamically generated method, and returns a delegate pointing to that method.

To test this, I wrote the following, which uses the Node class introduced earlier:

var get0 = LiftMemberAccessToNull((Node f) => f.Value);
var get1 = LiftMemberAccessToNull((Node f) => f.Next.Value);
var get2 = LiftMemberAccessToNull((Node f) => f.Next.Next.Value);
var get3 = LiftMemberAccessToNull((Node f) => f.Next.Next.Next.Value);

So we have four methods, which pick out the value of the current node, the next node, the node after that, and finally the node after that. I’m using the ordinary member access syntax, but I’m calling the code I wrote earlier to add lifting to the ‘.’ operator. To test these methods, I wrote the following (which, just for fun/added confusion, uses the ?? operator to help print out the results):

Node f1 = new Node { Value = "One" };
Node f2 = new Node { Value = "Two" };
Node f3 = new Node { Value = "Three" };
f1.Next = f2; f2.Next = f3;

Node[] foos = { f1, f2, f3, null };
foreach (Node f in foos)
{
  Console.WriteLine("Value: "                + (get0(f) ?? "null"));
  Console.WriteLine("Next.Value: "           + (get1(f) ?? "null"));
  Console.WriteLine("Next.Next.Value: "      + (get2(f) ?? "null"));
  Console.WriteLine("Next.Next.Next.Value: " + (get3(f) ?? "null"));

  Console.WriteLine();
}

This builds a list with three nodes, and iterates through this list, trying all four get methods on each of the nodes.

Normally, you’d expect this to throw a null reference exception. If you have a reference ‘f’ to the first node, the expression f.Next.Next.Next.Value is looking for the fourth node (i.e. three along from the first). There are only three nodes, so that final Next will evaluate to null. Normally you’d expect the attempt to access Value to throw an exception. But we’ve run these expressions through my lifting rewriter, so instead, we see this:

Value: One
Next.Value: Two
Next.Next.Value: Three
Next.Next.Next.Value: null

Value: Two
Next.Value: Three
Next.Next.Value: null
Next.Next.Next.Value: null

Value: Three
Next.Value: null
Next.Next.Value: null
Next.Next.Next.Value: null

Value: null
Next.Value: null
Next.Next.Value: null
Next.Next.Next.Value: null

This illustrates that the lifting is working as desired. This is consistent with the + operator: adding null to a number gives you null; likewise here, when I try to access a member via a null reference I get back null.

Limitations

One big problem with this is that it’s pretty clunky. I have to wrap up the member access in a lambda, and then pass it to my function to get it rewritten and compiled. If I want to make sure I perform that step just the first time my code runs, and yet I still want my lambdas to have access to local variables in scope, the code’s going to get even uglier. So while it might be a neat trick to illustrate the idea, I don’t see myself using this in production code.

This is a shame, because the idea of rewriting expressions is a pretty powerful one. Lifting member access is only one example of what you can do with this technique. It would be great if there were a clean syntax for this sort of idea.

(Also, beware that I’ve not tested this in any serious way – it’s only intended to illustrate an idea. It’s probably full of subtle flaws as well as the more glaring issues. I suspect it doesn’t play nicely with Nullable<T>, for example. Please don’t use it as is in your production code! That said, I offer this code under the MIT license, so feel free to do what you will with it within the terms of the license. Just consider yourself warned. You can grab the source from here: http://www.interact-sw.co.uk/downloads/LiftMemberAccess.zip)

A Monadic Footnote

When I first started to look at this, I had been considering a slightly different approach. The idea of a nullable value (whether it’s a Nullable<T> or just an ordinary reference) is pretty similar to Haskell’s Maybe type. I can use Maybe to write a Haskell equivalent to the Add function I wrote in C# above:

addMaybe :: Maybe Integer -> Maybe Integer -> Maybe Integer
addMaybe x y =
  do
    xVal <- x
    yVal <- y
    return (xVal + yVal)

This is slightly long-winded, because in Haskell, the addition operator is not suitably lifted – it doesn’t recognize Maybe types. (I could arrange for this lifting to happen in Haskell by plugging the code above into the right place and doing some other extra work, incidentally. But this entry’s long enough already.) So I had to write code to explicitly extract the values. A slightly closer C# version might look like this:

// Note: doesn't work properly!
public static int? AddNullable(int? x, int? y)
{
    int xVal = x.Value;
    int yVal = y.Value;
    return xVal + yVal;
}

However, this C# doesn’t work, because we’ve lost the lifting – it’ll throw an exception if you pass in null for either argument. But the Haskell version works fine, as we can see from this interactive GHCi session:

*Main> addMaybe (Just 18) (Just 24)
Just 42
*Main> addMaybe (Just 123) Nothing
Nothing
*Main> addMaybe Nothing (Just 345)
Nothing
*Main> addMaybe Nothing Nothing
Nothing

(Note, ‘Just’ is a constructor for a Maybe instance. So (Just 18) is roughly similar to new Nullable<int>(18) in C#. And Nothing constructs a Maybe that contains no value – it’s more or less like null.)

This works because Maybe belongs to Haskell’s Monad type class. The syntax involving ‘do’ and ‘<-’ is available on monadic types, and in the case of the Maybe monad, the <- operator provides lifted access to the value inside the Maybe – if either x or y turn out to contain Nothing then that whole ‘do’ block will evaluate to Nothing. (See this allegedly ‘gentle’ introduction to monads for more information on monads in Haskell.)

I had been thinking of going down a similar path as a result of reading Wes Dyer’s excellent explanation of monads. He points out that LINQ uses monadic constructs – the standard SelectMany query operator is essentially the monadic ‘bind’ operator by another name. He even implements a C# version of Maybe, and I could perhaps use that to rewrite my C# version roughly like this:

var r = from xVal in x.ToMaybe()
        from yVal in y.ToMaybe()
        select x + y;

Not only does this follow a fairly similar pattern to the Haskell version, it would also offer the same lifting behaviour – it would work correctly in the face of Nothing (null) values.

So it occurred to me that I could use this technique to perform the member access lifting I was looking for. Here’s a class that tries to do this by providing some extension methods that effectively define the monadic ‘bind’ operator for references:

public static class NullableExtensions
{
  public static V Select<T, V>(this T maybeT, Func<T, V> s)
  {
    if (maybeT == null) { return default(V); }
    return s(maybeT);
  }

  public static V SelectMany<T, U, V>(this T maybeT,
                          Func<T, U> k, Func<T, U, V> s)
  {
    if (maybeT == null) { return default(V); }

    U maybeBoundResult = k(maybeT);
    if (maybeBoundResult == null) { return default(V); }
    return s(maybeT, maybeBoundResult);
  }
}

The SelectMany implementation here just bails out early if it hits a null, and returns null without attempting to evaluate anything that follows. (And Select does much the same.) Note: I’ve used the phrase ‘maybe’ here to indicate potentially empty values – I’m not actually using Wes’s Maybe implementation. This particular code snippet stands alone.

Again, I would recommend NOT using this in production code – it effectively adds standard LINQ query operators to System.Object, which is probably not a sane thing to do in a real program. But with just that code (I don’t need any of the code from earlier) I can now write this sort of thing:

int? two = 2;
int? one = 1;
int? none = null;

var twoPlusTwo = from x in two
                 from y in two
                 select x + y;
Console.WriteLine(twoPlusTwo);

var onePlusNothing = from x in one
                     from y in none
                     select x + y;
Console.WriteLine(onePlusNothing);

var nothingPlusTwo = from x in none
                     from y in two
                     select x + y;
Console.WriteLine(nothingPlusTwo);

var onePlusTwoPlusOne = from x in one
                        from y in two
                        from z in one
                        select x + y + z;
Console.WriteLine(onePlusTwoPlusOne);

This prints out results for the calculations that have non-null results, and blank lines for the others. (Console.WriteLine just prints an empty line when given a null.) So lifted use of nullable types is working. But we can also use this to deal with member access through potentially null references.

The simplest example of working with member access wouldn’t work if I had only provided the SelectMany method. I added Select as well was so it would work even if there were only one from clause. That lets me write this sort of thing:

foreach (var item in doc.SelectNodes("//item").Cast<XmlNode>())
{
    var title = from attr in item.Attributes["title"]
                select attr.Value;

    Console.WriteLine(title);
}

This provides a solution of sorts to the member access problem we started with. I’ve run into this particular example on many occasions – I’ve used an XPath query against an XmlDocument and need to deal with the fact that some attributes may be missing on some of the elements that came back. (And yes, I could have written a more selective XPath query in this particular case. But when you’re dealing with lots of optional attributes, that gets tedious.) If the ‘title’ attribute is missing on a particular item, the code won’t hit an exception on the attr.Value expression, because my implementation of Select chooses not to evaluate the select clause at all if attr is null. (And the presence of SelectMany means the same will be true if I use multiple from clauses.)

So it’s a similar result to the expression-tree-based lifting of the ‘.’ operator: I get to write code that attempts to access members via references that may be null without needing an explicit null check. However, the resulting code looks pretty arcane, so again, I don’t think I’d use this in production code. In practice, the problem in this particular example is better solved in C# with an ordinary helper function.

Conclusion

I’ve used some tricks here to implement a couple of variations on my original theme: trying to avoid writing explicit tests for null references when accessing member variables, by lifting member access over null. Unfortunately, neither solution is pretty, so I consider this experiment an interesting failure. And perhaps that’s to be expected – I’m trying to subvert a fundamental feature of the language that was never designed to be modified. If C# had wanted to make it easy, either ‘.’ would have been overloadable, or the compiler would support compile-time hygienic macros.

Would I want to see either of those things in C#? I’m not sure I would – I’ve suffered too much ‘clever’ C++ that tries to redefine language behaviours. To be fair, a lot of the horror there was down to how the clunky macro system in C++ is entirely unrelated to the rest of the language. However, some of the horror was with better integrated features, such as the way you can in fact overload member access in C++.

There’s a lot to be said for a language where code does what it appears to do. It’s normal to spend more time reading code than writing it, and if you need to question fundamentals such as “is that thing that looks like field access really accessing a field?” every time you look at a line of code, that’s going to slow you down massively, and increase the chances of misunderstanding the code.

It reminds me of a troubleshooting consulting engagement I had years ago. A memory leak turned out to be the result of a couple of ‘clever’ behaviours interacting badly. I still remember the growing expression of disbelief on the developer’s face as I stepped into what looked like a single, straightforward statement, but which turned out to involve no fewer than seven implicit function calls! He had no idea it was doing all that, and he’d written it!

If I were going to have a syntax that means “get this thing from that object, unless we have a null reference, in which case return null”, I think I’d prefer a new, specialized syntax over a new behaviour for an established syntax.

So I think I’ll file these under “neat trick; DO NOT USE!”

Thanks to everyone who attended the Applied Silverlight course in London this week. As promised, here are the demos built during the week.

Jon Skeet recently observed that range variables in LINQ expressions are not like other variables. A range variable is one like foobar in the following:

var query = from foobar in source
            where foobar > 20
            select foobar;

Jon points out that the analogy people often draw – the iteration variable in a foreach loop – isn’t that close an analogy. I thought it would be interesting to come up with a concrete illustration of the differences.

Before I start, let me introduce a type:

List<Func<int>>

That’s a list of functions. Each function in the list takes no arguments, and returns an integer. Here’s a method that walks through such a list, invokes each function in turn, and prints the result:

static void PrintValues(List<Func<int>> funcList)
{
    foreach (Func<int> func in funcList)
    {
        Console.WriteLine(func());
    }
}

I’m going to use this to illustrate some differences between LINQ query expression range variables and foreach iteration variables.

Capturing Variables

In order to take a look at these variables, I’m going to capture them in lambdas, so we can poke them about a bit and see what happens. This example iterates through the numbers 1 to 10, building a function that captures the iteration variable each time round:

var source = Enumerable.Range(1, 10);

List<Func<int>> capturedForeachIterationVariables =
    new List<Func<int>>();
foreach (int i in source)
{
    capturedForeachIterationVariables.Add(() => i);
}

That lambda (the “() => i” expression) denotes a function that takes no arguments (hence the empty ()) and returns the value of i. And the i in scope here is the iteration variable of the for each loop.

We can print out the results using the method I showed earlier:

PrintValues(capturedForeachIterationVariables);

And here’s the outcome:

10
10
10
10
10
10
10
10
10
10

This illustrates that there is just the one iteration variable. Our list contains 10 functions, all of which captured that one same iteration variable. That’s why they printed out the final value.

Per Loop Scope

With a slight modification, we can give each lambda its own private variable to capture:

List<Func<int>> capturedForeachLocals =
    new List<Func<int>>();
foreach (int i in source)
{
    int iLocal = i;
    capturedForeachLocals.Add(() => iLocal);
}

While i remains in scope for the entire execution of the loop, iLocal goes out of scope at the end of each iteration. Consequently, each time round the loop, we create a lambda that captures a different iLocal. So when we print the results, we see something different:

1
2
3
4
5
6
7
8
9
10

Capturing a Range Variable

What about range variables? Here’s the rough equivalent

var query = from i in source
            select ((Func<int>) (() => i));

List<Func<int>> capturedRangeVariables =
    query.ToList();

The syntax is a little messy here because C# is unable to infer the lambda type. Remember, lambda expressions don’t have an intrinsic type, so the compiler needs more information. In the previous two examples, the compiler was able to infer what was required from the signature of the List<T>.Add method to which we were passing the lambda. I can’t work out a neat way of giving the compiler what it needs here, so I’ve settled for a rather ugly cast. (I suppose I could also have used a variation on the Funcify function from that recent blog, but it doesn’t look much better to me.)

Anyway, here are the results:

1
2
3
4
5
6
7
8
9
10

So the behaviour is not like the foreach iteration variable – it’s more like the case where we made a copy of the iteration variable. That’s a better analogy, but we can dig a little further.

Modifying the Range Variable

Expressions in LINQ query expressions can sometimes have side-effects. Whether that’s allowed depends on what flavour of LINQ you’re using. We’re using LINQ to Objects here, and that permits it. Here’s a modified version of the previous query expression:

var query = from i in source
            select ((Func<int>) (() => ++i)); 

List<Func<int>> capturedRangeVariables =
    query.ToList();

This increments the captured range variable. Now that won’t interfere with the overall progress of the iteration, because as we already know, the lambda here seems to be working with a copy of i rather than the ‘real’ i. So if we print it out once, we get what you’d probably expect:

2
3
4
5
6
7
8
9
10
11

And if we print it out the list a second time, as you may or may not have been expecting everything moves up by one:

3
4
5
6
7
8
9
10
11
12

(Note, I’m only evaluating the query once – that happens when I call ToList. And then I’m printing out the resulting list multiple times, and the numbers go up by one each time. If I were to re-evaluate the query, that’d be different. It would build a brand new set of functions that would start back from 2 again.)

We can represent this as a foreach:

foreach (int i in source)
{
    int iLocal = i;
    capturedForeachLocals.Add(() => ++iLocal);
}

The behaviour is the same. Again, this suggests that LINQ query expression range variables work more like a copy of a foreach iteration variable than the iteration variable itself. (Incidentally, if you were to try and use i++ in this foreach loop, it wouldn’t even compile. C# prevents you from modifying the iteration variable – another way in which iteration variables are different from range variables.)

But we can go further still.

Multiple References to the Range Variable

LINQ query expressions can contain many clauses, each of which will often want to access the range variable. This means I can build a query that attempts to modify the range variable in one clause before going on to use it again in a later clause:

var query = from i in source
            where ++i < 5
            select ((Func<int>) (() => i));

List<Func<int>> capturedRangeVariables =
    query.ToList();

On the face of it, this has a where clause that increments the range variable, and then only lets through items where the result is less than 5. So we’d expect only to see three items in the output. But what do we expect those output values to be? It would be reasonable expect to that the i in the select clause will have the incremented value – after all, it looks like the same variable, i, that got incremented in the line above. And yet the output is as follows:

1
2
3

That is, the i captured in the select clause does not seem to have been affected by the increment operator applied to the i in the where clause. Clearly the ++ has had an effect – we’ve only got three numbers out, and we would have expected four if that ++ were not present.

So it’s as though the two clauses are working on different variables, despite being in the same scope and using the same name. The foreach equivalent looks like this:

foreach (int i in source)
{
    int iLocal1 = i;
    if (++iLocal1 < 5)
    {
        int iLocal2 = i;
        capturedForeachLocals.Add(() => iLocal2);
    }
}

Phantom Variables

So query expression range variables clearly are quite different from foreach iteration variables. Each clause in a query expression that refers to the range variable gets its own local variable, so while it may look like one variable, it’s not. And as Jon points out the range variable never really exists in the compiled result – once a query expression is compiled, the only tangible variables in the code are the local copies. And you can see why by manually expanding any of these query expressions out into their equivalent method call forms: you’ll see that the range variable vanishes as part of this transformation.

So the range variable is really just a label that directs the transformation of a query expression into code at compile time. Once it has done this job, it vanishes.

Daniel Cazzulino recently wrote a blog entry whose main focus was on building pipelines using iterators in C#. Towards the end he showed a slightly irritating problem in C# 3.0. He wanted to write this:

var transformer = x => new { Original = x, Normalized = x.ToLower() };

However, the C# compiler complains because it doesn’t have enough information to infer the type of the transformer variable. The problem it reports is “Cannot assign lambda expression to an implicitly-typed local variable”.

Daniel doesn’t present a working solution to this particular problem – he ends up structuring his program differently to avoid the issue entirely. But in his discussion of this problem, he proposes something that he describes as ugly, and which, as he points out, doesn’t work anyway:

Func<string, {string Original, string Normalized}> transformer =
   x => new { Original = x, Normalized = x.ToLower() };

This is a direct approach to the problem described in the compiler error message. Can’t assign the expression to an implicitly-typed variable? OK, let’s make the variable explicitly typed. Unfortunately, you can’t specify the type because the expression involves an anonymous type. And that’s the thing about anonymous types: they don’t have names.

Daniel has made up a plausible syntax for denoting unnamed types: the first part in {} shows a possible solution. But that was just a hypothetical suggestion on his part – in fact C# 3.0 doesn’t provide a way to denote anonymous types.

But there is a solution. We carry on using an implicitly-typed variable, and fix the expression instead. The error says we can’t use a lambda expression. OK – let’s use something else. And as you’ll see, we don’t even have to get rid of the lambda. We can just wrap it in something.

Why Can’t I Use a Lambda?

To decide what to use in place of the raw lambda expression, we need to understand why C# isn’t letting us use a lambda here. The problem is that lambda expressions don’t have an intrinsic type. They’re more adaptable than that – a given lambda expression could mean several different things. The most straightforward demonstration of this is that the exact same expression could evaluate to either a delegate or an expression tree.

The compiler uses the context to determine the type. If you’re trying to assign or pass the lambda to something expecting a delegate, it’ll turn into a delegate. But if the target expects an expression tree, you’ll get one of those instead.

In short, a lambda’s expression’s evaluated type is determined by the context in which it’s used.

This gives us a problem with var: var says that its type is determined by the expression with which it is initialized. So when var meets a lambda, it’s like the two are stood by a doorway, each politely saying “No, after you”. (Apologies to my international readers – I suspect that might be a slightly over-British analogy...)

Something needs to decide the type. But the obvious approaches (ordinary explicitly-typed variables, or casts) end up hitting the problem Daniel hit: you can’t denote an anonymous type, so how do you specify the type? But it turns out that you don’t need to specify the type completely. All you really need to do is provide just enough information to let the C# compiler proceed.

Providing Just Enough Type Information

We can narrow down the C# compiler’s options without fully specifying the type. The trick is to go via a function call. Consider this apparently pointless function:

public static Func<T, TResult> Funcify<T, TResult>(Func<T, TResult> f)
{ return f; }

What use would that be, you may wonder? It takes any single-parameter function and returns that function without doing anything with or to it!

However, this provides the C# compiler with a little more information. If I pass a lambda as a parameter to Funcify, I’ve closed down an option: the C# compiler knows that whatever it produces needs to be a Func – it no longer has the option to produce an expression tree. This might be enough to disambiguate matters. Unfortunately, it’s not quite enough to fix our example, as we’ll see when we try it:

var transformer = Funcify(
      x => new { Original = x, Normalized = x.ToLower() });

This gets rid of the previous error, but replaces it with a new one:

“The type arguments for method 'Funcit.Program.Funcify<T,TResult>(System.Func<T,TResult>)' cannot be inferred from the usage. Try specifying the type arguments explicitly.”

But that’s progress. Honest! The compiler is no longer telling us that we can’t do this with var. It’s now telling us that it doesn’t have quite enough information.

The problem here is that it doesn’t know what ‘x’ is. And how could it? Any type with a ToLower method would do. The intent here is for it to be a string, so we just need to express that:

var transformer = Funcify(
      (string x) => new { Original = x, Normalized = x.ToLower() });

And now we’re good. The compiler is happy, because we gave just enough information to pin things down and it was able to infer the rest.

Mumble Types

I’ve seen in some C# team members’ blogs a suggestion for something they call ‘mumble types’. If I’ve understood them correctly, the idea is that you can provide a type specification, but you get to ‘mumble’ (i.e. be unclear) about parts of the type specification. This would give us an alternative solution:

Func<string, ?> transformer =
      x => new { Original = x, Normalized = x.ToLower() };

This doesn’t work today, but I would prefer this solution if it were available. It does exactly what I did – it lets me specify just enough to let the compiler’s type inference finish the job – but it doesn’t require the questionable hack of my Funcify function. And I think it’s clearer – with my example, it’s not particularly obvious that I was trying to fix some parts of the type specification while leaving other parts to be inferred. But with this ‘mumble type’ approach, it’s a lot more obvious that I’m saying ‘I want it to be a Func, and I want the input to be a string, but I want the compiler to infer the Func’s return type’. Also, this approach avoids the clutter of specifying the string type in the lambda parameter list itself, so I find it somewhat more aesthetically pleasing.

The use of the ‘?’ is based on what I’ve seen in C# team members’ blogs in the past year or two. However, here’s a different syntax to consider:

Func<string, var> transformer =
      x => new { Original = x, Normalized = x.ToLower() };

We already use the keyword var to say ‘the compiler should deduce this type’ for variable declarations. So I think it would be consistent to use the same keyword to mean the same thing in partially specified generic types too. That said, the ‘?’ has the benefit of standing out. And it seems reminiscent of Haskell’s use of ‘_’ to mean ‘whatever’. So I could see either approach, but I think I slightly prefer the idea of using var.

Gratuitous Plug for Training in London

And finally, some advertising. I’m teaching a couple of training courses for Pluralsight in London soon. I’ll be running the Silverlight course that Fritz Onion and I co-author at Old Street from 31st March to 3rd April. And I’ll be teaching our WPF course at the same location the following week – from 7th to 10th April.

I recently ran into what appears to be a bug in Silverlight 2 Beta 1’s handling of cross-domain web service access when using a clientaccesspolicy.xml file. I’m hoping this post might save a few other people the time it took me to work out what was going on here.

Here’s the executive summary: if the web service exposes resources whose URIs contain semicolons, you will not be able to access those resources cross-domain if you’re using clientaccesspolicy.xml. The workaround is to use crossdomain.xml instead.

Now for the more detailed version.

In case you’re not familiar with cross-domain web service access, here’s the basic idea. By default, a web browser won’t let client-side code go connecting to any old web site. Client-side code is allowed to make requests against the web site from which it was originally downloaded, and it should only have access to other sites if those sites opt in.

In pure AJAX sites, this is often achieved using a faintly smelly hack where web services return runnable script rather than simple data. Flash introduced a somewhat more formal mechanism by which a web site can declare that it’s happy to be accessed by client-side code from other domains. Silverlight now supports this feature as of v2 beta 1.

Here’s an example. If your web site offers a resource called /crossdomain.xml containing this:

<!DOCTYPE cross-domain-policy SYSTEM
   "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd">
<cross-domain-policy>
  <allow-access-from domain="*" />
</cross-domain-policy>

the site is declaring that it is open to anyone.

This policy is enforced on the client side by whatever runtime is performing the cross-domain network activity. Both Flash and Silverlight 2 beta 1 will look for this file when code in these runtimes attempts to access a web site from another domain. (This doesn’t help you from JavaScript, because today, no browser script engine supports this mechanism.)

One problem with this is that it’s a bit of a blunt instrument. You can be selective about who is allowed to access your site – you don’t have to use a wildcard. But you can’t be selective about what they can access. E.g., I might like to say that only content beneath a particular URI should be accessible to client code that originated from external domains.

Selective Cross-Domain Access: clientaccesspolicy.xml

As well as supporting Flash’s mechanism, Silverlight 2 beta 1 also introduces a more selective form of cross-domain policy. Before looking for a Flash-style crossdomain.xml resource, Silverlight will look for a clientaccesspolicy.xml file. This has a slightly more complex structure that enables you to constrain what a client can do. Here’s an example:

<access-policy>
  <cross-domain-access>
    <policy>
      <allow-from>
        <domain uri="*"/>
      </allow-from>
      <grant-to>
        <resource path="/extsvc/" include-subpaths="true"/>
      </grant-to>
    </policy>
  </cross-domain-access>
</access-policy>

This says that clients from any external domain are granted cross-domain access, but it limits it to resources that start with /extsvc/.

This is a little more flexible than the Flash approach – it lets you make only selected parts of your site available. With crossdomain.xml, you would need to expose services under a distinct domain name to get this selectivity. Obviously that’s not rocket science – it’s certainly possible to present various services under various different domain names. So Silverlight’s approach doesn’t exactly enable anything that was impossible under Flash. On the other hand, this is a whole lot more convenient for people whose web hosting arrangements might not make it so easy to crank up new domain names for each set of services requiring distinct client access policies. And editing a text file is usually a more lightweight process than partitioning a web site’s services across multiple domains.

Semicolons Not Welcome in Beta 1

Unfortunately, there seems to be a problem in beta 1 of Silverlight. If the service you wish to access has a semicolon in its URI, Silverlight won’t allow your client-side code to access it, no matter what the clientaccesspolicy.xml file says. This is unfortunate if you’re using a service written the way advocated in RESTful Web Services, which recommends delimiting order-independent parts of a URL with semicolons. And the service I was using did exactly that. It took me a while to work out that it was specifically the semicolons that were causing problems, hence this post.

This is presumably just a bug in the code that works out whether any particular URI is permitted by the policy. Since the simpler crossdomain.xml grants all-or-nothing access, it never has to examine the URI, so it doesn’t have this problem. That’s why the simpler but less flexible crossdomain.xml offers a workaround here.

Learn More about Silverlight 2

If you want to learn more about Silverlight, I’ll be teaching the Silverlight course that Fritz Onion and I co-author for Pluralsight in London in a couple of weeks. The 4 day course will be running at Old Street from 31st March to 3rd April. (And I’ll be teaching our WPF course at the same location the following week.)

I just finished the Cross Platform .NET on Silverlight talk at DevWeek. Demos can be downloaded from http://www.interact-sw.co.uk/downloads/DevWeek2008XPlatDemos.zip

I'm all done at DevWeek for this year. But if you want to hear more about Silverlight, I'll be teaching Pluralsight's Applied Silverlight course in London later this month - running from 31st March. (And the following week I'll be teaching our Applied WPF course, also in London.)

Fritz Onion and I just finished the pre-conference 'Day of Silverlight' talk at DevWeek. Demos can be downloaded from http://www.pluralsight.com/fritz/demos/DevWeek2008DayOfSilverlightDemos.zip

WPF’s MediaElement makes simple media playback pretty straightforward, but moving beyond the simple scenarios can sometimes raise surprising challenges. For example, I recently saw someone tripped up by the MediaElement when attempting to play several sounds concurrently.

As you’ll see, one solution would have been to use MediaPlayer instead of MediaElement. The difference between these WPF classes is fairly straightforward. MediaPlayer is the class that knows how to play media files – both video and audio. MediaElement is a wrapper around MediaPlayer that provides a simple way to connect it into a visual tree (i.e. a user interface), which in turn lets us hook it into things like the animation system or event triggers.

(Note: do not be misled by the class name. Although WPF and Windows Media Player depend on the same infrastructure for media decoding, the MediaPlayer class is not a wrapper around the Windows Media Player control. While they share codecs, the path by which decoded video gets onto the screen in WPF is significantly different from Windows Media Player.)

How would that get you into trouble when using MediaElement? If it’s a wrapper around MediaPlayer, surely you could use a MediaElement any place a MediaPlayer would work? In fact it’s not always that simple. To see why, we’ll start with a simple example.

One MediaElement

The simplest way to use MediaElement is to add it to a UI and point it at a media file:

<Window x:Class="MediaPlayback.Window1"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="Window1" Height="300" Width="300">

  <MediaElement Source="file:///c:/windows/media/tada.wav" />

</Window>

This will play the file soon as the UI loads. If you want a bit more control, you can tell it to wait until you’re ready:

<MediaElement x:Name="audioPlayer"
              Source="file:///c:/windows/media/tada.wav"
              LoadedBehavior="Manual" />

It’ll now hold off until you call audioPlayer.Play().

This approach is also often sufficient for playing multiple different sounds. you can change the Source property and call Play again. However, if you want to play multiple sounds simultaneously, this approach doesn’t work – setting the Source will stop playback if it is in progress. A single MediaElement or MediaPlayer can only play one thing at a time.

That’s OK, because we can always create multiple MediaElements.

Multiple MediaElements

Modifying the example above simply by adding multiple MediaElements to the Window will stop the Xaml from compiling, because Window can have only a single direct descendant. So we need to find something to hold the MediaElements. And this is where the example I saw tripped up: the developer put them into the UI’s Resources section.

On the face of it, this was a perfectly reasonable thing to do – the elements are all playing audio, so it doesn’t seem like they should need to be part of the visual tree, so why not make them resources? After all, WPF’s resource mechanism is designed to hold useful objects, right?

Well this is where the difference between MediaPlayer and MediaElement becomes important. Remember, the distinction is that MediaElement connects media playback into a visual tree. And it turns out that until it makes that connection, MediaElement won’t play the media. That makes sense for video – you don’t want that to start playing before you can see it. But while you might think a connection with the visual tree would be optional for audio, MediaElement sees it differently. (And there are reasons for that. For example, MediaElement can synchronize media playback with timelines of animations in the visual tree.)

So in this case, the extra functionality provided by the wrapper has worked against us.

One solution is simply to give the MediaElement what it wants. As long as we put it into the visual tree, it’s happy. So we can put the elements into a layout panel such as a Grid or Canvas:

<Canvas>
  <MediaElement x:Name="mediaElem1"
       Source="file:///c:/windows/media/tada.wav"
       LoadedBehavior="Manual" />
  <MediaElement x:Name="mediaElem2"
       Source="file:///c:/windows/media/Windows Logoff Sound.wav"
       LoadedBehavior="Manual" />
</Canvas>

The other approach is to go straight for the MediaPlayer – if we have no need for the visual tree integration features MediaElement offers, we may as well go straight to the underlying player. The only snag is that you can’t initialize MediaPlayer from Xaml – you must use the Open method to point it at the media file, and Xaml doesn’t do method calls. But it’s not a huge amount of effort:

MediaPlayer mp = new MediaPlayer();
mp.Open(new Uri(wavPath));
mp.Play();

That is all the code required; we don’t need anything at all in the Xaml. And to play multiple simultaneous sounds, you can simply create multiple MediaPlayers.