The Wayback Machine - https://web.archive.org/web/20110628234130/http://blog.abodit.com:80/category/programming/net/

.NET

Class-free persistence and multiple inheritance in C# with MongoDB

Much as I appreciate Object Relational Mappers and the C# type system there’s a lot of work to do if you just want create and persist a few objects. MongoDB alleviates a lot of that work with its Bson serialization code that converts almost any object into a binary serialized object notation and provides easy round tripping with JSON.

But there’s no getting around the limitations of C# when it comes to multiple inheritance. You can use interfaces to get most of the benefits of multiple inheritance but implementing a tangled set of classes with multiple interfaces on them can lead to a lot of duplicate code.

What if there was a way to do multiple inheritance without every having to write a class? What if we could simply declare a few interfaces and then ask for an object that implements all of them and a way to persist it to disk and get it back? What if we could later take one of those objects and add another interface to it? “Crazy talk” I hear you say!

Well, maybe not so crazy … take a look at the open source project impromptu-interface and you’ll see some of what you’ll need to make this reality. It can take a .NET dynamic object and turn it into an object that implements a specific interface.

Combine that with a simple MongoDB document store and some cunning logic to link the two together and voila, we have persistent objects that can implement any interface dynamically and there’s absolutely no classes in sight anywhere!

Let’s take a look at it in use and then I’ll explain how it works. First, let’s define a few interfaces:

    public interface ILegs
    {
        int Legs { get; set; }
    }

    public interface IMammal
    {
        double BodyTemperatureCelcius { get; set; }
    }

    // Interfaces can use multiple inheritance:

    public interface IHuman: IMammal, ILegs
    {
        string Name { get; set; }
    }

    // We can have interfaces that apply to specific instances of a class: not all humans are carnivores

    public interface ICarnivore
    {
        string Prey { get; set; }
    }

Now let’s take a look at some code to create a few of these new dynamic documents and treat them as implementors of those interfaces. First we need a MongoDB connection:

            MongoServer MongoServer = MongoServer.Create(ConnectionString);
            MongoDatabase mongoDatabase = MongoServer.GetDatabase("Remember", credentials);

Next we grab a collection where we will persist our objects.

            var sampleCollection = mongoDatabase.GetCollection<SimpleDocument>("Sample");

Now we can create some objects adding interfaces to them dynamically and we get to use those strongly typed interfaces to set properties on them.

            var person1 = new SimpleDocument();
            person1.AddLike<IHuman>().Name = "John";
            person1.AddLike<ILegs>().Legs = 2;
            person1.AddLike<ICarniovore>().Prey = "Cattle";
            sampleCollection.Save(person1);

            var monkey1 = new SimpleDocument();
            monkey1.AddLike<IMammal>();                 // mark as a mammal
            monkey1.AddLike<ILegs>().Legs = 2;
            monkey1.AddLike<ICarniovore>().Prey = "Bugs";
            sampleCollection.Save(monkey1);

Yes, that’s it! That’s all we needed to do to create persisted objects that implement any collection of interfaces. Note how the IHuman is also an IMammal because our code will also support inheritance amongst interfaces. We can load them back in from MongoDB and get the strongly typed versions of them by using .AsLike() which returns a value of type T or null if that object doesn’t implement the interface T. But that’s not all, we can even add new interfaces to them later allowing an object to change type over time! Now, of course, you could do a lot of this just with dynamic types but then you lose Intellisense and compile time checking.

So next, let’s take a look at how we can query for objects that support a given interface and how we can get strongly typed objects back from MongoDB:

            var query = Query.EQ("int", typeof(IHuman).Name);
            var humans = sampleCollection.Find(query);

            Console.WriteLine("Examine the raw documents");

            foreach (var doc in humans)
            {
                Console.WriteLine(doc.ToJson());
            }

            Console.WriteLine("Use query results strongly typed");

            foreach (IHuman human in humans.Select(m => m.AsLike<IHuman>()))
            {
                Console.WriteLine(human.Name);
            }

            Console.ReadKey();

So how does this ‘magic’ work? First we need a simple Document class. It can be any old object class, no special requirements. At the moment it does wrap these interface properties up in a document inside it called ‘prop’ making it just a little bit harder to query and index but still fairly easy.

        /// <summary>
        /// A very simple document object
        /// </summary>
        public class SimpleDocument : DynamicObject
        {
            public ObjectId Id { get; set; }

            // All other properties are added dynamically and stored wrapped in another Document
            [BsonElement("prop")]
            protected BsonDocument properties = new BsonDocument();

            /// <summary>
            /// Interfaces that have been added to this object
            /// </summary>
            [BsonElement("int")]
            protected HashSet<string> interfaces = new HashSet<string>();

            /// <summary>
            /// Add support for an interface to this document if it doesn't already have it
            /// </summary>
            public T AddLike<T>()
                where T:class
            {
                interfaces.Add(typeof(T).Name);
                foreach (var @interface in typeof(T).GetInterfaces())
                    interfaces.Add(@interface.Name);
                return Impromptu.ActLike<T>(new Proxy(this.properties));
            }

            /// <summary>
            /// Cast this object to an interface only if it has previously been created as one of that kind
            /// </summary>
            public T AsLike<T>()
                where T : class
            {
                if (!this.interfaces.Contains(typeof(T).Name)) return null;
                else return Impromptu.ActLike<T>(new Proxy(this.properties));
            }

        }

Then we need a simple proxy object to wrap up the properties as a dynamic object that we can feed to Impromptu:

        public class Proxy : DynamicObject
        {
            public BsonDocument document { get; set; }

            public Proxy(BsonDocument document)
            {
                this.document = document;
            }
            public override bool TryGetMember(GetMemberBinder binder, out object result)
            {
                BsonValue res = null;
                this.document.TryGetValue(binder.Name, out res);
                result = res.RawValue;
                return true;            // We always support a member even if we don't have it in the dictionary
            }

            /// <summary>
            /// Set a property (e.g. person1.Name = "Smith")
            /// </summary>
            public override bool TrySetMember(SetMemberBinder binder, object value)
            {
                this.document.Add(binder.Name, BsonValue.Create(value));
                return true;
            }
        }

And that’s it! There is no other code required. Multiple-inheritance and code-free persistent objects are now a reality! All you need to do is design some interfaces and objects spring magically to life and get persisted easily.

[NOTE: This is experimental code: it's a prototype of an idea that's been bugging me for some time as I look at how to meld Semantic Web classes which have multiple inheritance relationships with C# classes (that don't) and with MongoDB's document-centric storage format. Does everything really have to be stored in a triple-store or is there some hybrid where objects can be stored with their properties and triple-store statements can be reserved for more complex relationships? Can we get semantic web objects back as meaningful C# objects with strongly typed properties on them? It's an interesting challenge and this approach appears to have some merit as a way to solve it.]

Random names in C# using LINQ and a touch of functional programming

Today I needed some random names for testing. I wanted them to look like real names but to also look obviously made up so that nobody would ever treat them as a real record in the database. Below is a quick C# snippet to generate random names. It uses LINQ and a couple of anonymous functions to accomplish a task that would have taken many more lines in a traditional procedural style. Enumerable.Range() in particular is a really handy method for generating stuff.

Random r = new Random();
string alphabet = "abcdefghijklmnopqrstuvwyxzeeeiouea";
Func<char> randomLetter = () => alphabet[r.Next(alphabet.Length)];
Func<int, string> makeName =
  (length) => new string(Enumerable.Range(0, length)
     .Select(x => x==0 ? char.ToUpper(randomLetter()) : randomLetter())
     .ToArray());
string first = makeName(r.Next(5) + 5);
string last = makeName(r.Next(7) + 7);
string company = makeName(r.Next(7) + 7) + " Inc.";

How to get CrmSvcUtil.exe to work with Microsoft Dynamics CRM 2011 online

You’d think this would be easy – just download the SDK and run the utility, right? Sadly that’s not the case and the information to make it work is scattered around the web.

Here are the steps I’ve pieced together from various forum answers and some trial and error.

1. Install Microsoft WIF SDK
You can get it here: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=eb9c345f-e830-40b8-a5fe-ae7a864c4d76&displaylang=en

Hint: For Windows 7 and Windows Server 2008 R2, select the msu file with name starting Windows6.1. [Naming it Windows7and2008 would have been too easy I guess.]

2. Install Microsoft Dynamics CRM 2011 SDK

You can get it here: http://www.microsoft.com/downloads/en/confirmation.aspx?FamilyID=420f0f05-c226-4194-b7e1-f23ceaa83b69

3. Run the DeviceRegistration.exe utility to generate a device id and password

You can find it in the SDK Tools / DeviceRegistration directory.  Run it with command line /Operation:Register

Copy the values for device ID and device password, you’ll need them later

4. Now run CRMSVCUTIL from the MSCRM SDK under the bin directory (not the tools directory)

If you want to copy it to, say, your Utilities directory you’ll need to take all the DLLs with it.   [Someone was apparently too lazy to run ILMerge on it.]

The parameters you’ll need are:

crmsvcutil
   /url:https://<<<Organization>>>.crm4.dynamics.com/XRMServices/2011/Organization.svc
   /o:crm.cs
   /n:<<<Desired namespace name>>>
   /u:<<< your windows live id >>>
   /p:<<< your windows live password >>>
   /serviceContextName:<<<Desired service context name>>>
   /di:<<< Device ID >>>
   /dp: <<< Device Password >>>

5. That’s it!  You should have a file crm.cs that you can use in your Visual Studio Project to interface to MS-CRM.

I just wish it was one step!

6. To add this to your Visual Studio C# Project

Copy the crm.cs file to your solution, add a reference to System.Runtime.Serialization.

Add a /lib folder to your solution and copy the SDK /bin directory into it

Add a reference to the DLLs in there as necessary: Microsoft.XRM.sdk in particular.

Add a reference to System.ServiceModel.

Algorithm Complexity and the ‘Which one will be faster?’ question

Over and over on Stackoverflow.com people ask a question of the form ‘If I have an algorithm of complexity A, e.g. O(n.log(n)), and an algorithm of complexity B, which one will be faster?’ Developers obsess over using the cleverest algorithm available with the lowest theoretical bound on complexity thinking that they will make their code run faster by doing so. To many developers the simple array scan is anathema, they think they should always use a SortedCollection with binary search, or some heap or tree that can lower the complexity from O(n squared) to O(n.log(n)).

To all these developers I say “You cannot use the complexity of an algorithm to tell you which algorithm is fastest on typical data sets on modern CPUs”.

I make that statement from bitter experience having found and implemented a search that could search vast quantities of data in a time proportional to the size of the key being used to search and not to the size of the data being searched. This clever suffix-tree approach to searching was beaten by a simple dumb array scan! How did that happen?

First you need to understand that there are constants involved. An algorithm might be O(n) but the actual instruction execution time for each step might be A microseconds. Another algorithm of O(n.log(n)) might take B microseconds to execute each step. If B >> A then for many values of n the first algorithm will be faster. Each algorithm may also involve a certain set up time and that too can dwarf execution time when n is small.

But here’s the catch: even if you think the number of add and multiply instructions between the two algorithms is similar, the CPU you are executing them on may have a very different point of view because modern x86 CPUs have in effect been optimized for dumb programmers. They are incredibly fast at processing sequential memory locations in tight loops that can fit in on-chip RAM. Give them a much more complex tree-based O(n log(n)) algorithm and they now have to go off-chip and access scattered memory locations. The results are sometimes quite surprising and can quickly push that O(n log(n)) algorithm out of contention for values of n less than several million.

For most practical algorithms running on typical datasets the only way to be sure that algorithm A is faster than algorithm B is to profile it.

The other huge catch is that even when you have profiled it and found that your complex algorithm is faster, that’s not the end of the story. Modern CPUs are now so fast that memory bottlenecks are often the real problem. If your clever algorithm uses more memory bandwidth than the dumb algorithm it may well affect other threads running on the same CPU. If it consumes so much memory that paging comes into play then any advantage it had in CPU cycles has evaporated entirely.

An example I saw once involved someone trying to save some CPU cycles by storing the result in a private member variable. Adding that member variable made the class larger and as a result less copies of it (and all other classes) could fit in memory and as a result the application overall ran slower. Sure the developer could claim that his method was now 50% faster than before but the net effect was a deterioration in the overall system performance.

MongoDB C# Driver – arrays, lists and hashsets

Here’s a nice feature of the C# MongoDB driver: when you save .NET arrays, lists or Hashsets (essentially an IEnumerable<T>) to MongoDB you can retrieve it as any other IEnumerable<T>. This means you can migrate your business objects between these different representations without having to migrate anything in your database. It also means that any other language can access the same MongoDB database without needing to know anything about .NET data types.

For example, the following will all serialize to the same BSon data and any can be retrieved.

        public class Test1
        {
            [BsonId]
            public ObjectId Id { get; set; }
            public List<string> array { get; set; }
        }

        public class Test2
        {
            [BsonId]
            public ObjectId Id { get; set; }
            public string[] array { get; set; }
        }

        public class Test3
        {
            [BsonId]
            public ObjectId Id { get; set; }
            public HashSet<string> array { get; set; }
        }

MongoDB Map-Reduce – Hints and Tips

For anyone getting started with Map-Reduce on MongoDB here are a few pointers to get you started.

1. Guids are not a good choice for MongoDB identifiers: use the provided ObjectId instead.

Guids in javascript compare as binary objects and thus don’t work well as keys for Map-Reduce operations.

2. You can’t use an Array as the return type for the reduce operation.

This is actually documented clearly on the site for those of you that actually read the documentation before trying to get it to work but for everyone else it’s going to cause some frustration.

3. The output value emitted by the map function MUST be the same format as the value returned by reduce.

The documentation on this one says it ‘should’ be the same, but in practice anything but the same format is bound to cause problems. What you need to understand is that ‘map-reduce’ is somewhat of a misnomer, the reduce function may be called iteratively and that doesn’t work unless each reduce operation can have its results fed back in to another reduce operation.

4. Using .length on the values array passed into the reduce function is never the right thing to do

In your map operation you often output a value of ’1′ for each key. In the reduce operation you want to add up these ’1′s. It looks like you could use value.length to get the result. But, here too the iterative nature of the reduce operation means that you actually need to examine the values in the array passed in and accumulate them.

5. The print() function provides for some limited debugging assistance

When you need to see some intermediate results in your map or reduce functions the print() function can help.

6. The Reduce function automatically includes the key and the return value

When deserializing the results in C# you’ll want to deserialize them into a type that has an ‘_id’ property and a ‘value’ property. The following generic type can help:

    /// <summary>
    /// This is a useful type for dealing with MapReduce results.
    /// It takes two type parameters, one for the key and one for the value.
    /// The simplest possible result would use type parameters ObjectId and int
    /// </summary>
    public class MapReduceResult2<Tid, Tvalue>
    {
        public Tid _id { get; set; }
        public Tvalue value { get; set; }

        public override string ToString()
        {
            return string.Format("{0} {1}", _id, value);
        }
    }

7. Chaining Map-Reduce operations to get the result you need is quite normal

If you can’t see how to get to what you want in a single Map-Reduce cycle don’t worry, it’s quite easy and normal to pass the results of one Map-Reduce operation to the next.

Exception: An object with the same key already exists in the ObjectStateManager. The ObjectStateManager cannot track multiple objects with the same key.

With the Entity Framework 4 it’s relatively easy to get this exception but it can be hard to figure out why it’s happening and how to fix it.

An object with the same key already exists in the ObjectStateManager. The ObjectStateManager cannot track multiple objects with the same key.

One of the causes of this problem is a complex entity graph with multiple paths between two objects.

One simple trick that can work is to use the Foreign Key id values to establish new relationships instead of setting the actual Entity objects on a relation.

For example, instead of this

            Mention statement = new Mention
            {
                DateTimeMentioned = dateTimeMentioned,
                Collection = collection,
                Item = item,
                PhysicalEntity = thing
            };
            this.Mentions.AddObject(statement);

Do this …

            Mention statement = new Mention
            {
                DateTimeMentioned = dateTimeMentioned,
                CollectionId = collection.Id,
                ItemId = item.Id,
                PhysicalEntityId = thing.Id
            };
            this.Mentions.AddObject(statement);

Constrained parallelism for the Task Parallel Library

When developing .NET applications there is often the need to execute multiple background processes, for example, fetching and rendering different size thumbnails for images. Typically you queue actions like these onto the thread pool. But in the case of thumbnail generation you typically want to fetch a base image first and then perform the resize operations on it. If five web pages each request a different thumbnail size simultaneously you may end up fetching the same image five times before processing it. Of course, you can add file based locking around this to ensure that only the first once gets to fetch the data but it would be much better if you could instead instruct the Task Parallel Library to execute co-dependent tasks sequentially.

The new Task parallel library has continuations that allow one task to chain onto the end of a previous task but you still a way to track all the tasks currently active so you can find the other task to chain onto it. In a multi-threaded asp.net environment that’s not so easy.

Below is a TaskFactory that gives you constrained parallelism allowing you to queue up tasks in such a way that no two tasks with the same key will execute in parallel. To use it you simply create a new TaskFactorySequentiallyByKey and then call StartNewChainByKey() with a suitable key, e.g. “RENDERimage12345.jpg”. This method returns a normal Task object that you can Wait on or add more continuations. All the usual TaskFactory constructor options are provided so you can have a different TaskScheduler, common cancellation token, and other options.

Note also that it expects an Action<CancellationToken> not just a plain Action. This is so your Action can be polite and monitor the cancellation token to know when to stop early. If you don’t need that you can always pass in a closure that tosses the CancellationToken, i.e. (token) => MyAction().

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Threading;
using System.Diagnostics;

namespace Utility
{
    /// <summary>
    /// The TaskFactorySequentiallyByKey factory limits concurrency when actions are passed with the same key.  Those actions are executed sequentially
    /// and never in parallel.
    /// </summary>
    /// <remarks>
    /// For example, you have an action to fetch an image from the web to a local hard drive and then render a specific size of thumbnail for it.
    ///   The action includes code to check if the original image is already on disk, if not it fetches it.
    ///   It then checks if the correct size thumbnail has been rendered, if not it renders it.
    /// You want to be able to fire off requests for thumbnails from multiple different asp.net web pages and ensure that any two requests for the
    /// same original image are executed sequentially so that the image is only fetched once from the web before both thumbnail renders run.
    /// </remarks>
    public class TaskFactorySequentiallyByKey : TaskFactory
    {
        /// <summary>
        /// Tasks currently queued based on key
        /// </summary>
        Dictionary<string, Task> inUse = new Dictionary<string, Task>();

        public TaskFactorySequentiallyByKey()
            : base()
        {
        }

        public TaskFactorySequentiallyByKey(CancellationToken cancellationToken)
            : base(cancellationToken)
        { }

        public TaskFactorySequentiallyByKey(TaskScheduler scheduler)
            : base(scheduler)
        { }

        public TaskFactorySequentiallyByKey(TaskCreationOptions creationOptions, TaskContinuationOptions continuationOptions)
            : base(creationOptions, continuationOptions)
        { }

        public TaskFactorySequentiallyByKey(CancellationToken cancellationToken, TaskCreationOptions creationOptions, TaskContinuationOptions continuationOptions, TaskScheduler scheduler)
            : base(cancellationToken, creationOptions, continuationOptions, scheduler)
        { }

        protected virtual void FinishedUsing(string key, Task taskThatJustCompleted)
        {
            lock (this.inUse)
            {
                // If the key is present AND it point to the task that just finished THEN we are done
                // and can clear the key for the next task that comes in ...
                if (this.inUse.ContainsKey(key))
                    if (this.inUse[key] == taskThatJustCompleted)
                    {
                        this.inUse.Remove(key);
                        Debug.WriteLine("Finished using " + key + " completely");
                    }
                    else
                    {
                        Debug.WriteLine("Finished an item for " + key);
                    }

            }
        }

        /// <summary>
        /// Queue an action but prevent parallel execution of items having the same key.  Instead, run them sequentially.
        /// </summary>
        /// <remarks>
        /// This allows you to, for example, queue up tasks to fetch an image from the web to a cache and render a thumbnail for it at different sizes
        /// while ensuring that the image is only fetched to the cache once before each different size thumbnail is generated
        /// </remarks>
        public Task StartNewChainByKey(string key, Action<CancellationToken> action)
        {
            return StartNewChainByKey(key, action, base.CancellationToken);
        }

        /// <summary>
        /// Queue an action but prevent parallel execution of items having the same key.  Instead, run them sequentially.
        /// </summary>
        /// <remarks>
        /// This allows you to, for example, queue up tasks to fetch an image from the web to a cache and render a thumbnail for it at different sizes
        /// while ensuring that the image is only fetched to the cache once before each different size thumbnail is generated
        /// </remarks>
        public Task StartNewChainByKey(string key, Action<CancellationToken> action, CancellationToken cancellationToken)
        {
            CancellationToken combined = cancellationToken == base.CancellationToken ? base.CancellationToken :
                                CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, base.CancellationToken).Token;

            lock (inUse)
            {
                Task result;
                if (inUse.TryGetValue(key, out result))
                {
                    // chain the supplied action after it ...
                    result = result.ContinueWith((task) => action(combined), combined);

                    // And then schedule a completion check after that
                    result.ContinueWith((task) => FinishedUsing(key, task));

                    // Update the dictionary so that it tracks the new LAST task in line, not any of the earlier ones
                    inUse[key] = result;

                    Debug.WriteLine("Chained onto " + key);

                    return result;
                }

                // otherwise simply create it and start it after remembering that the key is in use
                result = new Task(() => action(combined), combined);

                inUse.Add(key, result);

                // queue up the check after it
                result.ContinueWith((task) => FinishedUsing(key, task));

                Debug.WriteLine("Starting a new action for " + key);

                // And finally start it
                result.Start(this.Scheduler);

                return result;
            }
        }
    }
}

Singleton tasks: A TaskFactory for the Task Parallel Library with ‘run-only-one’ semantics

When developing .NET applications there is often the need to execute some slow background process repeatedly. For example, fetching a feed from a remote site, updating a user’s last logged in time, … etc. Typically you queue actions like these onto the thread pool. But under load that becomes problematic as requests may be coming in faster than you can service them, the queue builds up and you are now executing multiple requests for the same action when you only really needed to do one. Even when not under load, if two users request a web page that requires the same image to be loaded and resized for display you only want to fetch it and resize it once. What you really want is an intelligent work queue that can coalesce multiple requests for the same action into a single action that gets executed just once.

The new Task parallel library doesn’t have anything that can handle these ‘run-only-one’ actions directly but it does have all the necessary building blocks to build one by creating a new TaskFactory and using Task continuations.

Below is a TaskFactory that gives you ‘run-only-one’ actions. To use it you simply create a new TaskFactoryLimitOneByKey and then call StartNewOrUseExisting() with a suitable key, e.g. “FETCH/cache/image12345.jpg”. This method returns a normal Task object that you can Wait on or add more continuations. All the usual TaskFactory constructor options are provided so you can have a different TaskScheduler, common cancellation token, and other options.

Note also that it expects an Action<CancellationToken> not just a plain Action. This is so your Action can be polite and monitor the cancellation token to know when to stop early. If you don’t need that you can always pass in a closure that tosses the CancellationToken, i.e. (token) => MyAction().

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Threading;
using System.Diagnostics;

namespace Utility
{
    /// <summary>
    /// A task factory where Tasks are queued up with a key and only one of that key is allowed to exist either in the queue or executing
    /// </summary>
    /// <remarks>
    /// This is useful for tasks like fetching a file from backing store, or updating local information from a remote service
    /// You want to be able to queue up a Task to go do the work but you don't want it to happen 5 times in quick succession
    /// NB: This does not absolve you from using file locking and other techniques in your method to handle simultaneous requests,
    /// it just greatly reduces the chances of it happening.  Another example would be updating a user's last logged in data in a
    /// database.  Under heavy load the queue to write to the database may be getting long and you don't want to update it for the same
    /// user repeatedly if you can avoid it with a single write.
    /// </remarks>
    public class TaskFactoryLimitOneByKey : TaskFactory
    {
        /// <summary>
        /// Tasks currently queued based on key
        /// </summary>
        Dictionary<string, Task> inUse = new Dictionary<string, Task>();

        public TaskFactoryLimitOneByKey()
            : base()
        {
        }

        public TaskFactoryLimitOneByKey(CancellationToken cancellationToken)
            : base(cancellationToken)
        { }

        public TaskFactoryLimitOneByKey(TaskScheduler scheduler)
            : base(scheduler)
        { }

        public TaskFactoryLimitOneByKey(TaskCreationOptions creationOptions, TaskContinuationOptions continuationOptions)
            : base(creationOptions, continuationOptions)
        { }

        public TaskFactoryLimitOneByKey(CancellationToken cancellationToken, TaskCreationOptions creationOptions, TaskContinuationOptions continuationOptions, TaskScheduler scheduler)
            : base(cancellationToken, creationOptions, continuationOptions, scheduler)
        { }

        protected virtual void FinishedUsing(string key, Task taskThatJustCompleted)
        {
            lock (this.inUse)
            {
                // If the key is present AND it point to the task that just finished THEN we are done
                // and can clear the key so that the next task coming in using it will get to execute ...
                if (this.inUse.ContainsKey(key))
                    if (this.inUse[key] == taskThatJustCompleted)
                    {
                        this.inUse.Remove(key);
                        Debug.WriteLine("Finished using " + key + " completely");
                    }
                    else
                    {
                        Debug.WriteLine("Finished an item for " + key);
                    }

            }
        }

        /// <summary>
        /// Queue only one of a given action based on a key.  A singleton pattern for Tasks with the same key.
        /// </summary>
        /// <remarks>
        /// This allows you to queue up a request to, for example, render a file based on the file name
        /// Even if multiple users all request the file at the same time, only one render will ever run
        /// and they can all wait on that Task to complete.
        /// </remarks>
        public Task StartNewOrUseExisting(string key, Action<CancellationToken> action)
        {
            return StartNewOrUseExisting(key, action, base.CancellationToken);
        }

        /// <summary>
        /// Queue only one of a given action based on a key.  A singleton pattern for Tasks with the same key.
        /// </summary>
        /// <remarks>
        /// This allows you to queue up a request to, for example, render a file based on the file name
        /// Even if multiple users all request the file at the same time, only one render will ever run
        /// and they can all wait on that Task to complete.
        /// </remarks>
        public Task StartNewOrUseExisting (string key, Action<CancellationToken> action, CancellationToken cancellationToken)
        {
            CancellationToken combined = cancellationToken == base.CancellationToken ?  base.CancellationToken :
                                            CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, base.CancellationToken).Token;
            lock (inUse)
            {
                if (inUse.ContainsKey(key))
                {
                    Debug.WriteLine("Reusing existing action for " + key);
                    return inUse[key];  // and toss the new action away
                }

                // otherwise, make a new one and add it ... with a continuation on the end to pull it off ...
                Task result = new Task(() => action(combined), combined);
                inUse.Add(key, result);

                // queue up the check after it
                result.ContinueWith((finished) => this.FinishedUsing(key, result));

                Debug.WriteLine("Starting a new action for " + key);

                // and finally start it
                result.Start(this.Scheduler);
                return result;
            }
        }
    }
}

GDI+ Image.FromFile has a problem – here’s how to fix it

In GDI+ you can call Image.FromFile to load an image from a file. BUT there are several issues with this call, the biggest being that GDI+ will keep the file open long after you are done with it. Here is an image loader that gets around this issue.

If you are running a high volume web site, and your images are on a SAN you’ll find this technique necessary to prevent an eventual exhaustion of filehandles.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Drawing;
using System.IO;
using System.Data;

namespace Utility
{
public static class ImageLoader
{
// This isn’t going to help much – you’ll run out of memory anyway on very large images – but if you are keeping several in memory it might …
public const int MaximumImageDimension = 10000;

///
/// Method to safely load an image from a file without leaving the file open,
/// also gets the size down to a manageable size in the case of HUGE images
///
/// An Image – don’t forget to dispose of it later
public static Image LoadImage (string filePath)
{
try
{
FileInfo fi = new FileInfo(filePath);

if (!fi.Exists) throw new FileNotFoundException(“Cannot find image”);
if (fi.Length == 0) throw new FileNotFoundException(“Zero length image file “);

// Image.FromFile is known to leave files open, so we use a stream instead to read it
using (FileStream fs = new FileStream(filePath, FileMode.Open, FileAccess.Read))
{
if (!fs.CanRead) throw new FileLoadException (“Cannot read file stream”);

if (fs.Length == 0) throw new FileLoadException(“File stream zero length”);

using (Image original = Image.FromStream(fs))
{
// Make a copy of the file in memory, then release the one GDI+ gave us
// thus ensuring that all file handles are closed properly (which GDI+ doesn’t do for us in a timely fashion)
int width = original.Width;
int height = original.Height;
if (width == 0) throw new DataException(“Bad image dimension width=0″);
if (height == 0) throw new DataException(“Bad image dimension height=0″);

// Now shrink it to Max size to control memory consumption
if (width > MaximumImageDimension)
{
height = height * MaximumImageDimension / width;
width = MaximumImageDimension;
}
if (height > MaximumImageDimension)
{
width = width * MaximumImageDimension / height;
height = MaximumImageDimension;
}

Bitmap copy = new Bitmap(width, height);
using (Graphics graphics = Graphics.FromImage(copy))
{
graphics.DrawImage(original, 0, 0, copy.Width, copy.Height);
}
return copy;
}
}
}
catch (Exception ex)
{
ex.Data.Add(“FileName”, filePath);
throw;
}
}
}