Posts

  • Lightmoon

    Jan 10, 2011

    The last year I tried to develop Web Push Technology for .NET ( Comet Style Http Push + Web Sockets for more modern browsers ). While the actual communication technology was relatively easy to implement, I ended up in a world where my precious web application had to suddenly keep state around about virtual connections or queues in order to deliver the correct data to the correct clients. This ended in a world of hurt because there was no ready made solution for this kind of problem. While certainly one could develop such a specific solution, I started to think further. In the modern world of application development be it in the web space or in backend development, we are taught to make our applications stateless. However in reality most of them are stateful and we only push the state as far back as possible ( usually the Database ) and use mostly stateless parts elsewhere. This essentially is a trick to get parts of our infrastructure to scale easier ( or cheaper ) while the costs for databases skyrocket ( albeit only if you hit a certain amount of data ) in terms of hardware/software and personnel . It also makes the lives of developers a lot harder since we constantly have to use tricks and workarounds to hide the fact that in fact we have state that we would like to access. Be it clever data access strategies, http cookies or Session State wrappers. Wouldn’t it be easier to just not have to think about it, and follow one simple programming model that comes natural to the average OOP developer ( POCOs ) ? Enter LightMoon, an application middleware I have started developing that essentially provides these features, and while doing so does not sacrifice scalability. This technology could be used in a lot of domains, not the least the development of MMO style games ( where my inspiration originally came from ). I plan to release a technology preview sometime in May 2011 ( I know still some time off ) and later on make it a commercial product that is dual licensed under the GPL and a commercial one. Following is a short list of features that will be supported in the initial CTP:

    • Simple programming model of disconnected actors that communicate via message passing similar to what Erlang/Scala provide
    • Lightweight views on actors ( auto generated from the implementation code, initially for JavaScript via HTTP communication and C# via TCP/IP )
    • Support for a huge number of simultaneously connected clients.
    • Transparent persistence
    • Transparent concurrency
    • Easy horizontal scalability

    The ideas i follow are not new, there is just no good implementation of them for .NET so far with modern technology backends.

  • 2011 update

    Jan 10, 2011

    I have to admit I did not spend as much time as I wanted initially on this blog, mainly because I am a lazy person. The last year was packed with a lot of work at my most recent contract gig, and since the rollout was scheduled for the years end I was pretty busy finalizing a lot of things and dealing with the general madness that ensues from such endeavours.

    But I also found time to investigate some new things and start two projects that I really hope will blossom into something useful, partially for the development community and partially for consumers.

    I spent a good deal of time on getting to grips with Scala and Erlang, and I have to say I love both languages dearly nowadays. While I don’t see Erlang breaking into the mainstream programming space anytime soon, Scala is something that could very well show us the path to a successor of current generation programming languages like C#,Ruby and so on.

    Pattern matching is so much more powerful than the traditional branching logic you write in most other languages, and the approach both take to concurrency ( message passing between isolated entities ) seems so far the only one that shows a path to a brighter tomorrow ( for problems that are not easily decomposed into simple data/task partitioning algorithms.

    This basically leads me to why I started the first project on my agenda which I will detail in my next post.

  • IOC ModelBinder

    May 21, 2010

    I had an idea a couple of days ago, that I decided to play around with, and maybe collect opinions on. The ASP.NET MVC ModelBinder infrastructure is quite exhaustive, and allows you to perform almost any action when a something the developer wants to get data from the request into the domain/data/business model. However recently I found myself in a situation where a controller got quite sophisticated and required a lot of dependencies from our Ioc Container of choice ( Autofac ). Since all of those were mandatory dependencies for execution for one or the other action method, our constructor arguments got a little bit out of hand ( I just don’t like constructors with more than, let’s say 4 arguments ). So I decided to build a model binder that injects dependencies into action method calls.

    Without further ado, here is the interesting part of the code:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    public class IocBindAttribute : CustomModelBinderAttribute
    {
        public override IModelBinder GetBinder()
        {
            return new IocModelBinder();
        }
    }
    
    public class IocModelBinder : IModelBinder
    {
        public object BindModel(ControllerContext controllerContext,
                                ModelBindingContext bindingContext)
        {
            try
            {
                return GetContainer().Resolve(bindingContext.ModelName,
                                              bindingContext.ModelType);
            }
            catch (ComponentNotRegisteredException)
            {
                return GetContainer().Resolve(bindingContext.ModelType);
            }
        }
    
        private IContainer GetContainer()
        {
            return (HttpContext.Current.ApplicationInstance as
                        IContainerProviderAccessor)
                        .ContainerProvider.ApplicationContainer;
        }
    }
    
    public class EmailController : Controller
    {
        public ActionResult Index()
        {
            return View();
        }
    
        public ActionResult Send([IocBind]IEmailService emailService,
                             string from, string to, string subject, string text)
        {
            emailService.Send(from, to, subject, text);
    
            return RedirectToAction("Index");
        }
    }
    

    Basically you use your container as you like and register the given model binder either globally or via the provided ModelBinder attribute on a more granular level. Note that this code is not meant to be a production implementation, it’s just here to show the idea. The same basic idea can be implemented for virtually any container, or using the common service locator if you choose so. However I am a bit torn with this idea in the context of architectural design:

    Arguments for using it:

    • Reduces a lot of boilerplate code for dependency handling like constructor arguments and storage.
    • If you see each action as a distinct piece of code that is grouped by a container, it better reflects the correspondence between your actions and the dependencies they need, you can see at a glance what dependencies are required by each action, which might not be so obvious when you follow the traditional approach.
    • In the same vein it eases both refactoring as well as unit testing, because you can now treat each action more easily in isolation, move it around to different controllers, or set it up for a test case.

    Arguments against using it:

    • If you have a controller whose action methods require such disparate dependencies, then you might have a deeper problem, either your controller is doing to much in terms of the single responsibility principle, or your dependencies are too granular and it might be a good idea to provide some bridge dependency for that case.
    • It mixes the concerns of data flow and the business layer of your application.
    • So please fire away with the comments to tell me what you think, or what I have done wrong with this.
  • Simple factory for open dynamic delegates

    Apr 18, 2010

    During the course of my development adventures I came to a point where I needed to call instance methods dynamically based only on information that can be obtained through reflection. It is well known that MethodInfo.Invoke is quite slow, and so I decided to create and cache delegates based on the given MethodInfo on first use, and use those to invoke the method.

    However it would be prohibitive to generate a delegate for every instance of an object, rather it would be nice to create a delegate per Method prototype and decide the instance to call it on during the final invocation. A little known feature that allows us to achieve this, is the ability of the CLR to create so called “open” delegates. In the CLR a call to an instance method is quite similar to the call of a static one, except that the first parameter on the stack is the instance on which to call the method. Open Delegates allow us to create delegates that for an instance method take an additional first parameter for the instance to call the method on, thus neatly wrapping the above concept.

    The feature is not common knowledge, basically because neither C# or VB offer the syntax to create such open delegates, one has to always use Delegate.CreateDelegate to create them dynamically. The overload to use is mainly Delegate.CreateDelegate( Type delegateType,MethodInfo method,bool throwOnBindFailure ).

    This is quite straightforward to use, a little complication stems from the fact that we need a delegate type to create the delegate instance. I decided to leverage the predefined Action<> and Func<> types for that, and bind the generic arguments to the parameter types of the method.

    Here is my simple utility class that wraps the presented ideas. Note that it does not support open generic methods and that it is limited to methods that have up to 16 parameters ( not that you should have methods that have more ). To use just call the CreateOpenDelegate method with the MethodInfo that represents your instance method. The returned delegate can then either be casted to an appropriate Action<> or Func<> or called via DynamicInvoke. Regardless of the call method used, one has to pass in the instance to call the method on as the first parameter. Here is the source code:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    public static class DelegateFactory
    {
        private static readonly Type[] actionTypes = new [] {
            typeof(Action),
            typeof(Action<>),
            typeof(Action<,>),
            typeof(Action<,,>),
            typeof(Action<,,,>),
            typeof(Action<,,,,>),
            typeof(Action<,,,,,>),
            typeof(Action<,,,,,,>),
            typeof(Action<,,,,,,,>),
            typeof(Action<,,,,,,,,>),
            typeof(Action<,,,,,,,,,>),
            typeof(Action<,,,,,,,,,,>),
            typeof(Action<,,,,,,,,,,,>),
            typeof(Action<,,,,,,,,,,,,>),
            typeof(Action<,,,,,,,,,,,,,>),
            typeof(Action<,,,,,,,,,,,,,,>),
            typeof(Action<,,,,,,,,,,,,,,,>),
        };
    
        private static readonly Type[] functionTypes = new [] {
            typeof(Func<>),
            typeof(Func<,>),
            typeof(Func<,,>),
            typeof(Func<,,,>),
            typeof(Func<,,,,>),
            typeof(Func<,,,,,>),
            typeof(Func<,,,,,,>),
            typeof(Func<,,,,,,,>),
            typeof(Func<,,,,,,,,>),
            typeof(Func<,,,,,,,,,>),
            typeof(Func<,,,,,,,,,,>),
            typeof(Func<,,,,,,,,,,,>),
            typeof(Func<,,,,,,,,,,,,>),
            typeof(Func<,,,,,,,,,,,,,>),
            typeof(Func<,,,,,,,,,,,,,,>),
            typeof(Func<,,,,,,,,,,,,,,,>),
            typeof(Func<,,,,,,,,,,,,,,,,>),
        };
    
       public static Delegate CreateOpenDelegate( MethodInfo method )
       {
            Contract.Requires(method != null);
    
            var closedType = GetClosedDelegateType(method);
    
            return closedType != null ?
                 Delegate.CreateDelegate(closedType, method, true)
                 : null;
        }
    
        private static Type GetClosedDelegateType( MethodInfo method )
        {
            Contract.Requires(method != null);
    
            var openType = GetOpenDelegateType(method);
    
            if (openType != null)
            {
                var parameterTypes =
                    new[] {method.DeclaringType}.Union(method.GetParameters()
                    .Select(p => p.ParameterType));
    
                if (method.ReturnType != typeof (void))
                {
                    parameterTypes = parameterTypes.Union(new[] {method.ReturnType});
                }
    
                return openType.MakeGenericType(parameterTypes.ToArray());
            }
    
            return null;
        }
    
        private static Type GetOpenDelegateType( MethodInfo method )
        {
            Contract.Requires( method != null );
    
            var parameterCount = method.GetParameters().Length + 1;
    
            if (parameterCount < functionTypes.Length &&
                parameterCount < actionTypes.Length)
            {
    
                return method.ReturnType != typeof (void)
                       ? functionTypes[parameterCount]
                       : actionTypes[parameterCount];
            }
    
            return null;
        }
    }
    
  • Enhanced persistent continuations

    Apr 11, 2010

    Continuations are the base for several useful idioms in programming like coroutines, cooperative multithreading and so on.

    In the vanilla .NET framework C# supports a way to emulate them using the yield keyword that was originally intended for custom enumerations, but was soon found out to be much more powerful. Using yield one can define methods that can be interrupted at well defined points and continued when desired, with only minor syntax inconveniences.

    Please note that Mono ( 2.6 or above ) include an even better Continuations implementation that works in conjunction wit the CLR itself, but of course this only works on Mono, and not on the vanilla .NET framework. In an project I am doing in my spare time I needed continuations that can also be migrated to different machines ( similar to what the SecondLife servers do ) in order to provide enhanced scalability and load balancing, and it needed to run on the normal Microsoft CLR implementation.

    The yield statement in C# generates an implicit state machine in a custom IEnumerator implementation, that executes the parts of your code based on the current state.

    These enumerators contain a number of state member fields: xxxstate is an integer that stores the state of the method, xxxcurrent stores the current value the enumerator returns and xxx_this contains the instance on which the method itself was called. Additional members contain hoisted out variables of the method code itself ( e.g the control variables for loops and such ) xxx is a compiled generated prefix that seems to be generated by the order in which the compiler generates the members.

    When a method that uses yield is called directly an instance of the compiler generated enumerator is created, the instance that was used for the call is stored in it and it is returned to the caller. The actual method code is only executed when MoveNext is called on the enumerator. This means that a call to the method itself has no side effects and can be done an arbitrary number of times without doing any harm ( except some memory allocations of course ).

    So in order to migrate the execution state of such a method we need to execute any number of steps from it, ( using MoveNext of the enumerator returned ), persist the state of the enumerator, load it again on some other machine, call the method again, apply the saved state to the enumerator, and call MoveNext again to continue execution. To achieve this I created a surrogate class for an enumerator that extracts and state members out of an instance and that is serializable, and that allows to apply that state again to a different enumerator of the same type.

    I do not serialize the xxxthis member since it gets assigned on the initial method call anyway, and the xxxcurrent member is also not needed since the current value is only interesting to me after a call to MoveNext ( in my case I always return an implementation of my IWaitCondition interface ) Note that this all works only with the style that the C# compiler implements yield based enumerators, and will probably break in VB or other .NET languages. It is also required that any control variables that get hoisted out of the method code are serializable as well but in most situations that is the case anyway, and otherwise can be worked around easily. In order to avoid the reflection overhead I create state member accessors using DynamicMethod that get used on any subsequent usage.

subscribe via RSS