The Inquisitive Coder - Davy Brion's Blog

Protecting Your Application From Remote Problems

Davy Brion — Sun, 05 Jul 2009 19:45:41 +0000

If you have a web application which communicates with a remote service, it’s important to protect that web application from any problems the remote service might be dealing with. For instance, if the remote service goes down (for whatever reason) you really don’t want your application to keep making calls to this service. These failing calls increase the load on the service, which is already having problems, and will also block your threads which takes away resources from your application to deal with other requests. One pattern which is very suitable to reduce the problems for this situation is the Circuit Breaker (read that unless you’re familiar with the circuit breaker).

The biggest issue i have with my previous implementation is that it required you to call it manually to protect potentially risky calls. I don’t like having to call my circuit breaker whenever i want to make a service call because as a consumer of a service proxy, i shouldn’t even know about the circuit breaker. I also don’t want any coupling between my service proxy and the actual circuit breaker. Sounds like a good candidate for some AOP magic, right?

We’re going to use Castle Windsor’s Interceptors to make this work. First, the implementation of the CircuitBreaker class:




    public class CircuitBreaker : IInterceptor
    {
        private readonly object monitor = new object();
        private CircuitBreakerState state;
        private int failures;
        private int threshold;
        private TimeSpan timeout;
 
        public CircuitBreaker(int threshold, TimeSpan timeout)
        {
            this.threshold = threshold;
            this.timeout = timeout;
            MoveToClosedState();
        }
 
        public void Intercept(IInvocation invocation)
        {
            using (TimedLock.Lock(monitor))
            {
                state.ProtectedCodeIsAboutToBeCalled();
            }
 
            try
            {
                invocation.Proceed();
            }
            catch (Exception e)
            {
                using (TimedLock.Lock(monitor))
                {
                    failures++;
                    state.ActUponException(e);
                }
                throw;
            }
 
            using (TimedLock.Lock(monitor))
            {
                state.ProtectedCodeHasBeenCalled();
            }
        }
 
        private void MoveToClosedState()
        {
            state = new ClosedState(this);
        }
 
        private void MoveToOpenState()
        {
            state = new OpenState(this);
        }
 
        private void MoveToHalfOpenState()
        {
            state = new HalfOpenState(this);
        }
 
        private void ResetFailureCount()
        {
            failures = 0;
        }
 
        private bool ThresholdReached()
        {
            return failures >= threshold;
        }
 
        private abstract class CircuitBreakerState
        {
            protected readonly CircuitBreaker circuitBreaker;
 
            protected CircuitBreakerState(CircuitBreaker circuitBreaker)
            {
                this.circuitBreaker = circuitBreaker;
            }
 
            public virtual void ProtectedCodeIsAboutToBeCalled() { }
            public virtual void ProtectedCodeHasBeenCalled() { }
            public virtual void ActUponException(Exception e) { }
        }
 
        private class ClosedState : CircuitBreakerState
        {
            public ClosedState(CircuitBreaker circuitBreaker)
                : base(circuitBreaker)
            {
                circuitBreaker.ResetFailureCount();
            }
 
            public override void ActUponException(Exception e)
            {
                if (circuitBreaker.ThresholdReached()) circuitBreaker.MoveToOpenState();
            }
        }
 
        private class OpenState : CircuitBreakerState
        {
            private readonly Timer timer;
 
            public OpenState(CircuitBreaker circuitBreaker)
                : base(circuitBreaker)
            {
                timer = new Timer(circuitBreaker.timeout.TotalMilliseconds);
                timer.Elapsed += TimeoutHasBeenReached;
                timer.AutoReset = false;
                timer.Start();
            }
 
            private void TimeoutHasBeenReached(object sender, ElapsedEventArgs e)
            {
                circuitBreaker.MoveToHalfOpenState();
            }
 
            public override void ProtectedCodeIsAboutToBeCalled()
            {
                throw new OpenCircuitException();
            }
        }
 
        private class HalfOpenState : CircuitBreakerState
        {
            public HalfOpenState(CircuitBreaker circuitBreaker) : base(circuitBreaker) { }
 
            public override void ActUponException(Exception e)
            {
                circuitBreaker.MoveToOpenState();
            }
 
            public override void ProtectedCodeHasBeenCalled()
            {
                circuitBreaker.MoveToClosedState();
            }
        }
    }

Notice how the CircuitBreaker class implements Windsor’s IInterceptor interface. The Intercept method will be called by Windsor whenever we try to call a method from a protected component. Within the Intercept method we can add the necessary logic to apply the Circuit Breaker pattern to the code that was originally called.

Now we just need to configure the Windsor IOC container to apply this bit of AOP magic for us.

First, we register the CircuitBreaker with the container:




            container.Register(Component.For<CircuitBreaker>().LifeStyle.Singleton
                                   .Named("serviceProxyCircuitBreaker")
                                   .DependsOn(new Hashtable { { "threshold", 5 }, { "timeout", TimeSpan.FromMinutes(5) } }));

Notice that we register the CircuitBreaker implementation with a Singleton lifestyle, a custom name and the required constructor parameters to create an instance of the CircuitBreaker.

Then we register our service proxy:




            container.Register(Component.For<IServiceProxy>().ImplementedBy<ServiceProxy>().LifeStyle.Transient
                                .Interceptors(InterceptorReference.ForKey("serviceProxyCircuitBreaker")).Anywhere);

Notice how we registered the service proxy as a transient component, while referencing the singleton CircuitBreaker interceptor. This means that each resolved instance of our service proxy will be protected by the same CircuitBreaker instance. If you have multiple services that you want to protect, simply register multiple CircuitBreakers with different keys and link each service you want to protect with the correct CircuitBreaker key.

Save Some Time With The Right Testrunner

Davy Brion — Wed, 01 Jul 2009 08:04:27 +0000

If you’re running your tests multiple times throughout the day, it’s in your best interest to keep those testruns as fast as possible. We have a solution with 2 test projects. The first testproject contains 2700 regular tests that don’t use any external resources so they should be very fast. The second testproject contains 1200 database tests (you do write tests for your queries, right?) which is a little bit slower, as is to be expected.

The problem is that with Resharper’s testrunner, running the database tests actually takes less time than running the ‘fast’ tests. The 1200 database tests take 1m14 seconds to run. The 2700 in memory tests take 1m20 seconds to run… so that’s almost 3 minutes of waiting time whenever you run the entire testsuite.

Just out of curiosity, i ran all of the tests with TestDriven.NET to see how much faster it would be… The 1200 database tests only took about 45 seconds, and the 2700 in memory tests only took 28 seconds to run. Running the entire testsuite with TestDriven.NET actually takes less than half the time it takes with Resharper’s testrunner…

Avoid Using NHibernate With NUnit 2.4.6

Davy Brion — Wed, 24 Jun 2009 17:35:42 +0000

We just spent about 2 hours trying to find out why our NHibernate tests were about 10x slower on our build server than they were on our local machines. I had noticed lately that the build for one of our projects was taking longer and longer but i hadn’t really timed the difference. This project has about 1200 tests that use NHibernate and they run in about 45-60 seconds on my local machine. It turns out they took around 15 minutes on the buildserver when running them through TeamCity.

I logged into the buildserver and ran the tests manually using nunit’s console runner (with an NUnit-2.4.7 build that i happened to have installed somewhere on the machine) and they only took about 45 seconds. After a lot of guesswork and screwing around, it turned out that we never modified our base build script (why yes, i do believe in build script inheritance) to use a newer version of NUnit. We set up the buildserver about 1 year ago, and at that time, the latest stable NUnit version that TeamCity supported was NUnit 2.4.6. Our base build script was still referring to NUnit 2.4.6, which apparently sets log4net to use debug level logging. Now, NHibernate logs a huge amount of information at the debug level, so this turned out to slow down all of our builds that had NHibernate tests.

We changed the the 2.4.6 version in our script to 2.4.7 and the build time of this particular project decreased from around 50 minutes to about 35 minutes. Yes, that’s still a lot but this is a huge project with a lot of legacy tests and the entire build process is pretty complex. Other projects went from build times from around 7 minutes to about 2 minutes.

That’s a pretty nice improvement for simply changing a “6″ to a “7″

Surface Development For Belgian TV

Davy Brion — Mon, 22 Jun 2009 12:20:48 +0000

Some of my coworkers recently developed 2 Surface applications for VTM, one of the commercial Belgian TV channels. This is one of the few high-profile usages of Microsoft’s Surface in Belgium so far. Be sure to check out the videos on our company website.

And We’re Back

Davy Brion — Mon, 22 Jun 2009 12:20:02 +0000

I got back from the Dominican Republic yesterday morning so i just wanted to post a short ‘review’ of the trip. Getting there took a looong time, because our plane first stopped in Cuba where we had to wait for another hour before we flew into Punta Cana. Unfortunately, we weren’t allowed to get off the plane for a short time which kinda sucks after 10 hours of flying. The little bit of Cuba i did get to see (through the window) did look pretty nice though. So after 13 hours we finally landed at Punta Cana’s airport and then we had to take a 2 hour bus ride to get to our hotel.

The hotel was great. Big, nice rooms. Lots of restaurants (4 plus a general buffet as well). Great pool, including a nice bar. Fantastic beach, also with a bar obviously. Plenty of activities and a few shops as well. If you really want to, you could have a pretty good time without even leaving the resort.

But that would be a bit of a shame because the country has so much more to offer than beautiful resorts. I went on two excursions while i was there. The first one was a snorkeling trip which was great. It was basically a whole day on a boat going along the Caribbean coastline, which in itself is already a sight to behold. Then there were 2 snorkeling sessions, one before noon, and one in the afternoon. For lunch, we went to the Saona island which you may know better as the Bounty Island (used in the commercials) to have a barbecue plus some relaxing on the beach. We also spent some time in a ‘natural swimming pool’ which is located within the sea on a huge sandbank (which was the result of a hurricane a few years back, as i’ve been told). It really is mindblowing to stand up in the middle of the ocean, with the water only reaching up to your waist.

The second excursion was a truck safari from the southern part of the island to the north side. I’d definitely recommend doing this because the sights you see along the way are unbelievably beautiful. And you don’t just spend the whole time in the truck either. We stopped along a sugar cane field and tried real sugar from a sugar cane. We also visited some people in the country side where we tried fresh coffee (from beans that had finished roasting in the sun just a few days ago), fresh cocoa (both in powder form after non-industrial processing as well as directly from the fruit), fresh Creole pine apple, coconuts, Mama Juana and other things as well. We had lunch on some ranch in the mountains (seeing large palm trees on top of high mountains is pretty cool as well) and after that we did some horseback riding. We also visited a tiny school (with only 2 classrooms) and a big church in Higuey which is apparently pretty famous. In the afternoon we stopped along a beach in the north side of the island (with the Atlantic ocean instead of the Carribean sea). We definitely saw a lot of beautiful things during this excursion. Funny side story: some British tourists that were also on the truck brought notepads and pens for the poor children we’d meet along the way. During one of our walks, they gave them to two kids who were sitting in front of their house. What these tourists didn’t realize was that those kids were sitting in front of a very nice house which actually had a beautiful pool in the backyard. Good thing these poor kids now also have notepads and pens

Combining these excursions with a lot of relaxing at the beach definitely makes for a great trip. I’m not really a going-to-the-beach-kinda-guy but the Bahayibe beach and the Carribean Sea are just too nice to pass up on. And the water is just incredible… it’s crystal clear, it’s very warm, and occasionally you’ll see a bunch of colorful fish swim by or even around you.

The one downside to the trip was that there was hardly anything to do at night where we were. The northern side apparently has a lot more going on at night (it’s only normal with all the mass-tourism over there) but on the southern side there’s really not much to do outside of what’s going on in your resort. I was there with my best friend so we were hoping for some interesting bars to go to at night, but we mostly stuck to the lobby bar in the resort for a lack of anything better. You could just go to local towns and visit the bars over there but that’s not really recommended unless you’re with a reasonably-sized group to avoid trouble.

But apart from that, everything was great and i’d definitely recommend it to anyone. And after all of that, we spent another 9 hours on a plane to land in Brussels with pouring rain and pretty chilly weather *Sigh*.

I still have to go through a couple hundred blog posts that i missed, a shitload of email and some other stuff i have to do as well. I’ll start posting and coding again in the next couple of days though

Time For A Much Needed Break

Davy Brion — Sun, 07 Jun 2009 19:55:37 +0000

As you have probably noticed, i haven’t been very active lately. Mostly because of some situations in my personal life lately which took up a lot of my spare time. Because of this, i haven’t been posting as much as i used to, nor did i contribute anything to NHibernate in the past couple of weeks. Luckily for me, i’m leaving for a trip to the Dominican Republic in a couple of days where i intend to explore the limits of the concept “relaxation”. As much as i love software development and computers, i’m really looking forward to not even touching a computer (or any electronic device for that matter) for about 10 days . Who could think of computers when you’re hanging out a place that looks like this:

I’ll see you all in about 2 weeks

How Can This Test Fail?

Davy Brion — Wed, 03 Jun 2009 11:46:55 +0000

We have a certain class which sometimes makes some tests fail… the code is old (consider it legacy) but it’s not that bad… but we just can’t figure out why it sometimes causes tests the fail.

Here’s an example of a test that occasionally fails:



        [Test]
        public void TestGet_CacheableObjectReturned()
        {
            TestCacheableObject cacheableObject = new TestCacheableObject("test", 60000);
 
            CacheProvider.GetInstance().Put(cacheableObject);
 
            try
            {
                Assert.AreSame(cacheableObject, CacheProvider.GetInstance().Get("test"));
            }
            finally
            {
                CacheProvider.GetInstance().Remove("test");
            }
        }

If it fails, it’s on the following line:



                Assert.AreSame(cacheableObject, CacheProvider.GetInstance().Get("test"));

Sometimes, the Get method of the CacheProvider returns null in this test, even though the cacheable object has just been added to the CacheProvider.

Here’s the code of the CacheProvider:



    public class CacheProvider
    {
        private static object monitor = new object();
 
        private static CacheProvider _instance;
 
        private IDictionary<object, CacheableObject> _cacheDictionary;
 
        private CacheProvider()
        {
            _cacheDictionary = new Dictionary<object, CacheableObject>();
        }
 
        public static CacheProvider GetInstance()
        {
            lock (monitor)
            {
                if (_instance == null)
                {
                    _instance = new CacheProvider();
                }
            }
 
            return _instance;
        }
 
        public void Put(CacheableObject value)
        {
            if (ContainsKey(value.Key))
            {
                Remove(value.Key);
            }
 
            value.Provider = this;
            _cacheDictionary.Add(value.Key, value);
            value.StartTicking();
        }
 
        public CacheableObject Get(object key)
        {
            if (_cacheDictionary.ContainsKey(key))
            {
                return _cacheDictionary[key];
            }
 
            return null;
        }
 
        public void Remove(object key)
        {
            _cacheDictionary.Remove(key);
        }
 
        public bool ContainsKey(object key)
        {
            return _cacheDictionary.ContainsKey(key);
        }
    }

This is the code of the CacheableObject class:



    public abstract class CacheableObject
    {
        private object _key;
        private CacheProvider _cacheProvider;
 
        private Timer _timer;
 
        protected abstract double GetTimeToLive();
 
        public CacheableObject(object key)
        {
            _key = key;
        }
 
        public void StartTicking()
        {
            _timer = new Timer(GetTimeToLive());
            _timer.Elapsed += Timer_Elapsed;
            _timer.Start();
        }
 
        private void Timer_Elapsed(object sender, ElapsedEventArgs e)
        {
            _timer.Stop();
            Provider.Remove(this.Key);
        }
 
        public object Key
        {
            get
            {
                return _key;
            }
        }
 
        public CacheProvider Provider
        {
            get
            {
                return _cacheProvider;
            }
            set
            {
                _cacheProvider = value;
            }
        }
    }

And this is the code of the TestCacheableObject that is used in the failing test:



        public class TestCacheableObject : CacheableObject
        {
            private double _ttl;
 
            public TestCacheableObject(string key, double ttl)
                : base(key)
            {
                _ttl = ttl;
            }
 
            protected override double GetTimeToLive()
            {
                return _ttl;
            }
        }

I can’t for the life of me figure out why the test above fails sometimes. It happens very occassionaly, sometimes it even takes weeks or months before it fails again. It’s not just the test above, it’s basically every test that uses the CacheProvider’s ContainsKey method (either directly or indirectly) that fails sometimes.

Important note to consider: when these tests fail, they take about 10ms…

If anyone has a clue what could possibly cause this, i’d be very willing to hear it

What Microsoft Should Do For .NET Open Source

Davy Brion — Mon, 25 May 2009 20:37:06 +0000

I recently read Rob Conery’s “What Should Microsoft Do For .NET Open Source” post where he invites people to answer the question he poses in his post. Obviously, i could not resist posting my thoughts and views on this.

Before i start, i would like to state that i appreciate all of Microsoft’s recent efforts to move to a more open development model. I think they’ve done a pretty good job already but nothing’s perfect and there’s always room for improvement. So let’s just get to the list of things that i would love to see from Microsoft with regards to dealing with Open Source Software in the .NET world.

First of all, i’d like to see a completely transparent development model. I love the fact that ASP.NET MVC is released under an Open Source License, but it would be better if everyone could track the development. With that i mean access to the internal bug/issue tracker and source code repository. I want to know what problems are known, and how they are being dealt with. I want to see the progress in the code. I want the ability to either run off the trunk, or to merge specific pieces of the trunk into a released version if that enables me to avoid problems that i might run into with an officially released version. Getting the source code to a released version is (IMO) only a small part of what Open Source development is all about.

I also want them to be open to outside contributions. I truly hate running into annoying little bugs that won’t be fixed until the next major version (which sometimes takes a year or 2 to come out) but which could be avoided with a simple patch. Allow people to contribute patches and you’d be amazed at the amount of goodwill and support you’ll create among a vocal group of users. Obviously, this only works when the development model in general is as open as i mentioned earlier. Nobody is going to submit patches based on a released version if the actual trunk might already contain fixes or has been modified heavily after the latest release. And being able to patch a released version for your own use isn’t always a viable option to most people since you really don’t want to deal with having to merge the patch when newer versions are released (and all the regression testing that comes with it).

Microsoft should also play nice with other Open Source projects. Sure, they included JQuery with ASP.NET MVC which is a good move. But there are a lot of examples where they developed some library or framework for which a generally accepted Open Source alternative was already present. I still cringe whenever i’m forced to use MSTest (which to this day is still a pretty crappy testing framework IMHO) while they could’ve just as easily supported NUnit’s development. And if you’re not happy with NUnit, consider the state of MSTest before you complain about my choice of NUnit over say… MBUnit or any of the other testing frameworks. Does anyone remember NDoc? Was Sandcastle really necessary or would it have made more sense to just invest some effort into NDoc instead of starting from scratch? My biggest issue with this is that Microsoft’s alternatives can often be used without resistance in certain workplaces, just because it’s from Microsoft. It doesn’t even matter if there are actually better Open Source alternatives available… In Microsoft we trust, thus Microsoft you’ll use. Not always the smartest choice, as i’m sure many of you will agree with.

Would it really be so bad for Microsoft if they would just assign some developers to help out with projects that already exist? We now generally have to wait until the third version of any Microsoft product before we really get great quality, while in some cases, a high-quality Open Source alternative was already available from the start. This often leads to about 3 to 4 years of crappy releases that many people will adopt solely because it’s from Microsoft and because “it’ll improve in the next versions”. Even if you don’t want to assign developers to Open Source projects, it might already be very useful to provide helpful things such as servers, bandwidth, anything that could help out. Hell, references in official Microsoft documentation on MSDN would already be a huge step up.

There also needs to be some kind of PR cleanup campaign for all of the FUD that Microsoft has spread about Open Source software in general in the past few years. While it can be argued that that FUD was mainly targeted at GPL software, to many managers in big companies the message came across as “Do not use Open Source because it’s a legal minefield”. That is a misconception that many .NET developers are still fighting on a daily basis which isn’t good for anyone.

Then there is the issue of support. I’m not going to repeat myself so i’ll just link to a previous post of mine which covers that topic: Support Of Commercial Software vs Open Source Software.

And as for the host of legal aspects that Microsoft is probably afraid of, i would suggest looking into the reasons why other (large) companies are able to be cooperative citizens within the Open Source world without legal troubles. Hell, commercial software can be a legal minefield as well so is the Open Source world truly worse? Maybe they should ask companies like IBM… surely they have plenty of experience in both cases.

Do i expect or even want Microsoft to open source everything in the .NET world? No. But i would definitely appreciate it if Microsoft would play nice with other Open Source projects, be a more active participant and be open to outside participation as well. I’m sure i won’t be the only one who thinks so.

Comment Spam

Davy Brion — Sun, 24 May 2009 20:05:59 +0000

I used to go over the comments that were marked as spam on a regular basis to check for false positives. This was pretty doable in the past since i got between 30-50 spam comments on a daily basis and you could usually scroll through them pretty quickly to identify real comments. Lately though, i’ve been getting several hundred of spam comments per day and i really don’t have the time to go through all of that. Unlike some other bloggers, i refuse to disable commenting in general due to spam so some legit comments might accidentally be flagged as spam and won’t show up. So, if you leave a comment and it’s not showing up immediately, just send me an email (preferably from the same address you filled in for the comment) and i’ll try to rescue it from the spam filter. I do delete spammed comments pretty regularly so don’t wait around if your comment isn’t displayed immediately

Continuous Integration On A Real, Big Project

Davy Brion — Sun, 24 May 2009 19:45:54 +0000

Some of you may remember a post of mine where i showed the complete lack of attention that was being paid to the Continuous Integration of one of our projects. This particular project is pretty big. The development of this project has gone on for years, and will keep going for years to come as it is a strategically important project for us. The actual system is used internally by our company as well as a growing list of customers.

There are essentially 2 big problems with this project. One is a mountain of legacy code with so much technical debt (due to the evils of code generation) it sorta resembles the current economic recession (as in: it’ll take a long time to get everything sorted out). The other problem is more a matter of organization. We’re a pretty small company and while we have great ideas for products, we simply can’t assign a steady, stable team of developers to work on any of these projects on a continuous basis since we all often need to work on stuff that is simply more lucrative at that particular point in time. The result is that this particular project typically has an ever rotating group of developers working on it, usually for short periods of time. Some people work almost full-time on it, but that list is pretty limited.

Not exactly the ideal situation for a large project to apply CI and other agile development practices to, right? Luckily for us, we’re quite stubborn and we try to make the best out of every situation. So back when this project’s CI success rate was only showing a lack of success, we all agreed to follow the CI rules and at this point, i’m pretty proud to be able to show you guys the following picture:

Note: we moved to Team City in July 2008 so i can’t show you any earlier data than that.

Anyways, as you can see, after the dismal months of February and March, the success rate of the CI build gradually started improving again. The average time to fix failing tests also decreased sharply. We still have failing tests from time to time, but at least now they’re all dealt with in a timely fashion. We still have build failures, though those have decreased a lot as well and are always dealt with pretty soon now. I’m not sure if this is because of my ‘Continuous Bitching’ whenever the build fails, or that everyone bought into the concept of CI again (for my own sake, i’ll just assume that it’s the latter instead of the former) but i’m pretty happy with the results that we’re getting.

Also, take a look at the number of tests (we’re at 16000+ now) and the duration of the build. The build time is about 48 minutes on average now, which is obviously way above the recommended 10 minutes. I’d love to see this go down to about 20 minutes (which i’d find very acceptable considering the size of the project) but that’s gonna take a long while. Of those 16000 tests, there are about 13000 tests that cover the legacy code and they all use the database. And since those 13000 tests use a generated data layer, we can’t just let it run on an in-memory database nor can we mock the database in those tests because all of that generated code, and pretty much everything that was written on top of it, is coupled more tightly than Siamese Twins. We also lose a couple of minutes of build time due to our Genesis processing but that is simply something that we can’t go without anymore so we don’t really mind the extra build time of that part.

So there you have it… the reason i wanted to post this is because when the topic of CI comes up, you always read about ‘instant feedback’ and really quick builds and things like that. It’s simply not always like that in the real world. But with a bit of effort and focus, you can get many of the benefits that are usually attributed to CI, even on huge projects with lots of legacy code.

Keep An Eye On Those Indexes

Davy Brion — Thu, 21 May 2009 12:14:57 +0000

We have a multi-tenant application, where each tenant has its own database. We recently were informed about a particular performance problem that one tenant (which we’ll refer to as Tenant A) was experiencing in every screen where data of a certain type needed to be shown. None of the other tenants experienced this problem though.

We tracked down the query that was causing the bad performance and ran it on the database of Tenant B. Tenant B actually had a lot more data in the main table that was used in the query and the query executed immediately whereas it took about 25 seconds to complete for Tenant A. So the query runs fast on another database that actually has more data… at this point i was convinced that it had to be related to indexes.

Turns out that someone recently ran an import process to import a bunch of data in Tenant A’s database. I know very little about databases, but one thing i’ve seen time and time again (with both Oracle and SQL Server) is that you really need to make sure that your indexes are in good shape after any process that performs a lot of inserts (or removals). A couple of years ago, i had a very intensive nightly import process for a particular project that used an Oracle database. As time went on, the application’s queries became painfully (unacceptably even) slow. I managed to restore the performance of those queries by simply instructing Oracle to recalculate all of the statistics of the indexes of tables that were affected heavily during the nightly import.

With that in mind, we simply rebuilt the indexes for Tenant A’s database, and the same query that took 25 seconds completed almost instantly from then on. Now, we did had a weekly job running on that database server to keep the indexes in a healthy shape but that job didn’t really do a good umm… job of it, apparently.

Lessons learned: make sure that you:

Have a proper maintenance job set up which keeps your indexes healthy and schedule it to run regularly
Run that job manually if you need to perform a manual import process
Execute that job in an automated fashion whenever an intensive automated import process has completed

Oh, and consult with your DBA’s or at least people who know what they’re doing when it comes to your particular database on how to keep those indexes healthy. In this case, we rebuilt them. In other cases it’s sufficient to recalculate the statistics… i’m not sure which way is the best but you should at least keep an eye on this possible problem

Using The Guid.Comb Identifier Strategy

Davy Brion — Thu, 21 May 2009 12:00:31 +0000

As you may have read by now, it’s a good idea to avoid identity-style identifier strategies with ORM’s. One of the better alternatives that i kinda like is the guid.comb strategy. Using regular guids as a primary key value leads to fragmented indexes (due to the randomness of the guid’s value) which leads to bad performance. This is a problem that the guid.comb strategy can solve quite easily for you.

If you want to learn how the guid.comb strategy really works, be sure to check out Jimmy Nilsson’s article on it. Basically, this strategy generates sequential guids which solves the fragmented index issue. You can generate these sequential guids in your database, but the downside of that is that your ORM would still need to insert each record seperately and fetch the generated primary key value each time. NHibernate includes the guid.comb strategy which will generate the sequential guids before actually inserting the records in your database.

This obviously has some great benefits:

you don’t have to hit the database immediately whenever a record needs to be inserted
you don’t need to retrieve a generated primary key value when a record was inserted
you can batch your insert statements

Let’s see how we can use this with NHibernate. First of all, you need to map the identifier of your entity like this:




    <id name="Id" column="Id" type="guid" >
      <generator class="guid.comb" />
    id>

And that’s actually all you have to do. You don’t have to assign the primary key values or anything like that. You don’t need to worry about them at all.

Take a look at the following test:




        [Test]
        public void InsertsAreOnlyExecutedAtTransactionCommit()
        {
            var insertCountBefore = sessionFactory.Statistics.EntityInsertCount;
 
            using (var session = sessionFactory.OpenSession())
            using (var transaction = session.BeginTransaction())
            {
                for (int i = 0; i < 50; i++)
                {
                    var category = new ProductCategory(string.Format("category {0}", i + 1));
                    // at this point, the entity doesn't have an ID value yet
                    Assert.AreEqual(Guid.Empty, category.Id);
                    session.Save(category);
                    // now the entity has an ID value, but we still haven't hit the database yet
                    Assert.AreNotEqual(Guid.Empty, category.Id);
                }
 
                // just verifying that we haven't hit the database yet to insert the new categories
                Assert.AreEqual(insertCountBefore, sessionFactory.Statistics.EntityInsertCount);
                transaction.Commit();
                // only now have the recors been inserted
                Assert.AreEqual(insertCountBefore + 50, sessionFactory.Statistics.EntityInsertCount);
            }
        }

Interesting, no? The entities have an ID value after they have been ’saved’ by NHibernate. But they haven’t actually been saved to the database yet though. NHibernate always tries to wait as long as possible to hit the database, and in this case it only needs to hit the database when the transaction is committed. If you’ve enabled batching of DML statements, you could severly reduce the number of times you need to hit the database in this scenario.

And in case you’re wondering, the generated guids look like this:

81cdb935-d371-4285-9dcb-9bdb0122f25f
a44baf99-58e9-4ad7-9a59-9bdb0122f25f
a88300c2-6d64-4ae3-a55b-9bdb0122f25f
032c7884-da2f-4568-b505-9bdb0122f25f
….
70d7713c-b38d-4341-953d-9bdb0122f25f

Notice the last part of the guids… this is what prevents the index fragmentation.

Obviously, this particular test is not a realistic scenario but i’m sure you understand how much of an improvement this identifier strategy could provide throughout an entire application. The only downside (IMO) is that guid’s aren’t really human readable so if that is important to you, you should probably look into other identifier strategies. The HiLo strategy would be particularly interesting in that case, but we’ll cover that in a later post

Why Are Mutable Value Types Even Possible?

Davy Brion — Sun, 17 May 2009 12:06:07 +0000

A coworker of mine was recently trying to figure out why some code wouldn’t compile.

Consider the following code:




            var point = new System.Windows.Point(5, 7);
            point.X = 6;
            point.Y = 8;
            Assert.AreEqual(6, point.X);
            Assert.AreEqual(8, point.Y);

This compiles and works. Now consider the following class:




    public class SomeClass
    {
        public System.Windows.Point SomePoint { get; set; }
    }

And then this small piece of code:




            var myObject = new SomeClass
            {
                SomePoint = new Point(2, 3)
            };
 
            myObject.SomePoint.X = 5;

You’d typically expect that this code would compile, right? Well it doesn’t. It fails with the following compiler error:

error CS1612: Cannot modify the return value of ‘SomeApp.SomeClass.SomePoint’ because it is not a variable

Not exactly what you would expect, right? Well, the compiler error in this case is actually a good thing, though the message isn’t as clear as it should be. See, System.Windows.Point is actually a value type instead of a reference type, and as you should know, value types behave very differently from reference types when passed around. You basically get a copy every time instead of the actual instance.

Let me just borrow the explanation of the “DO NOT define mutable value types” guideline, from Microsoft’s Framework Design Guidelines:

Mutable value types have several problems. For example, when a property getter returns a value type, the caller receives a copy. Because the copy is created implicitly, developers might not be aware that they are mutating the copy, and not the original value.

Now the compiler error makes sense, right? Well, the message still sucks, but it makes sense that the compiler would protect you from making this mistake.

It won’t protect you from the following mistake though:




            var myObject = new SomeClass
            {
                SomePoint = new Point(2, 3)
            };
 
            var point = myObject.SomePoint;
            point.X = 5;

This code will compile, though if you expect that myObject.SomePoint.X will return 5, you’re in for a treat. Hopefully nobody writes code like this, but imagine having to debug something like this in some legacy code that was never properly tested.

But really, why are mutable value types even possible? In what situation does a mutable value type make sense? I’d really like to know, so please point out those situations in the comments if you know of one. The only reason i can think of is performance but the number of situations where a mutable value type is going to give you a significant performance boost is probably so small that it would’ve made more sense to enable mutability for value types with a language keyword or something like that. Unless there is another benefit that i’m not aware of yet?

Who’s In Charge Of The Time?

Davy Brion — Wed, 06 May 2009 16:53:07 +0000

I recently noticed that 2 tests were suddenly failing for no apparent reason. This was in a large codebase where we have a mix of two development approaches. The project is simply too big to convert everything at once to the latest development approach so we’re using Ayende’s Obsolete In Isolation approach on this. The 2 failing tests were in the old parts of the code, so i cringed and then set out to investigate.

The tests of the old part of the code aren’t always easy to comprehend and they use the database all over the place. I figured i’d quickly run them on the database that we use when we run those tests on our local machines, and after that, run them on the database that the buildserver uses (which is rebuilt entirely with each build) to examine the differences between both test runs. They both passed in both cases. They were still failing on the buildserver though.

Upon first inspection of the code of those tests, it didn’t seem like there was any blatantly obvious reason why those tests were failing on one machine, and not on the others. They were inserting some records with some manipulated DateTime values, and then calling some code which would execute a query which should have retrieved these values. The records inserted by the test were not retrieved for some reason, which caused the tests to fail.

DateTime values and tests can be tricky sometimes so i started looking in that direction. The queries that where executed in those tests would use the current date/time together with a margin of a couple of minutes to retrieve records where a certain date/time column’s value would fall in the range of the current date/time with the given margin. The records that were inserted in the tests would use the current date/time, minus 2 minutes, or plus 2 minutes. Hmm… this could possibly be flaky depending on how the query determines the current date/time. If the query receives the current date/time from the calling code, then this wouldn’t cause problems but if it uses the current date/time of the database, then this is not only a flaky test, but a real bug in the code.

I then checked the system clock of the buildserver, and as i suspected, it was about 4 minutes ahead of the system clock of the database server. Normally, all of our servers’ system clocks are synchronized, but for some reason, the buildserver hadn’t had its system clock synchronized since the day before. The code was inserting records in the database with modified date/time values, based on the system clock of the system the code was running on, and was then querying the database using the database’s system clock. Since the time difference was big enough, the query would not return the expected rows and this made the tests fail on the buildserver.

Obviously, this never would’ve happened if the query would’ve used a parameter supplied by the caller which would’ve represented the current date. And that is the real bug that these flaky tests (which up until this particular incident never failed) highlighted. A situation like this could’ve occurred in production if the application server’s clock and the database server’s clock would’ve gotten out of synch. Can you imagine the pain you’d go through when investigating a bug report like this? :p

Date/Time values are often tricky, especially when there is more than one source which could determine the current Date/Time. So do yourself, and anyone who will maintain your code down the line, a favor by making sure that only ONE source is in charge of Date/Time values in your system. Using the database for all date/time values is definitely not always possible, so it’s best to have your application server be in charge of all date/time values. If you need the current date/time in a query, simply supply it as a parameter to the query and you’ll avoid problems like this.

Though this advice does leave me wondering what the best approach to deal with this is when you have multiple application servers which could assign date/time values.

On Getting Rid Of Common Libraries… What About Don’t Repeat Yourself?

Davy Brion — Sat, 02 May 2009 17:26:49 +0000

Just wanted to post a small reaction to Ayende’s dismissal of Util & Common Libraries. In his post, Ayende goes over everything that Rhino.Commons (his own library) offers and why he’s not using it anymore. If you go over the list of everything in the library, i can definitely see why. As Ayende says, it’s the garbage bin for anything that he came up with in the past couple of years, regardless of how many projects would actually reuse each specific part.

At work we have also have something like this, though in some ways it’s smaller in scale, and in some ways it offers a bit more than Rhino Commons. A lot of it is based, or are further developed versions of, the infrastructural bits that i’ve posted on this blog in the last year. We also have some other interesting things in there that my coworkers wrote. We took the opposite direction however.

We started off with putting these classes in each project where we could use them. But since a lot of that stuff was still being modified or extended on a semi regular basis, we were constantly making the same changes or additions in multiple places, which violates the Don’t Repeat Yourself principle. Essentially, it was duplicate code, just spread in multiple projects. Obviously, this brought along all of the same problems that duplicate code usually brings with it.

So we set out to figure out to best way to deal with this, and i even asked you guys about it. We eventually ended up moving these bits to some sort of in-house library/framework that all (or most) of our applications now use. While i was the one who was hesitant at first to go with a reusable library/framework (due to some bad experiences with it in the past), i’m now very glad we actually did do it.

We did go for multiple assemblies (ie: web, silverlight, service layer, …) instead of putting everything in one big assembly though we did try to follow my ‘one assembly per physical deployment‘ rule.

Bugs get fixed in one place and one place only. The only repetitiveness we have to endure when fixing a bug in the library is making sure we can merge the fix to previous versions and updating the referenced binaries in each project that uses it. That’s it. New stuff only gets added when it’s been proven to be useful in more than one project. We don’t just add new things like that, unless it really makes sense to do so.

I think most of the problems that Ayende now sees with Rhino Commons can be avoided by splitting things up a bit more, and by being more restrictive as to what actually makes it into the library. Versioning suddenly becomes pretty important as well. These are all things that i always felt where somewhat missing with Rhino Commons. That’s not to criticize the project because i do think there’s a lot of valuable stuff in there, but in it’s current form it’s just not very well suited to be reused by other projects.

Having said all that, i’m not saying that reusable libraries/frameworks are the best thing since sliced bread either. They require a lot of discipline and it’s easy to get tripped up simply by adding something or making a minor change where you didn’t really think all of the consequences through. But the downsides of dealing with these libraries are usually smaller than the downsides of having the same code spread out over multiple projects.

Using SQL Functions in Criteria Restrictions

Davy Brion — Thu, 30 Apr 2009 11:31:53 +0000

A coworker needed to use a SQL function in the where clause of a query that he was creating with NHibernate’s ICriteria API. Most examples of this on the web use HQL instead of the ICriteria API and since we primarily use the ICriteria API we looked into how to do this.

Turns out it is pretty simple to do, though the syntax isn’t really straightforward. Suppose you want to query all of your employees who are born in a specific year. You could mess around with some DateTime parameters, but most databases have SQL functions to get the year from a date. Using the ICriteria API, this would look like this:



            var employeesBornIn81 = session.CreateCriteria<Employee>()
                .Add(Restrictions.Eq(Projections.SqlFunction("year", NHibernateUtil.DateTime, Projections.Property("BirthDate")), 1981))
                .List<Employee>();

which adds the following where clause to the SQL statement (on SQL Server 2005):

WHERE datepart(year, this_.BirthDate) = @p0;

The Good, The Bad And The Ugly In The .NET World

Davy Brion — Wed, 29 Apr 2009 21:21:37 +0000

I’ve gotten quite a bit of feedback on my previous post, and there are a couple of things i’d like to address. It seems that some people kinda took offense to my statements about the rarity of .NET developers with knowledge of certain practices/principles. First of all, i would like to state that i have been working in the .NET world for about 4 years now and have worked with quite a few .NET developers during those years. Most of you already know this and i feel stupid for even mentioning it, but apparently there were people who read the previous post thinking i didn’t know anything about the .NET world and that i had based my statements on just the one .NET developer that i mentioned in the post.

Anyways, back to the topic at hand… the quality of developers in the .NET world. As some people mentioned in the comments of the previous post, Microsoft itself thinks of developers in 3 categories: Mort, Einstein and Elvis. I couldn’t really find an ‘official’ Microsoft link about these personas but they are pretty well known by now. Here’s a quick summary:

Mort, the opportunistic developer, likes to create quick-working solutions for immediate problems and focuses on productivity and learn as needed. Elvis, the pragmatic programmer, likes to create long-lasting solutions addressing the problem domain, and learn while working on the solution. Einstein, the paranoid programmer, likes to create the most efficient solution to a given problem, and typically learn in advance before working on the solution.

I believe that Mort is the single most influential individual in the .NET world. Everything revolves around Mort, and this is where a lot of the strengths and weaknesses of the .NET platform in general come from.

People who are new to .NET can usually be productive in a pretty short timeframe. The tools are often relatively easy to use and there is plenty of documentation and Getting Started material available. No matter how you turn it, that is a strength of the platform. However, what is generally lacking here is some kind of follow-through on the fundamentals of healthy, maintainable software development. We’ve all seen the countless simplified, hello-world examples that are often demonstrated at Microsoft events. Some people consider me arrogant when i say this, but i truly believe that this kind of Drag-N-Drop Development is generally unhealthy. Does it have its place in the right context? Sure, as much as just about anything else has within its context. Take it outside of its context and things can get messy pretty fast.

Getting Mort hooked on this kind of development, without following through on Mort’s education is like teaching construction workers how to quickly put up a wall without educating them about making sure that wall is gonna hold up over the years through all of the forces that will act upon it. But hey, the wall is there, right? Time to move on to the next wall! Or better yet, let’s have Mort do some electrical work while he’s at it. After all, we already know he’s adept at construction and he finishes his work quickly. Whenever he has a few minutes time, we’re going to have him work on the plumbing as well. He’s got a great set of tools at his disposal, so he should be able to do all of this pretty quickly without investing too much time in his educational process.

There are tons of Morts in the .NET world. And Microsoft is constantly building and releasing more and more libraries and tools to make sure that Mort can quickly complete any task that is assigned to him, no matter how fit or unfit he is to complete that task in a sufficient manner. I think we all know that there is a lot of horrible .NET code that has been written over the years and i truly believe that a lot of that could have been avoided if more effort had been put into properly educating Mort.

Steven Smith left the following gem in his comment on my previous post:

In the Microsoft space rather than respond to the increasing demands for greater productivity by always seeking more productive and powerful development practices, that is how can we use our tools more effectively, the general mindset seems to be that Microsoft is bringing out a new version of .NET or some new visual tooling, what we perceive to be a more powerful tool and will solve our problem. But a powerful tool misused can cause a great deal of damage. Developer’s continuing professional development seems to consist of learning new Microsoft or third party APIs or attending Tech Ed rather than learning development principles and practices that are language and provider independent.

I think i can safely say that this will sound very familiar to a lot of you who are reading this.

So, is all hope lost? Certainly not. In the past 2 years, we’ve seen some tremendous improvements in the .NET community, mostly brought on by the ALT.NET movement which focused more on writing clean, maintainable code using proven design practices and software development principles in general. Regardless of what you (or me) personally think of the ALT.NET movement in its current form (or lack thereof), there is no denying the impact that it has had already. Microsoft appears to be focusing more and more on the topics that are typically associated with ALT.NET. All of a sudden, Microsoft has libraries for Inversion Of Control, for MVC web development, Object Relational Mapping, and more. These tools might not appeal to everyone who’s already used more mature/powerful alternatives, they are still a very good sign. It not only shows that Microsoft is willing to invest more effort into this, but it also increases the exposure of these principles and practices to Mort. It’s a slow process for sure, but the wheels do appear to have been put in motion.

If we can combine this with a more concentrated educational effort, we might be able to get to where Steven Smith and myself would like to get:

I would love to work with kindred spirits, people who consider their effectiveness as a developer is largely based on the power of the practices they employ and are always seeking more powerful practices that are being developed in the wider development community and apply them in their .NET teams.

Perhaps someday the majority of us, instead of a very small minority, will.

At This Point, I’d Prefer Java Developers Over .NET Developers

Davy Brion — Mon, 27 Apr 2009 20:37:18 +0000

I had to do 5 technical screenings today. 4 of these guys where Java developers, one of them a .NET developer. A couple of weeks ago, i also had to screen a Java developer. I’m gonna be blunt about this: at this point, i would much rather hire a Java developer than a .NET developer.

Let me just summarize these screenings to illustrate my point. Out of the 5 java developers i had to screen recently, 4 of them were very familiar with topics such as Inversion Of Control, Dependency Injection, Object Relational Mapping, and Aspect Oriented Programming. One of them was a ‘junior’ developer with about 2 years of experience, and we actually talked about the differences between compile time AOP weaving vs runtime weaving, even going into the differences between dynamic proxies and more traditional proxies to achieve AOP and how that would work and how it can be tied into your IOC container. This wasn’t just me talking about this stuff… i was leading into certain things and he was filling in the blanks. During a technical screening. With a guy who only has 2 years of experience on his resume.

The other 3 java developers i screened where pretty impressive as well. These guys truly realized the benefits of using an Inversion Of Control container, of using Dependency Injection and felt it was only natural to use an ORM when working on typical business applications. And you know what the really cool part about all this was? All 4 of those java developers that impressed me wanted to learn more about TDD, and writing automated tests in general. They hadn’t really experienced Continuous Integration yet. They wanted to improve their TDD skills. They genuinely seemed to care about the fact that their TDD skills weren’t as well-rounded yet as they could be. In case you haven’t noticed, these people truly believe in DI and IOC without even getting into the testability aspect.

In the .NET world however, most people who get into DI and IOC usually do it for testing purposes, only to move on to the other benefits of it later on. And that is if you’re lucky enough to find a .NET developer who even cares about these things. Generally speaking, you should consider yourself lucky if you get to screen a .NET developer who’s interested in these ‘alternative’ practices (in the .NET universe anyway), let alone one who actually has experience with them. The odds of finding .NET developers like this in interviews are so incredibly low it’s not even funny anymore. It’s downright pathetic actually. And then you get 3 of these guys in one day, though they are java developers. 4 of them if you count the one from a few weeks ago. These numbers don’t truly mean anything, but still… i think it’s worth thinking about.

Now, i’m not going to say that Java developers are by definition better than .NET developers. But i really do think that, generally speaking, they know a lot more about proper design principles and coding practices than your average .NET developer.

Help Us Name This Method

Davy Brion — Fri, 24 Apr 2009 13:48:35 +0000

This worked pretty good last time so i’m gonna try it again. We have a Repository base class (yes we still use it, and no, i’m not going to get into the discussion that’s going on about in on other blogs) which uses NHibernate underneath. We have the typical Get method which retrieves an object based on an ID. This method uses NHibernate’s ISession’s Get method, which either returns the entity if it’s present in the database, or it returns null.

Some people want to put an equivalent to NHibernate’s ISession’s Load method into the base repository class, but we just can’t come up with a good name for it. In case you don’t know, ISession.Load either returns the actual entity if it’s already in the session cache, or it returns a proxy for the entity but it does not immediately hit the database. The functionality of the Load method is thus very different from the Get method. Now, i think this is a case of extremely bad naming within NHibernate, and i don’t want to have the same naming issue in our Repository implementation.

So basically, the question is: how would you name the equivalent of the Load method? Remember: this method returns a proxy to an entity if it’s not yet in the session cache, whereas the Get method actually retrieves the entity from the database immediately or returns null if the entity for the given ID does not exist.

Small Milestone

Davy Brion — Tue, 21 Apr 2009 11:20:57 +0000

Finally