Performance on-demand; Giving your ops team runtime flexibility

Performance On Demand

Pretend you are in an operations position, in which your job is to maintain the infrastructure that routes traffic and the servers that serve requests. Wouldn’t it then be nice, if you suddenly had a surge in traffic or a drop in available server hardware (be it expected or unexpected), you could alter the performance characteristics of your web applications?

This is a problem we’ve been tackling with our new set of web apps, and we think we’ve got a pretty good solution in place.

Operations Administration Panel for Runtime Configuration

For starters, we’ve created an administration web application for our operations folks, whose primary purpose is that of runtime configuration. Operations can control various aspects of our systems from this application, including:

• Logging levels
• Caching TTLs
• Database masters/slaves and replication strategies
• Application settings
• Network locations for editorial assets
• Logical service bus participants
• Etc.

When any of these settings are updated, we send a message on the service bus informing subscribers of changes in the settings they care about. Let’s analyze the one we made reference to above, which will provide ops with a way to dial in performance on demand.

Output Caching

In the administration panel, we’ve provided a settings page where the output caching TTL and data caching TTL can be set for a given application. When this setting is updated, we publish a message on the service bus, which our front end rendering ASP.NET MVC application can subscribe to.

Creating a handler in the rendering application then is pretty easy. We listen for the settings type that corresponds to caching:

    public class CacheSettingsUpdater : SettingsChangedHandler<CacheSettingsData>
    {
        protected override bool ShouldHandle(string id)
        {
            return string.Equals(
                id,
                CacheSettingsData.StorageId,
                StringComparison.OrdinalIgnoreCase);
        }
 
        protected override void Update(CacheSettingsData settingsData)
        {
            CacheSettings.UpdateSettings(settingsData);
        }
    }

As you can see, the handler can then inform a settings class by calling its “UpdateSettings” method, who keeps a reference to the latest data.

    public static class CacheSettings
    {
        private static CacheSettingsData Data = new CacheSettingsData();
 
        public static int OutputCacheDurationSeconds
        {
            get
            {
                return
                    Data.CurrentAppCacheParameters
                        .OutputCacheDurationSeconds;
            }
        }
 
        internal static void UpdateSettings(CacheSettingsData data)
        {
            Data = data;
        }
    }

Leveraging it with OutputCacheAttribute

Now, in ASP.NET MVC, there is an action filter for output caching: OutputCacheAttribute. This attribute can be applied at the controller level, or at the individual action level. When an action is run the first time, the framework will cache the result, such that the next request won’t require processing again, and will be delivered from cache. The cached item will be delivered from cache until the TTL/Duration expires. The effect of this is that your application won’t be processing for every request, and will be able to therefor serve more requests.

The issue with connecting up our runtime configuration class from above (CacheSettings) to the OutputCacheAttribute, is that the settings for a filter can only be specified by constants, like so:

    [OutputCache(Duration = 10)]
    public ActionResult Index()

So, we need to instead create our own action filter, which inherits from OutputCacheAttribute, so we can control where it gets its values from. I’ve simplified this for brevity to just illustrate the Duration extensibility point.

    public class ConfiguredOutputCacheAttribute : OutputCacheAttribute
    {
        public new int Duration
        {
            get { return base.Duration; }
            set
            {
                throw new NotSupportedException(
                    "Duration cannot be set directly. " +
                    "Set from runtime config.");
            }
        }
 
        public ConfiguredOutputCacheAttribute()
        {
            base.Duration = CacheSettings.OutputCacheDurationSeconds;
        }
 
        public override void OnActionExecuting(
            ActionExecutingContext filterContext)
        {
            base.Duration = CacheSettings.OutputCacheDurationSeconds;
            base.OnActionExecuting(filterContext);
        }
    }

As you can see, when we hit OnActionExecuting, we check the CacheSettings class for the current output cache duration, and set it on the base OutputCacheAttribute class we inherited from. The effect of this, is that during day to day traffic, operations can control the cache TTL.

Then we just apply it where we want to cache:

    [ConfiguredOutputCache]
    public ActionResult Index()

Well, how did we do?

Let’s see what it looks like if I simulate light traffic load. The red line indicates request execution time.

Turning on output caching results in a dramatic drop off in request execution time.

The dramatic drop off occurred when I went into the operations administration panel and changed the TTL. The spikes every 10 seconds following the drop off are when the cache duration TTL expired, forcing the page to actually process again.

There are a number of things we can do to enhance the flexibility of this system. For example, we could specify groupings in the operations administration panel that correspond to cache policies, and then simply specify on each instance of our attribute which policy we’d like to use:

    [ConfiguredOutputCache(CachePolicy = "FooCachePolicy")]
    public ActionResult Index(string streamSlug)

We think this feature will be particularly valuable in situations where we need more performance on demand, and look forward to extending it to have more flexibility as needed.

Happy coding!

Javascript Encapsulation

After having read Douglas Crockford’s book “Javascript: The Good Parts”, I had learned some ways to leverage objects and information hiding, in the following manner:

var fooInstance = function() {
 
	var privateAlpha;
 
	return {
 
		getAlpha: function() {
			return privateAlpha;
		},
 
		setAlpha: function(a) {
			privateAlpha = a;
		}
 
	};
 
}();
 
fooInstance.setAlpha(10);
console.log(fooInstance.getAlpha()); //10
fooInstance.setAlpha(20);
console.log(fooInstance.getAlpha()); //20

However, if you do a quick search on StackOverflow, you’ll quickly encounter a huge number of answers on javascript questions which say not to bother with private variables. Their reasoning is “well, you can just look at the javascript source in the browser, so what’s the point?”. This is missing part of the point.

One of the primary points of encapsulation is to control the internal state of the object, by only providing a limited interface. This protects the integrity of the state.

BUT… the other purpose of private variables and encapsulation in any language is not simply security or protection from nefarious people. It’s abstraction. An external entity should not need or care to know how the internals are accomplished. The smaller the footprint of the interface, the less complexity exists for the consumer to understand. You should expose no more than is necessary to leverage the functionality. The result of doing so is code that has looser coupling and is easier to understand by those who wish to consume it.

The Old Republic at Ben’s

While not really relating to Software Development, I should point out that I’ve created another blog covering Star Wars: The Old Republic, the new MMO coming out in December.

Check it out here: http://benlakey.com/tor/

Introducing List Indexer

I decided to create a simple web app using ASP.NET MVC 3, Backbone.js, and Raven DB. The result is List Indexer (listindexer.com). It’s very simple, and similar to things like bit.ly, except that its intent is to persist and share lists. Examples: Grocery list, wish list, favorite things, top 10 tech companies.

Give it a try, and let me know what you think. Feel free to log issues or suggestions at it’s github site here: ListIndexer at GitHub

Complexity

“Computer programs are the most complex things that humans make. Programs are made up of a huge number of parts, expressed as functions, statements, and expressions that are arranged in sequences that must be virtually free of error. The runtime behavior has little resemblance to the program that implements it. Software is usually expected to be modified over the course of its productive life. The process of converting one correct program into a different correct program is extremely challenging.”

-Douglas Crockford

How do you measure programmer productivity?

How do you measure programmer productivity?

When the Lisa team was pushing to finalize their software in 1982, project managers started requiring programmers to submit weekly forms reporting on the number of lines of code they had written. Bill Atkinson thought that was silly. For the week in which he had rewritten QuickDraw’s region calculation routines to be six times faster and 2000 lines shorter, he put “-2000″ on the form. After a few more weeks the managers stopped asking him to fill out the form, and he gladly complied.

Feedback Loops

I originally posted this on our development blog over at MSNBC.

Are you familiar with feedback loops? A feedback loop is a loop in which information encountered by the system is fed back into the system, and the system can then respond to it. You’ve probably seen a diagram similar to this:

Feedback loops are very important for a system since they allow the system to evolve and adapt to change. A great example of this seen in nature is the human body’s temperature regulation:

Without feedback, the body would have no way of making the minor adjustments it needs.

Feedback loops turn out to be extremely valuable in software development as well. Agile development practices all relate to the notion of a feedback loop:

• Unit tests
• Pair Programming
• Sprints
• Stand-up meetings
• Code reviews

You may notice something about these practices; The feedback loops are very fast. Feedback loops that are slow are of little value, because by the time you respond to one set of feedback, the next may have already arrived, which would render your initial response meaningless.

In the example of body temperature regulation, it would be pretty useless to have the body react with sweat hours after a temperature change occurs.

We want feedback loops to be as fast as possible so that we can find out whats working, whats not working, and what needs to change right now.

1. Change something
2. Find out how it went
3. Learn from it and adjust
4. Rinse and repeat

Automation is a great way to achieve our feedback loop goals in software development. In addition to the agile practices above, we can also try to implement the following whenever possible:

• Continuous integration
• Increasing the frequency at which code is released
• Automated acceptance tests
• Automated code analysis

A set of tight feedback loops like these can allow us to respond to change rapidly. Every time someone checks in code, the automation can tell us immediately whether we are in a healthy state, which will allow us to immediately correct any problems if they occur.

The Law of Demeter

Hello boys and girls, it’s been a while. Let’s talk about OO design.

Have you heard of the Law of Demeter? If you develop software, you should. You may actually be familiar with its concepts without actually knowing that it had a formal name. At it’s core, the Law of Demeter is the explicit capture of the intent of encapsulation. The rules are as follows.

A method FooMethod of a class Foo should only call the methods of these:

• Foo
• An object created by FooMethod
• An object passed as an argument to FooMethod
• An object reference held as an instance variable of Foo

Uncle Bob summarizes it nicely: “In other words, talk to friends, not to strangers.”

This all sounds dandy and obvious, but it’s easy to forget and slip up when dealing with complicated designs, or designs that are poorly understood.

Happy coding.

Criticality and Communication

The title of this blog post was originally to be ‘criticism’, but This Developer’s Life already used that. I suppose it’s for the best anyhow, since the idea behind this post is to address not only criticism, but the ability to communicate as well. It stems from a discussion my wife and I had as we walked home from a screening of the extended edition of Lord of the Rings: Fellowship of the Ring last night.

Being a nerd, engineer, computer scientist, programmer, coder, web developer, or any other such title that you can lump us into, is hard. It’s not only hard because of the increasing complexities of emerging technologies. Some of the biggest difficulty for us is actually around interpersonal relationship skills. We find others to be strange, alien creatures, who we do not understand. We marvel at how easily it can come to some, but at the same time look with a perplexed eye, since we do not think as they do. Why is this? It’s because we see the world through monochrome tinted glasses; We see a world of boolean logic, in which something either is, or it isn’t. It is with this polarizing conviction that we derive our deep seated views, holy wars about technologies, and get extremely testy when confronted. Further, because the world is of course not black and white, true or false, we find middle ground to be… uncomfortable. So then what you end up with is a personality type that is not only polarized, but also critical. We are critical of those things which are not tidied, tied off, cleaned up, made parallel, and understood. It’s really a double edged sword.

So, when you confront someone with a different view from their own, and that view is firmly seated in their mind, what happens? The reaction is, perhaps surprising, perhaps not, that they actually will stick to what they know more adamantly than before. This is known in the psychology community as the ‘backfire effect‘. My wife posits that this could actually be the cause of many of the world’s serious problems, such as religious wars, government policy stance on moral issues, etc. So what is one supposed to do to attempt to pry open anothers mind? How do we do it without seeming like we’re forcing our views on theirs? We just want to crack through ever so slightly, so as to allow alternative perspectives to enter their locked mind. I’m not sure I have the answer on how to do this. Maybe you do.

Nerd psychology is a vast, complex topic. The personality of the ‘alpha geek’ is so pervasive in the industry of software engineering, that you can often times learn how to deal with many, by just basing your interactions off of one. The attitudes of “I know it all”, and “this way is best”, and “obviously this technology X is not as good as Y”, are like a poison, a poison which threatens to turn our programmers world of creation into a cesspool of bickering, snapping at one another, dismissing eachother as not intelligent, and an all around gnashing of teeth. Too often this poison presents itself, and when it does, its typically accompanied by a very unpleasant, snarky, “I’m better than you” attitude. This attitude leaks through in a very obvious way, even when it is not vocalized or intended to be communicated. We must drive this poison out, and attempt to make eachother see the world through a spectrum of colors. The value of other perspectives in software engineering cannot be overstated, and is too often a blind spot for otherwise gifted developers.

Const vs readonly

In C# when you want to have a named constant you have a choice between ‘const’ and ‘readonly’. For example:

public const string FooConst = "Bar";
public static readonly string FooReadonly = "Bar";

Which is appropriate?

Assembly Recompilation

You should only ever use ‘const’ on private fields. Using ‘const’ on a public field means that constant must be compiled into any consuming assembly. You might ask why having a constant compiled into consuming assemblies is a problem: The issue is that if you change that field, all consuming assemblies must be recompiled. Using ‘static readonly’ will not require consuming assemblies to be recompiled.

Flexibility/Mutability

Using ‘const’ is only possible with primitives, other value types, and strings, as its value is burned directly into the assembly. ‘Readonly’ on the other hand has no restrictions whatsoever, since it’s just a one time allocation of memory that has measures in place to prevent modification.

Also important to note is that ‘readonly’ only prevents the reference from being modified, not the actual memory being pointed to by the reference. To illustrate, the following is legal:

    public class Foo
    {
        public static readonly IList<string> barReadonly = new List<string>();
 
        public static void AppendString(string newString)
        {
            barReadonly.Add(newString);
        }
    }

Run time vs Compile time

A ‘const’ must be initialized immediately in code, and is initialized at compile time. A ‘readonly’ can be initialized at runtime, so long as it’s taken care of before the ctor exits.

Summary

In general, use ‘const’ when you know for certain that a value will never change, ever, or if the value is private. Otherwise, use ‘static readonly’.