The simple and yet complex case of KISS

I was blown away when first introduced to KISS principle. At the time, I was frustrated with how overly complicated our enterprise software was designed. It was a rewrite of a legacy system and our architect single-handedly wrote a custom framework on top of ASP.NET MVC framework, with the goal to make it more succinct and extensible.

The only problem, was that everybody was so confused about how it worked, that we lined up to the door of the architect with questions on how to implement particular logic. I carried that bitter taste of over-engineering for a long time, and have rigorously asked myself every time I make any design decisions, if my design was not simple enough...until now.

Today, a very smart and capable colleague criticized a simple piece of my code for being over-engineered, it was a surprise.The problem was fairly simple, so I skipped over things like IoC container, unit testing and the like, implementing only a very simple MVC design, with model layer logic located in separate project, using a single-class Micro ORM (PetaPoco) as opposed to a full-blown ORM.

In my mind, things cannot get simpler than this. Well, technically it can, like skipping the overhead of unit-testability, keeping all logic within single project, using built-in SqlDataReader instead of Micro ORM. However those measures only reduces complexity marginally, while making future refactoring exponentially more difficult. In my book, those are bad trade offs.

What I failed to recognize was that, you see, I have been writing code this way for many projects by now and all those concepts and code structures are intuitive to me. Writing and reading code structured this way is as straight-forward to me as code structured in the simplest possible way (TM).

This was not the case for my colleague.

He has different background than I do, and has grown different habits. All the "intuitive" project structure, the MicroORM, and unit testable code was extra complexity to him. While he recognizes the value of all those things, those are extra complexity that has no potential return due to simplicity of the project.

In the end, after some discussion with him and mental struggle with myself, I got rid of most of the "fluffy" stuff. I also lost the bitter taste for the "over-engineering" architect, after all, the gigantic custom framework that I found so overly-complicated was probably "simple enough" for him.

Lesson learned: Simplicity, like anything, is subjective -- what is simple to one person may not be the same for another. And when working in a team, you have to consider the whole team when making design decisions.

Premature Abstraction

Everyone knows that premature optimization is root of all evil and in extreme cases the programmer who does such thing will be stoned to death.

But not many people seem to be bothered with premature abstraction, or encapsulation or whatever people fancy it. Here is what I mean by premature abstraction: creating a train of complex framework to perform simple tasks to make them future proof.

This is a natural tendency after becoming more and more fluent in OO design, as one of the main advantage of OO is abstraction, to make code reusable and extensible. But like making a dish, if you put in too much of any sauce, it will ruin the taste. Abstraction is no exception.

Here is what will actually happen when you design something to be "future-proof": most of the anticipated requirements will be forgotten, and many new ones will show up from no where and on things you never thought of. When such thing happens, it'll be a painful experience to refactor that neatly designed and implemented framework to accommodate new requirements. If the project happen to be behind schedule at the time, which is always, guess what, you may not get the leisure and time to refactor the framework and hacks/workarounds will be applied to make it even harder to refactor. In fact once enough duct-tape is being applied to that originally neatly designed framework, nobody will dare to touch it any more.

Now I think about it, there is actually a proper name for it: it is called over-engineering. And it is quickly surpassing premature optimization to become the new root of all evil.

I will stone you next time I see you do that.

Does Inheritance breaks Encapsulation?

Because inheritance exposes a subclass to details of its parent's implementation, it's often said that 'inheritance breaks encapsulation' (Gang of Four 1995:19)

No I did not read the GoF book, I'm quoting them because I believe quoting important people makes my point more correct.

So we know that inheritance breaks encapsulation because it exposes a subclass to details of its parent's implementation. But what is the harm of exposing parent's implementation to subclasses? Why does it matter?

Imagine this situation: you are coding a very core class for a system, and decided to make a field protected so it's convenient for subclasses to access it. After all, it is a core member field that all subclasses need to use and declaring it as protected seemed like reasonable thing to do. Everything worked fine and smoothly in the beginning. Then 5 years passed, assuming the system is actually making money for your company, it has grown substantially through a steady stream of feature requests. And many subclasses were created inheriting the core class you wrote. Now imagine you have to change that protected field, and *bam*, you're stuck! Because there are many subclasses sprang across many assemblies, and nobody has a headcount of them all! So you either end up not changing it, or do the change and fix as many subclasses as possible, then prey you don't miss any.

Does this sound familiar? Yes, this is the exact scenario when someone accidentally declared a field public for convenience when it should have been properly encapsulated, I hope it's not you, oops! (I'll be honest, I did this before!)

I believe this is the context when those influential people declared that inheritance breaks encapsulation. But a class does not have to expose its implementation to subclasses, take Hashtable for example: it is carefully designed to hide all implementation details from not just the rest of the world, but also its subclasses. So if there is a change to the default hashing method, how data is stored internally or any of the implementation details, all the subclasses created by developers across the world will not be broken! I believe a more proper statement would be:

Overusing protected members breaks encapsulation.

And the remedy to that? Don't overuse the protected keyword. :)

The problem of too many layers of indirection (abstraction)

So everyone has probably heard this by now

All problems in computer science can be solved by another level of indirection

and the corollary

...except for the problem of too many layers of indirection

While it sounds smart and witty, I have never quite figured out what the corollary is referring to. After all, our society is built based on layers and layers of indirection abstraction, and that's how we advanced into modern society. It is a proven concept.

Well, let me step back a little and talk about an interesting issue I had recently. It was decided that our enterprise contract management software is not responsive enough and a few engineers were tasked to take a look at the problem.

Thanks to the advancement of software development, we now have awesome tools like dotTrace, among others, to simplify the daunting task of inserting time stamping instructions into every single method in the application. Looking at the profiling result, an interesting method quickly grabbed my attention -- it is a heavy lifter which makes up 40% of the overall page load time. Upon closer investigation, I realized that it is our new navigation tree which loads a gigantic metadata file that contains all information there is to know (like data dependency, context relation, data validation rule, permission rule and etc). To be more specific:

1. it was reloading the (static) data every page load.
   
2. only a small amount of data (those related to current page) is actually required.
   
3. it was executed twice on every page load.

Let's forget about the first two, and focus on the third problem. How did this happen? I mean, this looks like a simple problem that even junior programmers would know to avoid.
Well, the truth is, the original developer implemented the navigation module nicely, with caching and stuff, so the heavy-lifting method will never be called twice. Then a few months later, someone had to fix a caching bug -- the cached navigation tree became corrupted for some unknown reason. He looked around, and found a little method that was nicely packed and seems harmless, which will solve his problem by rebuilding the corrupted data. It was a perfectly logical choice on his side, although little did he know that about 3 abstraction layers down the road, it reads a gigantic xml file and create a few hundred objects on the fly.

Maybe the original developer should have documented this code better, maybe the other developer should have been more cautious when using other people's code. But the real issue is, abstraction hides so much detail that gives you a false sense of confidence. It makes you believe you know everything, after all, the method name and comment will be sufficient to describe what it does right? (hint, no) If it does, it would have to explain what all its function calls do, and the functions called by those functions it calls, and the functions called by those functions called by those functions it calls...

Every abstraction layer does not only adds a little over head to the CPU, but also to the poor human who has to read that code. Be careful, those little overheads may come and bite you one day.