Friday, December 21, 2012

Slicing Concerns: Implementations

In Slicing Concerns And Naming Them I posed a question about how to go about separating different concerns while still maintaining a clean and relatable code base.  Some interesting conversation resulted, and I wanted to follow up by investigating some of the different approaches to this problem that I'm aware of.

Inheritance
public class Task : ActiveRecord
{
  public string Name { get; set; }
  public int AssignedTo_UserId { get; set; }
  public DateTime DueOn { get; set; }
}

public class NotificationTask : Task
{
  public override void Save()
  {
    bool isNew = IsNewRecord;
    base.Save();
    if (isNew)
      Email.Send(...);
  }
}

public class TasksController : Controller
{
  public ActionResult Create(...)
  {
    ...
    new NotificationTask {...}.Save();
    ...
  }

  public ActionResult CreateWithNoEmail(...)
  {
    ...
    new Task {...}.Save();
    ...
  }
}
This works, and the names are reasonable. But of course, inheritance can cause problems... I wont go into the composition over inheritance arguments as I assume this isn't the first time you've heard it!

Decorator
public class Task : ActiveRecord
{
  public string Name { get; set; }
  public int AssignedTo_UserId { get; set; }
  public DateTime DueOn { get; set; }
}

public class NotificationTask
{
  Task task;

  public NotificationTask(Task t)
  {
    this.task = t;
  }

  public void Save()
  {
    bool isNew = t.IsNewRecord;
    t.Save();
    if (isNew)
      Email.Send(...);
  }
}

public class TasksController : Controller
{
  public ActionResult CreateTask()
  {
    ...
    new NotificationTask(new Task {...}).Save();
    ...
  }
}
This is not really the decorator pattern... At least not as defined by the GoF, but I have seen it used this way often enough that I don't feed too terrible calling it that. Really this is just a wrapper class. It's similar to the inheritance approach, except because it doesn't use inheritance, it opens us up to use inheritance on the Task for other reasons, and apply the email behavior to any kind of task.

The naming is a bit suspect, because NotificationTask is not really a task, it just has a task. It implements only one of the task's methods. If we extracted an ITask interface we could make NotificationTask implement it and just forward all the calls. This would make it a task (and a decorator), but would also be crazy tedious.

Service
public class Task : ActiveRecord
{
  public string Name { get; set; }
  public int AssignedTo_UserId { get; set; }
  public DateTime DueOn { get; set; }
}

public class CreatesTask
{
  Task task;

  public NotificationTask(Task t)
  {
    this.task = t;
  }

  public void Create()
  {
    t.Save();
    Email.Send(...);
  }
}
This service represents the standard domain behavior for creating a task. In an edge case where you needed a task but didn't want the email, you would just not use the service.

The naming is pretty nice here, hard to be confused about what CreatesTask does... However, this path leads to a proliferation of <verb><noun> classes. In the small it's manageable, but as they accumulate, or as they start to call each other things get confusing. For example, if you know nothing about Task and you have to start working on it, would you know you should call the CreatesTask service? Would you know it exists? And would you be sure it was the correct service for you to be calling?

Dependency Injection
public class Task : ActiveRecord
{
  public string Name { get; set; }
  public int AssignedTo_UserId { get; set; }
  public DateTime DueOn { get; set; }

  INotifier notifier;

  public Task(INotifier notifier)
  {
    this.notifier = notifier;
  }

  public override void Save()
  {
    bool isNew = t.IsNewRecord;
    t.Save();
    if (isNew)
      notifier.Send(...);
  }
}

public class TasksController : Controller
{
  public ActionResult Create(...)
  {
    ...
    new Task(new EmailNotifier()) { ... }.Save();
    ...
  }

  public ActionResult CreateWithNoEmail(...)
  {
    ...
    new Task(new NullNotifier()) { ... }.Save();
    ...
  }
}
I'm going to ignore all the complexity around the fact that this is an ActiveRecord object which the ActiveRecord framework will usually be responsible for new-ing up, which makes providing DI dependencies difficult if not impossible...

The idea here is to pass in an INotifier, and then when you find yourself dealing with a task you'll build it with the notifier you want it to use.  If you want no notification, you use the Null Object pattern and pass in an INotifier that doesn't do anything (called NullNotifier in the code example).

But this has the ORM-framework draw back I mentioned above.  Plus it requires the code that is constructing the task to know what behavior the code that is going to save the task will require.  Most of the time that's probably the same code, but if they aren't, you're out of luck.

Operational vs Data Classes
public class TaskInfo
{
  public string Name { get; set; }
  public int AssignedTo_UserId { get; set; }
  public DateTime DueOn { get; set; }
}

public class TaskList
{
  public TaskInfo Create(TaskInfo t)
  {
    t.Save();
    notifier.Send(...);
    return t;
  }
}
Here I've separated the data class from the operational class. I talked about this in the Stratified Design series of posts.  This separation hides ActiveRecord, giving us the control to define all of our operations independently of the database operations they may require.  If we needed to save a task without sending an email we could just call TaskInfo.Save() directly from whatever mythical operation had that requirement.  Or we could do some extract method refactorings on the Task.Create method to expose methods with just the behavior we need.  Or we might extract another class.  Naming is going to be hard for these refactorings, but at least we have options.

If I missed anything, or if you see an important variation I didn't think of, please tell me about it!  As always you can talk to me on twitter, and you can still fork the original gist.

Monday, December 17, 2012

Slicing Concerns, And Naming Them

Naming is hard.  Especially in OO.  To name something, you have to understand it at it's deepest level. You must capture it's true essence.  This is hard when you're giving a name to a thing that already exists, but it's orders of magnitude harder when you're simultaneously creating the thing out of thin air, and trying to decide what to call it.  Which is after all what we do when we're designing code.

The "essence of things" correlates closely with concepts like Separation of Concerns and the Single Responsibility Principle.  You can slice any object into ever smaller concerns or responsibilities.  You can slice it right down to it's constituent atoms!  Many design problems, like tight coupling and loss of flexibility, are in large part due to having concerns and responsibilities defined at too high a level.  Could this be so common simply because it's so hard to find names for the smaller concepts?  It's frequently easy to see what those separate concepts may be, but terribly hard to think what to name them!

Let's have an example:
This is entirely fictional code, but it's not so different from a lot of real code I've seen in the wild. And it illustrates this problem of slicing concerns very well.

At first glance, it seems very simple. The domain has a Task concept which has a default due date (set in the constructor), and which sends a notification email after it's inserted (using an ActiveRecord hook).  This very nicely and completely describes what a task is and how it behaves in our system.  And the names make it very intuitive.

Or do they?  Is it really the case that every single time we insert a task in the database it should send an email?  Unlikely.  We should slice that behavior out and put it somewhere else:
public class _WhatShouldThisBeCalled_
{
  public class _WhatShouldThisBeCalled_(Task t)
  {
    t.Save();
    Email.Send(t.AssignedTo_UserId, "New Task", "You have been assigned a new task");
  }
}
This is an incredibly simple refactoring, but I have no idea what this class should be called. The method is a bit easier, it could be InsertAndNotify(Task t) or something similar. But what is this class? What concern does it represent?

No, really, I'm actually asking you.  What would you call it?

Or how else would you write it?  Maybe you'd do something like a fire an event and have someone hook it?  How would they hook it?  Maybe we need an EventAggregator?  This is getting awfully complex for such a simple requirement!

And it's not done, because it's not really so great that it defaults the DueOn date in the constructor.  Is every single task really due tomorrow?  Or is it just a certain kind of task, or tasks created in a certain way?  And where will we put that code, what will it be called?

I sincerely believe this is both a significant design problem, and a significant naming problem.  I want to know how you'd tackle it.  Please do leave a comment or tell me on twitter or even better, fork the gist on github!

These concerns need to be separate!  But what a cost we pay for it!  The simple OO domain model of a Task has turned into something much less relatable.  Either it's event driven spaghetti code with strange infrastructure objects like EventAggregators.  Or it's a hodge-podge of service or command classes, none of which actually model a relatable thing...  They only model functions, features, behaviors, use cases.  Or maybe we try applying inheritance, and then we end up in a whole different world of confusing names and surprising behaviors.

Can't we do better?  Is there some way we can do the slicing of concerns we need but still maintain the modeling of real relatable things?  Even if that may require a different way of thinking or not using the design patterns that led us here (Active Record, in this case).

Friday, December 14, 2012

The Fundamental Software Design Problem

The most fundamental software design problem, that this the the most important problem which underlies all design decisions, is:

Choosing the right amount of abstraction


Say you're starting a brand new project that you don't have any previous experience with.  What sort of architecture should we apply?  We have a lot of choices, some listed here, ordered in increasing complexity:
  • SmartUI
  • MVC w/ Active Record
  • Ports and Adapters
  • SOA
  • CQRS
For some problems just a glance is enough to know it needs a more abstract and complex solution.  Equivalently, some problems quite clearly should be as simple as possible.  But most problems lie somewhere in between.  And generally there's really no way up front to know exactly where on the complexity scale it will lie.

Worse still, in a large enough application different portions of the application might be more or less complex.  Some areas could be simple crud with no logic, while other areas involve heavy data processing and complex workflow and queries.

And even worser, this is a moving target.  If I had a dollar for every time something I thought was pretty straight forward became much more complicated either because of changing requirements, scope creep, or just misunderstanding...  Well, I'd have quite a few dollars!

As I see it, there are basically two strategies for dealing with this problem:
  1. Start as simple as you possibly can, and evolve to more complicated designs as things change
  2. Start slightly more complex than may be strictly necessary so that it's easier to make changes later
I would expect people from the Agile and Lean communities to balk at the very mention of this question.  They'd probably bring up stuff like YAGNI and evolutionary design.  And I agree with this stuff, I agree with it completely!

But I also think boiling frog syndrome is a real thing.  Even a great team with the best intentions can easily find themselves stuck in the middle of a big ball of mud.  That's just life.  Little things change, one little thing at a time, and you do "the simplest thing that could possibly work" because hey, ya ain't gonna need to do a big overhaul now, this will probably be the last tweak.  And next thing you know, everything is a tangled mess and all your flexibility is gone!

To add insult to injury, when you find yourself wanting to do a significant refactoring to a more abstract design, it's frequently your unit tests that are the primary problem spot holding you back.  Those same tests that were so useful when you were building the code in the first place are suddenly locking you into your ball of mud.

I can hear you now.  You're looking down your nose at me.  Huffing and puffing that if I'd had more experience it never would have come to this!  If I'd just listened to my tests, the ball of mud wouldn't have happened.  If I'd just understood the right way to build software!  blah blah blah.  Sorry, I don't care.  I build real software for real people with a real team, I'm not interested in idealism and fairy tales.  I'm interested in practical results!  I'm interesting in making the correct compromises to yield the best results while constantly striving to do better!

And that's ultimately my point!  No matter what design I start out with, I want it to allow me to strive to do better.  If the simplest thing that could possibly work is going to be hard to evolve into something more flexible, that's a problem.  Accounting for change doesn't necessarily mean doing the simplest thing, in some cases it means doing something a little more complicated, a little more abstract, a little more decoupled, or a little more communicative.

If this ticks you off, please come argue with me on twitter!

Thursday, December 13, 2012

The Dizziness of Freedom

"A man stands on the edge of a cliff and looks down at all the possibilities of his life.  He reflects on all the things he could become.  He knows he has to jump (i.e. make a choice).  But he also knows that if he jumps, he'll have to live within the boundaries of that one choice.  So the man feels exhilaration but also an intense dread."  - Jad Abumrad quoting Kierkegaard

Wednesday, December 12, 2012

Neat F#: Custom Operators

F# has support for custom operators.  The best use of this I've seen so far is in the canopy web testing library.  Canopy allows you to write code like:
"#firstName" << "Kevin"
"#firstName" == "Kevin"
That code is the same as this code written with the Coypu web testing library:
browser.FillIn("#firstName").With("Kevin")
browser.FindField("#firstName").Value.ShouldEqual("Kevin")
As you've likely deduced, the "<<" operator has been overridden to lookup the field and set it's value, while the "==" operator has been overridden to lookup up the field and assert on it's value.

In this case, both of these operators do exist already in F#, but they obviously aren't usually used to drive a web browser.  So this is a powerful use of operator overloading.  But F# allows you to define custom operators that have no definition in F# as well.  They can be any combination of a certain set of characters.

For example, there is no "=~" operator in F#, but you could define one to do a regex match as follows:
open System.Text.RegularExpressions
let (=~) input pattern = Regex.IsMatch(input, pattern)
And you'd use it like:
"some input" =~ ".*input"
And you could also define one that is case insensitive:
let (=~*) input pattern = Regex.IsMatch(input, pattern, RegexOptions.IgnoreCase)
These operators are not overloaded, they're just custom defined.

There is clearly a tradeoff here between explicit code and concise code.  Look back at the first example from Canopy again.  If you knew that was web testing code, and you recognized "#firstName" as a css selector, you would probably figure out what it was doing.  And this conciseness is going to be really nice in a situation where you're executing the same type of operations over and over and over again (say, like, in a Selenium web test!).  So while there's no mistaking what the Coypu code is doing, I'd rather write the Canopy code!

However, in the regular expression example, since =~ and =~* are not part of the language, how would you know what they do.  Certainly there's a similarity to ruby, but I've never seen a =~* operator.  So introducing stuff like this to your code base runs the risk of making your code harder to understand.

In the end, I think it's an awesome feature to have at your disposal.  And I think a good rule of thumb is to be willing to try some custom operators when you have a high and dense repetition of operations.  That is, it's not a one off operation, or it's not used always by itself in far flung sections of code.

In any case, this another powerful, and very neat, feature of F#.

Tuesday, December 11, 2012

Neat F#: Inferred Return Types

What is the return type of this F# function?
let hello name = sprintf "Hello %s" name
If you guessed string, you're correct! I know this syntax can be confusing at first glance, so here it is one element at a time:
  • let hello name =
    let: the "let binding's" job is to associate a variable name with a value or function.  It BINDS things to names
    hello: hello is the name of the function
    name: hello takes one parameter, and it called name, and it's type will be inferred
  • sprintf "Hello %s" name
    sprintf: basically F#'s version of .net's String.Format, it's a function that takes a format string with placeholders and values as arguments and returns a string.  In fact, this function is so neat it deserves it's own post.
    "Hello %s": the format string, %s tells the compiler a string parameter is required
    name: argument to sprintf
sprintf returns a string, therefore the hello function returns a string.  Notice there's no explicit return statement, whatever the last statement returns the function returns.  However, this is made more interesting by the fact that just about everything in F# is a statement that returns a value, including if statements:
let hello name =
    if name = "kwb" then
        "'sup KBizzle!"
    else
        sprintf "hello %s" name
This function still returns a string, because the if statement returns a string. Note this means in F# the if and the else must return the same type!

Also note, there's nothing wrong with that last code sample, but it's my impression that if statements are generally frowned upon in F# in favor of pattern matching. A true F# dev would probably have written that last using the pattern matching function syntax like this:
let hello =
    function 
        | "kwb" -> "'sup KBizzle!"
        | _ as name -> sprintf "hello %s" name
That's really outside the scope of this post, but it makes me happy!

So this brings us to the _really_ neat part: functions with changing return types.  All the functions we've seen so far have had a single static return type.  But what about this function?
let crazy f =
    f 4
What is it's return type? Maybe this will help?
let somestring x = sprintf "the number %d" x
let someint x = x

crazy somestring
crazy someint
Crazy's return type is different depending on what function we pass to it!
mind blown

Monday, December 10, 2012

Neat F#: Pipe Forward

Functional programming dates back to the 1950s, but from my perspective it seems to have been garnering more attention in the software engineering community recently.  I first got really interested in it last year at CodeMash when Ben Lee introduced me to a little bit of Erlang.  It was so fascinating that I decided I had to dive in deeper.  F# being a .NET language, it was the obvious choice.  So I bought an ebook of Programming F# and started writing a few little programs.

Along the way there have been a few things that completely blew my mind, or that I thought were just flat out neat.  You could (and should) learn about these things from much better sources, but I love to share!

So to kick it off, this first post will cover the first thing in F# that really truly blew my mind: the Pipe Forward operator.  You see it used in F# all the time, and what it allows you specify the last parameter's value first, thus writing your statements in a more logical order.

So for example, this:
let testInts = [1;2;3;4;5;]
let evenInts = List.filter (fun i -> i % 2 = 0) testInts
Can be re-written as this:
let testInts = [1;2;3;4;5;]
let evenInts = testInts |> List.filter (fun i -> i % 2 = 0)
This example is trivial of course, but where this really starts to shine is when you can effectively describe an entire program in one chain of function calls:
let oddPrimes = 
    testInts
    |> filterOdds
    |> filterPrimes
Basically what's happening here is that the value on the left of the pipe forward operator is being passed as the last parameter to the function on the right. In the first example, List.filter takes two parameters. The first is a function, and the second is a list. You'll find that all the functions in F# modules are structured so that the parameter most likely to be passed down a chain like this is defined as the last parameter.

At first I didn't think this was mind blowing at all. It just looked like a simple compiler trick. But then I learned this isn't in the compiler at all. In fact, |> is just a single line F# function. It's definition is nothing more than:
let (|>) x f = f x

F# also has a pipe backward operator:
let evenInts = List.filter (fun i -> i % 2 = 0) <| testInts
And it is defined as:
let (<|) f x = f x
If you've followed along this far, I hope you are boggling your mind as to what possible purpose this could serve! I know I was. The answer comes from the fact that operators have higher precedence in F#. So the pipe backwards operator allows you to avoid wrapping an expression in parenthesis. For example:
let output = "the result is: %d" (2 + 2)
Those parenthesis are such a drag. So we can rewrite this without them as:
let output = "the result is: %d" <| 2 + 2

Monday, December 3, 2012

Stratified Design

The last posted ended by presenting a style of OO in which the objects only exposed operations which communicated via data classes.  We arrived at this design by thinking more deeply about encapsulation.  I asserted that there were a number of benefits to an object structured this way, but promised to also talk about the architectural benefits of applying this practice throughout your code.

A Stratified Design means writing all of our objects in this behavior-only style, and passing data classes between them.  There are lots of detailed decisions to be made around what exactly those data classes should contain, but for now I'm going to stay at a higher level.

You may be thinking to yourself, "Hey!  Those are just layers!  I've been duped, there's nothing novel or interesting in this except a fancier name!"   While what I'm advocating here is very similar to the traditional layered architecture there are some very critical differences.
  1. Some definitions of layers include a restriction that lower layers may not reference higher layers.  When applied to domain modeling this tends to lead to ridiculous restrictions where your data layer is not allowed to return domain objects because the domain is a higher level concern than the database.  A stratified design agrees that lower stratum should not use behaviors of higher stratum, but that doesn't mean they can't share the same data classes!
  2. Layered architectures usually prescribe an exact number of layers for specific purposes.  In a stratified architecture, there aren't any consistent named layers.  Instead, there's just a series of classes calling into each other as needed.  Each of those classes is defined at some level of abstraction, and calls into it's dependent layers as needed.  And that's as prescriptive as it gets.
There are a lot of awesome things that follow from having objects calling other objects in this way:
  • Decoupled: clean interfaces communicating with data is about as decoupled as you can get (and therefore insanely easy to unit test!)
  • Simple, not complected: each object knows only about the interface of it's dependencies.  And it accepts small data classes, and outputs small data classes.  There's no static or global knowledge, no god objects.  And each object represents one concept defined at a consistent level of abstraction.
  • Behavior only where needed: the simplest example is you will only be passing small data classes into your view, not ActiveRecord objects (which expose query and data persistence behavior).
  • Somewhat side-effect free: not entirely side-effect free, but because there is little to no shared state, it's difficult to be surprised by a side effect.
  • Intuitive: If you do a good job separating your levels of abstraction, you will find that when you are looking for something, it's always right where you expect it.  Or if it not right there it's one explicit function call away.  Contrast this with OO designs that are riddled with inheritance and misapplied strategy and state patterns...  Or compare it to "light weight" Active Record based designs where logic might be in an AR hook, or might be in a service class, or might be in a controller...
The inspiration for this Stratified Design came from a number of different sources.  But the primary ones were:
  1. Rich Hickey's Simple Made Easy and The Value of Values
  2. Bob Martin's Architecture the Lost Years
  3. David West's book Object Thinking (which I reviewed before)
These don't spell this out exactly, but they contributed certain concepts.  And as always, remember that "architecture" is dangerous.  Lots of people might be excited about CQRS, but that doesn't mean it should be applied to a mostly read only content management site.  And Rails might be an efficient platform for quickly building a web app, but that doesn't mean it's right for building a space shuttle control panel.  And the same goes for Stratified Design.  The architecture of your application should reflect the nature of your application.

But that said, if you give Stratified Design (or something similar) a try, I'd love to hear about your experiences with it!