Top-Down Vs. Bottom-Up Software Architecture

More pragmatic advice from James McGovern, this time about top-down versus bottom-up IT initiatives. In other words, CEO mandate versus workers-know-best. Both are needed, but when is which one appropriate? My favorite quote:

One of the more interesting things is that a person debating top-down vs bottoms-up needs to take into consideration is historically which one has had better results. Of course, top-down has the advantage that even mediocre delivery will be declared success while bottoms-up mediocrity will probably result in throwing daggers.

Consideration? Sure, OK. That's the politically expedient thing to do for bottom-up... but it misses a vital point: is the technology something that people will absolutely love? If so, mediocre implementation will be forgiven, and the top-down criticism will eventually be won over. As I always say, its easier to get forgiveness than permission.

I was a part of multiple skunk works projects at Stellent, some of which got me into a teensy bit of trouble with executives. Some of these projects languished, but many of them took off like gangbusters because people loved them. In the words of Cathy Sierra, these systems helped people kick ass. In the words of Clay Shirky, they were systems where people took care of one another, and that more than anything else guaranteed their existence the next day.

James feels that Outsourcing, CMMi and/or PMI, and Identity Management are best done top down... I agree with everything but Outsourcing, unless its on a large scale. There's nothing wrong with giving your middle managers a budget to outsource. Heck, I see nothing wrong with giving every developer a $5,000k budget, and let them "outsource" some tricky code to another developer who can do it in half the time. Or buy a awesome monitor. As long as there's modest oversight, and your developers aren't idiots, there's very little harm that can be done.

James also says Agile, SOA, and Open Source are best done bottom-up... whereas I agree its easier from a bottom-up perspective to sneak-in cutting edge technology, I'm not sure if its best. This problem is by no means unique to the software world. Its the same age-old question: when should I let my employees take the initiative? The same rules apply across the board, whether the "initiative" is a software methodology, an innovative marketing plan, or sales incentives. The two most important questions are:

  • How does this decision affect the rest of the organization?
  • Do the people making this decision fully understand its risks and benefits?

No matter what you're doing, the answer is always the same: unless your employees are bound by a genuine shared purpose, bottom-up initiatives will do more harm than good! In a typical company, you'll be OK if your IT department sneaks in Agile techniques... but in a dysfunctional company, you run a significant risk of making things worse. This is bad, because in dysfunctional companies, bottom-up initiatives are sometimes the only way to get anything done.

Unless you have some folks at the bottom who have a 50,000 foot view of the entire company, or are genuinely dedicated to helping your customers kick ass, then bottom-up initiatives are just spinning wheels on systems that go nowhere. If your IT department wants to take the initiative, they better be keenly aware how their changes effect everybody else. They had better understand that all change has positive- and negative-effects, and keep track of them all. Finally, they had better have a plan to roll-back the changes, should the negatives dominate for too long.

This way, you're more likely to have bottom-up initiatives that kick ass! Also, you'll have a nice paper trail of successes to prove its value to your inevitable critics.

comments

No silver bullet

I work at a large company that promotes top down approach using Mode Driven Architecture in order to build a large SOA system.

Top down approach has its merits that are easy to understand and it's no wonder the management swears by it. However it has its limits, which can be summed as this: if you need to decompose complex systems into smaller components, top down method is well suited. But if you want high performance, top down approach simply won't buy it. Worse, you may completely overlook these issues, because in the top down approach, you usually leave the implementation details to later stages of design.
But when you want high performance, implementation details have to be kept in mind from the VERY beginning, at all stages of the design, when the architecture can still be changed in order to accomodate performance requirements. Because another issue with the top down approach is that it's not iterative. Once a design is set, it's very hard to change it.
So the top down method requires that extremely skilled and experienced architect, who have a good understanding of the performance issues of the technologies they use. Because at later stages of developments, it's simply too late, you can't get high performance from a bad design.
The bottom up approach, by focusing on foundations, allows performance considerations from the start, during API design.

A typical mistake is to believe that designing a database backed high performance system using UML tools and object relational mapping by even experienced Java architects is a good thing. What they usually do is build the business model in UML that is generally sound in a Java architecture (even though the less experienced architects often build a completely concrete model, overlooking all the design qualities of abstraction and therefor building very rigid and hardly extensible systems). Once the business model is set and meets all the business requirements, they generate the database model and the ORM mapping for JPA, Hibernate or another ORM.

Done.

But when it comes to performance, what is sound in UML can be catastrophic or at best mediocre in the database.
Because the assumption is, with object-relational mapping, you don't need to know SQL (false) nor database performance issues (false again).
Basically, to get high performance out of a large database (tables with millions of rows, and near real time response), one must first design the database model with performance in mind, thinking such details as how to retrieve/update data efficiently. This type of thinking is typically bottom up (implementation details first). Here, MDA development should be database model based, and NOT business model based, because again, once an inefficient database model is set, and hundreds of thousands of lines of business code have been written, it's simply too late, you will never get high performance from the system.
And in large systems, that can mean failure.

In conclusion, as usual, there is no silver bullet. One needs both top down and bottom up at the same time. For business requirements, top down is the way to go, for performance requirements where they are needed, bottom up must not be overlooked.

all good points...

There are analogies between the top-down/bottom-up question, and the architecture-first/functionality-first question.

The value of thinking at the high level is that you can determine if success on this project will lead to problems elsewhere... however, since you are divorced from the implementation details, you don't have a complete idea of the complexity of the problem.

Some folks throw up their arms and go agile... just keep iterating and iterating until it works. That sometimes works, but there are flaws.

I prefer the concept of feedback loops in addition to iteration... that allows the implementation guys to experiment in order to find the "optimal" solution, but it also ensures the experimental results flow back to the top level decision makers and stakeholders.

Of course, this means you need somebody on your team who understand iteration, feedback, implementation details, and high-level strategy... and has the ability to communicate with everybody on all levels. This, apparently, is something of a tall order.

wats is the difference

wats is the difference between topdown and bottomup approach when it comes to programming

Recent comments