Chapter 4: Gear Your Organization for Reuse

"One of the greatest pains to human nature is the pain of a new idea."

Earlier chapters have indicated that reuse is common in engineering and much needed in software engineering. The technology of adaptable frames embodies a principle on which we can build effective reuse techniques. I have suggested the changes in software development practices that moving to reuse necessitates and supports. All these elements are already in place.  There are two more elements to treat. To get the full benefits of reuse implies changing the infrastructure of the organization and the culture of its people. Infrastructure and culture set the ultimate limits on reuse in the large.

Reuse maturity levels

Table 4-1. Organizational Reuse Maturity 

*Footnote: The reuse maturity model is not to be confused with the Software Engineering Institute's capability maturity model. While the two models are correlated, they group organizations according to different criteria.

Level Zero — Ad hoc
This level provides a point of reference. As most organizations are unaware of the benefits of reuse, their PIs (process productivity indexes) currently center around QSM's industry norm of 16.9. With ad hoc reuse, individual programmers typically copy and modify source modules, usually their own. 

Level One — Latent
These organizations, while they use frame technology, don't write their own reusable frames, other than simple data definition frames. In spite of their relative lack of interest, they do get 70 to 80 percent reuse from vendor-supplied frame sets. And they enjoy PIs well above the industry norm. 

Level Two — Project
Most organizations using frame technology are at Level Two. In fact, all but one of the participants in the QSM study were at this level. Project level reuse means project developers write frames to be reused within the scope of their projects. Those projects achieve very desirable results — 80 - 90 percent reuse and PIs from 24 - 29. But reuse is not systemic. Incompatible tools, components, and skill sets continue to coexist — so called islands of technology. There is neither an inventory of mature, corporate standard parts, nor an infrastructure for fostering them. Level-two reuse is not self-sustaining. 

Level Three — Systemic
Reuse becomes institutionalized across an IS organization having multiple project areas within its span of control. At least 40 percent of developers are routinely designing systems based on a common component architecture. Forty percent seems to be a threshold beyond which resistance crumbles and there is a rapid shift to the reuse paradigm. 

Systemic reuse is characterized by having a defined (in the SEI capability maturity model sense) process to produce and support standard components. These components are robust and mature, and constitute a technical architecture for designing and building systems. Developers are portable — because they share a common vocabulary they can readily move between different application areas. With these characteristics process productivity is in the low 30s, and reusable frames account for 90 to 95 percent of the functionality. 

You might wonder why anyone would worry about going from 90 to 95 percent reuse — five percent sounds quite marginal. Turn your view upside down, and you are going from 10 percent of the work to five percent. You just doubled your productivity. 

With the Information Systems function achieving Level Three, the surrounding enterprise cannot sustain the pace and volume of change unless it also undergoes a fundamental attitudinal and behavioral shift. Level Three organizations typically foster incremental improvements to existing processes (as in Total Quality Management). Level Four organizations foster the invention of new processes and infrastructure (as in Business Process Reengineering). For Information Systems this shift entails reengineering core systems accordingly. 

And a Systemic Reuse systems organization acquires the means to revamp core systems without putting the enterprise out of business. With ever more functionality being captured in technical and business architectures, the time, cost and risk of overhauling outmoded systems becomes ever more acceptable. 

Level Four — Cultural
The distinguishing feature of a Level Four organization is the "do better with less" attitude that the entire enterprise reflects. For example, they use metrics comprehensively to forecast each new opportunity's return on investment. Then they do whatever it takes to get that ROI.

Unlike level three, cultural reuse makes no distinction between frame engineer and application developer. Moreover, the entire IS group is quite compact, and often centralized. (They have gone from portable people to portable SWAT teams — James Martin's name for Skilled With Advanced Tools). The focus has shifted from development to orderly evolution. With so much reusable functionality available off-the-shelf, most user requirements are small "deltas" from what already exists. 

Cultural reuse is associated with enormous process productivity. Remember that the process productivity index reflects an exponential scale. Every three index numbers amount to doubling process productivity. Thus, Level Four organizations are as far beyond Level Two as Level Two is beyond Level Zero. That is a big jump. Because relatively few organizations can achieve PI levels of 33 - 36 they can dominate their competitors with nimbleness and sophistication. 

Level Two is not hard to reach. And, for selected projects, enjoying 70-percent reductions in time-to- market and 84-percent reductions in costs is certainly not shabby. But trying to move beyond Level Two may be like hitting a brick wall. 

Barriers to systemic and cultural reuse

Conceptual barriers

Confusing reuse with use-as-is. Not perceiving the "same as, except" nature of reuse prevents people from capturing in component architectures the tremendous amounts of similarity that exist across organizations. 

Confusing assemblers with generators. Code generators shrink-wrap their data structures and algorithms at the factory — one size fits all. Fighting with clumsy and inefficient generators has given productivity tools a bad reputation. Letting you control every assembled line of code was a prime objective of frame technology. 

Believing reuse requires big investments up front. It's not only unnecessary but wrong to try to perfect reusable components before reusing them in real systems. Reuse can and should pay for itself as it goes.  

Believing diverse lines of business embody few commonalities. We share 97.5 percent of our genes with chimpanzees; sheep and cows share 89 percent*! Operating units that appear to be quite diverse can still overlap their cores by 90 percent or more. (Much more challenging is to induce quasi-independent business units to share common components with each other.)

*Footnote: Ledyard Stebbins, Darwin to DNA, Molecules to Humanity, W.H. Freeman and Company, San Francisco, 1982, pp 129.  

Designing software as a kind of hardware, thus building systems late, over budget, and misfitting current needs. We must use software's exquisite adaptability and self-modeling ability to keep our well-defined solutions suitably close to our ill-defined problems. In other words, reuse across time — the process of iterative design refinement. 

Technological barriers

Brittle, obsolete systems. Today's great new systems are tomorrow's euphemistically termed "legacy systems." The trick is to get out of the resource-sucking, maintenance tar pit and stay out. The pit may lack life buoys and winches, but tools and techniques do exist for the arduous task of finding components worth salvaging and of reconditioning them to be adaptable and reusable.

Missing or incomplete and inflexible architectures. Our industry has had decades of experience designing run-time architectures. Client-server, distributed processing, and object oriented environments exemplify modern run-time architectural styles. But what of construction-time? What about equipping ourselves to deal with changes — customizations, interfaces, enhancements — to our run time executables? At construction time we need component architectures that are adaptive. This approach is natural to reuse. Its absence, conversely, is a barrier. 

Metrics. Reuse has a major contribution to make to many bottom-line objectives, and is easy to measure to boot. Tom DeMarco's famous truism "You can't manage what you don't measure" is no truer than the fact that few information systems organizations quantify quality, productivity, and flexibility. Fewer still explicitly use what they do measure to improve processes. With today’s tools for measuring objectively and unobtrusively, excuses for not doing so have worn thin.

Managerial barriers

There is too little consensus on common tools and standards. Project managers, in particular, are paid to complete their projects on time and on budget. In serving this mandate a project manager resists anything she perceives as jeopardizing her critical path, such as standards that might get in her way, or off-the-shelf components that she does not trust. (If the components are not well supported she is probably right to resist!) The net result: the greater good of the organization suffers. 

Major risks go uncontrolled. Examples: We set requirements in ignorance of what designs and components are available off the shelf. Result — we miss opportunities to reuse and hence to avoid construction work; we increase duplication and hence downstream support costs. We automate yesterday’s view of the business, not what it will need tomorrow. We write code any which way, its future adaptability be damned. We expend excessive efforts designing and implementing highly normalized relational models, and ignore performance questions until it’s too late to save the project. I could go on. 

Decisions focus on technology trends rather than business criteria. By its nature, reuse technology is infrastructural, not directly in the face of customers and end-users. This implies that the ability to interconnect people and systems, induce commonality, enforce standards, and promote stability are the dominant business criteria in choosing reuse technology. Trendy, "flavor of the month" technologies need not apply. 

Managers represent the established order. I do not mean to be too harsh. They are responsible for getting the day-to-day work done, and monumental change can upset the apple cart. 

Infrastructural barriers

The ideal team size is one. Beyond three or four a team suffers diminishing, even negative returns. In my experience, small teams are successful tackling large projects because they adapt the available frame architectures. At AFS, for example, one person built an 800,000 line system in 8 months. At Noma Industries and other companies, one person teams are the norm. In the above shops, two and three man teams are formed when the problem requires a lot of (non-SPC) frame writing. 

Monopolistic information systems departments. When an organization's users form a captive market for its IS department, even the best of intentions degenerate into monopolistic practices — too expensive and unresponsive, self-perpetuating — a barrier to innovation. 

Obsolete processes. The waterfall methodology and user intermediaries, as I explained in the last chapter. 

Obsolete supporting infrastructure. For example, when technology automates tasks on software development's critical path, previously non-critical tasks become critical. To take an example, suppose a gap in requirements normally takes stakeholders two weeks to clarify — analyze, discuss, build consensus, get necessary approvals, and give developers a go ahead. No problem. There is plenty of work to keep developers busy. But with automated software construction, a requirements gap can put developers into thumb-twiddling mode in a matter of hours. 

Cultural barriers

Hay Pay Plans. Hay is a consulting firm that provides management salary surveys and compensation plans that, while in principle take contribution into account, in practice, compensate executives based on head count (number of staff reporting to them). Under such a plan small is not beautiful. 

Cowboy mentality. By this I mean the pervasive inattention to professionalism — programmers who cut corners, code obscurely, generalize poorly, document badly, ignore efficiency issues, and so on. Reusability is predicated on people who possess the opposite of these characteristics. The good news is reuse technology makes professionalism easier to practice. 

Cultural barriers are hardly changeable overnight. As I said earlier, don't despair if your organization is not ready to achieve systemic or cultural reuse (Level 3 or 4). The benefits of project reuse are considerable and quite accessible. 

Business effectiveness through systemic reuse

- Major reduction in time-to-market 
- Treat IS as strategic enabler 
- Major reductions in information systems size and cost 
- Reengineer the business 

Is Your Business Case Plausible?

1. What are your enterprise's defacto "spans of control" — limits of cooperative behavior? If two managers who should be cooperating don’t normally talk to each other, or talk but don’t trust each other, they exist in two different defacto spans of control — no matter what the organization chart says. Systemic reuse can be successful only within a given span of control, say an I.S. group devoted to one department or line of business. 

If individual project teams are the largest units across which you can induce cooperative behavior, you are stuck at reuse maturity Level Two. If you plan to enlarge one span, be prepared to curtail others. Such power shifts may require a combination of carrots and sticks: decision making may have to become less consensus oriented; people’s self-interests may have to be "readjusted" towards the greater good of the organization. 

2. For a given span, what are your milestones? How much of each goal can you achieve and by when? Quantifying your goals forces objectivity, and of course, the use of metrics. It also drives the urgency and aggressiveness with which you must make changes.

4. Of the various roll-out plan details (described in the next section), which are the most problematic for your organization? How do you plan to achieve them and what are the costs? 

5. When should the anticipated returns-on-investment pay for the costs of change? Are the ROIs adequate for the risks involved? If not, consider increasing the span of control, and/or the goals and pace of the roll- out. 

6. What are the risks? Can they be managed, or do they appear to jeopardize critical aspects of the business? If so, consider a smaller scale effort which can be aaccelerated in the context of a better understanding of what it takes to move ahead. 

The Elements of Your Roll-out Plan

No matter how aggressive your plan may be, it will encompass the following elements, discussed below. 

- Involve senior executives 
- Educate/Train/Coach 
- Rollout projects 
- Move to a hybrid organization 
- Create a new career path: Frame Engineers 
- Create a frame engineering department 
- Evolve system and business architectures 
- Measure and reward 

Involve Senior Executives. They are the only element in the organization with the clout to overcome the barriers to change. They form a Reuse Steering Committee, with representation from user departments, information systems, and an ex officio reuse expert. They set policies that induce behavior and attitude changes. They also, monitor and guide progress, and remove barriers. The buck stops here. 

Beyond clout and strong support, senior management has to project the vision of the new organization and what it will mean to be employed there. As I said, this vision has to be strong enough in everyone's mind to make the pain well worth bearing. And executives should be perceived to be sharing in that pain. Unless their own careers are seen to be at risk, the rest of the organization will cynically hunker down until the storm blows over. You may have already seen this phenomenon a few times. 

Educate/Train/Coach staff and managers (including those who manage users). Educate everyone to overcome conceptual barriers, inspire vision, and imbue the "do better with less" attitude; train generalists both to know the business and to assemble systems from reusable components; coach novices (by experts) how to apply theory to practice. Schedule courses to precede practical application just-in-time. This is all common sense. But in reality the education and coaching elements are often not well synchronized, putting rollouts at serious risk. 

Roll out projects, medium sized with high reuse potential. Initial projects should not be "betting the bank," but they should be real — laboratory experiments bring out the tire kickers who tell you all the reasons this new-fangled idea won't work in your environment. Real projects with tight deadlines clarify the mind and focus people on what really matters. Moreover, from real projects emerge realistic components. 

Seed projects with at least one competent person per four novices. This seeding will ensure frames get written and properly reused, and novices receive proper skills transfer. The sooner you can achieve self- sustaining reuse — at least 40 percent of resources practicing reuse routinely — the better. 

Move to a hybrid organization. A centralized infrastructure is required to support reuse across multiple projects and departments. On the one hand, it is centralized because it is where your corporate standards are defined and your common parts are managed. It is where your software staff is recruited, trained, and allocated to various projects. It is where their performance is evaluated. 

On the other hand, software projects are decentralized, so developers can immerse themselves in the business problems their skills are trying to solve. This immersion includes the frame engineers. They should not work centrally because of the ivory tower effect — designing elegant frames that don't cut it on the shop floor. By doing their work as members of project SWAT teams, frame engineers can ensure the corporate standards are observed locally, work shoulder to shoulder with the reusers of their frames, and coach novices. 

Frame engineers form the glue that binds the hybrid organization together. While allocated to a project their salaries are paid from that project's budget. But their solid-line accountability is back to the central infrastructure, with dotted line accountability to the project manager. Experience has taught me this lesson. Let me explain it. 

The project-centric power structure, traditional in information systems departments, must be readjusted to ensure that the greater good of the enterprise is properly balanced with project imperatives. Project managers must agree up front that their own interests are best served when the frame engineers are not pressured into other duties, such as doing Johnny's work because Johnny is not pulling his weight. Yes, priorities often have to be juggled. But the solid line is there to stop expedience from becoming habitual. 

Create a new career path: Frame engineers. Frame engineers are to developers as developers are to end- users. They are a small group of people, who are known to be highly proficient programmers with a flair for generalizing solutions well. Why not give them a career path that maximizes their value to the organization? Rather than rewarding such people with Peter Principle promotions into management, provide them with equivalent salaries and highly respected roles as frame engineers. 

Those roles not only include developing standard frameworks but also: acting as consultants to project managers and coaches to novices, managing the corporation's reusable assets, defining system standards, architecting software, and ensuring corporate frames are properly deployed and reused. 

Create a frame engineering department. Here are a few of its functions: 

- Define standards for frames, systems, development processes, reuse metrics,   measurement, reporting, and process improvements. 

- Mature frames, catalogue them, and ensure their effective deployment in various   projects. 

- Manage the evolution of the standard frame set.

- Manage the careers of frame engineers. 

- Contract frame engineers to projects. 

One way to help the frame engineering department manager act as a counterweight to project managers is to make frame engineering a profit center. With a good return on investment, he gains support from senior management. He has a strong incentive to insist that his frame engineers tend to frame business, unless a project manager can make a convincing case that there is more profit to be made pulling a project out of the hole. Even then, the frame manager will reevaluate the equation every Monday morning. 

Evolve system and business architectures. As described under technological barriers, these layered frameworks provide a common infrastructure for your systems. They make interfaces and platforms transparent; they embody corporate standards. They make possible the combination of high-speed development and high quality — faster, cheaper, better. They also make possible the cost-effective replacement of obsolete systems. 

There is a process (described in Chapters 19 and 20) to design, write, test, and mature these architectures, but it may take two or three years to complete. So the plan must provide milestones and resources for this process. 

Measure and reward. Measure to improve processes, not staff. You first need to know your starting point — a baseline. Then to calculate your ROIs, you baseline compare each reuse-based project's time-to-market, cost, and user satisfaction. 

Again, a baseline of component reuse frequencies will enable you to spot anomalies, prove benefits, and suggest which frames justify further investments. 

Based on real benefits, reward people for their reuse efforts, either psychically or materially. Vision gets reuse rolling, but incentives keep it going until it becomes self-sustaining. 

It has been done

How many software developers do you think they have? Typical for their revenues would be a department of several thousand. Their number blows my mind: 40 people. That's worldwide. And they use no outside contractors. They have hundreds of systems serving all these far-flung units. Yet with these 40 people they process over 28,000 requests for changes and enhancements per year, as well as absorbing new subsidiaries. 

Many organizations would be happy to reduce their information systems staff from 3,000 to 2,400, a 20- percent cut. That is the level one reads about in downsizing stories. Getting down to 40 is something else. It involves fundamental changes, also known as a paradigm shift. 

Of course, these 40 are not your average developers. They are all excellent frame engineers. They don’t need the centralized-decentralized hybrid of Level 3. They are all in one room! Time and again they told me "We can do what we do because we are so small." 

It took only three of them, for example, to implement a network of VAXes that supports their entire European operations. My point is that in any large enterprise, there are at least 40 people of their caliber. Support them with good reuse technology and infrastructure, surround them with a company culture that rewards doing better with less, and diligently measures every ROI, then 40 top people are all they need! 

Going from thousands to 40 is arduous and may not be desirable in many circumstances. Still, we now have a grasp of the magnitude of the effect that is possible. Set your goals appropriate to your circumstances and progress from there.