I’ve always believed that architecture is a strategic function for an enterprise. But when I tried to uproot that belief from the big-up-front-design soil it was nurtured in and transplant it to newly-tilled agile soil I realized I believed in something I did not fully understand.

Today’s rumination is on strategy.

Two perspectives on strategy, both drawn in contrast to tactics, seem to form the basis of our conventional understanding – one in terms of time, the other magnitude.

‘Tactics are your plans for right now. Strategy is your plan for the future.’

This makes tactics and strategy seem part of some temporal planning continuum in which at some point in time, a year out, eighteen months out, a tactical plan morphs into a strategic one.

‘Strategy is a plan of action to achieve a major goal. Tactics are plans to achieve minor goals.’

This on the other hand makes tactics and strategy seem part of some magnitude-of-objective planning continuum where at some point a tactical objective is important enough to become a strategic one.

Neither one sounds all the way right, does it? My intuition is that there must be a qualitative difference between strategy and tactics, not just a quantitative one.

In previously trying to speak to that qualitative difference – in describing the strategic nature of the architect’s role to both business and IT people to whom it was unfamiliar – I have made frequent resort to two analogies.

The first is a sailing analogy. Strategy is picking a lighthouse as a desirable destination, and charting a course; tactics is dealing with sail and wind and water to stay the course as best you can and get there.

That’s better. Charting a course calls on boating knowledge, navigation skills, an understanding of weather, time, and tide, and experience blending them. But it is still not quite right. In the real world there are no fixed lighthouses by which to guide a business. There is very little not in some state of flux.

The second is a chess analogy based on the Silman method. To figure out your next move, first mentally remove all the remaining pieces from the board save the kings and pawns. Now replace your pieces on any squares you’d like, keeping bishops on their original colors. Can you position your pieces to create a marked advantage? Based on where your pieces actually are right now, can you get them to their desired destinations? How long will it take? Can you get them there while your opponent is both trying to prevent you as well as executing a plan of his own? If you can’t achieve the desired end state you must go back and choose a different one, perhaps a less ambitious one, and keep iterating in this manner until you find a realizable plan.

Better still, I think. It calls out the need for a strategy to be practical, to be realizable. It acknowledges that there are slow-to-change features of the environment to deal with – in your mind’s eye you leave the pawns on the board while you are rearranging your pieces. And it recognizes that you are not alone out there on the water – you have, in business or in war, competitors. But in the real world even the rules of the game can change over time.

The sailing analogy is probably more accessible than the chess one. I am generally unable to leave well enough alone and go on to belabor them, talking about tacking against the wind versus running with it, throwing that your boat needs to be ‘yar’ in for good measure, or describing the Stenitzian chess trade-offs of space for time and time for material. I will continue to exercise these analogies in what follows here.

Both simple definitions have something of the truth. And both analogies resonate. Let’s try to find an underlying structure that supports them all.

To get at the true nature of strategy let us imagine a world without it. Why do we need strategy? What would we lose if it didn’t exist?

Sun Tzu points the way when he says that ‘tactics without strategy is the noise before defeat.’

While we might make some short term tactical gains, giving us the illusion of progress, without strategy our path would be ultimately aimless and reactive. Dude, even surfers have to plan to catch a wave – without a strategy surfing is just floating on a board.

We use strategy because it is smart to do so.

Smart.

There is an interesting discussion of intelligence in Steven Pinker’s ‘How the Mind Works’ (mentioned previously in my rumination on BI). He quotes a passage about what makes good science fiction. Believable aliens seem intelligent because of their purposeful behavior.

Paraphrasing Pinker, one useful definition of intelligence is the ability to achieve goals in the face of obstacles. That is, intelligence is the ability to have goals, make a plan to achieve them, and change plans in the face of obstacles.

Sounds exactly like strategy, doesn’t it?

Let’s try a working definition of our own: strategy is the systematic application of intelligence by a volitional entity – person or group – to improve their lot over time.

Strategy helps us – individually or collectively – come up with the best answer to the question ‘What should you do next?’

The simple answer is ‘something to help you achieve your goal(s).’

So how do we pick the goals? What is a desirable end-state?

First, which end-state are we talking about? That is to say, end when? Next year? Five years from now? Fifty?

All of them of course. It is clear that at some level the process of creating a strategy must be ongoing, that we must redefine our strategy as we age.

Are there goals that do not change over time? Is there a fundament?

We can start with existence. The most obvious answer to ‘what is a desirable end-state’ is ‘one where we still exist.’ This must be the fundamental goal of any entity. Some argue this is why intelligence evolved, and so, according to our working definition, why we have strategy.

Beyond survival – even I suppose including it in some supererogatory circumstances – our values, vision and purpose provide structure for all our stratagems.

Our values are our moral primitives, our moral postulates.

Our visions are our values writ large. They are unrealizable goals projected into ideal resolutions. World Peace. An End to Poverty.

Visions perform the same function as vanishing points in perspective drawings. They are virtual elements that provide real organizing structure and consistency to all that follows.

Our purpose is our highest level potentially realizable goal beyond continued existence. As businesses or organizations it is what we were formed to accomplish. As individuals determining this is called ‘finding yourself,’ which illustrates that the font of our purpose flows from within and is ultimately an expression of our values.

Once we have our purpose, how do we get achieve it?

Generally speaking you can’t do it in one big bite.

You break it down into smaller bites – you create a series of sub-goals where the sub-goals lead or build up to the overall goal. So the answer to ‘what do we do next’ becomes ‘something to achieve one or more of our sub-goals.’

This implies a linear goal structure at its simplest, and a singly rooted lattice structure – assuming a single purpose – at its most complex. This also means that to some extent our goal structure is likely self-similar – that little branches look like big branches. Fancifully we may find it has fractal dimension (remember when fractals and chaos were sexy?).

But what is the structuring model for the lattice?

And how do we find the right sub-goals? How do we find any sub-goals for that matter?

Let’s look at an extension to get a sense of this.

If your vision is Elvis Forever, and your purpose is To Let The World Never Forget The King, you may make it a goal to visit Graceland to shoot some video for your blog. Natural sub-goals might be to save up enough money to get gas for the hooptie and a buy a big SD card for your sister’s camcorder. Then, after getting gas and borrowing the camcorder, you plan to get to a series of each-one-closer-to-Memphis rest stops where you can stop and rest. Eventually you will get to Graceland. Which you will find surprisingly small and unimpressive. But you will not let that get in the way of your purpose, so you will choose camera angles that make it look bigger.

It gets a lot harder to find sub-goals when the rules are more complex than the speed limit is 65, drive on the right and pass on the left (this means you left lane campers), and the domain is larger than I-55 or I-69 and their rest stops.

We can infer a number of things from this example.

A prerequisite for goal setting is to understand the comprehensive landscape in which a future state would exist. If Graceland is going to be closed for re-velveting by the time you get there you need to come up with a new desired future state, or at least a different time to achieve it.

You must predict the future at the nexus of the domains which form the arena in which your plans play out. Our arenas exist in a kind of Hilbert space, shaped by many dimensions of different type and different magnitude. While the dimension of time is one of those structuring our goal lattice, it is not the only one. Contracts, law, regulation, supply, demand, technology, infrastructure, public opinion, politics… And those domains may not be discrete, but may be cross-connected. A given sub-goal may not straddle all of the domains.

What can be easy to miss is that we are also agents of change. This is the lesson of Steve Jobs. We can shape our own future.

Change is neither necessarily continuous, nor as we find from the example of evolution necessarily toward increasing complexity or value. Change in different domains happens at different rates, with different periods and different punctuation.

External changes may impact near-term sub-goals. Some may strike higher in the goal lattice, up to and including survival.

There are external changes to which you must react. There are those things to which you may react. There are those things to which you should not react.

There are domains that evolve slowly or not at all – the laws of physics. Domains that evolve slowly – traditions and institutions. Domains that evolve more frequently – laws and regulations. Other that evolve even more frequently – the advance of technology, style, fashion. Others that advance moment to moment – stock prices.

And it is the evolution with respect to us that is important. Physics doesn’t change, but our understanding of it does. The invention of a new packing algorithm, while unlikely, would potentially revolutionize part of the arena for shippers.

Understanding the relative pace of change is even more important to the selection of lower-level near-term sub-goals.

Like weather forecasts (an analogy borrowed from my ‘role of EA in Agile’ research), sub-goals closer to the present are more reliably chosen than those further away. We have a better understanding of the near-term arena which gets rapidly fuzzier as we move out in time. We know where the hurricane will be in ten minutes. We have a good idea where it will be in an hour. We have only some idea of where it will be tomorrow.

So as we’ve seen the best uber-strategy is to frequently refresh our strategy.

The pace of change of business in this age of technology has tended to shrink the strategic horizon because of a perception of our inability to forecast the arena with any degree of accuracy even a few years out. While some companies still plan five to eight years out, and a few ten or longer, it seems that more have fallen back to a three to five year horizon. We need to remember though that technology, while important, is only one dimension of our arena.

But the most important takeaway for me from the weather analogy is this one: your near-term sub-goals are well-defined only if you understand where you are right now. In our chess analogy it is easy: you just look at the actual board. In business it is much harder. Very few enterprises have an accurate and comprehensive idea of where they are right now. Those that do have a leg up on strategy.

Similarly knowing when you have achieved your sub-goals is necessary to working your way up the lattice efficiently. Near-term goals in particular must have associated success metrics.

Finding sub-goals sounds like a daunting task.

But ultimately the question of how you find sub-goals has a simple answer.

Remember our working definition of strategy: the systematic application of intelligence to improving our lot over time. Finding sub-goals is part and parcel of being intelligent. How do you learn to find sub-goals? The same way you learn to speak. You are built to by evolution. Singly and collectively you are a goal-finding beast. Immerse yourself or your organization into the best information you have – the best research, the best forecasts – about the churning reaction in which you swim, and think about it, and you will precipitate out goals.

If you are insecure about trusting to your innate abilities, there are a number of formal methods that can help ensure you are being comprehensive in your strategic considerations, such as the wonderfully acronymed STEER, or its woefully acronymed cousin PEST.

‘What should you do next?’ also sounds like the question answered by tactics. I have been maintaining there is a qualitative difference between strategy and tactics. If they are both answering the same question how can there be a qualitative difference?

Let’s reuse our tactic of defining something by imagining the impact of its absence.

To get at the true nature of tactics let us imagine a world without them. Why do we need tactics? What would we lose if they didn’t exist?

We would still do something. And we could still follow our strategy. But we would make a lot more wrong turns. Without tactics we would not be able to recognize and take advantage of recurring patterns in our environment. It would take us longer to achieve our strategic objectives.

Without tactics we could not force things to happen. Tactics are not just single moves, nor are they plans without branches. In chess forced sequences that inevitably result in material gain are called ‘combinations.’ I do this, then my competitors may do x or y or z. If they do x I do p, if y q, etc. No matter my opponent’s response to each of my moves, there is an incontrovertible path that wins me material.

Forced sequences don’t have to end in material gain as combinations do. They can also be a single move or a series of moves that improves our control of space, or gives us an advantage in time, or improves the scope of our pieces.

So tactics are not just random acts. They are single acts or sequences of acts that improve the quality of your position. Recognizing what constitutes an improvement means understanding the nature of advantage. And understanding which advantages may be turned into other kinds of advantage. In chess you can generally turn an advantage in space to one in time, one in time to one in material, and one in material to checkmate.

We see this transformation of advantage in business: the Amazons of the world know that if you establish market dominance, profits will – likely – eventually come.

This is a place where strategy and tactics come together. A transmutable advantage as contributed to by a specific tactic is a valid and useful strategic sub-goal.

So a wide and deep knowledge of tactical patterns is a prerequisite to best executing strategy.

There is a skill involved in being able to find instances of those patterns in the real world. Not everyone can see a knight fork or the sting of a discovered check several moves down the road with equal facility. But everyone can learn the patterns, and improve their ability to see them with practice.

Every chess player knows the strength of ‘pigs on the seventh’ – having your rooks both occupying the opponents second rank. Every chess player has marveled at Fischer’s ‘windmill’, the sweeping scythe that Byrne graciously played out for posterity.

For a chess player the ability to succeed at deep tactics is partly a function of the ability to hold those possible futures in your head with clarity. This is what the non-chess player assumes is the fundament of chess strength as they ask ‘how many moves ahead can you see?’

But computers can do that better than humans – they can see many more moves into the future accurately. So how are we able to compete with them at the highest level?

A human player’s strength is in the ability to eliminate whole branches of moves based on a well-developed aesthetic – a sense of the beauty of a position. If a branch of moves is ugly it is generally safe to prune it.

Articulating an opinion on the relative aesthetics of alternate futures based on insufficient information with which to make a deterministic decision is a function of strategy. We can choose among alternate plans when the forced sequences have not been worked out in detail.

It is also one of the functions of an architect. They (we) are almost always very senior software and systems engineers with a well-developed sense of technical aesthetics. We can and do make or recommend good decisions based on what is ugly versus what is beautiful.

The challenge of playing to this strength in the corporate world is that it is ultimately an argument from authority – an argumentum ad verecundiam, aka ‘ipse dixit’ – and suffers from that argument’s failings, so well-known that it is on the short list of common logical fallacies. ‘Because I said so’ usually doesn’t get it done unless one is the king, or carries the king’s seal. (I actually said it once in a meeting, only partly in jest. It did not go over well.) So the architect’s contributions to strategy may be underutilized. (I know. Wahhhhh. /digression.)

Selecting the best tactic is not just a function of that which best advances us to the nearest sub-goal. There is another strategic expression in place, that, like the kiddie bumpers at the bowling alley, helps keep us out of the gutter. Those are our policies.

Policies are our bible. They codify our values into guidance for how we do the things we do. They help preserve our integrity by ensuring we don’t justify our means solely by our ends.

Policies are a mechanism for our higher-level goals to directly influence our actions without that influence having to trickle down the goal lattice. They help ensure that we don’t let the temporal dimension of our goal lattice dominate.

Sometimes we find that we need to perform the same actions repeatedly. Like building a pyramid we have to aggregate our achievements over time toward our goals. These repeated actions are our procedures. They take place in the lanes formed by our policies.

Procedures have specific objectives – measurable states of affair reached at a particular time or range of time. These objectives when accumulated reach a goal in our goal lattice.

Have we gotten anywhere? Let’s sum up.

Strategy is the systematic application of intelligence by an entity to to improve its circumstances over time. Tactics is the systematic application of learning by an entity to improve its circumstances right now.

Strategy deals with the novel. Tactics deals with the recurring.

Strategy and tactics both require an understanding of the environment as an n-dimensional evolving arena in which an entity’s actions play out.

Strategy requires the recognition of potential futures and the creative synthesis of desirable end-states in them.

Tactics require the recognition of recurring patterns and the successful execution of learned responses to them.

Strategy when best done evokes aesthetics. Strategy is an art.

Tactics when best done evokes calculation. Tactics is a science.

Policies are actionable rules encoding our values.

Procedures are rule-governed repeating activities, constrained by our policies, to achieve objectives we believe will aggregate into the achievement of higher-level objectives.

To make the most efficient progress toward our higher-level goals we must must be able to measure both where we are now and when we have achieved a goal.

Our near-term strategy is always more accurate and reliable than our long-term strategy. Therefore we must regularly renew our strategy to keep its edge sharp. As our higher-level objectives change we must evolve our procedures to meet them.

Our strategy and tactics must be consistent with our values. Selecting what to do next, and next, and next again is each and every time as much a moral action as is executing to our plan.

Strategy is the intersection of reason and righteousness.

As a Big_New_EDI_System developer

I want to be able to employ a standard router service

So that I can determine using the EDI transaction type, customer data, customer id prefix, and customer transaction date which back-end system to route the transaction to.

This rang really wrong for me. A developer shouldn’t be the user in an agile user story unless you are making a development tool, right?

So I decided to dig a little deeper.

I did a quick survey of all the user stories on the Big_New_EDI_System project. In 100 user stories there were 29 different users. Categorizing them as humans, transactions, developers, or systems, this is what I found:

100 total user story users =

81 developer users, such as ‘BPEL Orchestration Developer’

9 transaction users, such as ‘Order Status Inquiry’

4 system users, such as ‘Big_New_EDI_System’

6 human users, such as ‘Store Owner who has submitted a customer account validation request via our online tool.’

I would say my comfort level at this point was, oh, about a 6 on a scale of 100.

I could see how a system might be a user. But a developer? A transaction? Could it be that 9 out of 10 user stories on the project had been created with invalid users?

If so, what did that mean for the stories themselves, the team’s ability to work them, and their ability to finish the project? And if they did finish, what would be the impact to the nature and quality of the system they delivered? Time for a little pranayama.

If I was going to raise the hue and cry over the possible validity of the user stories on a hundred-plus resource eight-figure project I needed to laser-align my ducks.

I needed to understand exactly what a user story user was.

Today’s rumination is on the true nature of agile users.

In ‘On the true nature of use case actors‘  I described how poorly the use case actor has been defined in use case theory, and attempted a working definition.

The user story email made me suspect that the user story user might be similarly ill-defined. And given the similarities between use case actors and user story users I thought I might be able to leverage some of the same thinking I had employed in analyzing actors.

When I researched agile user story users I found the same kind of slim pickings I had found researching use case actors. There is very little original work on the methodology.  The overwhelming majority is simply parroted from elsewhere, usually without attribution.  What passes for expertise seems very shallow.

So my research approach was to start with work from key signatories of the Agile Manifesto and move forward in time.

Here is what I was able to glean.

Kent Beck (inventor of the term ‘user story’) has a number of wiki pages on c2.com where he and fellow Manifesters Alistair Cockburn, Ron Jeffries and other posters discuss various aspects of user stories. There is no single clear description of what a user story user is – users are generally referred to simply as the authors of user stories. In fact, there is a lot of discussion but little clarity on what user stories themselves are or are not.

Jeffries is the pithiest: “[a] UserStory is a story, told by the user, specifying how the system is supposed to work, written on a card, and of a complexity permitting estimation of how long it will take to implement.”

Jeff Sutherland – co-inventor of Scrum, and another original signatory of the Agile Manifesto, has published a presentation deck called ‘User Stories Done Right.’  But in it he largely borrows the actual user story stuff from one Mike Cohn.

And it is Cohn, Mr. Mountain Goat Software (cool logo and great website), who seems to have emerged as the leading acknowledged expert on user stories. He was a founding member of the Agile Alliance. His 2004 book ‘User Stories Applied for Agile Software Development,’ published in Addison Wesley’s Kent Beck ‘Signature Series’, has become a standard reference. According to the back cover blurb from ‘Applied’ “you’ll learn what makes a great user story, and what makes a bad one…”.  He has since published two other popular books on agile practice.

And we have Cohn to thank for what has become the most widely used template for user stories – it was used in the story we began with above:  “As a <type of user> I want to <feature> so that <benefit>”.

Cohn, who is not only an author but also an instructor and a popular speaker on the conference circuit, is definitely one of the parent-most nodes in the agile IP parrot tree.

So I got ‘Applied.’

What does Cohn have to say in it about user story users?  Let’s dive in together and see.  We are looking not only for explicit descriptions of user story users, but also for descriptions of user stories from which we might infer things about their users.

“What is a user story? A user story describes functionality that will be useful to either a user or a purchaser of a system or software.”

But then rather than trying to describe the difficult-to-describe, Cohn jumps right away to the ‘you’ll know it when you see it’ extensional device of some quick user story examples from the fictional used-for-illustration BigMoneyJobs website project:

“A user can search for jobs.

A company can post new job opening.

A user can limit who can see her resume.”

And he provides two examples of ‘bad’ user stories:

“The software will be written in C++.

The program will connect to the database through a connection pool.”

The users in two of the ‘good’ examples are people who will use the system, simply called ‘user’. We assume that then is the fundamental definition of user.

The user in the other good example is ‘company’. Since the company has no fingers, we assume that the actual user is a company employee, but that the objective is the company’s – the person is the company’s amanuensis, or, prefiguring our later explanation, the instrument of the company’s collective will. We will call the person-or-company entity ‘Party’ hereafter after Fowler.

Cohn says the first example is not a good user story because BigMoneyJobs users would not care which programming language was used. I can’t see how to squeeze a reasonable user at all into the passive ‘the software will be written in C++,’ no matter how acrobatic my grammar skills.

Cohn then tries to clarify. “If this were an application programming interface, then the user of that system (herself a programmer) could very well have written that ‘the software will be written in C++’.”

Ok. If you are making an API, then the developers who are going to be writing against it are the users. That _might_ make sense. (Shouldn’t the user be the system that will be calling the API?) But specifying it be written in C++ just seems out-of-bounds for a user requirement. The interface – CORBA IDL comes to mind as an interface frequently implemented in C++ – yes.

He says the second example is no good because the _real_ users, i.e. we can infer that he means the ‘program’ is not _really_ a user, “do not care about the technical details of how the program connects to the database.”

So he says on the one hand that it is ok to have technical details (‘C++’) in a user story, but on the other hand it’s not (‘database connection pool’).

Why is it ok for one and not the other? He gives us a clue: “Stories should be written so that customers can value them.”

So we assume the programmer user of an API values that it is in C++, but the business user of BigMoneyJobs does not value that its back-end data is accessed via a connection pool. He says that it if that actually _is_ a requirement (thought that is what the user stories were elucidating) that there are ways to express the requirement that are valuable to the business user. Which he leaves us with as the ‘What’s a User Story’ cliffhanger – tune in to Chapter 2 for the exciting conclusion!

Before we get there, though, Cohn goes on to advocate that the customers and users be the primary authors of the user stories. Which means projecting themselves into the ‘user’ slot.  (So _their_ answer to the question ‘What is an agile user’ is easy – ‘I am!”) But that where the users are not available, something called ‘User Role Modeling’ can help.

Cohn then reinforces the tacit fundamental definition: “A user story represents…something a user would be likely to do in a single sitting.”

In expanding on the ‘value to the users’ idea, closing the cliffhanger, Cohn says that stories that are valuable to the actual intended users are ok, and that stories that are valuable to the purchaser are ok (“The dev team will product documentation suitable to an ISO 9001 audit.”), but that stories that are valued only by developers (“all connections to the database are through a connection pool”) are not. He suggests that developer-valued stories can be written to be customer-valued, i.e. instead of speaking of database pooling, we should say something like “Up to 50 users should be able to use the application with a 5-user database license.”

Cohn goes on to talk about how to divide up stories that are too large to do in an iteration. There are two kinds of too-big stories, he says, compound and complex. Compound is a just a bunch o’ separate user stories, which split naturally.

In talking about dealing with complex stories Cohn give an example, splitting ‘A company can pay for a job posting with a credit card’ into ‘investigate credit card processing over the web’ and ‘a user can pay with a credit card.’

Here he has introduced a new kind of story – not one that speaks directly to a user goal at all, but one that describes a ‘spike’ that will prepare the project team for handling a subsequent story. Who is the ‘user’ of the “investigate credit card processing over the web”? It simply doesn’t fit our working definition of user story without stretching it beyond reason.

Surely a user story can’t simply be whatever you’d like.

In Chapter 3, Cohn suggests that it is a fallacy to write stories from the perspective of a single uber-user, that that approach can lead to undiscovered stories, and suggests user roles as the solution.

Using his example project, BigMoneyJobs, he asks – his italics – “when we talk about user stories, who is the user we’re talking about?”

Finally!

But rather than providing intensional rigor, he simply identifies a few possible users of the system, both job seekers and job posters, by name and brief description – sketchy (in the non-teenage-sense) Alan Cooperian personae.

He suggest that “[w]hile each user comes to the software with a different background and different goals [my italics] it is still possible to aggregate individual users and think of them in terms of user roles” – “collections of defining attributes that characterize a population of users and their intended interactions with the system.”

So we are still working in terms of the fundamental definition – humans who will interact with the software/system.

While our system will satisfy its set of high level goals – in his example the primary user goals are Find a Job and Find an Employee – they can be satisfied by a series of sub-goals which may be more or less important to particular segments of the highest level user demographic. And those segments may have other secondary goals in common, after Cooper, “some users favor convenience, others favor a rich experience, and so on.”

Cohn does call out system users in a boxed sidebar:

“As much as you can, stick with user roles that define people, as opposed to other systems. If you think it will help, then identify an occasional non-human user role.”

‘Occasional’ makes it sound as if there is a choice you are making willy-nilly among alternatives. Surely there is some set of ideal user roles for a given system. There must be a way of determining in a given instance what the appropriate users are.

Cohn does draw a perimeter of a kind:

“[w]e don’t need user roles for every conceivable user of the system, but we need roles for the ones who can make or break the success of our project. Since other systems are rarely purchasers of our system, they can rarely make or break the success of our system. Naturally, there will be exceptions to this, and if you feel that adding a non-human user role helps you think about your system, then add it.”

Really? Write stories for the people writing the checks, and add a system user if it helps you think?

Cohn suggests capturing these user roles on user role cards. He then goes on to suggest that some teams may find value in fleshing out Cooperian personae – detailed vignettes describing individual users based on role characteristics to help bring them to life, and/or Djajadiningrat (great name!) et al’s ‘extreme characters’, a technique where you use fringe user characters, almost caricatures, such as hookers and Popes, to effectively explore the skinny ends of your user demographic space’s bell-shaped curve. But in neither approach are the users usefully compared to Cohn’s base user story users.

We are provided one last chance to find out what bad users might be…Cohn catalogs ‘story smells’ in Chapter 14.

Unfortunately none of them captures the peculiar pong of a ‘bad user.’

He does advocate that we capture non-functional requirements as system user stories (with no user): “[t]he system must support peak usage of up to 50 concurrent users”

Let’s try to synthesize Cohn’s definition of user:

“An agile user story user is a human or company who is paying for a System or a human or company working in a given role who will interact with the System, and who can make or break the System’s success, and occasionally is a System user if it will help the developers think or capture a non-functional requirement.”

So I think we can see that there is a fundamental lack of rigor in the agile methods around the definitions of users and consequently a lack of rigor around what are valid and appropriate user stories.

In his chapter on User Role Modeling Cohn acknowledges a debt not to any of the Agile Manifesters but instead to the usage-driven design theory of Larry Constantine and Lucy Lockwood and the goal-directed design theory of Alan Cooper.

Let’s look at Cohn’s sources and see if we can find more clarity on users there.

Larry Constantine’s most recent concise description of users is in his 2005 article “Users, Roles and Personas.”

Constantine and Lockwood’s highest level methodology structure looks like this:

Role Model -> Task Model -> Content Model -> Visual and Interaction Design

C&L’s original articulation of what are now called ‘task cases’ came about because use cases were seen to have “too many assumptions and premature decisions about the user interface being designed.” He says that task cases have had widespread adoption, and actually cites the Cohn book we have been discussing as an example.

Forgive the extended quote, but after reading dozens of agile parrots’ prose I find Constantine downright lyrical:

“A task case represents a single, discrete intention carried out by a user in some role (my italics.)

“Task cases are also called essential use cases because they are stripped down to the barest essentials of the task. Each task case is expressed as a dialog in which user intentions and system responsibilities are abstract, simplified, and stripped of all assumptions about technology or implementation. This form of description is intended to get closer to the essence of what the task is really about from the perspective of the user in a role and to avoid making unintended or premature assumptions about the form of the user interface to be designed.

“The need to anchor the task model in an appropriate understanding of users quickly became apparent to those working with essential use cases. User roles emerged as an elaboration of the software engineering construct of Actors (Jacobson et al., 1992), which, like use cases, required adaptation to better suit the needs of user interface designers.”

“User roles capture and carry the essential understanding about users.”

“In the broadest sense, a user role has been defined as a collection of characteristic needs, interests, expectations, and behaviors in relation to a particular system (Wirfs-Brock, 1993).

“In its most compact form, the form now most commonly used, a user role is represented by its Context, Characteristics, and Criteria, that is, the context in which it is played, the characteristics of its performance, and the criteria that must be met by the design to support successful performance of the role. Context includes the overall responsibilities of the role and the larger work and environmental context within which it is played. Characteristics refer to typical patterns of interaction, behaviors, and attitudes within the role. Criteria include any special functional facilities needed for effective support of the role along with design objectives of particular importance to that role. Such criteria for effective support of a role are sometimes referred to as usability or user experience attributes.”

Ahhhh….sweet like pure oxygen, isn’ t it?

Larry Constantine is also a lot clearer than Cohn about the prioritization/ranking of user roles.

“In usage-centered design, user roles are usually ranked by anticipated frequency and by business importance. The agile modeling technique for making these rankings is to sort index cards representing the roles into order (Constantine and Lockwood, 2002). On the basis of the combination of these two potentially different views of priority, a small subset of roles is distinguished as “focal” roles. Focal roles serve as a central focus for the rest of the design process, but not to the exclusion of other user roles.”

Unfortunately in Cohn’s popularization this vital closure has been lost – focal roles become the vital ones “who can make or break the success of our project”.

The overall rigor we have been starving for in Cohn is supplied by Constantine:

“The complete task model is expressed by the narrative bodies of all the individual task cases along with a task case map similar to a user role map but showing the interrelationships among task cases. The task case model can be checked against the user role model to verify that all user roles can be fully performed with the identified task cases and that every task case is genuinely needed for the performance of some one or more roles. The objective is to be truly comprehensive, to cover all tasks needed to fully perform all identified roles. In principle it might seem that a complete task model is an unobtainable ideal, but in practice we have found that, with well-timed and thoughtful reviews along the way, it is rare to discover missing task needs late in the game.”

So our fundamental conclusion is this. There was a lack of clarity in the original exposition of agile methodologies around users and user stories. Early practitioners such as Cohn who popularized the user story approach based their work in part on the ‘essential use cases’ or ‘task cases’ in the usage-directed design tenets of Larry Constantine et al. But the potential rigor present in Constantine’s full method was lost in their adaptations. While there is clear value in Cohn and other popularizers’ approaches for greenfield projects, there is insufficient guidance and rigor around the definitions of users and user stories to help agile methodology adopters for non-greenfield projects make good decisions.

Here I would like to draw on the work I’ve done on use case actors. To do that, I need to distinguish use cases from user stories.

But I find this is a very difficult task.  And I am not alone.

Alistair Cockburn, the Eldest Son of Use Cases (if Jacobson the pater familias) has practically thrown up his hands trying.

“I’ve tried answering this question so many times and in so many ways: “What’s the difference between a use case and a user story?

I know what each is, I know how to describe each; I’ve described each and even how to relate the two. I’ve finally concluded that a user story is to a use case as a gazelle is to a gazebo.

See if you can describe the difference without just enumerating what each is.

As the Mad Hatter asked: “Why is a raven like a writing desk?”

And a few dozen subsequent comments on his webpage do nothing to illuminate.

Fellow Manifester Martin Fowler says that

“[Use cases and user stories ] have a complex correlation. Stories are usually more fine-grained because they have to be entirely buildable within an iteration (one or two weeks for XP). A small use case may correspond entirely to a story; however a story might be one or more scenarios in a use case, or one or more steps in a use case. A story may not even show up in a use case narrative, such as adding a new asset depreciation method to a pop up list”

But the most clarity comes, as we might expect, from Larry Constantine:

“A task case represents a single, discrete intention carried out by a user in some role.”

Based on the relationship of task cases to use case, we can infer this:

• A use case represents the collective scenarios to achieving a high-level goal for a system user.
• A user story is one or more task cases that can be solved in a given time box, where each task case represents a single, discrete intention carried out by a use case user in a particular role consistent through the story.
• An intention is an attempt to satisfy a sub-goal, where a series of sub-goals adds up inexorably to an overall user goal as described in a use case.

So a user story user is one of the facets of a use case user…the multiple personalities of all the user story users sum up to the use case users.

So a user story user IS A a use case user – IS A in a generalization-specialization sense.  It is a complex sub-type derived from many partitions on use case user.  The partitions themselves may be categorized as Constantine’s context, characteristics and criteria.

Progress.

As we have demonstrated, a use case actor is a direct or indirect volitional entity.

So a user story user is a role played by a direct or indirect volitional entity with respect to a system.

A user story is an attempt by such an entity in a given role to satisfy an intention, where said intention represents the achievement of a sub-goal participating in the aggregate accomplishment of the goal of the volitional entity.

So whither our Big_New_EDI_System Project?

Cohn’s sample project in ‘Applied’ is a modest web-based job board.

He ends up with 31 use cases. Only three of them are neither ‘a user’, ‘a repeat user’, ‘an administrator’ or ‘a report viewer.’

“The system must support peak usage of up to 50 concurrent users.”

“Orders made from the website have to end up in the same database as orders made by phone”

“The site always tells shoppers what the last 3? items she viewed are and provides links back to them

In Big_New (sounds political like Zbiegniew or Agnew : ) there are 100 user stories. Ninety percent of the users are ‘developers’ or ‘transactions’.

An EDI project will obviously have many fewer direct volitional entity users than a web-based job board.  And it will be much more challenging to define the user roles well.  There will be a lot of system users. That is why we need the definition rigor – it is not a happy-path vanilla-case project for agile.

There might be an objection raised from the touchy-feely user story school that says a user story is just a promise for a conversation to be held later.  So that not much rigor is required, and I should untwist my panties.

My first response is that they should go build a doghouse to hang their crystals in and stay away from my skyscraper project.

Biting that back let me just ask this.  For those 81 user stories with developer users, who are the conversations going to be with?

If you are building a compiler or an IDE you will have developer users to talk to.  Otherwise developer users are just plain wrong.  And the user stories made from them are immediately suspect.

Time to raise the hue and cry.

When a tool is right for a job, the tool disappears from the user’s active experience.

Our goal in IT is to equip our business users with disappearing tools.

Our goal as IT architects is to anticipate the needs of the business accurately, and with enough lead time, so that we can buy – or build – and integrate tools into the business work experience…so those tools, too, can disappear.

This is the very definition of ‘aligned.’

A technology roadmap is the enterprise architect’s forecast of the information tools needed at certain times in order to support the business’s anticipated tasks.

Today’s rumination is about how to make such a roadmap.

I am a chess player.  Somewhere in the family store of dusty stuff there may still be the trophy I won for finishing second in the Indiana Junior (under 18) Chess Championship in 1977 (I think it was ’77.)  Which I mostly only say to give some weight to what follows.  (And for what it’s worth I should have finished worse…in the five round Swiss system tournament I drew my fourth round game from a won position – connected passed pawns – and so did not have to play The Prodigy in the final round.   So I won my last round game, and instead of finishing 4-1 – The Prodigy would have eaten my lunch – I finished 4 1/2 – 1/2, good for second.  If I had known the outcome in advance I might have thrown the fourth round win to get the draw I ended up with.  I was saved from the moral quandry by ignorance of the overall picture.  And fwiw the memory of drawing with connected passers grates more than not winning.)

Aaaaanyway, one of the best chess instruction books ever is Jeremy Silman’s ‘How to Reassess Your Chess.’  In it, the International Master (the best teachers are not the best players – Silman is not a Grandmaster) outlines the Silman Thinking Technique.

Paraphrasing broadly, here ’tis. In your mind’s eye, take all of the remaining pieces off the board, leaving only the pawns.  Now arbitrarily place your pieces anywhere you want.  Can you find a way of placing your pieces that gives you a marked advantage?  Can you create a checkmate?  Can you create a preponderance of force against a key point?  Understanding the optimal placement requires not only an understanding of the rules of the game, it also requires an understanding of tactical patterns.   Is there a last rank mate by a rook?  A windmill?  A discovered check?

Having placed your pieces in an optimal configuration, you then need to figure out if you can achieve it…how long will it take to you get there from where you are at now?  And remember, your opponent will be working against you all the while.

Finally, given the constraints in your position – the slow to change pawn structure – and given your opponent’s best efforts to foil your plan, can you achieve your objective?  If so, rock on.  If not, go back to the beginning and make a less ambitious objective.

So what is your best plan?  And given that, what is the best first move to effect that plan?

Roadmapping is like that.

Keeping our chess analogy in mind, let’s look at roadmaps.

Remember we have said that a technology roadmap is the enterprise architect’s forecast of the information tools needed at certain times in order to support the business’s (an awkward possessive)’s anticipated tasks.

The business’s anticipated tasks obviously must be inferred from their goals and objectives.  Not so obvious, perhaps, is that they must also be inferred from the career expertise of the business users and from their training plans.  The tool an expert finds ready-to-hand and that of a beginner are not the same tool.  Tiger Woods plays with forged irons which permit precise control, but are not forgiving of error.  The amateur on the other hand plays with cavity back irons which assume off center hits, and minimize their impact.

So this is where our roadmap starts. We must work with the business to capture their goals and objectives.  There may be more or less work to do in this regard depending on the maturity of the business’s planning approach.  And we must work with the business to understand the nature of their expertise, their professional development plans, and their future expectations with respect to their tools.

From their goals and objectives we synthesize a picture of the business enterprise at some future date.  Providing the tools to support the business processes in that picture is our objective.  The full picture, including the tools, is where our roadmap leads.

We must pick a waypoint far enough in the future to make it possible for us to steer our ship toward that star.   Turning a supertanker involves holding the tiller hard over..and waiting, and waiting, and waiting for the ship to turn.  Turning an enterprise is even harder.  Project plans, with rare exception, will only take us out a year or eighteen months.   To make informed decisions about which projects to approve as aligned we need to go further out on the timeline.

If we get too far out, on the other hand, we obviously lose the ability to describe the expected future environment with any degree of accuracy.  In 1999 there was no MySpace, no Facebook, no Twitter.   But in 2004 all existed in some nascent form and a wise and careful observer might have projected their success with some confidence.

For the same reasons, existing corporate strategic plans are likely articulated over five years, or sometimes three years in rapidly changing industries.

So our first task is to take the business’s strategic plan (or if needed to help the business to create that plan), turn it into a to-be process picture, and figure out what the technology picture needs to be at that time to support those processes.

High level business use cases suggest themselves for the to-be process picture, following the familiar goal-driven approach espoused by Alastair Cook.  Which naturally leads us to describing our tools in the form of the ’4′ in Phillipe Krutchen’s 4+1 architecture approach, where the ’1′ is the use cases.

Having done so (ok, some handwaving here), we are ready to turn out attention to the roadmap.

We are not starting with a blank slate.  Not only do we have an existing IT infrastructure that represents our starting point, there are a number of constraints of which we must be cognizant and to which our roadmap, and our destination, must conform.

Nothing in the corporate existing or ‘as-is’ environment is fixed.  All is malleable.

But the cost and impact of changing a given feature varies.

Those things that are slower to change, and less likely to be changed, act as constraints on our changing other, more malleable features which are dependent on them.

Many of these are features that are managed in lifecycles, where there are changes punctuated with some rhythm or frequency:

• Hardware->Depreciation.  Lease terms.  End of life of support.
• Software->Licensing. Compatibility. End of life of support.
• Contracts->Terms and conditions
• Laws->Elections.

And of course Trends, Buzz, Fashions…those things with mind share in our decision makers.

We should model our constraints in terms of their speed of change, and the punctuation of their change, in order to determine…their contribution to the viscosity of our environment, and their projected influence at a given waypoint.  This approach to constraints should be part of the requirements assessment on every project.

In reality, of course, there is a continuous capability technology provides the business.

That technology evolves as new capabilities are brought on line, old ones are retired, problems are fixed, functions refined.

But as we have described, we know that our budget cycles are generally yearly.

So rather than an continuous roadmap, we should consider a series of one year roadmaps, each simply articulating the sub-goals, the sub-objectives, that add up to our overall goal.  We can then speak to the alignment of the slate of potential projects fielded for the next year.

This is arguably the path to an intelligent enterprise.  As we have seen in our earlier rumination on BI, the ability to create and work toward new sub-goals when faced with obstacles – including our evolving constraints -  is the very definition of business intelligence.

My daughter Zoe also likes to make up games – ever since she learned as a little girl that she who controls the rules controls the game.

In that spirit I contributed a new game to the family library last year. One person names three to five things, and the other players compete to see who can first name a category to which all the things named belong. For example, someone might say ‘vichyssoise, revenge, and beer’ and the category would be ‘things that are best served cold.’ A lame example, but then again, I usually lose. Trust me when I tell you there are wonderfully subtle and challenging lists.

Zoe asked me what we should call the game…and I thought of the seminal work on categorization by Berkley prof George Lakoff, with one of the best titles ever:  ‘Women, Fire and Dangerous Things.’

And that’s what we call the game now….’Women, Fire and Dangerous Things.’

So I was at a Barnes and Nobel in San Antonio, Texas a few weeks ago killing time on my way to the airport. I had to check out of my hotel at noon, but my flight back to Medford via Denver wasn’t until 3, so I had some time to kill. I still had a B&N gift certificate in my wallet from Christmas, so I decided to stop there on the way and find a book for the trip back home.

Maybe because I was away from home and missing my family I thought of Lakoff.  So I asked the bookseller if they had a copy of ‘Women, Fire and Dangerous Things’…not really expecting them to have a twenty-year-old cognitive science book.  But there it was.  Not only that, they also had Lakoff’s book on metaphor he did with Mark Johnson – subject of a future rumination related to Folk IT.  And both are now in my current ‘inner library’ – that floating set of a dozen or so books that never quite get shelved.

And today I was thinking about my previous posting, ‘On Intension, Extension, Wittgenstein and Exceptional Software‘, in which I argued that we should always make our type partitions incomplete so that we could accommodate Wittgenstein’s family resemblance categories, and I realized that Lakoff provides even more direction as to the nature of non-classical categorization that would be very useful to incorporate into our approach to modeling:

“The idea that categories are defined by common properties is not only our everyday folk theory of what a category is, it is also the principle technical theory – one that has been with us for more that two thousand years. The classical view that categories are based on shared properties is not entirely wrong. We do often categorize things on that basis. But that is only a small part of the story. In recent years it has become clear that categorization is more complex than that.”

“From the time of Aristotle to the later work of Wittgenstein, categories were thought to be well understood and unproblematic. They were assumed to be abstract containers, with things either inside or outside the category. Things were assumed to be in the same category if and only if they had certain properties in common. And the properties they had in common were seen as defining the category.”

But as a result of the seed planted by the later Wittgenstein about family resemblance categories, as well as later work by Eleanor Rosch,

our understanding of categorization has changed.

Rosch looked at two fundamental problems with classical theory:

• If categories are only defined by properties, then no members should be better examples of the category than any other.
• If categories are only defined by properties of their members, then the categorizers’ capabilities or attributes should have no bearing on the resulting category

Of course, as soon as one starts looking into it, Ms Elenor is obviously onto something profound.

What has emerged is something called ‘prototype theory.’  Thanks in large part to Lakoff, it has gotten all bollixed up in the Anglo vs. European philosophy,  objectivism vs. subjectivism debate spaces.   Which has unfortunately tended to marginalize it.

But for our purposes here, we should simply note this:

Our classical understanding of what a ‘Class’ is is simply wrong in many cases. There are classes which are constellations of other classes, not all the attributes of which which overlap.  And there are members of of a class that are best examples – not all members of a class are created equal.  As Rosch points out, a robin is a much better example of ‘Bird’ than an ostrich…or a chicken. And an expert in a field can classify a set not classifiable by a rule. Which may be related to Godel’s Theorem if set membership is  a truth-valued relation.  There is an interesting example in Dreyfus’s ‘Why Computers (Still) Can’t Think’: an expert gift-giver who can give something everyone agrees is perfect..even though no one has ever given it before or included it in a list of potentially desirable gifts or projected it from a model of gift-giving.

Because our classical object-oriented approach to classes starts to fail as the users’ perspective is brought in.  One of the practical challenges of use case realization is in the data mapping…and we are starting to realize why.

A revision of object-oriented design tenets that accomodates prototype theory will help software more closely meet users’ needs in the real world.

There must also be a ‘folk computer science’ – a ‘folk I.T.’ – the conceptual framework that end users use to explain and understand their interactions with computers.  Understanding the nature of Folk I.T. might revolutionize the ability of IT shops to effectively capture and understand requirements, to promote new technology solutions, and to succeed in creating true business alignment.

Snapshots from Wikipedia (the links will take you to the cited passage):

Folk psychology (also known as common sense psychology, naïve psychology or vernacular psychology) is the set of assumptions, constructs, and convictions that makes up the everyday language in which people discuss human psychology. Folk psychology embraces everyday concepts like “beliefs”, “desires”, “fear”, and “hope”.

Naïve physics or folk physics is the untrained human perception of basic physical phenomena. In the field of artificial intelligence the study of naïve physics is a part of the effort to formalize the common knowledge of human beings.

Many ideas of folk physics are simplifications, misunderstandings, or misperceptions of well understood phenomena, incapable of giving useful predictions of detailed experiments, or simply are contradicted by more thorough observations. They may sometimes be true, be true in certain limited cases, be true as a good first approximation to a more complex effect, or predict the same effect but misunderstand the underlying mechanism.

Even people who start to become educated in the underlying sciences persist in keeping their folk systems of understanding.

In Steven Pinker’s ‘The Stuff of Thought’ he relates that “[m]any psychological experiments have shown that when people have a pet theory of how things work (such as that damp weather causes arthritis pain) they will swear that they can see those correlations in the world, even when the numbers show that the correlations don’t exist and never did…”

In ‘The New Cognitive Neurosciences’ Michael S. Gazzaniga and Emilio Bizzi summarize the folk sciences:

Folk psychology is our everyday ability to understand and predict an agent’s behavior in terms of irrational states such as goals, beliefs and desires. … Folk physics is our everyday ability to understand and predict the behavior of inanimate objects in terms of principles related to physical causality.

Folk IT is our everyday ability to understand and predict the behavior of software applications in terms of …. ?

I don’t have the answer.

But it is clear that rather than trying, point by point, to correct and educate end users about the true nature of software and computer systems, we should consider instead systematically capturing the world of their existing understanding…to capture the Folk I.T. framework by which, in which, they understand how software and computer systems behave.

Then and only then would we be in a position to successfully add to, to augment, to extend, their framework by integrating a new or changed component on their terms.

As is well known, Jacobson’s use cases went mainstream in ’92 without benefit of a rigorous definition.  While that choice was arguably key to their successful initial adoption, as their use has matured a consensus formalism (well, formal enough – call it a ‘soft’ formalism) has emerged, driven first by Alistair Cockburn’s ‘Structuring Use Cases with Goals’ and later by the adoption of use cases in UML.

But that consensus has a significant gap that is still in my experience the source of endless debate and the cause of practical problems: the actor.

Closing that gap – getting at the true nature of use case actors – is the goal of today’s rumination.

There is simply not a common working understanding of what an actor is.  I have had very senior systems analysts tell me they really only did use cases for human actors – anything else was an integration problem.  As a partial result, that shop’s requirements docs confusingly include both SSA-style context diagrams as well as use case diagrams.  (Of course SSA’s ‘is it an internal or external system’ is simply the same issue in a different guise.)

It’s so bad that in a recent (2006) book on UML 2.0 (‘Learning UML 2.0′ by Russ Miles and Kim Hamilton) the authors start to throw in the towel and say you’ll know an actor when you see one:

“Deciding what is and what isn’t an actor is tricky and is something best learned by experience.”

Then in some sort of pole shift they throw the reader a bone in the form of an actor-identification algorithm:

“Identify a Thing from your requirements.  Is it an actual person?  If yes, it is _probably_ an actor – but be careful, some people are a actually part of the system.  If the thing is not an actual person, then it is something you can change in the system’s design?  If yes, then it is probably _not_ an actor.  If no, then is probably is an actor.”

Sorta maybe kinda.

And a little further on…in a section called ‘Tricky Actors’ (you can’t make this stuff up):

“Not all actors are obvious external systems or people that interact with your system.  An example of a common tricky actor is the system clock.  The name alone implies the clock is part of the system.  But is it really?  The system clock comes into play when it invokes some behavior within your system.  It is hard to determine if the system clock is an actor because the clock is not clearly outside of your system. As it turns out, the system clock _is_ often best described as as actor because it is something you can’t influence.”

Come again? (It’s a red flag for me when what I’m reading starts to sound like an Abbott and Costello routine.)

And from Doug Rosenberg and Matt Stephens’s 2007 ‘Use Case Driven Object Modeling with UML‘:

“An actor is represented on the diagram by a stick figure and is analagous to a ‘role’ that users can play.”

“Actors can represent nonhuman external systems as well as people. Sometimes people are confused by this notion; we’ve found that drawing a ‘robot stick figure icon’ seems to clear this up.”

Evil Robots!  Cool.

In fairness to the authors – who I don’t mean to impugn, I’m just having a little fun at their expense – the Schadenfreude is yours, gentle reader – earlier, more authoritative, less florid works did not much improve on Jacobson.

From the OMG’s UML standard we take the following:

“An actor in the Unified Modeling Language (UML) “specifies a role played by a user or any other system that interacts with the subject.”

“An Actor models a type of role played by an entity that interacts with the subject (e.g., by exchanging signals and data), but which is external to the subject.”

“Actors may represent roles played by human users, external hardware, or other subjects. Note that an actor does not necessarily represent a specific physical entity but merely a particular facet (i.e., “role”) of some entity that is relevant to the specification of its associated use cases. Thus, a single physical instance may play the role of several different actors and, conversely, a given actor may be played by multiple different instances.”

That last one was pretty circular, wasn’t it.  Craig Larman, hedging his bet carefully, gives us an ‘informal definition’ in his Applying UML and Patterns from 2002:

“First for some informal definitions.  An actor is something with behavior, such as a person (identified by role), computer system, or organization ; for example, a cashier.”

And in a work citing Jacobson himself as an author,  ‘Use Case Modeling,’ the 2003 book by Kurt Bittner and Ian Spence and Ivar Jacobson , we find the prosaic ‘[a]ctors represent the people or things that interact with the system; by definition, they are outside the system. We focus on the actors to ensure the system does something useful.”

Cockburn – if Jacobson is the father of use cases Cockburn is the oldest son – says in his seminal ‘Structuring Use Cases with Goals‘ that

“[a] ‘’primary’’ actor is one having a goal requiring the assistance of the system. A ‘’secondary’’ actor is one from which the system needs assistance to satisfy its goal. One of the actors is designated as the system under discussion.

Each actor has a set of ‘’responsibilities’’. To carry out that responsibility, it sets some goals. To reach a goal, it performs some actions. An ‘’action’’ is the triggering of an interaction with another actor, calling upon one of the responsibilities of the other actor (see also Figure 3)[not shown here - Ed.]. If the other actor delivers, then the primary actor is closer to reaching its goal. The entity-relationship diagram corresponding to this is shown in Figure 2.

Figure 2 says that there are internal and external actors. An external actor can be a person, a group of people or a system of any kind. The internal actor may be the system in design, a subsystem or an object. The system in design consists of subsystems, which consist of objects. Actors have behavior(s). The top-level behavior is a responsibility. It contains goals, which contain actions. An action triggers an interaction. The interaction is one actor’s goal calling upon another actors (or its own) responsibility.”

This is much better…although a goal calling on a responsibility doesn’t really work, does it?  The previous authors have attempted to identify an actor by listing its attributes – a dog is furry, has four legs, and barks.  But we need to understand something like an actor by placing it in a context, such as Cockburn has done.  But is has to be a logical context, one that makes sense.   And as we’ll see, Cockburn’s is both incomplete and inaccurate.   And we are completely missing an understanding of an actor’s essense.  For better or worse, our Folk IT understanding (the subject of the next rumination!) is based on essences.

So what do we have so far?   A collective definition synthesized from the aforementioned sources would be something like this:

“An actor is an person, system, or clock external to the in-scope system with goals which require them to engage in behaviors interacting with the in-scope sytem to achieve.”

Let’s analyze this.

First, can a system really have a goal?  Of course not.  Systems are tools…no matter how sophisticated (and leaving the Strong AI debate for a different day), they have purposes, not goals.  A cup is designed and built to achieve the user’s goal of having a drink.

So if a system can’t have a goal, how can a system be an actor?  It can’t, at least as described by Cockburn’s framework.  So the framework fails.

Who or what has goals, and what is their relationship to the system in question?

Simply put, humans, both singly and in groups, have goals.   The pursuit of a goal is an act of will.   The ability to achieve goals in the face of obstacles is intelligence.  The pursuit is a moral act – the agent is responsible for the outcome.

This is so fundamental it is reflected in language itself.  In Pinker’s ‘The Stuff of Thought’ we find this:

“Linguists sort verbs into classes, each called an ‘Aktionsart’, German for “action type,” based on their temporal conture….Each of the four major action classes, state, activity, culmination, and accomplishment, smuggles in a concept of will….Activities and accomplishments are voluntary,  state and culmination involuntary….Indeed, with an accomplishment it is the actor’s goal that determines the exact event that consummates it.  Once again, this is not just a fine point of grammar but a key to our moral sense.”

So the fundamental actors are those humans and groups of humans who can exercise will.  We shall call them ‘volitional entities.’  And the fundamental relationship of volitional entities to our systems is a moral one – they are responsible for the actions they take with the system to achieve their ends.

The practical relationship of an actor to an in-scope system is one of communication.  There must be some direct chain of events from a physical action – clicking a mouse, pushing a button, inserting a card, speaking a word – that triggers a state change in the in scope system.

When a volitional entity is a person, they may interact directly with the in scope system.  But when a volitional entity is a group, they have no direct ability to interact…a company doesn’t have a voice, or fingers.  So they have to communicate indirectly.

So what are system actors?   When a system is an actor, it is only so because it in turn has been driven by a volitional entity who has triggered its actions – or by another system which in turn was put in motion by a volitional entity, etc.    So the volitional entity is interacting indirectly with the in scope system – so in a sense the in-scope system is simply acting as a component of a larger system – and the volitional entity, the primum movens, is still responsible for the actions of the in-scope system.

A clock actor simply refers to a system actor whose purpose is narrowly to express the deferred will of the volitional entity as to when something occurs.  For actions whose period is arbitary, such as daily batch jobs, the will may not be just the timing, but that is happens at all, that is, the clock is a way to express the will of the group.

A rule actor would be the more general case, where the volitional entity has codified their will to have particular things happen in particular circumstances.  Unlike the clock actor, whose triggering domain is just the clock, the rule actor’s domain includes a larger set of states, usually including some business subdomain.

Ok.  We seem to have made some progress.  Let’s recap.

A volitional entity is human or group of humans – Fowler’s ‘Party’ – which can attempt to achieve goals by interacting with systems.  They are morally responsible for the outcomes.

A volitional entity’s will can be exercised in pursuit of a goal with respect to a given system either directly or indirectly.

When it is expressed directly, the volitional entity is a human engaged in physical communication with the system.

When it is expressed indirectly, some other system is engaged in physical communication with the in scope system.  That system in turn has either been directly or indirectly engaged by a volitional entity.  The primum movens, the ‘first mover,’ will always be either a human or an event-triggering system that has been programmed by a human or group to initiation an action according to a set of environmental rules – at a given time, or at a give barometric pressure, etc.

So a use case actor is an object which represents a direct or indirect volitional entity’s communication with an in-scope system.  A complete use case picture would have at its periphery only volitional entities, which are both the primum movens and moral owners of any system behaviors.

Software architecture is building and using models of software to help us understand the reality of software.

Software itself is based on models. But insofar as one of the primary types of data we deal with is indicative, data that indexes and describes something in the world, it is important for us to remember, as Rene Magritte famously reminds us, a picture of a pipe is not a pipe. And a record of a person is not a person.

What I’d like to explore in this rumination is the gap.

Underneath both the relational model and the object-oriented model, the two dominant models in software today, we find Zermelo-Fraenkel, or ZF, set theory.

There are two deterministic ways of describing the members of a set, intension and extension. Intension applies rules to a population to carve out set membership. Red-headed step children is an intensional set. Extension simply lists the members: the world’s greatest guitar players are Paco de Lucia, Pat Metheny, John McLaughlin, and Eric Clapton. (Ok, maybe not the same as your list.)

But, as Wittgenstein pointed out, humans have the ability to categorize, to make sets, based not on explicit rules but on family resemblances. We can include or exclude things in sets without applying exact rules. These conceptual categories are how we interact with the world.

So when we design and build our model-based, rule-based, logic-based software, we leave out stuff our users _know should be there_, things they _know should be included_.

How do we deal with the gap?

I don’t have a fully formed idea. But I think it may be to always build extension into our model of set descriptions to complement intension…to always have a way that users can say, ‘I know your software says birds fly, but a penguin is a bird, so just deal with it.’

This means no complete partitions. If you partition people on gender, and your subtypes are ‘male’ and ‘female’, build your software to enable users to extend it so it will accommodate the real world where we have hermaphrodites.

These aren’t exceptions. We should not have to deal with them as such. It is our software that is imperfect, not the world.

Chris Date is also an opinionated cuss. His introduction of new terminology (quick, what’s a RELVAR?), and his sharp opinions presented in his instantly recognizable, slightly supercilious style tend to polarize his audience. So taking pot shots at Date’s opinions is a common sport in the industry. But Date is smart as a whip, and his acolytes are legion, so if you are going at him you’d better bring your A game.

My turn.

Date first described something he called ‘The First Great Blunder’ in a paper titled ‘Introducing…The Third Manifesto’ he and his frequent co-author Hugh Darwen wrote back in ’95. The ‘Blunder’ struck me as wrong at first reading. But it was wrong in some low level conceptual way that was, that has been, difficult to articulate. And Date, who seems to be the most fond of his opinions when they skewer some conventional praxis, has put references to said Blunder in many of writings since then – not only the full book version of ‘Third Manifesto,’ but into updates of ‘Intro to Database Systems’ as well – so that it is In My Face All The Damn Time.

So the time has finally come to drive a stake through Date’s Blunder, and show that it is really a case of the First Great Blinder. And the refutation will take us through the pretty interesting territory that lies under databases and data models – the land of set theory, logic and identity – and I think we’ll end up not only with a refutation of Date, but also an interesting conceptual perspective on the world of object-relational mapping.

Ok, so what is The Blunder? The Blunder comes at the intersection of The Relational World and The Object World. Date suggests the key question in object-relational mapping is this: ‘What concept is it in the relational world that is the counterpart to the concept object class in the object world’? And he characterizes the most common answer, mapping a table (in Date-Speak a relational variable or ‘RELVAR’) to a class, as The First Great Blunder.

Date understands that that mapping is a very natural one to make. If we look at the DDL for a Person table

CREATE TABLE PERSON {
FirstName VARCHAR2(20),
LastName VARCHAR2(20),
Gender CHAR(1),
Birthdate DATE
}

and the (Java) code for a class definition

class Person {
String FirstName;
String LastName;
char Gender;
Date Birthdate
}

it looks like a no-brainer. What could be more obvious? So what problem could Date have with this?

The answer requires a longer explanation of the relational world. Date’s vocabulary is full of ‘domains’ and ‘attributes’ and ‘tuples’ and ‘relations’ and ‘relvars’ instead of ‘data types’ and ‘columns’ and ‘rows’ and ‘resultsets’ and ‘tables.’ I’ll try to use the latter vocabulary so that those who haven’t had the KoolAid can follow without a dictionary, and those of us who found the KoolAid too bitter to swallow can follow without doing the internal simultaneous translation of the non-native speaker.

Date believes the correct mapping is not from class to table, but from class to data type.

Sounds like another no-brainer. A class is a data type.

The natural answer given our two no-brainers is that a class can map to a data type or a table. So the key question to understand Date is why does he think a class can’t map to a table?

Because he believes a table is a variable, and a class is a type. And that a class has methods and no public instance variables, whereas a table has public instance variables and only optionally methods.

Because he believes the fundamental value of an instance of a table is the set of all rows – a resultset or Date’s ‘relation’ – and the fundamental value of an instance of a class is a single object.

A couple of threads are entertwined here that we need to tease apart. The first is the database side distinction between data types and tables – between ‘domains’ and ‘relvars.’ The second is the single instance to set question.

What Date has missed on the database side is that the distinction he is drawing between data types and tables is an implementation distinction, not a conceptual one. And the important conceptual distinction he needs to visit and does not is that between entities – those instances that have identity – and non-entities. His mistake – the First Great Blinder – comes from not opening up his mind wide enough. No no no no he’s ou-out side, looking in…

Taking our inspiration from Jorge Luis Borges’ short story ‘The Library of Babel‘, let’s imagine in place of each of the data types we have a table holding all possible values of that data type. So for integers, we have a table ‘Integer’ with two columns, ‘SurrogateKey’ and ‘Value’, and an infinite number of rows, one row each for each integer.

“A moment” says the perceptive gentle reader. “What data type is ‘SurrogateKey?’ And more importantly, what data type is ‘Value?’”

The type of SurrogateKey we shall leave undefined except to say it is a unique identifier with aleph-null possible values. And the type of ‘Value?’ Why, integer, of course, says the White Rabbit. In infinite regress. But that is in the nature of the definition of number. And for each possible width of VARCHAR, such as 1 or 223, we have a table such as VARCHAR_1 or VARCHAR_223 whose columns are ‘SurrogateKey’ and ‘Value’ where the datatype of Value is ‘VARCHAR(1) or ‘VARCHAR(223). And the rows contain all the possible permutations of all the valid characters for VARCHARs. So that our table VARCHAR_1′s rows might be

KEY VALUE

1 a

2 b

etc.

And EVERYDATE is a table with two columns, ‘SurrogateKey’ and ‘Value,’ and its rows contain all possible date values. (Those of you who have designed data warehouses may have just had an ‘A Ha’ moment.)

So now, at our database design level, our tables consist only of sets of foreign keys with constraints binding them to other tables.

And our Person table definition becomes

CREATE TABLE PERSON {
FirstName mystical_surrogate_key,
LastName mystical_surrogate_key,
Gender mystical_surrogate_key,
Birthdate mystical_surrogate_key,
CONSTRAINT fkFirstName FOREIGN KEY (FirstName) REFERENCES Varchar_20(SurrogateKey),
CONSTRAINT fkLastName FOREIGN KEY (LastName) REFERENCES Varchar_20(SurrogateKey),
CONSTRAINT fkGender FOREIGN KEY (Gender) REFERENCES Varchar_1(SurrogateKey),
CONSTRAINT fkBirthdate FOREIGN KEY (BirthDate) REFERENCES EveryDate(SurrogateKey),
}

Ok. So now, in place of the surrogate keys just imagine that each actual value of a datatype, each varchar, or each integer, is not a literal value, but a foreign key to a table holding all possible values of that datatype…but because those tables have no other data, only the single columns, we never instantiate them explicitly. We can make direct references from one key value, such as the integer 23, to the same key value, 23, anywhere they occur.

The long-winded point here is that the distinction between data types and tables is not a necessary one, not a base conceptual distinction. There is no difference. The distinction is a practical one, one of convenience. All there are at the bottom are sets and sets of sets. It’s turtles all the way down.

The First Great Blinder is Date’s attempt to map from one implementation to another without reference to an underlying conceptual model. This is the most common error made in integration projects. We map one tool’s export file directly to another tool’s import file, and, rather than creating a common logical domain model to guide the mapping, we let Junior Programmer do it in their cube. And inevitably we end up with mapping errors.

The more important distinction we should be making in both realms is between entities and non-entities. An entity has identity. It is unique in its space. We describe an entity with a set of attributes (which as we’ve seen are sets of sets of sets of …), just like we describe something without identity. So what is the difference? An entity has continuity over time. It has a continuous existence. This notion is fundamental to OR-Mapping, not the distinction between data type and tables.

I’ll have a lot more to say about the nature of identity in another long-promised rumination.

The second mistake Date makes is to use the single table and single class as his canonical example for the mapping. As anyone who has ever worked in the trenches of OR mapping, or model-driven architecture tools, knows, single tables and single classes are just simple cases of the much more useful, and _more fundamental in most important ways_, relations – not Date’s relations but queries written to access joined tables, and – and there does not seem to be a good word here – graphs of related classes.

What I’d like to point out here – I am wrapping this up for now, but will try to flesh this out more soon – is the important mapping is not happening at the individual class or table level. Date make an error in characterizing the mapping of the relational world to the object world with a the mapping from a single class to a relation. That’s obviously not a rich enough example. And the people who have actually designed and used the tools of OR mapping know what’s right.

So, Mr. Date, here we are. The First Great Blunder is not a blunder at all. It is a partial truth. And your preferred mapping is but another piece of the same truth. You asked one of the key questions, but you were simply wearing the First Great Blinders and couldn’t see the arena where the full truth plays out. To a hammer everything is a nail.

A class can map to a data type. A class can map to a table. Your tuple – a row of a table – can map to an object. An instance of table data – a resultset, your relation – can map to a set of objects, generally in some container managing access to them.

But in the larger arena, these are the fundamental mappings:

A resultset (Date’s relation) maps to an object graph.
A query definition (Date’s relational variable) maps to the metadata description of an object graph – let’s coin an expression and call it a class graph.
A row of a resultset (Date’s tuple) maps to a set of individual objects in an object graph connected by pointers.

The fundamental importance of the object graph has become more and more clear over time. It has emerged at the heart of Service Data Objects, which are object graphs that can be disconnected from the mothership, modified, and phoned home.

More on object graphs soon.

And what exactly do knowledge workers do?  That’s easy.  They make decisions in the area of their expertise.  They are immersed in a situation made intelligible by their experience and training, and apply their knowledge and reason to make decisions which result in actions.

A user sitting in front of a software screen can only select things.  (We can take care of edge cases by describing writing text as selecting one text out of all possible texts, and by describing drawing with a paint program as selecting one picture out of all possible pix, etc.)

Each selection represents a decision – check the box, set that radio button, select this item from a combo box – and the little decisions are rolled up into bigger ones: save, start, do.

There is a range of complexity and sophistication of decision making, and a wide range of the type and depth of expertise a user has to bring to the arena.   At the low end, there are binary decisions in simple spaces – open a file? ok.  At the high end, there are sophisticated tools that can be used to help analyze complex situations using queuing theory, game theory, probablility, simulation, and other formal techniques.  More about the nature of decisions in a future rumination.

So what is the true nature of a use case?  A use case is a decision context.  It describes an expert engaged in a decision scenario.  The overall goal of the use case is reached by a series of such decisions.  The expert needs to be immersed in the environment where the decision is possible.  They have to be provided with the information, the research, on which to base their decision.  And they need to be presented clear and correct selections that correspond to qualifying and making the decision.

As always, more to come.

There are a number of useful ways of structuring requirements. We’ll touch on one intially, then augment this over time.

One very useful categorization is by the frequency of change or possible change.

The laws of physics, our understanding of how the physical world works, are fundamental requirements for all projects, even if frequently not articulated.   If there was a big bang, there may have been some change near the bang, but it is safe to assume the laws of physics will be stable for the lifetime of one of our software projects.  Sometimes our understanding of them can change, though  – if x number of objects of a given size fit in a particular box, that is unlikely to change, but the invention of a new lattice packing algorithm might do the trick.  Or making all the chips the same size and shape so they stack in a tennis ball can.

Federal and state laws are generally slow to change.   But they do, and if our project is potentially impacted by such changes during its lifespace, we should take the possibility of change into account.

Contracts with our business partners change at the end of the contract period – or sometimes sooner if there is a dispute.

Our technical infrastructure changes.  New technologies are introduced and adopted.

Business objectives change much more frequently, driven by changes in the market or  changes in leadership.

Categorize the requirements by their frequency of possible change.  Qualify that with the period of the change and its nature.  Find logical gaps in the resulting picture and fill them.  Then account for the temporal volatility of requirements – you are designing for the lifespan of your system, not just for the snapshot of it when it rolls out.