Craft, not Manufacturing

The book “The Pragmatic Programmer” has as its subtitle “from journeyman to master” and on the cover a picture of a hand plane.  (The book itself is highly recommended).  The point is quietly made: good software development is more akin to craftwork than to manufacturing.  

Software developers are smart people, they are employed for their ability to create and engage with the logics that lie within our systems, and good software developers are at their best when they are being creative, and can exercise creativity.

This means:

• Developers who engage with what they are doing take on a feeling of personal ownership, of responsibility. They don't need to be held to timesheets or working hours, but will make their work part of their lives, their self-identity.
• Developers will want to take chances to look a little further than just the immediate problems they are dealing with. This allows opportunities for refactoring (which improves design and may increase the capabilities of the software), and for sweeping up of related problems, etc.
• Developers should be recognised for the work they do, they way they do it, and the unique contributions they make.
• Software development becomes enjoyable, and (speaking personally, but I think this is generally true) what this means is doing work that is useful, clever and efficient: “neat” in all senses of the word
• The developer who is proud of their work will spread the word. If they have just extended the capabilities of a service in some way, they will encourage its use and re-use.
• Likewise, working in a happy shop (and this may be a virtual workplace) will inspire and stimulate innovation, creativity and quality.
• Reduced staff turnover. If our working environment is highly rewarding, and great fun, and in particular if it allows us to do good work that makes us feel good about ourselves, who would look elsewhere?

Everything Changes ... but some things change faster than others

Some Requirements are Enduring - Others are Volatile

Enduring Requirements are Coded, Volatile Requirements are supported as Configuration Models

When dealing with requirements and change the "best practice approach" could be summarised as:

• Gather current requirements
• Establish known points of variation
• Investigate likely future requirements
• Build a highly parameterised application to cope with this anticipated level of change

This is all well and good: difficulties arise when:

• Unanticipated change occurs (often driven by regulatory change)
• Anticipated change does NOT occur (the system is left with unused and confusing configuration mechanisms)
• There is an attempt to extend the scope of the existing system to areas not covered by initial analysis (new lines of business, new territories)

Very often, despite the best efforts of talented developers, systems will not have adequately been prepared for the change that comes. Chaos ensues.

How can we do things differently? We can take a very wide definition of which requirements should be regarded as volatile, and plan to support this in a structured way:

• Anything driven by regulation is volatile
• Anything driven by “products” is volatile
• Anything driven by markets and competition is volatile
• Anything unique to the line of business, market sector, or territory should be regarded as volatile.

We can then analyse these volatile requirements and capture their variation in open-ended and structurally sophisticated configuration.  We call this process modelling, and the configuration definitions thus obtained are models.  Every effort goes into making modelling easy, so that as far as is possible, model change becomes a non-programming skill.

Enduring requirements are those aspects which are unlikely to change over a period of decades. “In a contract, we need to know who the contracting parties are”, “We will need to keep track of financial movements” are examples of enduring requirements.  Enduring requirements are supported by software applications that we call engines.

Engines provide raw, generic capability.

Models shape that capability into purpose-specific services.

When volatile requirements change, as we expect them to do, these changes are managed without change to the engines, purely as model change.

This is not to say that engines do not change: they do. But engines change only to extend generic capability.  This means that engine change is exclusively a positive thing. This is hugely important. It means that as time progresses, we expect the engines to improve: extended capabilities, better performance, more flexibility in deployment.

Engines + Models = Services

THINK

Meetings, Planning, and Corporate Process Can’t Build Better Systems

IBM used to make extensive use of the simple catchphrase “THINK”.  

Wikipedia : At an uninspiring sales meeting Watson interrupted, saying The trouble with everyone of us is that we don't think enough. We don't get paid for working with our feet — we get paid for working with our heads. Watson then wrote THINK on the easel.

Too often we act as if process is a substitute for thinking.   The response to a tricky problem is often to call a meeting.  While discussion is essential, the best discussions are preceded by and followed by periods of thought.  

This blog will strive to redress this balance. The posts here will represent principles, attitudes and practices that are the result of experience, followed by analysis, reflection, thoughtful design, implementation and testing.

No claims are made that these will all be final answers, or beyond the scope of change: after all, these are "Working Propositions". I will write about things that on balance, on the basis of experience and THINKing, seem to work.

Building Enterprise Systems that Last, that Improve With Age

I come from a background of financial services and specifically insurance systems.  It is a world of frightening levels of compromise and muddle.  Once (50 or so years ago) it was this industry that led computerisation. Now after generations of mismanaged solutions, bewildering requirements change, and growth in technological possibility it is struggling to even achieve stability in a changing world.     It is a world where every decade or so, a new layer of technology gets overlaid on what went before, a layer that typically is out of date before it is fully implemented.

The plight of the industry is made even more apparent when compared to the astonishing rush of new consumer computer technologies that put the equivalent of a supercomputer in everyone's phone or tablet, which has capabilities that Science Fiction writers would have scarcely dreamed of a couple of decades ago.

The purpose of this blog is to explore why enterprise systems in financial services companies is in such a poor state, and to offer some alternatives.

Let's do a thought experiment.  Cast your mind ahead 20, 30, even 50 years ahead.  What will the financial services architecture of the mid 21st Century look like?

On a bad day I can imagine it looks a lot like today's, just with more layers of failed solutions, new languages, hugely faster computers running the same bodged set of applications, messing things up more thoroughly than ever before.

On a good day I can believe that today's mess is a sign of immaturity:  that 50 years is an awfully short time to progress from Assembler through COBOL and PL/1 to Java to {insert whizzy language of choice here}. We have been learning, all this time.  Maybe it would be useful to see what we have learned.  Maybe it would be useful to think about what system design / architecture principles we can articulate that we can believe will still be valid in 50 years time.

As computerisation / automation / data processing / information technology reaches maturity, I can hope that we can stop reinventing the wheel.  This is not to say that the pace of change will slow: rather, as the ground firms beneath our feet we can hope to run faster and faster.

This blog will focus on the principles, attitudes and practices that I think stand a chance at the 50-year survival criterion.  The lessons that we think will stand the test of time.  Some of them will be wrong: that's why I call the blog "Working Propositions".  True for now, but not guaranteed for all time. Still, solid enough to build on.