Examples that break the model

Break the model is a process to identify a representative set of examples. The process involves creating a model through an evolutionary process of describing examples that otherwise would not fit in the model and effectively break it.

There are two types of example that break the model:

1. Examples that are different but have the same description in the model.

2. Examples that do not fit in the model.

To explore this concept, lets consider the inventory system for a comic book shop (like Stewie’s in big bang theory)

Our model for the description of a comic book consists of:

a. The “Title” ( e.g. Amazing Spiderman )

b. The issue Number ( e.g. “1” )

c. The price.

Examples that are different but have the same description in the model.

Consider the two photo’s below. They have the same description according to the model. Are they the same comic?

The answer is no. We discover that the condition of the comic is important. This is actually an extreme example. The comic on the left sold for $104,200. The one on the right sold for $2,750. These examples would break the model. We would need to include a condition indicator ( or Grade ) on the comic description. The model helps us compare examples. It allows us to sort the examples into piles. We can then compare the examples in the piles to see if any of the examples are different. The comparison of the examples often needs the expertise of the subject matter expert. The subject matter expert can tell the difference between two examples that a lay person may not be able to detect. The subject matter expert also knows where the bodies are buried. The SME knows the examples that are likely to be overlooked. They are aware of those examples that look the same but need to be treated differently. To illustrate this point, compare the two comic books below.

The one on the left is the same one with a grade of 9.4 and sold for $104,200. The one on the right is ever so slightly poorer condition with a grade of 9.2, and sold for $57,523. Only a trained expert can tell the difference. A difference worth $50,000. As fields develop, they become more sophisticated. The rare comic book industry now compares comics of the same grade. Two 9.4 comics might be judged as having “white pages” or “off-white pages” which can also have an impact on the price.

Another important point is that the differences we find between the examples are orthogonal to the model. They are things we never considered.

Examples that do not fit in the model.

Some examples do not fit in the model at all. These we have a tendency to ignore. In software systems, we would develop a manual or spreadsheet work around outside of the system. Typically this involves shoe-horning the example into the system and then managing the differences off-line. Once again, our subject matter experts will be able to tell us where the bodies are buried. From the systems perspective, it is blind to the examples that do not fit into its model.

As we are developing the model, we may explicitly exclude examples that do not fit the model. This can be risky as we may exclude examples that should be included. It is better to include them in the learning model but exclude them from the implementation model. This way we can better understand the implications of changes to the requirements.

Non-Software Models.

Not all models exist in software systems. Some exist in people (cognitive models) and some exist in markets. Making our cognitive models explicit can help us spot the differences for things we consider the same. The interesting examples are the ones that do not fit our cognitive models. The examples that have no value to us. How many of you have walked past a shop / bar / building for months or years and only notice it when you need what it sells, or a friend points it out. How often have you said “I did not realise you were here!”.

Not only does this apply to shops and the like, it also applies to people. Who is that world renowned expert who sits two desks away, but in a different team or department? The examples that make it through our filters of perception (model) have much to do with the value we ascribe the examples. As we receive information, or learn more (our filters change), we change the value we ascribe to things.