Business rule testing

I work in a field where we thrive on supporting professionals with the tasks they need to accomplish. We gather the information and complex logic that is needed to make decisions. Naturally I work a lot with business rules and have a lot of discussion with my customers to implement them and create an application to support the professionals in their work.

Due to my focus on execution of business rule and complex logic I regularly work together with companies specializing in rule management. Not rarely do these companies also market their own Business Rule Management System. The linked Wikipedia article also gives a—questionable—definitiion:

A BRMS includes, at minimum:

• A repository, allowing decision logic to be externalized from core application code;
• Tools, allowing both technical developers and business experts to define and manage decision logic;
• A runtime environment, allowing applications to invoke decision logic managed within the BRMS and execute it using a business rules engine.

Although I do not agree all these items should be included in a BRMS, this definition serves to emphasize my position on testing business rules. Depending on the project the 3 points above are supported by 0, 1 or more different systems. The reason to use multiple systems is (or should be) to use specialized system and use the strength of each system. However, if you—as a supplier of a BRMS—do not comply with the definition above, you should be clear about this and the impact on the testing strategy.

As noted before, I do not think you should comply completely with the definition: focus on your strength. However, I can’t agree with some propositions. Some suppliers market their BRMS as a one-stop-solution while the run-time environment is not available. And, depending on the goals of the project, this doesn’t even have to be problem. In the market I work in, governmental organizations, it is important to translate the law and regulation as close as possible to business rules. The law and regulations are—as for as of concern for the business being modelled—transformed one-to-one supporting tracing and auditing of the resulting rules to their source.

Since in governmental projects business rules often don’t need discovery or creation, but are already stipulated in laws and regulations these are special kinds of projects. Projects where the testing needs to be adjusted, or can be absent completely (as long as the supplier of the BRMS fulfills the promise of a one-to-one transformation). I’ll expand on this below, taking the article Testing Rule-Based Systems — Protect Customer Focus as guideline for testing.

Verification

Verification is intended to check that a product, service, or system meets a set of design specifications.

The article starts with the position that the execution engine shouldn’t be tested. A position a heartily agree with: it is the responsibility of the execution engine supplier to test the execution engine and guarantee the correct working. Therefor the focus is on testing the operationalization of laws and regulations into business rules. Let’s start to review the verification methods given:

• Syntactical correctness: is to ensure the result is executable. A BRMS is responsible to provide a syntactical correct and provable (domain specific) language. As long as these conditions are meet, this may be an existing language, or a homegrown language. If a BRMS generates executable rules it should be the responsibility of the BRMS to ensure the input (the rule as described by the rule engineer) and the output (the executable rules) are syntactically correct and equivalent. If the law or regulation is interpretable in several ways or cannot be translated into the domain specific language, than there is a problem with the law and regulation;
• Decision model quality: is the duty of the rule engineer. He (she) should create quality decision models. However, considering the premise the rules are extracted one-to-one from the laws and regulations into business rules, one could argue it is up the writers of the laws and regulations to make sure these are readable and understandable (something I would welcome);
• Logical consistency: should be warranted by complying to the other checks given in this list. Except when explicitly circumvented by e.g. rolling a digital dice having the same outcome under the same circumstances and input should be a no-brainer.
• Ambiguity: should not occur. It should always be clear which rule of several to select—two or more rules should have no conflict with each other. This is one of the quality aspects a BRMS should be able to verify. Of course if a problem us found by the BRMS the problem is already present in the law or regulation.
• Completeness: ensures that for each situation there should be an outcome. Again if a problem is found—by the BRMS—in the completeness of the business rules, the problem is already in the law or regulation.
• Redundancy: to avoid applying the same condition at most once. Again something a BRMS should be able to detect, and a problem in the law and regulations.
• Logical loops: should be prevented at all costs, as the resulting application wouldn’t provide an answer in this case.

Did you notice a pattern in the above? All problems are traceable to the source—the law and regulation—being operationalized. Therefor if the laws and regulations are sound and complete /and/ the BRMS is equipped to fulfill the promise of a one-to-one translation to business rules there is no need to verify the operationalization. The BRMS can, however, ensure the laws and regulation are sound and complete, by performing the checks described above. A practice I applaud and in which these BRMS can make a difference—verifying the written laws and regulations.

Validation

Validation is intended to ensure a product, service, or system meets the operational needs of the various users and stakeholders.

This is the point where, in a law and regulation operationalization, I start to differ from the article regarding the needs:

• Compliance: is always adhered to if operationalizing laws and regulations. It is this compliancy itself that is being operationalized. Of course, it may happen, other laws or regulations should be complied to, but probably these should be operationalized too. However, compliancy is not only important during the creation of business rules. I’d argue they’re even more important during the execution of the business rules. Decisions should be explainable, justifiable and traceable to source. This should be included in the business rule execution engine and—with a one-to-one translation—is already warranted for.
• Operational validation: makes sure the decisions and rules are applicable in a practical manner. However, with the law and regulations already stipulated and a one-to-one translation to business rule we’re restrained in any changes we can make. This, again, is the responsibility of law and regulation makers. And this is actually correct, after all the judge has to apply them as well in a practical manner.

The validation step is important during testing, however the supplied checks should already be included in the law and regulation creation. It doesn’t make sense to create law and regulations, operationalize them, create an execution model and then start validating and start over again. This be done from the start. Therefor a BRMS used to operationalize existing laws and regulations (as seen a lot in my practice) shouldn’t perform these checks. If the BRMS is used to create new laws or regulations these checks should be performed. Unfortunately, using a BRMS to instantiate new laws or regulations doesn’t seem to be practice.

Simulation

Simulation is getting information about how something will behave without actually testing it in real life.

With the lead time of law changes simulation /after/ the law has passed is not fit. Simulation should be used right from the start. Therefor in projects where the current law is operationalized a simulation is not suitable.

Conclusion

Although all these testing activities provide value in creating or discovering new business rules and operationalizing them, in a project where the business rules are already written in a more or less formal language—like law and regulation—they provide little benefit. E.g. validation and simulation are /after the operationalization/ activities meant to adjust and optimize the business rules. However, as they are stipulated in law and regulations (and a one-to-one translation has been performed) these become /after the fact/ activities. One cannot change the business rules as this would result in either changing the laws and regulations, or not operationalizing the laws and regulations one-to-one truthfully.

I still advise to use the verification possibilities provided by the BRMS to ensure the provided requirements (laws and regulations) are sound and complete. However, I don’t agree with the position of some BRMS suppliers you need the execution engine to prove the correct working of the business rules in a laws and regulation project. There verification process and provide verification options should guarantee the correct translation from law and regulation to business rules.

Please note, above I’m talking about projects where the law and regulations are already stipulated. If you are lucky enough to be able to create the rules from scratch, please, by all means do use all the above described test facilities. However, if you don’t create the rules from scratch, it doesn’t make sense to use all techniques.

Addendum (24. January 2020)

It seems I’ve been not clear enough, I’m describing a theoretical, optimal transformation from law and regulations to business rules—a view some BRMS suppliers like to pretend is always the case. The above works, but only if the law is written in some kind of natural, literate programming language. At this moment, I haven’t seen a law or regulation written in such a way that this is the case and that semi-automatic transformation can be performed. Thus, yes, operationalizing is possible, but not with the guarantees some suppliers are pretend to give.