An article from University of Oslo
New mathematical logic could have averted the attack on Saddam
A completely new type of mathematical logic has the potential to improve intelligence services worldwide. The US Army has already expressed keen interest.
Imagine that you are the head of US military intelligence services immediately prior to the invasion of Iraq in 2003. You have a wealth of intelligence to help you figure out whether Saddam Hussein really has weapons of mass destruction or not. You have access to satellite photos and huge amounts of information from spies and defectors.
Each of these sources is fraught with some level of uncertainty. Some pieces of evidence are more reliable than others.
The amount of information is so enormous that nobody can get a complete overview. You need computational tools to interpret all the information.
One of your hypotheses is that Saddam has weapons of mass destruction. The other hypothesis says the opposite. It is your job to determine which hypothesis appears to be the most correct.
The head of the US intelligence services got it wrong. He erroneously determined that Saddam had access to weapons of mass destruction.
"Current intelligence analyses are often based on information that is subject to a considerable degree of uncertainty. Intelligence analysts are constantly struggling with the reliability of circumstantial evidence. The sources may be unreliable or directly misleading. When intelligence services in one country attempt to find out what another country is planning to do, they need to take into account the credibility of the information," says Audun Jøsang, professor at the Department of Informatics, University of Oslo.
The fact that the information tends to be incomplete and the circumstantial evidence often is contradictory does not make things any easier.
"It is therefore essential to assess all information, evidence, facts and circumstances in a way that reflects this situation," says the researcher.
The new Logic
Jøsang has developed a completely new type of mathematical logic that may improve the ability of intelligence services to deal with unclear evidence while identifying intelligence areas that merit further investigation.
This new form of intelligence analysis is based on subjective logic. This type of logic can explicitly handle degrees of uncertainty, and allows analysts to create intelligence analysis models that are far more realistic than those produced by current interpretation methods.
In current intelligence analysis models, all circumstantial evidence must be weighted with a specific probability. Jøsang says this is not enough. Instead, he says, the models need to include an estimate of the certainty of this probability.
"Most people are unaccustomed to the fact that a probability in itself may be uncertain," he says.
Certainty
To understand the importance of certainty, assume that an intelligence agent estimates the probability of two different events to be one half. Even though the probability is the same, their certainty and uncertainty may be quite different.
If you flip a coin, the probability of heads is one half. There is a very high certainty that this probability estimate is correct. Unless the coin is biased or manipulated, the certainty is one hundred per cent.
The probability that Oswald shot Kennedy in 1962 may also be estimated as one half, but this probability is fraught with a great deal of uncertainty. Even though the probability is one half here as well, it is highly uncertain and the probability estimate might be totally wrong.
Quantifying uncertainty
Intelligence analysts struggle with these kinds of problems all the time. They need to be able to take uncertainty into account, but they don't currently have the tools to do intelligence analyses that consider the degree of uncertainty in each probability estimate.
With the aid of subjective logic, however, the certainty and uncertainty that are inherent in all probability estimates can be quantified.
"Unless this is done, the analysis sweeps uncertainty under the carpet. We humans are stuck in our preconceived notions and always stick to the beaten track. We are unable to see things objectively. If the Americans had used this new mathematical logic, they would have seen that the uncertainty as to whether or not Saddam had access to weapons of mass destruction was too large," Jøsang said.
To express the uncertainty of their probability estimates, statisticians use a tool called the confidence interval. A confidence interval describes the likelihood that an event will occur with a probability within a certain interval. However, there are no tools that can simply handle confidence intervals as input arguments in complex models. As a rule, confidence intervals are used only to present results.
Today, all input arguments in intelligence models must be entered with a specific probability, even if this figure is uncertain.
"What they really ought to say is 'we don’t know'. However, these input arguments are not permitted in traditional analytical tools. With subjective logic, the input arguments may be completely uncertain and estimates can be made with these probabilities, even though they are fraught with uncertainty," Jøsang said.
Mathematical repertoire
The certainty and uncertainty of each probability can be represented by triangles, where the horizontal line describes the magnitude of the probability and the vertical height represents the uncertainty of the probability estimate.
In order to describe situations, Jøsang is now working to expand the repertoire of mathematical operators, such as deduction and abduction, and how these mathematical operators can be combined.
By emphasizing the uncertainty of the input arguments, decision makers can visualize the degree of uncertainty of their analyses in order to make better decisions. If the result is fraught with large uncertainty, the individual making the decison will be reluctant to make significant, important decisions, but might instead call for more intelligence work.
"When we implement this logic, we can see the aspects of the theory that are incomplete and need to be straightened out. If the Americans had had access to this tool, perhaps they would have found that there was too much uncertainty with regard to the hypothesis that Saddam had weapons of mass destruction before taking such a momentous decision to invade Iraq. Unless the uncertainty of the circumstantial evidence is taken into account, the analysis tool may erroneously conclude that there was a clear probability that Saddam had access to weapons of mass destruction," says Jøsang.
The gist of the matter is:
"In order to balance all circumstantial evidence it is crucial to describe the uncertainty of each individual piece of information. If this is not done, there is a risk that evidence with a large degree of uncertainty is compared to evidence that is basically certain. One should be cautious in taking large, momentous decisions if the results are fraught with a large degree of uncertainty."
Professor Jøsang emphasizes that his theory is sufficiently developed to be adapted to analytical tools.
Intelligence services: Applicable
The Norwegian military intelligence service has expressed an interest in this new, mathematical logic, but the intelligence officer who had familiarized himself with the theory did not want to comment on the tool to Apollon.
The intelligence officer referred Apollon to Dean Tore Pedersen, who is head of intelligence studies at the Norwegian Intelligence College. He made it clear to Apollon that his statement was of a general, academic nature.
"Subjective logic can be used to explore complex problems that include a large element of subjectivity and uncertainty," Pedersen wrote to Apollon in an email.
The US Army is interested
American intelligence services are currently using an analytical framework called ACH. The US Army Research Lab, the American equivalent of the Norwegian Defence Research Establishment at Kjeller, supports this research at UiO with NOK 2 million to explore how subjective logic can be implemented in their intelligence analyses.
"By incorporating uncertainty, subjective logic has the potential to revolutionize automated probability reasoning and improve intelligence operations. The method may enable the decision maker to realize when the responses are too uncertain and that more information needs to be collected. We still need answers to a number of fundamental questions. The US Army Research Lab is therefore collaborating with Professor Jøsang through the project 'Advanced Belief Reasoning in Intelligence' to determine whether and how his idea can be realized," says Lance Kaplan in the Networked Sensing & Fusion Branch, US Army Research Laboratory.
