The eighth commandment forbids misrepresenting the truth in our relations with others. This moral prescription flows from the vocation of the holy people to bear witness to their God who is the truth and wills the truth. Offenses against the truth express by word or deed a refusal to commit oneself to moral uprightness: they are fundamental infidelities to God and, in this sense, they undermine the foundations of the covenant. (Catechism 2464)

Table of Contents

A Tragic Tale

Consider, if you will, the following timeline of events:

Is this just some hypothetical far-fetched tale?

No! Unfortunately is not a far-fetched tale at all. It is a tragic tale! The events detailed are actual events (summarized, of course) in the life - and death - of Sally Clark.

Sally Clark was a young professional lawyer who along with her husband, Steve, lived in England. Their story, in particular, Sally’s, is well known not only in England, but around the world. If a woman murdering one infant child catches everyone’s attention, then the same woman murdering two infant children is certainly foder for sellable news. But, what really makes the Sally Clark case renowned is that the rationale by which she was convicted was not only wrong, but totally unavoidable. Her life was ruined, not only by the deaths of her two children, but by her ordeal. She never fully recovered. She died of alcohol poisoning at the age of 42.

So the questions abound. Was this a miscarriage of justice? Why was she convicted in the first place - or even arrested for that matter? Why was her appeal denied? What caused her ultimate conviction to be overturned?

There is a lot to the details of the Sally Clark case and there are any number of resources available which detail the case. The area of focus here will be two gross errors, which when combined were basically the ultimate nail in her coffin, so to speak, that caused her guilty verdict, incarceration, and ultimately resulted in her death. Indeed, her case is cited in many judicial writings and has brought forefront the issue of the Prosecutor’s Fallacy.

top

1 in 73 Million

Though there were a lot factors and (agruably non-conclusive) evidence presented at Sally Clark’s trial, the testimony given by Professor Roy Meadow took front and center. Professor Meadow was a pediatrican who seemed to focus his career on child abuse, often to the absurd. He was called by the prosecution as an expert witness.

His testimony was that his data showed that the chance of an infant dying of SIDS to a family like the Clark’s was approximately one in 8500 (a family like the Clark’s meant that they did not have any of the three main risk factors for SIDS: a smoker in the family, non-affluence and the mother under the age of 26). Let’s call the event, a child dying of SIDS, event A, so \(P(A) = \frac{1}{8500}\). Meadow then testified that the chance of a second infant dying of SIDS would be the same as the first. Let’s call this event, event B, so \(P(B) = \frac{1}{8500}\) His conclusion, then, that the probability of the first child dying of SIDS and the second child dying of SIDS was

\[ P(A \text{ and } B) = \left(\frac{1}{8500}\right)\left(\frac{1}{8500}\right) \approx \frac{1}{73,000,000}. \]

In other words, Meadow testified that there was a one in 73 million chance that two infants from the same family like the Clark’s could die of SIDS (or perhaps more specifically, unexplanable reasons). This was the first gross error made.

Even assuming the number of 1 in 8500 was correct (the overall number was more like 1 in 1300) for the chance of the first infant dying of SIDS, the multiplication of these two probabilities is down right wrong. In order for the probabilities of two events to be multiplied together, the events must be independent. That is, the probability of one event occurring has no effect on the ocurrance of the other. If two events are not independent, their individual probabilites can not be multiplied. It is only under the conditon of independence, and only then, that one conclude

\[ P(A \text{ and } B) = P(A)P(B). \]

The 1 in 73 million chance, according to Meadow, equates to a double infant SIDS death about once a century (based on birth rates in the UK). But,the two events are not independent. Besides any technical or medical reasoning, common sense dictates that the assumption of independence here is not justified. There could be genetic factors or other family factors (perhaps not yet known) that may affect the probability of SIDS. Numerous studies on SIDS deaths support this (even on out in the UK at the time). Yet, Professor Meadow presented his testimony of 1 in 73 million and to the best of my knowledge has never retracted that testimony.

top

Prosecutor’s Fallacy

The gross error of multiplying two non-independent probabilities was compounded by the prosecutor. It was inferred (or perhaps outright stated) that this 1 in 73 million chance was, in effect, saying that there was a 1 in 73 million chance Sally Clark was innocent! In other words, there was no way that Sally Clark could be innocent. She must be guilty. The jury agreed, hence the guilty verdict.

This was the second gross error. The Prosecutor’s Fallacy!

In essence, the prosecutor’s fallacy is the lack of a clear understanding of conditional probabilities (and good statistical reasoning). Whether this is intentional or not is not known nor is it being implied. That said, prosecutors do their best to win their case (that is, prove beyond a reasonable doubt that the defendent is guilty).

So, what does this lack of a clear understanding of conditional probabilities mean? For two events, \(I\) and \(E\), the conditional probability of \(I\) given \(E\) is the probability that the event \(I\) occurs given that the event \(E\) has already occurred. The notation for conditional probability is: \(P(I|E)\) (the probability of \(I\) given \(E\)). It is unreasonable to conclude that \(P(I|E) = P(E|I)\). Yet, too often that conclusion is made. In general, the Prosecutor’s Fallacy is the belief that \(P(I|E) = P(E|I)\).

In the Sally Clark case, the testimony that there was a 1 in 73 million chance that two infants from the same family like the Clark’s could die of SIDS (again, wrong) was the probability that given innocence, could match the evidence - that is \(P(E|I)\). The prosecutor took this to imply that this was the same as the probabilty that the defendent, in this case, Sally Clark, was innocent given the matched evidence - that is \(P(I|E) = P(E|I)\). The prosecutor’s fallacy.

What the prosector was implying was that the chance of an apparently extreme rare event (the death of two infants from the same family like the Clark’s could die of SIDS) is exactly the same as the chance that Sally Clark was innocent. Since 1 in 73 million is an extremely small number, then it is virtually impossible that she is innocent, therefore she must be guilty! The chance actually was not 1 in 73 million, but more likely around 1 in 100,000. Though still small, it is significantly higher than 1 in 73 million. Regardless, in a large population, such as the UK, it is certainly possible that rare events can, and do, occur. So, with no other incriminating evidence available, is the mother who fell victim to this chance occurance, this tragedy, a murderer?

And, therein lies the rub. Other incriminating evidece was basically nonexistent. Indeed, in the absence of the fact that two of Sally Clark’s infants died suddenly and unexpectantly, each death would have would not have been suspicious in-and-of-themselves. Thus the 1 in 73 million chance argument was the one which weighed heavily on the jury, especially given that the prosecutor was equating this statistic as being equal to the chance of Sally Clark being innocent. Thus she was convicted and sentenced to two life terms.

top

Epilogue

While Sally Clark languished in prison, work was done to appeal her conviction, citing the 1 in 73 million statistic as a major reason. The appeal failed. Of interest is the Court of Appeal, in upholding her conviction, wrote: “The point on statistics was of minimal significance - a sideshow - and there is no possibility of the jury having been misled.”

Fortunately, in partnership with Steve Clark, lawyers succeeded in obtaining the medical examination records of the two boys, which were never made available to the defense. It was discovered that Sally’s second son had a blood infection which certainly could have caused his death. Eventually, armed with this new evidence and the statistical blunders, an appeal was successful. Sally Clark’s conviction was overturned and she was set free in late January, 2003.

Sally Clark was later diagnosed as having an “enduring personality change after catastophic experience”. She succumbed to alcohol on March, 16, 2007.

Roy Meadow was found guilty by the British General Medical Council (GMC) in 2005 of professional misconduct as a direct consequence to his misuse of statistics. While he was recognized by the GMC as a qualified doctor, “he should not have strayed into areas that were not within his remit of expertise”. Meadow was removed from the medical registry. In 2006, Meadow’s eventual appeal was successful and was reinstated.

Roy Meadow was an expert witness in several other cases of note. Angela Cannings lost three of her babies to SIDS and was sentenced to life in prison after being found guilty of murder. Her conviction was later overturned. Trupti Patel also lost three of her babies to SIDS but fortunately for her Meadow’s chastement became quite public after Sally Clark’s conviction was overturned and was acquitted of murder.

Roy Meadow has always maintained having done nothing wrong and to best of knowledge has never apologized to the innocent women (and their families).

The Sally Case has brought attention to the use, or mis-use, of statistics and probabilities in courts of law. In the years since, work has been done to address this issue. As usual, a lot of good has come from bad, thanks be to God!

Sally Clark (1964-2007) and Husband, Steve

May God have mercy on your soul. RIP

(Photo: Copyright © 2003, BMJ Publishing Group Ltd)

The above accounts of the Sally Clark case are my summarization and based on my understanding.

top

Post Mortem

A post mortem is a term I’ve used in my career to refer to time taken after the conclusion of a project or activity, to reflect on lessons learned, what could have been done differently and, perhaps most important, what is the path forward. So, having concluded the above summarization, here are my reflections.

Obviously two gross errors as described above were made. First, Roy Meadow’s multiplication of two probabilities which were not independent (thus producing the outrageous chance of 1 in 73 million). And, second, the prosecutor’s fallacy - the equating of the chance of a rare occurence with guilt.

Given the lack of incriminating evidence, was even the right question put before the court? Instead of asking whether this absurd 1 in 73 million chance implies guilt, perhaps the question should have been which was more likely, the death of Sally Clark’s sons by natural causes or by murder? Said another way, what is the likelihood, again given the lack of incriminating evidence, that murder was actually committed?

This is the main take-away. Always be sure to be asking the right questions and to always seek the truth.

Given my passion for Bayesian analysis (I am not an expert for sure, but have been engaged in study and will continue to do so and will write more in the future) it seemed like the use of Bayes’ Theorem is need to help address the question of what is the likelihood that murder was actually committed. In short, Bayes’ Theorem allows for the separation of the likelihood of an event happening at all from the likelihood of other explanations of an event. In other words, given that Sally Clark’s two boys died and their death was unexpected and sudden, what is the probability that they died of SIDS versus having died due to some other reason - murder. These are conditional probabilities and this is the power behind Bayes’ Theorem. Had this been done (and using more correct data) it would have been determined that this probability was (approximately) two-thirds. Yes, two-thirds. That is, the probability that the cause of death was SIDS given their unexplained deaths was two-thirds. Sally Clark, again in lieu of any other incriminating evidence (which there wasn’t) should never have been tried in the first place.

Please see my companion instructional article, Bayes’ Theorem - An Introduction, on Bayes’ Theorem where, in additon to providing the technical details of the theorem, a worked example of the Prosector’s Fallacy is given. At the risk of boring the reader, however, to see how the (approximate) two-thirds was achieved and Bayes’ Theorem in action, will present a quick summary (the numbers used are the most accurate estimates):

Given the hypothesis, \(H\), that the two children died of SIDS (died suddenly and unexpectantly) and data, \(D\), both died suddenly and expectantly. Then:

Apply Bayes’ Theorem:

\[ \begin{align} P(H|D) &= \frac{P(D|H)P(H)}{P(D|H)P(H) + P(D|\text{~}H)P(\text{~}H)} \\ \\ &= \frac{0.000007692}{0.000007692 + 0.000004615(0.999992308)} \\ \\ &= \frac{0.000007692}{0.000012308} \\ \\ &= 0.625001803 \\ \\ &\approx \frac{2}{3} \end{align} \]

Again, this is the probability that the cause of death was SIDS given their unexplained deaths. Had the right question been asked Sally Clark would still be alive today.

top

Resources

There is a wealth of commentary on the Sally Clark case. Two particular resources are worth mentioning:

top

Seeing God

As is normally case, the finished product is nothing like it was envisioned at the start. Let me explain.

I am fascinated by Bayes’ Theorem and have been engaged in deep study for a while and will continue to be (I admit I am not an expert on Bayesian analysis though am committed to further study and will present subsequent posts, for sure). I am fascinated by inferential statistics, especially Bayesian inference where one can start with some subjective belief and as new data or observations are obtained update the prior belief to a posterior belief. But above all, I am fascinated by truth - God’s truth - and constantly seek a deeper understanding of, appreciation for, and living of God’s truth. Yes, even in mathematics, God’s truth is revealed, at least to the extent we seek it and to the extent that God reveals it. The same applies for statistics and how God’s truth is revealed to us, especially in the probabilistic nature of inferential statistics.

Many months ago, as it happened, I stumbled across an article from Nature.com entitled Conviction By Numbers, by Mark Buchanan. The subtitle is what really caught my eye: Statistics have the power to trip everyone up — including judges and juries. I was intrigued and after reading the short two page article, I was hooked! (See Conviction By Numbers for the article. It apparently was written after Sally CLark’s conviction was overturned but just before she died.)

My thought processes were churning and the phrase prosecutor’s fallacy stuck in my brain. I was vaguely familiar with the Sally Clark case but certainly was not intimate with it. After doing some initial research I was struck, not only by the tragedy of the events, but by how the erroroneous use of statistics by someone, or someones, played such a key role in the outcome. How a life was ruined. It got me thinking about how often this can, might, or has happened. How many men and women have been, are, or will be incarcerated due to these erroroneous uses, or misunderstandings of statistics. Or worse! How many men or women were put to death as a consequence.

So, my mission became to research this idea of prosecutor’s fallacy (where a prosecutor equates the chance of a rare event occuring to the chance of innocence) in general, and with respect to the Sally Clark case in particular. The idea was to glean insight into how the misuse (or misunderstanding) of statistics can, in some cases, life or death consequences.

While doing my research it soon became clear to me that Bayesian analysis and the prosecutor’s fallacy can be combined. How wonderful!

Once I began really inserting myself into the Sally Clark case, the name Roy Meadow kept rising to the forefront. Now, please understand, I am in no way making any assertions on Mr. Meadow, his character, his motivations, etc. I just had a gut reaction to the power an alleged expert witness has and how an ill-informed jury, or even a prosecutor, can be misled.

And, that brought me to the Eighth Commandment: “Thou Shalt Not Bear False Witness”. And that brought me to where I am now. This was not even in my consciousness at the start of this endeavour.

The Eight Commandment is more than just perhaps telling a lie. According to the Catechism (reference the Catechism of the Catholic Church, Section Two: The Ten Commandments, Chapter 2: You Should Love Your Neighbor As Yourself, Article 8: The Eighth Commandment which can be found on the Vatican’s Website). The Eigth Commandment encompasses six components:

Paragraph 2476, under Offenses Against Truth, states: “False witness and perjury. When it is made publicly, a statement contrary to the truth takes on a particular gravity. In court it becomes false witness. When it is under oath, it is perjury. Acts such as these contribute to condemnation of the innocent, exoneration of the guilty, or the increased punishment of the accused. They gravely compromise the exercise of justice and the fairness of judicial decisions”.

A statement contrary to the truth jumped out to me. Intentional lying certainly would fit this description. If one knows an event happened (it rained yesterday), but speaks to the contrary (“No, it did not rain yesterday”) then that is a deliberate making of a statement contrary to the truth. If in a court of law, where a witness is giving testimony under oath, then this raises to the level of perjury. A clear-cut case of bearing false witness.

But what about an expert witness? In most cases, I suspect, an expert witness is giving testimony to their perception of the truth given their education, experience, etc. Assuming for the moment, that this expert witness does not outright lie, how can any statement made be taken as the truth? Isn’t it a subjective truth based on the expertness of the witness? That is one purpose of a jury, I suppose, to filter out what is the truth - to decide the credibility of the testimony made by the expert witness. But even that is quite subjective if the members of the jury are not as well informed of the subject matter as perhaps is required.

This is not a commentary on our judicial system (which, arguably is the best in the world) or a jury of alleged peers. What this is a commentary on is how it is the duty of an expert witness to not bear false witness. That is, to freely and openly admit their limitations in knowledge of the truth. This to me is becoming even more important as our technical knowledge is expanding at a seemingly exponential rate. Is it a reasonable expectation for a jury, again, of alleged peers to keep up with this tremendous growth rate? Or even a prosecutor?

The Sally Clark case brought this to light for me. The expert witness gave testimony that was blatantly wrong. Neither the jury, judge, prosecutor or even apparently, the defense was able to spot the error. The prosecutor then compounded that false testimony by making a gross error of his own. The assumption here is that neither the expert witness or the prosecutor deliberately chose to make these errors (that would be quite serious). Rather, the point is shouldn’t the expert witness have readily admitted that the testimony given was not as expert as it was being protrayed to be? Likewise for the prosecutor? Or was pride involved by the expert witness? Or was the prosecutor more concerned about winning the case than seeking the truth? Who knows.

These are questions that I have been pondering, especially as they pertain to the Eighth Commandment. Bearing false witness against the truth. Which truth? To me, it is God’s truth.

Even though the topic here is probabilistic in nature, there is still God’s truth. God has blessed us - over time, of course - to further our understanding of His creation, and that includes inferential statistics. Given our increasing knowledge we are able to use inferential statistics (probability) to make what we believe to be well informed decisions. To decide the effectiveness of drug therapies. The list can go on. Yet, we must not forget that these decisions are probabilistic. What does that mean? It means that just because something is likely (or not likely) to happen that does not mean it will (or won’t) happen. Said another way, it means that based on our level of understanding it seems likely that it will happen (or won’t) so we should make our decision based on that understanding, fully recognizing that God is control!

Just because there is a 1 in a 73 million chance of something happening does not mean that it will not happen. It just might if it is God’s will. Even though something might seem “statistically” improbable, one must remember that, as scripture says NOTHING IS IMPOSSIBLE WITH GOD! God’s truth has been revealed to us (in His time and in His manner) and probability is one of those truths to help us make sense of our world, to help guide us. Yet, we must always remember that a probability of zero is not part of God’s truth, His will! There is no such thing as \(P(A) = 0\) with God!

We must not discount truth based on a probabilistic improbability, for, again, nothing is impossible with God. And, we must not apply our preconceived truth - or understanding - haphazardly, especially where there is human life, and the consequences, thereof, involved. Nor should we allow our intellectual pride stand in the way of seeking truth.

It is our duty to constanly seek the truth, God’s truth, to continue to grow in our knowledge and education, but it is also our duty to understand our limitations and to recognize God is in control. Nothing we can say or do is absolute in our terms. though something might seem quite improbable to us is not with God. This is one reason I’m fascinated by Bayesian inference which is based on the prinicple of starting with a belief and applying new data (observations) to update that belief - a means of constantly seeking the truth.

When making statements as an expert witness, when assessing the validity of statements as a juror, especially where human life is involved, we must never bear false witness. But, remember, we do not have to be in a court of law to act as an expert witness or juror. Do we not assume these roles in our every day lives? Are we humble enough to understand (admit) our limited knowledge? Do we laud our perceived superior intellect over others? Do we totally discount someone’s circumstances as being too improbable, therefore disbelieve them? Do we speak or act in absolute certainity, never considering we may be wrong and then how it might affect others? Do we think that just because something seems impossible, it has to be? Do we seek to rectify any wrongs caused by our failure to consider our limited knowledge? Do we seek the truth (Yes! Even in statistics)? Do we speak the truth in love and not to satisfy some self serving agenda? Do we bear false witness? Or do we remember:

Thou Shalt Not Bear False Witness

top