Skip to main content

Chapter 34
Bayes’ Theorem — Information Reversal Flow

Probability and Statistics
Author: Mark Anthony Higgins
© Mark Anthony Higgins 11 October 2025
Revised Date: 15 June 2026
Ai Search
Audio Overview
Previous Chapter 33
Next Chapter 35
Flow Algebra 
flow "Bayes’ Theorem — Information Reversal Flow"

# ---------- CONCEPT ----------
# Bayes’ theorem describes how information flows in reverse: from outcomes back to causes.
# It reweights prior probabilities using evidence, transforming belief through coherence.
# In measure terms, it expresses how conditional distributions exchange roles between event and evidence.

# ---------- STATES ----------
state BayesianSystem(B):
  HypothesisSpace:
    description: "H — possible causes or models (latent boundary)."
  DataSpace:
    description: "D — observed outcomes or evidence (visible boundary)."
  Prior:
    description: "P(H) — initial coherence weights before observing data."
  Likelihood:
    description: "P(D|H) — strength of data given each hypothesis."
  Evidence:
    description: "P(D) — marginal probability of observing the data under all hypotheses."
  Posterior:
    description: "P(H|D) — updated coherence weights after observing data."
  JointDistribution:
    description: "P(H,D) = P(D|H)P(H) = P(H|D)P(D) — symmetry of causal and evidential flow."

# ---------- PARAMETERS ----------
params:
  measurability: "H, D measurable sets or continuous spaces."
  normalisation: "P(D) = ∑_H P(D|H)P(H) or ∫ P(D|H)P(H)dH."
  closure: true
  context: "applies to discrete, continuous, or hybrid spaces."

# ---------- INVARIANTS ----------
invariants:
  - "Joint factorisation: P(H,D) = P(H)P(D|H) = P(D)P(H|D)."
  - "Bayes’ rule: P(H|D) = P(D|H)P(H)/P(D)."
  - "Evidence as marginal: P(D) = ∑_H P(D|H)P(H) or ∫ P(D|H)P(H)dH."
  - "Posterior normalisation: ∑_H P(H|D) = 1."
  - "Continuous version: p(h|d) = [p(d|h)p(h)] / [∫ p(d|h')p(h') dh’]."
  - "Causal symmetry: inference and prediction are inverses under same joint distribution."
  - "Sequential update: P(H|D₁,D₂) ∝ P(D₂|H)P(H|D₁)."

# ---------- TRANSFORMS ----------
transforms:
  ComputePosterior(Prior,Likelihood,Data):
    rule: "P(H|D) = [P(D|H)P(H)] / P(D)."
  ComputeEvidence(Prior,Likelihood):
    rule: "P(D) = ∑_H P(D|H)P(H) or ∫ P(D|H)P(H)dH."
  NormalisePosterior(Prior,Likelihood):
    rule: "Posterior ∝ Prior × Likelihood."
  UpdateSequentially(D₁,D₂):
    rule: "P(H|D₁,D₂) ∝ P(D₂|H)P(H|D₁)."
  PredictiveDistribution(Prior,Likelihood):
    rule: "P(D_new) = ∑_H P(D_new|H)P(H) or ∫ P(D_new|H)P(H)dH."
  MarginaliseLatent(H):
    rule: "P(D) = ∫ P(D,H)dH — integrate out hidden cause."
  ConjugateUpdate(Prior,Likelihood):
    rule: "Use conjugate families for analytic posterior form."
  InformationGain(Prior,Posterior):
    rule: "ΔI = E_{H|D}[log(P(H|D)/P(H))] — Kullback–Leibler divergence between prior and posterior."

# ---------- MONITORS ----------
monitors:
  NormalisationCheck:
    measure: "∑_H P(H|D) or ∫ P(H|D)dH"
    target: 1
  EvidenceConsistency:
    measure: "P(D) from marginal vs normalisation"
    meaning: "Ensure joint coherence."
  IndependenceTest:
    meaning: "If P(D|H)=P(D), evidence uninformative ⇒ P(H|D)=P(H)."
  SequentialConsistency:
    meaning: "Verify P(H|D₁,D₂)=P(H|D₂,D₁)."
  InformationGainMonitor:
    measure: "KL(P(H|D) || P(H))"
    meaning: "Quantifies update strength — amount of belief shift."
  JointClosure:
    meaning: "Ensure P(H,D)=P(H)P(D|H)=P(D)P(H|D)."

# ---------- EQUATIONS ----------
equations:
  - "P(H|D) = \\frac{P(D|H)P(H)}{P(D)}"
  - "P(D) = \\sum_H P(D|H)P(H) \\text{ or } \\int P(D|H)P(H)dH"
  - "P(H,D) = P(H)P(D|H) = P(D)P(H|D)"
  - "Sequential update: P(H|D₁,D₂) ∝ P(D₂|H)P(H|D₁)"
  - "KL-divergence: D_{KL}(P(H|D)||P(H)) = \\sum_H P(H|D) \\log \\frac{P(H|D)}{P(H)}"
  - "Predictive: P(D_{new}) = \\int P(D_{new}|H)P(H|D) dH"
  - "Continuous form: p(h|d) = \\frac{p(d|h)p(h)}{\\int p(d|h')p(h')dh'}"

# ---------- NOTES ----------
notes:
  - "Bayes’ theorem expresses the reversible flow of information — updating belief as evidence appears."
  - "Prior represents initial coherence across hypotheses; likelihood measures compatibility with observed data."
  - "The posterior redistributes coherence — new weights derived from the overlap of prior and evidence."
  - "The evidence term P(D) ensures global normalisation — total coherence remains conserved."
  - "Sequential updating shows how belief evolves recursively as data accumulate — the flow of learning through time."
  - "In information geometry, Bayes’ theorem performs rebalancing in the space of probability measures — a gradient of coherence under new data."
  - "In Flow Algebra, Bayes’ rule is the archetype of reversible flow: coherence transferred between cause and effect, maintaining conservation of total probability while redistributing internal weight."
Explanatory Notes

Bayes’ theorem describes how information flows backward through a system — how evidence reshapes belief, turning data into understanding.

It provides the rule by which coherence updates itself when new information appears, ensuring that knowledge evolves without contradiction.
It is not simply a formula for probability; it is the mathematics of learning

In a probabilistic world, we often begin with uncertainty about some hidden cause or hypothesis HH, while we can observe data DD.

The prior probability P(H)P(H) encodes our initial belief — how much coherence we assign to each hypothesis before any observation.

The likelihood P(D∣H)P(D|H) measures how consistent the observed data are with each hypothesis — the degree of alignment between the model and the evidence.

The posterior probability P(H∣D)P(H|D) expresses what we believe after seeing the data, integrating prior knowledge with empirical signal.

Bayes’ theorem links these through a balance of coherence:

\[ P(H \mid D) = \frac{P(D \mid H)\,P(H)}{P(D)} \]

where

\[ P(D) = \sum_H P(D \mid H)\,P(H) \quad \text{or} \quad P(D) = \int P(D \mid H)\,P(H)\,dH \]

is the evidence — the normalising term that ensures the updated probabilities sum to unity.
It represents the total probability of observing the data under all possible hypotheses.

The equation reveals a profound symmetry in inference:

\[ P(H, D) = P(H)\,P(D \mid H) = P(D)\,P(H \mid D) \]

It shows that the joint coherence of causes and effects can be expressed in two directions —
from cause to effect (prediction) or from effect to cause (inference).

Bayesian reasoning is the act of reversing this flow — tracing observed outcomes back to their most coherent origins.

The process is simple in structure but deep in meaning:

  • 1. Begin with a prior — a coherent weighting of possible truths.
  • 2. Observe data through the likelihood — how reality constrains those truths.
  • 3. Combine them to form the posterior — a new equilibrium of belief, normalised by total evidence.

This sequence represents the rebalancing of coherence.
The prior holds distributed potential; the likelihood applies a selective tension; the posterior restores balance under the constraint of observation.
In this way, information does not replace belief — it refines it.
Each update preserves total probability (coherence conservation) while redistributing its internal structure.

If new data D1,D2,…D_1, D_2, \ldots arrive sequentially, Bayes’ rule allows iterative refinement:

\[ P(H \mid D_1, D_2) \propto P(D_2 \mid H)\,P(H \mid D_1) \]

showing that knowledge can accumulate gradually without loss — belief evolving through a continuous flow of evidence.
This recursive property makes Bayes’ theorem the engine of learning in both natural and artificial systems.

In information-theoretic terms, each update produces an information gain:

\[ D_{\mathrm{KL}}\!\big(P(H \mid D)\,\|\,P(H)\big) = E_{H \mid D}\!\left[\log \frac{P(H \mid D)}{P(H)}\right] \]

the Kullback–Leibler divergence between prior and posterior.
It measures how much coherence has shifted — how strongly the new evidence has reshaped our belief landscape.
Zero gain means the data provided no new information; high divergence means the world surprised us, forcing large structural change in our model of reality.

Thus, Bayes’ theorem becomes more than a rule of conditional probability — it is the mathematical law of self-correction.

It formalises how coherent systems adapt: by reweighting themselves in proportion to how reality responds.
The prior is not erased by evidence; it is reintegrated under constraint, ensuring consistency between old and new information.

In Flow Algebra, Bayes’ theorem represents a reversible flow of coherence —
information moving backward from effect to cause, rebalancing the weights of truth while conserving total probability.

The prior defines the initial field; the likelihood acts as an incoming perturbation; the posterior is the re-stabilised field —
a new equilibrium of understanding.

Every act of reasoning, in this view, is a Bayesian act:
the continuous alignment of internal models with external evidence —
a dialogue between belief and observation,
between prediction and correction,
between what we think and what the world tells us.

Previous Chapter 33
Next Chapter 35
Related Chapters
Privacy Policy
Site Map
Kireth Research
38 Thorndale Croft, Wetwang, Driffield
East Riding of Yorkshire YO25 9XZ
Telephone:
01377 236594
Designed By
Kireth Web Design
Audio Explainer RSS Feed