Skip to main content

Chapter 28
Entropy as Information Divergence

Thermal and Entropic Flows
Author: Mark Anthony Higgins
© Mark Anthony Higgins 11 October 2025
Revised Date: 23 February 2026
Audio Overview
Previous Chapter 27
Next Chapter 29
Flow Algebra 
flow "Entropy as Information Divergence" v1.0

# ---------- PURPOSE ----------
# Expresses entropy as a measure of information divergence — the mismatch between
# the system's actual microstate distribution and its coherent reference distribution.
# Links thermodynamic entropy (σ, S) to informational coherence via Kullback–Leibler form.

# ---------- PARAMETERS ----------
params:
  units: "SI"
  k_B: 1.380649e-23          # Boltzmann constant [J/K]
  T:   positive               # system temperature [K]
  IB:  0.0..1.0               # boundary coherence index
  reference_state: "equilibrium"   # target distribution type (eq, coherent, etc.)
  symbolic: true              # enables symbolic use of p, q distributions

# ---------- STATE ----------
state System:
  p_i[]: prob                 # actual microstate probabilities
  q_i[]: prob                 # coherent reference probabilities
  S_phys: real                # thermodynamic entropy [J/K]
  D_info: real                # informational divergence [dimensionless]
  sigma: real                 # entropy production rate [J/(K·s)]

# ---------- DEFINITIONS ----------
invariants:
  # Shannon / Gibbs entropy
  S_phys = - params.k_B * Σ_i( p_i * ln(p_i) )

  # Kullback–Leibler divergence (information distance)
  D_info = Σ_i( p_i * ln(p_i / q_i) )

  # Link between thermodynamic and information forms
  S_rel = - params.k_B * D_info   # relative entropy = -k_B * divergence

  # Entropy production rate in coherence form
  sigma = params.k_B * Σ_i( dp_i/dt * ln(p_i / q_i) )
  sigma >= 0

  # Coherence link
  params.IB = exp(-D_info)   # high divergence => low coherence

# ---------- TRANSFORMS ----------
transforms:
  coherence_entropy_relation:
    S_phys = S_ref + (1 - params.IB) * ΔS_max
    # ΔS_max: maximal entropy difference between coherent and fully random states

  info_dynamics:
    # Rate of coherence loss as information divergence
    ddt(D_info) = Σ_i( dp_i/dt * (1 + ln(p_i / q_i)) )
    sigma = params.k_B * T * ddt(D_info)

  reversible_limit (when params.IB ≈ 1):
    D_info → 0
    sigma → 0
    note: "Perfect coherence — no divergence between p and q distributions."

# ---------- MONITORS ----------
monitors:
  positivity:
    assert all(p_i >= 0)
    assert abs(Σ_i(p_i) - 1) < 1e-9
  entropy_increase:
    assert sigma >= 0
  coherence_bound:
    assert 0 <= params.IB <= 1

# ---------- SCENES ----------
scenes:
  coherent_equilibrium:
    given:
      p_i: [0.25, 0.25, 0.25, 0.25]
      q_i: [0.25, 0.25, 0.25, 0.25]
      T: 300
      IB: 1.0
    expect:
      D_info = 0
      sigma = 0
      S_phys = 1.386 * params.k_B
      comment: "Perfect symmetry — no information divergence."

  decohered_system:
    given:
      p_i: [0.1, 0.2, 0.4, 0.3]
      q_i: [0.25, 0.25, 0.25, 0.25]
      T: 300
    compute:
      D_info = Σ_i( p_i ln(p_i / q_i) )
      IB = exp(-D_info)
      sigma = params.k_B * T * ddt(D_info)
    expect:
      D_info > 0
      IB < 1
      sigma > 0
      comment: "Information asymmetry manifests as entropy production."

# ---------- NOTES ----------
notes:
  - Entropy is the geometric distance between the actual system state (p) and its coherent baseline (q).
  - In HST terms, this is a *boundary divergence*: a phase-space curvature mismatch between expected and realised fields.
  - Coherence (IB) is the exponential of the negative information divergence, linking structure and thermodynamics.
  - When p = q, coherence is complete (IB = 1, D_info = 0); as p diverges, σ rises, reflecting information loss.
  - This formalism unifies physical and informational entropy within the same algebraic flow framework.
Overview

In classical physics, entropy is treated as a measure of disorder or uncertainty.
In Flow Algebra and Holographic Space-Time, entropy instead becomes a measure of coherence deviation — the distance between what the system is and what it could be if perfectly aligned.
It quantifies information divergence: the geometric gap between the system’s actual probability distribution pip_i and its coherent reference distribution qiq_i.

Core Equations

The classical and information-theoretic forms are unified as:

$$ S_{\mathrm{phys}} = -k_B \sum_i p_i \ln p_i, \qquad D_{\mathrm{info}} = \sum_i p_i \ln \frac{p_i}{q_i}, \qquad S_{\mathrm{rel}} = -k_B D_{\mathrm{info}}. $$

Here SphysS_{\mathrm{phys}} measures the absolute entropy of the system,
DinfoD_{\mathrm{info}} measures its relative divergence from coherence,
and SrelS_{\mathrm{rel}} converts that divergence into an entropy offset.
When pi=qip_i = q_i, the two align perfectly — no information is lost, no entropy is produced.

Entropy Production as Divergence Growth

The rate of entropy generation follows directly from the time derivative of divergence:

$$ \sigma = k_B\, T\, \frac{dD_{\mathrm{info}}}{dt} \ge 0. $$

Entropy production is therefore the temporal slope of information loss —
how fast the system’s microstate probabilities drift away from their coherent baseline.

Coherence Mapping

In this framework, the boundary coherence index IBI_B becomes an exponential measure of alignment:

$$ I_B = e^{-D_{\mathrm{info}}}. $$

Thus:

  • \( I_B = 1 \;\Rightarrow\; \text{perfect coherence} \;\Rightarrow\; D_{\mathrm{info}} = 0. \)
  • \( I_B < 1 \;\Rightarrow\; \text{partial coherence} \;\Rightarrow\; \text{measurable divergence} \;\Rightarrow\; \text{entropy generation.} \)
  • \( I_B \to 0 \;\Rightarrow\; \text{total decoherence} \;\Rightarrow\; \text{pure noise distribution.} \)

The connection between information and thermodynamics becomes exact:

$$ S_{\mathrm{phys}} = S_{\mathrm{ref}} + (1 - I_B)\, \Delta S_{\max}. $$

linking coherence loss to an entropy increase within the system.

Interpretation in HST Terms

In Holographic Space-Time, entropy arises not from disorder but from boundary misalignment —
the angular divergence between two field maps in phase space.
Each microstate carries a small vector of curvature; entropy accumulates as these curvatures lose collective alignment.
Information divergence DinfoD_{\mathrm{info}} is therefore the phase-space angle between expected and realised geometries.
As coherence decays, the boundary forgets its internal state — information becomes curvature noise.

Conceptual Translation
DomainVariableInterpretation
Thermodynamics
S,σ
Energy-weighted
information divergence
Information theory
\( D_{\mathrm{info}} \)
Statistical distance
between p and q
Coherence theory
\( I_B \)
Alignment index \( = e^{-D_{\mathrm{info}}} \)
Geometry (HST)
curvature drift
Spatial representation of divergence

Entropy is thus the shadow of coherence loss — not a mystery of randomness, but the visible trace of structural deviation from perfect alignment.

Coherence–Entropy Pair

\( I_B = e^{-D_{\mathrm{info}}}, \quad \sigma = k_B\, T\, \dfrac{dD_{\mathrm{info}}}{dt}. \)

These two equations complete the loop:
entropy production is simply information divergence evolving through time.

Summary

Entropy is not randomness itself — it is the measure of how far a system has drifted from perfect self-knowledge.

In Flow Algebra, that drift becomes quantifiable geometry.
When Dinfo=0D_{\mathrm{info}} = 0, the system remembers itself completely; when Dinfo>0D_{\mathrm{info}} > 0, it begins to forget.
Entropy, in this sense, is memory loss measured in energy units — the universal signature of coherence decay across physics, computation, and consciousness.

Previous Chapter 27
Next Chapter 29
Related Chapters
Privacy Policy
Site Map
Kireth Research
38 Thorndale Croft, Wetwang, Driffield
East Riding of Yorkshire YO25 9XZ
Telephone:
01377 236594
Designed By
Kireth Web Design