Backpropagation Entity Ontology

FLOW ALGEBRA v1.0 — BACKPROPAGATION ENTITY ONTOLOGY
SOURCE: https://kirethresearch.org/booksai-core/entity-graph/ai-entities/training-process-entities/backpropagation-entity-ontology.html
AUTHORS: Mark Anthony Higgins — Sky Recursive AI
STATUS: Topic-based formalisation; source page content not independently verified.
LANGUAGE: en-GB

1. PURPOSE
Formalise backpropagation as a computational entity and a reverse-mode automatic differentiation process for propagating loss sensitivities through a differentiable computational graph.

2. STATE MODEL
S_t = (I_t, R_t, B_t, K_t)
I_t = input activations, targets, graph topology and parameter values
R_t = forward operations, differentiation rules and optimiser rules
B_t = graph connectivity, tensor shapes, numerical precision and gradient constraints
K_t = stored activations, adjoints, gradients and training metadata
E_t = expand(I_t, R_t)
V_t = {x in E_t | satisfies(x, B_t)}
x*_t = argmax_{x in V_t} P(x | I_t, R_t, B_t, K_t)
I_(t+1) = update(I_t, x*_t)
NOTE: The probabilistic selection expression is a general Flow Algebra abstraction, not a necessary step of ordinary backpropagation.

3. ENTITIES
BackpropagationProcess: reverse traversal of a differentiable graph
ComputationalGraph: directed acyclic graph of tensor operations for one forward evaluation
Node: variable or differentiable operation
Parameter: trainable tensor theta
Activation: intermediate forward value
Loss: scalar objective L
LocalJacobian: derivative of node output with respect to node input
Adjoint: derivative of scalar loss with respect to an intermediate value
Gradient: derivative of loss with respect to parameter
GradientAccumulator: sum of contributions from multiple downstream paths
Optimiser: separate entity consuming gradients to update parameters

4. RELATIONSHIPS
ComputationalGraph contains Node
Node consumes Activation and produces Activation
Loss depends_on Output
BackpropagationProcess traverses ComputationalGraph in reverse topological order
LocalJacobian transforms downstream Adjoint into upstream Adjoint
GradientAccumulator combines contributions at shared ancestors
Optimiser consumes Gradient; optimiser update is distinct from backpropagation

5. FORMAL MATHEMATICS
Forward: a_0 = x; z_l = W_l a_(l-1) + b_l; a_l = phi_l(z_l)
Objective: L = ell(a_L, y)
Chain rule: dL/dx = (d f/dx)^T (dL/d f) for column-vector gradients
Output adjoint: delta_L = (d ell/d a_L) ⊙ phi'_L(z_L)
Hidden adjoint: delta_l = (W_(l+1)^T delta_(l+1)) ⊙ phi'_l(z_l)
Weight gradient: dL/dW_l = delta_l a_(l-1)^T
Bias gradient: dL/db_l = delta_l
Branch aggregation: adjoint(v) = sum over consumers u of J_(u,v)^T adjoint(u)
Example optimiser step (SGD): theta_(t+1) = theta_t - eta * grad_theta L
For non-smooth operators, implementations may use defined subgradients or other differentiation conventions.

6. PATHWAYS
P1: validate graph, inputs, targets, shapes and differentiability assumptions
P2: evaluate forward operations and cache required intermediates
P3: compute scalar loss
P4: seed loss adjoint (usually 1)
P5: traverse nodes in reverse topological order
P6: apply local vector-Jacobian products
P7: accumulate gradients for shared ancestors and parameters
P8: check finite values, shapes and optional gradient clipping
P9: hand gradients to optimiser; optionally apply parameter update
P10: log metrics and repeat for the next training batch

7. MINI-RECURSION ENGINE
function BACKPROP(node, upstream_adjoint):
  for each parent in node.parents:
    contribution = VJP(node.operation, parent, upstream_adjoint)
    parent.adjoint += contribution
  return accumulated parent adjoints

Execution on a DAG should use reverse topological scheduling or equivalent dependency-aware accumulation to avoid incorrect duplicate processing.

8. INFERENCE RULES
IF node is disconnected from loss THEN its loss gradient is zero (or absent in implementation).
IF multiple loss-dependent paths reach a node THEN sum their gradient contributions.
IF a parameter is frozen THEN do not apply an optimiser update to that parameter.
IF gradient is NaN or infinite THEN flag the step and invoke numerical safeguards.
IF a local operation is not differentiable THEN use a supported differentiation rule or report unsupported gradient.
IF gradient is computed THEN it does not imply the parameter has already changed.

9. INVARIANTS
Gradient tensors conform to parameter shapes.
Gradient accumulation respects graph dependency order.
Forward and backward refer to consistent parameter values for the evaluated pass.
Loss gradient is computed with the same reduction convention used in the objective.
Parameter update is not part of the mathematical definition of the backward pass.

10. TRI-MAPPING
ENTITY: BackpropagationProcess -> graph nodes, parameters, activations and adjoints
RELATION: depends_on, differentiates_through, accumulates_into, updates_via
BEHAVIOUR: forward evaluation -> reverse differentiation -> gradient accumulation -> optional optimisation

11. MAPPING TABLE
Loss -> scalar seed -> initiates reverse traversal
Activation -> cached tensor -> enables local derivative
LocalJacobian -> derivative operator -> propagates sensitivity
Adjoint -> reverse sensitivity -> combines downstream influences
Gradient -> parameter sensitivity -> input to optimiser
Optimiser -> update rule -> modifies trainable parameters

12. CONDITIONAL EXECUTION GUARANTEE
Given a finite acyclic graph with valid tensor shapes, supported derivative rules, consistent forward values and numerically valid arithmetic, reverse-mode differentiation computes the chain-rule gradient of the specified scalar objective (subject to the differentiation conventions and floating-point error). This is not a guarantee of convergence, accuracy or training success.

13. AI SEARCH FLOW
QUERY: What is backpropagation?
MATCH: BackpropagationProcess
EXPAND: ComputationalGraph, Loss, LocalJacobian, Adjoint, Gradient, Optimiser
FILTER: distinguish gradient computation from optimiser updates
ANSWER: Backpropagation computes gradients of a loss with respect to upstream variables and model parameters by applying the chain rule backwards through a computational graph.
CONFIDENCE: conceptual formalisation; verify implementation-specific details against the target framework.

14. CITATION
Source: https://kirethresearch.org/booksai-core/entity-graph/ai-entities/training-process-entities/backpropagation-entity-ontology.html
Attribution: Mark Anthony Higgins — Sky Recursive AI