Training Batch Ontology

FLOW ALGEBRA v1.0 — TRAINING BATCH ONTOLOGY
Semantic Library | AI Core | Entity Graph | Training Process Entities
Source: https://kirethresearch.org/booksai-core/entity-graph/ai-entities/training-process-entities/training-batch-ontology.html
Attribution: Mark Anthony Higgins — Sky Recursive AI
Status: Topic-based formalisation; source page content not independently retrieved.
Date: 2026-10-29

01. PURPOSE
Represent a training batch as a finite, versioned collection of training examples selected for one model-update step or one gradient-accumulation contribution. Preserve sample identity, preprocessing provenance, weighting, ordering, and masking.

02. STATE MODEL
S_t = (I_t, R_t, B_t, K_t)
I_t = batch data, sample identifiers, labels/targets, masks and weights
R_t = sampling, collation, validation, forward/backward and update rules
B_t = batch-size, memory, privacy, precision and schema constraints
K_t = model parameters, optimiser state, random state and data provenance
E_t = expand(I_t, R_t)
V_t = {x in E_t | satisfies(x, B_t)}
x*_t = argmax_{x in V_t} P(x | I_t, R_t, B_t, K_t)
I_{t+1} = update(I_t, x*_t, feedback_t)
Note: This is an ontology-level selection abstraction, not an assertion that ordinary batch sampling computes argmax probabilities.

03. ENTITIES
TrainingBatch: identified collection of examples used in a training iteration
TrainingExample: one input-target pair or unsupervised training item
Dataset: source population of examples
Sampler: selection and ordering policy
BatchSampler: groups sample indices into batches
Collator: converts examples into model-ready tensors
InputTensor: encoded model input
TargetTensor: expected output when applicable
AttentionMask: marks valid input positions
SampleWeight: per-example contribution coefficient
BatchSize: count of examples in a batch
Microbatch: memory-bounded subdivision of a logical batch
GradientAccumulator: combines microbatch contributions
DataLoader: retrieves and prepares batches
TrainingStep: consumes batch contributions and may update parameters
Epoch: traversal convention over a dataset
RandomSeed: reproducibility input
BatchManifest: sample IDs, schema, transforms, hashes and provenance

04. RELATIONSHIPS
Dataset --contains--> TrainingExample
Sampler --selects--> TrainingExample
BatchSampler --groups--> TrainingBatch
Collator --transforms--> TrainingBatch
TrainingBatch --contains--> InputTensor
TrainingBatch --mayContain--> TargetTensor
TrainingBatch --mayContain--> AttentionMask
TrainingBatch --mayContain--> SampleWeight
TrainingBatch --belongsTo--> TrainingRun
TrainingBatch --contributesTo--> TrainingStep
Microbatch --partOf--> TrainingBatch
BatchManifest --documents--> TrainingBatch

05. FORMAL MATHEMATICS
D = {z_i}_{i=1}^N
B_t = {z_i : i in J_t}, J_t subseteq {1,...,N}
|B_t| = b_t, where b_t >= 1
L_B(theta) = [sum_{i in J_t} w_i * ell(theta; z_i)] / [sum_{i in J_t} w_i]
Require w_i >= 0 and sum_{i in J_t} w_i > 0.
g_B(theta) = grad_theta L_B(theta)
theta_{t+1} = OptimiserStep(theta_t, g_B, optimiser_state_t)
For equal weights, L_B = (1/b_t) sum_{i in J_t} ell(theta; z_i).
For accumulated microbatches M_j partitioning B_t, combine loss sums and total weights before normalisation; do not average unequal microbatch means without weighting.
Effective examples per optimiser step = sum of actual microbatch sizes across accumulation, subject to distributed duplication and sampling policy.

06. BATCH LIFECYCLE
1. Define dataset version and training split.
2. Choose sampler, seed, ordering and replacement policy.
3. Select example IDs for the current batch.
4. Retrieve examples and apply documented transforms.
5. Collate tensors; pad variable-length data and create masks.
6. Validate dimensions, data types, finite values and provenance.
7. Split into microbatches if memory constraints require.
8. Compute loss contributions and backpropagate.
9. Aggregate gradients correctly across devices and microbatches.
10. Apply optimiser update when accumulation boundary is reached.
11. Record manifest, metrics, step ID and any errors.

07. INVARIANTS
INV-01: Every example references an authorised dataset version.
INV-02: Batch sample count matches manifest count.
INV-03: Input, target and mask shapes satisfy the model schema.
INV-04: Padding is excluded from loss where required.
INV-05: Weighted-loss denominator is positive.
INV-06: Training and evaluation partitions respect leakage controls.
INV-07: Optimiser step is distinguished from microbatch iteration.
INV-08: Random state and sampler policy are recorded for reproducibility.
INV-09: Missing, corrupt or non-finite inputs trigger an explicit policy.
INV-10: Distributed duplicate samples are accounted for in weighting.

08. INFERENCE RULES
IF batch has targets THEN supervised loss may be computed.
IF batch lacks targets THEN use a compatible self-supervised or unsupervised objective.
IF memory limit exceeded THEN reduce microbatch size or apply compatible accumulation.
IF variable sequence lengths THEN collate with padding and appropriate masks.
IF accumulation count > 1 THEN defer optimiser step until intended boundary.
IF last batch smaller than nominal THEN use actual sample/weight count.
IF batch schema invalid THEN reject or quarantine according to policy.
IF loss is non-finite THEN skip or halt and record diagnostic evidence.

09. MINI-RECURSION ENGINE
state = (dataset_cursor, sampler_state, rng_state, batch_manifest, model_state, optimiser_state)
repeat while training_budget_remaining:
  indices = sample_next(sampler_state)
  batch = collate(load(indices))
  assert validate(batch)
  contributions = forward_backward(batch)
  accumulate(contributions)
  if update_boundary:
    gradients = normalise_by_effective_weight(accumulator)
    model_state, optimiser_state = optimiser_step(gradients)
    reset(accumulator)
  record(batch_manifest, diagnostics, state)
  advance(dataset_cursor, sampler_state)
stop when budget exhausted or stopping condition triggered

10. TRI-MAPPING
Entity: TrainingBatch
Structure: ordered or unordered collection of examples with tensors and metadata
Process: sample -> load -> collate -> validate -> train -> record
Semantics: bounded unit of evidence contributing to parameter optimisation

11. MAPPING TABLE
Dataset -> source population -> provenance boundary
Sampler -> selection rule -> coverage and stochasticity
TrainingExample -> atomic data item -> learning evidence
TrainingBatch -> grouped examples -> training-step input
Collator -> tensor constructor -> schema conformity
Microbatch -> batch subdivision -> memory management
GradientAccumulator -> gradient combination -> update semantics
BatchManifest -> recorded metadata -> traceability

12. EXECUTION GUARANTEE (CONDITIONAL)
Given valid input schemas, deterministic operations where supported, controlled random states, and successful computation, the pipeline produces a traceable batch and correctly defined loss/gradient contributions. This does not guarantee convergence, accuracy, bitwise reproducibility across hardware, or freedom from bias.

13. AI SEARCH FLOW
QUERY: What is a training batch?
ANSWER: A finite group of training examples processed together to calculate model loss and gradient contributions.
QUERY: How does batch size differ from microbatch size?
ANSWER: Batch size describes the logical group; microbatch size describes a subdivision processed at once, often to fit memory.
QUERY: Does every batch cause an optimiser update?
ANSWER: No. Gradient accumulation may combine multiple microbatches before one optimiser step.
QUERY: Why store batch provenance?
ANSWER: To support auditing, debugging, reproducibility and detection of data leakage.

14. SEMANTIC EQUIVALENCE
Training batch ~= minibatch when used for a subset of training examples.
Training batch != entire dataset unless full-batch training is explicitly used.
Training batch != optimiser step when gradient accumulation is active.
Batch size != token count for variable-length sequences.

15. UPDATE MODEL
On changes to sampling, dataset, transforms, collator, masks, weighting or accumulation: increment ontology/pipeline version, record change rationale, and revalidate affected invariants.

END FLOW ALGEBRA