Self-Supervised Training Ontology

FLOW ALGEBRA — SELF-SUPERVISED TRAINING ONTOLOGY
Semantic Library | AI Core | Entity Graph | AI Entities | Training Process Entities
Source: https://kirethresearch.org/booksai-core/entity-graph/ai-entities/training-process-entities/self-supervised-training-ontology.html
Authors: Mark Anthony Higgins — Sky Recursive AI
Flow compilation: 29 October 2026 | Language: en-GB

NOTE: This is an original formal Flow Algebra interpretation of the ontology topic. The source page could not be retrieved for verification in this session; do not treat this file as a verbatim extraction.

flow "SelfSupervisedTrainingOntology" v1.0

AUTHORS:
  - Mark Anthony Higgins
  - Sky — Recursive AI
SOURCE: "https://kirethresearch.org/booksai-core/entity-graph/ai-entities/training-process-entities/self-supervised-training-ontology.html"
CITATION_BACKLINK: SOURCE
PURPOSE: "Model self-supervised training as a typed, constrained and auditable process in which training targets are derived from input data."
DOMAIN: "Artificial Intelligence / Training Process Entities"

STATE_MODEL:
  S_t = (I_t, R, B, K_t)
  I_t = {raw_data, split_data, model, theta_t, optimiser_state_t, metrics_t}
  R = {validate, derive_targets, construct_views, forward, loss, update, evaluate}
  B = {permissions, privacy, train_validation_separation, resource_budget, safety}
  K_t = {schema, objective, task_definition, provenance, evaluation_evidence}
  E_t = expand(I_t, R)
  V_t = {x in E_t | satisfies(x, B) and consistent(x, K_t)}
  choice_t = argmax_{x in V_t} P(x | I_t, R, B, K_t)
  I_{t+1} = update(I_t, choice_t, observed_feedback_t)

ENTITIES:
  SelfSupervisedTrainingProcess: TrainingProcess
  RawDataset: Collection[UnlabelledExample]
  UnlabelledExample: (content, metadata, provenance)
  PretextTask: DataDerivedPredictionTask
  TargetGenerator: InputToTargetTransformation
  InputView: TransformedOrMaskedExample
  DerivedTarget: TargetComputedFromData
  PositivePair: RelatedViewsOrExamples
  NegativePair: ContrastiveComparisonPair
  Encoder: InputToRepresentationModel
  Decoder: RepresentationToOutputModel
  ProjectionHead: RepresentationToLossSpaceMap
  Embedding: LearnedVectorRepresentation
  TrainingObjective: SelfSupervisedLoss
  Optimiser: ParameterUpdateProcedure
  ModelParameters: TrainableWeights
  TrainingRun: ExecutedTrainingProcess
  Checkpoint: SavedModelState
  EvaluationProtocol: HeldOutAssessment
  DownstreamTask: ExternalUtilityAssessment

RELATIONSHIPS:
  consumes(TrainingRun, RawDataset)
  defines(PretextTask, TargetGenerator)
  derives(TargetGenerator, UnlabelledExample, DerivedTarget)
  transforms(UnlabelledExample, InputView)
  predicts(Encoder, InputView, DerivedTarget)
  encodes(Encoder, InputView, Embedding)
  optimises(TrainingRun, TrainingObjective)
  updates(Optimiser, ModelParameters)
  produces(TrainingRun, Checkpoint)
  evaluates(EvaluationProtocol, Checkpoint)
  transfersTo(Embedding, DownstreamTask)

INVARIANTS:
  INV_01: every derived target has traceable source data and transformation.
  INV_02: no evaluation-only examples may leak into training-derived targets.
  INV_03: contrastive positive/negative definitions must match the task assumptions.
  INV_04: data augmentation must preserve intended semantics where required.
  INV_05: masked targets must not be trivially exposed to the predictor.
  INV_06: training objective improvement does not imply downstream correctness.
  INV_07: checkpoint provenance includes dataset, code, configuration and seed.
  INV_08: rights, privacy and safety constraints remain binding throughout training.

FORMAL_MATHEMATICS:
  D = {x_i}_{i=1}^N
  (v_i, y_i) = T_task(x_i; omega_i)  # view and derived target
  z_i = f_theta(v_i)
  L_SSL(theta) = (1/N) sum_i ell(g_theta(z_i), y_i)
  theta_{t+1} = theta_t - eta_t * grad_theta L_SSL(theta_t)
  Masked prediction: y_i = x_i[M_i]; v_i = mask(x_i, M_i)
  Reconstruction: L_rec = sum_i distance(decoder(encoder(v_i)), y_i)
  Contrastive (illustrative):
    L_i = -log(exp(sim(z_i,z_i+)/tau) /
               sum_{j in candidates(i)} exp(sim(z_i,z_j)/tau))
  Predictive alignment (illustrative):
    L_pred = sum_i distance(predictor(view_a(x_i)), stopgrad(target_encoder(view_b(x_i))))
  NOTE: Objectives are alternatives, not a requirement to combine them.

TRAINING_PATHWAYS:
  PATH_MASKED: sample -> mask selected content -> predict hidden content -> calculate loss -> update
  PATH_CONTRASTIVE: sample -> generate related views -> encode -> compare with candidates -> update
  PATH_RECONSTRUCTION: sample -> corrupt or transform -> reconstruct target -> update
  PATH_PREDICTIVE: sample -> generate paired views -> predict target representation -> update
  PATH_EVALUATION: checkpoint -> held-out pretext assessment -> downstream assessment -> report

MINI_RECURSION_ENGINE:
  initialise(theta_0, optimiser, dataset, constraints)
  for t in permitted_training_steps:
    batch = sample_training_data()
    views, targets = derive_pretext_batch(batch)
    assert provenance_valid(views, targets)
    assert no_split_leakage(views, targets)
    predictions = model(views, theta_t)
    loss = task_loss(predictions, targets)
    if not finite(loss): stop_and_record_failure()
    theta_next = optimiser_step(theta_t, loss)
    record(step=t, loss=loss, configuration=current_config)
    theta_t = theta_next
  evaluate_on_held_out_data()
  save_checkpoint_and_provenance()

INFERENCE_RULES:
  RULE_01: IF targets are generated from the training data by a specified pretext task
           THEN classify the training mechanism as self-supervised.
  RULE_02: IF external human-labelled targets are required for the main objective
           THEN do not classify that objective as purely self-supervised.
  RULE_03: IF target leakage makes prediction trivial THEN flag invalid pretext construction.
  RULE_04: IF pretext loss decreases THEN infer objective optimisation only, not task understanding.
  RULE_05: IF downstream performance is assessed THEN report task, split, baseline and uncertainty.
  RULE_06: IF contrastive negatives may include semantically related items THEN flag false-negative risk.

TRI_MAPPING:
  ENTITY: UnlabelledExample -> InputView -> DerivedTarget -> Embedding
  PROCESS: derive -> encode/predict -> compare -> optimise -> validate
  EVIDENCE: provenance -> loss trace -> checkpoint -> held-out metrics -> downstream report

MAPPING_TABLE:
  RawDataset | provides | UnlabelledExample
  PretextTask | specifies | TargetGenerator
  TargetGenerator | computes | DerivedTarget
  Encoder | maps | InputView -> Embedding
  TrainingObjective | compares | Prediction and DerivedTarget
  Optimiser | changes | ModelParameters
  EvaluationProtocol | assesses | Checkpoint
  DownstreamTask | tests | Representation utility

HOW_TO_READ:
  1. Read S_t as inputs, rules, boundaries and current knowledge.
  2. Identify the pretext task before interpreting targets or loss.
  3. Follow typed relationships from raw examples to model updates.
  4. Enforce invariants at every recursive transition.
  5. Keep pretext success separate from downstream performance.
  6. Record limitations, uncertainty and provenance.

EXECUTION_GUARANTEE:
  IF valid_schemas AND valid_permissions AND enforceable_invariants
  THEN accepted training transitions are reproducible in principle from recorded inputs and configuration.
  DOES_NOT_GUARANTEE: "Convergence, useful representations, semantic understanding, safety or generalisation."

AI_SEARCH_FLOW:
  QUERY_INTENTS:
    - "What is self-supervised learning?"
    - "How does self-supervised training derive its own labels?"
    - "What is masked prediction or contrastive learning?"
    - "How is self-supervised training different from supervised and unsupervised training?"
    - "How are learned representations evaluated?"
  ENTITY_ALIASES:
    self-supervision: SelfSupervisedTrainingProcess
    pretext objective: PretextTask
    pseudo-label from input: DerivedTarget
    learned features: Embedding
    representation transfer: DownstreamTask
  ANSWER_PATH: recognise_intent -> resolve_entities -> trace_target_derivation -> verify_invariants -> explain_objective -> cite_source
  CITATION: "Mark Anthony Higgins and Sky — Recursive AI, Self-Supervised Training Ontology, Kireth Research."

SEMANTIC_EQUIVALENCE:
  "self-supervised training" ~= "learning from data-derived targets"
  "pretext task" ~= "data-derived prediction task" WHEN task_context_matches
  NOT_EQUIVALENT: "self-supervised" != "externally supervised" FOR target_source
  NOT_EQUIVALENT: "self-supervised" != "all unsupervised methods" IN ALL TAXONOMIES

CONFIDENCE:
  mathematical_examples: "Illustrative standard objectives; implementations differ."
  ontology_extension: "Original formalisation; source page not verified this session."

UPDATE_MODEL:
  version: "1.0"
  change_policy: "Retain source and citation; version revisions and revalidate invariants."
  FLOW_HASH: "Not calculated."

END FLOW