AI Training Data Ontology

FLOW ALGEBRA — AI TRAINING DATA ONTOLOGY
Semantic Library | Training Process Entities
Source: https://kirethresearch.org/booksai-core/entity-graph/ai-entities/training-process-entities/ai-training-data-ontology.html
Authors (source page): Mark Anthony Higgins — Sky Recursive AI
Companion formalisation; embedded page specification not retrievable. Not a verbatim transcription.
Encoding: UTF-8 | Date: 2026-10-29

flow "AITrainingDataOntology" v1.0

PURPOSE:
  Represent AI training data as typed, provenance-aware entities and formal
  transitions from acquisition through validation, partitioning and use.

STATE_MODEL:
  S_t = (I_t, R_t, B_t, K_t)
  I_t = available data entities, metadata and observed training state
  R_t = admissible data transformations and relationships
  B_t = privacy, licensing, quality, leakage and resource boundaries
  K_t = provenance, schema, ontology and validation knowledge
  E_t = expand(I_t, R_t)
  V_t = {x in E_t | satisfies(x, B_t)}
  x*_t = argmax_{x in V_t} P(x | I_t, R_t, B_t, K_t)
  I_(t+1) = update(I_t, x*_t)

ENTITIES:
  Dataset(id, version, modality, task, licence, provenance)
  DataRecord(id, dataset_id, features, target?, metadata)
  Annotation(id, record_id, annotator, label, confidence)
  Source(id, origin, collection_time, permission)
  Partition(id, role in {train, validation, test}, member_ids)
  Transformation(id, operation, parameters, lineage)
  QualityAssessment(id, metrics, findings, status)
  TrainingRun(id, dataset_version, model_version, configuration)
  DataManifest(id, hashes, schema_version, partition_hashes)

RELATIONSHIPS:
  Dataset CONTAINS DataRecord
  DataRecord DERIVED_FROM Source
  Annotation LABELS DataRecord
  Partition SELECTS DataRecord
  Transformation PRODUCES DataRecord
  TrainingRun CONSUMES Partition(role=train)
  Evaluation CONSUMES Partition(role=validation or test)
  DataManifest IDENTIFIES Dataset(version)

FORMAL_OPERATIONS:
  acquire(source) -> records with collection provenance
  validate(records, schema, constraints) -> accepted, rejected, reasons
  deduplicate(records, equivalence_policy) -> unique_records, linkage_map
  partition(records, split_policy, group_constraints) -> train, val, test
  transform(records, operation, fitted_state) -> derived_records
  audit(records, policy) -> findings, admissibility
  snapshot(dataset, version) -> immutable manifest

INVARIANTS:
  unique(record.id) within each dataset version
  partition(train) INTERSECT partition(validation) = EMPTY
  partition(train) INTERSECT partition(test) = EMPTY
  partition(validation) INTERSECT partition(test) = EMPTY
  grouped or temporal splits preserve required separation constraints
  fit(preprocessing_parameters) uses training partition only
  validation/test targets never inform model fitting or feature selection
    except through explicitly documented evaluation/tuning procedures;
    final held-out test remains isolated until final evaluation
  all derived records preserve traceable lineage
  records without lawful/authorised use are excluded

MEASURES:
  N = count(records)
  coverage(c) = count(records satisfying c) / max(1, N)
  missingness(f) = count(records missing feature f) / max(1, N)
  label_agreement = agreements / max(1, comparable_annotations)
  class_frequency(y) = count(records with label y) / max(1, labelled_records)
  leakage_count = count(prohibited_cross_partition_links)
  quality_score = weighted_aggregate(validated_quality_metrics)

INFERENCE_RULES:
  IF schema_invalid(record) THEN quarantine(record)
  IF licence_prohibits_use(source, intended_use) THEN exclude(source)
  IF personal_data_requires_controls(record) THEN enforce_privacy_policy(record)
  IF duplicate_group_spans_partitions THEN flag_leakage
  IF transformation_is_fitted THEN fit_on_training_partition_only
  IF data_distribution_shift_detected THEN flag_reassessment
  IF manifest_hash_mismatch THEN reject_unverified_snapshot
  IF label_conflict THEN route_to_annotation_review
  IF insufficient_provenance THEN mark_unverified; do_not_claim_verified

MINI_RECURSION_ENGINE:
  initialise S_0 from dataset inventory, requirements and policies
  repeat until stable or iteration_limit:
    candidates = expand(I_t, R_t)
    admissible = filter(candidates, B_t)
    chosen = rank(admissible, K_t)
    I_(t+1) = update(I_t, chosen)
    record provenance, validation outcomes and state transition
  stop when no admissible improvement or a boundary is reached

TRI_MAPPING:
  SEMANTIC: training data, samples, labels, sources, partitions, provenance
  STRUCTURAL: typed entities, membership edges, manifests, constraints
  OPERATIONAL: acquire -> validate -> split -> transform -> train -> audit

HOW_TO_READ:
  Each entity is a typed node; each relationship is a directed constraint.
  Each operation transforms a validated state while preserving provenance.
  A failed invariant blocks the transition rather than silently repairing it.

DIAGRAM:
  Sources -> Acquisition -> Raw Records -> Validation -> Dataset Snapshot
                                            |
                                            v
                                Leakage-Aware Partitioning
                                   /       |        \
                                Train   Validation   Test
                                  |        |          |
                            Fit/Train   Tune      Final Evaluate
                                  \        |          /
                                   Provenance + Audit

AI_SEARCH_FLOW:
  QUERY -> INTENT(training-data concept) -> ENTITY_RESOLUTION
        -> RELATIONSHIP_EXPANSION -> POLICY_FILTER -> EVIDENCE_RANKING
        -> ANSWER_WITH_PROVENANCE
  confidence = calibrated_evidence_score, not an invented certainty
  semantic_equivalence requires compatible meaning, role and provenance

EXECUTION_GUARANTEE:
  Guarantees are conditional on supplied metadata and implemented checks.
  The flow does not guarantee dataset accuracy, absence of bias, legality,
  privacy compliance or model performance without external verification.
  On missing evidence, return UNKNOWN or NEEDS_REVIEW rather than fabricate.

UPDATE_MODEL:
  New source or changed records -> new dataset version and manifest.
  Preserve previous versions, transformation lineage and audit trail.

CITATION:
  Mark Anthony Higgins and Sky — Recursive AI,
  Kireth Research Semantic Library, AI Training Data Ontology.
  Source URL above.
END FLOW