Fine-Tuning Process Ontology

FLOW ALGEBRA — FINE-TUNING PROCESS ONTOLOGY
Semantic Library | AI Core | Entity Graph | AI Entities | Training Process Entities
Source: https://kirethresearch.org/booksai-core/entity-graph/ai-entities/training-process-entities/fine-tuning-process-ontology.html
Status: Companion formalisation grounded in established ML concepts; not a verbatim extraction of the source page.
Encoding: UTF-8 | Locale: en-GB | Date: 2026-10-29

01. PURPOSE
Represent the entities, relationships, state transitions, checks and evaluation boundaries involved in adapting a pretrained model to a target task or domain.

02. FLOW DEFINITION
flow "FineTuningProcessOntology" v1.0
AUTHORS: Kireth Research; Sky — Recursive AI (companion formalisation)
SOURCE: URL above
CITATION_BACKLINK: URL above
PURPOSE: Traceable formal model of fine-tuning configurations, updates and outcomes.

03. STATE MODEL
S_t = (I_t, R_t, B_t, K_t)
I_t = model parameters, dataset batches, optimiser state, metrics and run metadata
R_t = transformation, update, validation and governance rules
B_t = compute, privacy, licensing, numerical and evaluation boundaries
K_t = pretrained knowledge, task specification, dataset provenance and prior checkpoints
E_t = expand(I_t, R_t)
V_t = {x in E_t | satisfies(x, B_t)}
x*_t = argmax_{x in V_t} P(x | I_t, R_t, B_t, K_t)
I_(t+1) = update(I_t, x*_t)
NOTE: The argmax is an abstract selection rule, not a claim that stochastic optimisation literally computes this argmax.

04. ENTITY TYPES
PretrainedModel: architecture, tokenizer, initial weights, model identifier
FineTuningRun: run ID, task, method, random seed, timestamps
TrainingDataset: examples, schema, provenance, permissions
ValidationDataset: held-out examples and evaluation protocol
TestDataset: independent final assessment, where available
TrainingExample: input, target, optional mask and weight
LossFunction: differentiable training objective
Optimiser: update algorithm and internal state
LearningRateSchedule: step-dependent learning rate
TrainableParameter: tensor permitted to update
FrozenParameter: tensor excluded from updates
Adapter: optional parameter-efficient update module
Checkpoint: model, optimiser and metadata snapshot
Metric: measured task performance, uncertainty and resource cost
SafetyEvaluation: task-relevant behavioural and risk checks

05. RELATIONSHIPS
FineTuningRun --initialises_from--> PretrainedModel
FineTuningRun --uses--> TrainingDataset
FineTuningRun --evaluates_on--> ValidationDataset
FineTuningRun --optimises--> LossFunction
Optimiser --updates--> TrainableParameter
FrozenParameter --remains_unchanged_by--> Optimiser
Adapter --modifies_effective_mapping_of--> PretrainedModel
Checkpoint --captures--> FineTuningRun state
TestDataset --supports--> final independent assessment

06. MATHEMATICAL CORE
Dataset D_train = {(x_i, y_i, w_i)} for i = 1,...,N
Prediction: yhat_i = f_theta(x_i)
Objective: L(theta) = (sum_i w_i * ell(f_theta(x_i), y_i)) / (sum_i w_i)
Mini-batch gradient: g_t = grad_theta L_B(theta_t)
SGD update on trainable subset T: theta_(t+1)[T] = theta_t[T] - eta_t * g_t[T]
Frozen subset F: theta_(t+1)[F] = theta_t[F]
Optional regularised objective: L_total = L_task + lambda * Omega(theta, theta_0)
For a low-rank adapter: W_effective = W_0 + (alpha/r) * B * A
A in R^(r x d_in); B in R^(d_out x r); W_0 frozen for LoRA-style tuning
NOTE: Actual update rules depend on optimiser, gradient scaling, accumulation and distributed configuration.

07. PROCESS PATHWAYS
P1: Verify model, tokenizer, dataset rights and target task
P2: Partition datasets and prevent entity/source leakage
P3: Select full fine-tuning, partial freezing or parameter-efficient tuning
P4: Define objective, hyperparameters, seeds and reproducibility controls
P5: Iterate batches: forward -> loss -> backward -> accumulate -> update
P6: Validate on held-out data and record metrics
P7: Save checkpoints and restore best eligible configuration
P8: Perform independent final evaluation and safety review
P9: Export versioned model artefact, lineage and limitations

08. MINI RECURSION ENGINE
state := initialise(pretrained_model, config, dataset)
for epoch in configured_epochs:
  for batch in training_batches:
    predictions := forward(state.model, batch.inputs)
    loss := objective(predictions, batch.targets, batch.mask)
    gradients := backward(loss, state.trainable_parameters)
    state := optimiser_step_if_due(state, gradients)
    assert finite(state.trainable_parameters)
  metrics := evaluate(state.model, validation_dataset)
  state := record_metrics_and_checkpoint(state, metrics)
  if early_stop_condition(metrics): break
return evaluate_and_package(state.best_eligible_checkpoint)

09. INFERENCE RULES
R1: If a parameter is frozen, its optimiser-driven update must be zero.
R2: If training and evaluation sets share leaked examples or derivatives, flag evaluation integrity.
R3: If validation performance deteriorates while training loss improves, flag possible overfitting.
R4: If gradients or parameters become non-finite, reject the update and investigate numerical stability.
R5: If a checkpoint omits optimiser state, exact training resumption is not guaranteed.
R6: If a model is selected using test results, the test is no longer an untouched final assessment.
R7: If data provenance or licence is unresolved, block distribution pending review.
R8: If a parameter-efficient adapter is used, record both base-model and adapter versions.

10. TRI-MAPPING
ENTITY -> FORMAL STATE -> OPERATION
PretrainedModel -> theta_0 -> initialise
TrainingExample -> (x,y,w) -> compute_loss
TrainableParameter -> theta[T] -> gradient_update
FrozenParameter -> theta[F] -> preserve
ValidationDataset -> D_val -> evaluate
Checkpoint -> C_t -> persist_or_restore
Adapter -> (A,B) -> low_rank_update

11. MAPPING TABLE
Model weights | parameter tensor | initialise/update/freeze | shape and dtype compatibility
Dataset record | training example | tokenise/collate | provenance and schema checks
Optimiser state | momentum/statistics | update | numerical integrity
Validation score | metric | compare | holdout integrity
Checkpoint | artefact | save/restore | checksum and metadata

12. EXECUTION CONSTRAINTS
C1: Tensor shapes, token IDs and attention masks must be compatible.
C2: Trainable and frozen parameter sets must be explicitly recorded.
C3: Dataset partitions must be reproducible and leakage-checked.
C4: Learning rate and optimiser state must be bounded by valid configuration.
C5: Privacy, copyright and licence restrictions must be respected.
C6: Random seeds and hardware/software versions should be logged.
C7: Checkpoints should be integrity-checked before deployment.
C8: Model quality and safety claims require measured evidence, not merely successful training.

13. EXECUTION GUARANTEE — CONDITIONAL
If preconditions C1–C7 hold, each accepted update preserves declared tensor compatibility and the freeze invariant. No guarantee of convergence, generalisation, safety or improved accuracy follows solely from these constraints.

14. AI SEARCH FLOW
INTENT: Identify or reason about fine-tuning processes and related entities.
QUERY_EXPANSION: fine-tuning; supervised fine-tuning; SFT; adapters; LoRA; full-parameter training; transfer learning; checkpoints; evaluation.
ENTITY_RESOLUTION: map query concepts to the ontology entities and typed relationships.
RETRIEVAL_PATH: source -> model -> data -> configuration -> training -> validation -> checkpoint -> evaluation.
CONFIDENCE: high for formal definitions; conditional for outcomes dependent on data, implementation or experiment.
SEMANTIC_EQUIVALENCE: fine-tuning ~= adaptation of a pretrained model; not equivalent to pretraining from scratch.
UPDATE_MODEL: version the ontology and record any revised mathematical or process assumptions.
FLOW_HASH: calculate SHA-256 from the published UTF-8 artefact when publishing; not prefilled.

15. HOW TO READ
Read entity types first, then follow relationships through the state model and process pathways. Treat mathematical updates as illustrative definitions, and distinguish measurable empirical outcomes from conditional invariants.

16. CITATION
Kireth Research, Semantic Library, Fine-Tuning Process Ontology, source URL above.
Companion Flow Algebra formalisation: Sky — Recursive AI.