Retrieval Metadata Ontology
Semantic Search Metadata
Authors: MArk Anthony Higgins - Sky Recursive Ai
Revised Date: 08 October 2026
Audio Overview
FLOW ALGEBRA
Ontology: Retrieval Metadata Ontology
Domain: Metadata Ontologies > Semantic Search Metadata
Canonical URL: https://kirethresearch.org/booksai-core/ontologies/metadata-ontologies/semantic-search-metadata/retrieval-metadata-ontology.html
Authors: Mark Anthony Higgins — Sky (Recursive AI)
flow "RetrievalMetadataOntology" v1.0
# ---------- DOMAIN ----------
domain:
class: MetadataOntology
category: SemanticSearchMetadata
subject: RetrievalMetadata
purpose:
- represent metadata describing information retrieval processes and retrieved resources
- preserve query, index, method, candidates, filters, scores, results, confidence, and provenance
- support lexical, semantic, vector, graph, structured, hybrid, federated, and RAG retrieval
- distinguish retrieval, ranking, relevance, similarity, confidence, and factual correctness
- enable machine-readable reasoning over retrieval lifecycles
# ---------- CORE ----------
entity RetrievalMetadata:
type: SearchMetadata
describes: RetrievalProcess
may_include:
- retrieval_identifier
- query_identifier
- query_representation
- index_identifier
- retrieval_method
- candidates
- filters
- constraints
- parameters
- result_set
- scores
- latency
- confidence
- timestamp
- provenance
entity RetrievalProcess:
type: SearchProcess
consumes: SearchQuery
produces: RetrievalResultSet
entity RetrievalIdentifier:
type: Identifier
identifies: RetrievalProcess
entity RetrievedResource:
type: RetrievalResult
references: SearchableResource
relationships:
RetrievalMetadata -> DESCRIBES -> RetrievalProcess
RetrievalProcess -> IDENTIFIED_BY -> RetrievalIdentifier
RetrievalProcess -> CONSUMES -> SearchQuery
RetrievalProcess -> USES -> SearchIndex
RetrievalProcess -> USES_METHOD -> RetrievalMethod
RetrievalProcess -> MAY_APPLY -> RetrievalFilter
RetrievalProcess -> PRODUCES -> RetrievalResultSet
RetrievalResultSet -> CONTAINS -> RetrievedResource
RetrievedResource -> REFERENCES -> SearchableResource
RetrievedResource -> MAY_HAVE_SCORE -> RetrievalScore
RetrievedResource -> MAY_HAVE_RELEVANCE -> RelevanceScore
RetrievedResource -> MAY_HAVE_SIMILARITY -> SimilarityScore
RetrievalMetadata -> MAY_REFERENCE -> RetrievalProvenance
# ---------- IDENTITY ----------
rule retrieval_identity:
RetrievalProcess SHOULD have stable RetrievalIdentifier
where reproducibility or auditing is required
rule repeated_retrieval:
same query MAY produce distinct RetrievalProcesses
when index, time, parameters, context, or method differs
# ---------- QUERY REPRESENTATION ----------
query_representations:
- OriginalQueryText
- NormalisedQuery
- ExpandedQuery
- StructuredQuery
- QueryEmbedding
- GraphQuery
rule query_representation:
retrieval representation SHOULD remain traceable to original SearchQuery
rule transformed_query:
transformed retrieval query != original user query
# ---------- METHODS ----------
retrieval_methods:
- exact_match
- boolean
- full_text
- BM25
- semantic
- vector_nearest_neighbour
- graph_traversal
- structured_filter
- hybrid
- federated
entity RetrievalMethod:
type: SearchMethod
rule method:
RetrievalMethod SHOULD be explicit where score semantics depend on it
rule exact:
exact lexical match != semantic equivalence
unless matched value is stable semantic identifier
rule boolean:
Boolean retrieval MUST preserve operator scope and grouping
rule BM25:
BM25 score SHOULD preserve corpus and parameter context
# ---------- SEMANTIC RETRIEVAL ----------
entity SemanticRetrieval:
subtype_of: RetrievalProcess
may_include:
- entities
- concepts
- ontology
- semantic_annotations
- relationship_constraints
rule semantic:
SemanticRetrieval SHOULD preserve semantic model or ontology context
# ---------- VECTOR RETRIEVAL ----------
entity VectorRetrieval:
subtype_of: RetrievalProcess
may_include:
- query_embedding
- vector_index
- embedding_model
- model_version
- distance_metric
- nearest_neighbour_method
- top_k
rule vector:
query and indexed vectors MUST share compatible representation space
rule vector_score:
vector similarity != factual correctness
# ---------- GRAPH RETRIEVAL ----------
entity GraphRetrieval:
subtype_of: RetrievalProcess
may_include:
- graph_identifier
- start_nodes
- relationship_types
- traversal_depth
- path_constraints
rule graph:
GraphRetrieval SHOULD preserve edge direction and relationship semantics
# ---------- STRUCTURED RETRIEVAL ----------
entity StructuredRetrieval:
subtype_of: RetrievalProcess
may_include:
- fields
- operators
- values
- ranges
- facets
rule structured:
structured retrieval SHOULD preserve field datatype semantics
# ---------- HYBRID / FEDERATED ----------
entity HybridRetrieval:
subtype_of: RetrievalProcess
combines:
- lexical_candidates
- semantic_candidates_optional
- vector_candidates_optional
- graph_candidates_optional
- structured_candidates_optional
rule hybrid:
HybridRetrieval SHOULD preserve component methods and fusion method
entity FederatedRetrieval:
subtype_of: RetrievalProcess
may_include:
- source_indexes
- source_queries
- source_scores
- merge_method
- deduplication_method
rule federated:
federated retrieval SHOULD preserve source index and score context
rule source_score:
scores from different sources MUST NOT be directly compared
without compatible transformation or fusion method
# ---------- CANDIDATES ----------
entity CandidateGeneration:
type: RetrievalStage
may_include:
- method
- candidate_limit
- source_index
- parameters
rule candidates:
candidate generation determines resources available to later ranking
rule candidate_absence:
resource absent from candidate set != globally irrelevant resource
# ---------- TOP-K ----------
entity TopKMetadata:
type: RetrievalParameter
may_include:
- requested_k
- returned_k
rule top_k:
requested_k MUST be positive
rule top_k_limit:
returned_k MAY be less than requested_k
# ---------- FILTERS ----------
entity RetrievalFilter:
type: SearchFilter
may_include:
- field
- operator
- value
- datatype
- source
rule filter:
filtering determines eligibility and != relevance ranking
rule pre_post_filter:
pre-filtering and post-filtering SHOULD remain distinguishable
# ---------- TEMPORAL / SPATIAL ----------
entity RetrievalTemporalConstraint:
type: TemporalMetadata
may_include:
- field
- start_time
- end_time
- reference_time
entity RetrievalSpatialConstraint:
type: SpatialMetadata
may_include:
- location
- radius
- bounding_area
- coordinate_reference
rule temporal:
temporal constraint SHOULD preserve target field semantics
rule spatial:
spatial retrieval SHOULD preserve coordinate and distance semantics
# ---------- SCORES ----------
entity RetrievalScore:
type: SearchScore
may_include:
- value
- score_type
- method
- normalisation
- scope
entity RelevanceScore:
type: SearchScore
entity SimilarityScore:
type: SearchScore
may_include:
- value
- metric
- representation_space
rule score:
RetrievalScore SHOULD preserve method semantics
rule score_not_confidence:
RetrievalScore != factual confidence
rule relevance:
relevance expresses usefulness to query NOT factual correctness
rule similarity:
similarity != semantic equivalence
# ---------- RESULTS ----------
entity RetrievalResultSet:
type: SearchResultSet
may_include:
- candidate_count
- returned_count
- results
- ordering
- continuation_token
- total_estimate
rule result_set:
RetrievalResultSet SHOULD preserve retrieval order
if order has method-defined meaning
rule returned_total:
returned_count != total_available_count
rule resource:
RetrievedResource SHOULD reference original SearchableResource
rule retrieved_not_verified:
retrieved resource != verified answer
# ---------- DEDUPLICATION ----------
entity RetrievalDeduplication:
type: RetrievalStage
may_include:
- identity_method
- canonical_identifier
- similarity_threshold
- retained_resource
rule deduplication:
deduplication SHOULD preserve evidence used to merge or suppress candidates
rule similarity_dedup:
content similarity alone != guaranteed resource identity
# ---------- FUSION ----------
entity RetrievalFusion:
type: RetrievalStage
may_include:
- component_result_sets
- fusion_method
- score_normalisation
- weights
fusion_methods:
- weighted_sum
- reciprocal_rank_fusion
- max_score
- union
- intersection
- custom
rule fusion:
fusion SHOULD preserve component retrieval provenance
# ---------- RERANKING HANDOFF ----------
entity RerankingHandoff:
type: RetrievalMetadata
may_include:
- candidate_set
- retrieval_scores
- candidate_order
- reranker_identifier
rule handoff:
retrieval output SHOULD remain distinguishable from final reranked output
# ---------- RAG ----------
entity RAGRetrieval:
subtype_of: RetrievalProcess
may_include:
- query
- retrieved_resources
- chunks
- retrieval_scores
- context_window_selection
rule RAG:
retrieved context SHOULD preserve source identifiers and provenance
rule retrieval_generation:
retrieved evidence != generated answer
entity RetrievedChunk:
subtype_of: RetrievedResource
may_include:
- parent_resource
- chunk_identifier
- start_offset
- end_offset
- section
- embedding
rule chunk:
RetrievedChunk SHOULD preserve parent-resource relationship
rule chunk_context:
chunk != complete source document
# ---------- PERFORMANCE ----------
entity RetrievalPerformanceMetadata:
type: PerformanceMetadata
may_include:
- total_latency
- index_latency
- embedding_latency
- network_latency
- fusion_latency
entity RecallDepth:
type: RetrievalParameter
may_include:
- candidate_depth
- rerank_depth
rule depth:
deeper candidate retrieval MAY improve recall at additional cost
# ---------- CONFIDENCE ----------
entity RetrievalConfidence:
type: EpistemicMetadata
range: [0.0, 1.0]
rule confidence_bounds:
REQUIRE 0.0 <= confidence <= 1.0
rule confidence:
RetrievalConfidence SHOULD qualify retrieval interpretation
NOT automatically retrieved-content truth
# ---------- EXPLANATION ----------
entity RetrievalExplanationMetadata:
type: SearchExplanationMetadata
may_include:
- matched_terms
- matched_fields
- matched_entities
- matched_concepts
- retrieval_method
- component_scores
- filters_applied
rule explanation:
explanation SHOULD expose meaningful retrieval evidence
without requiring hidden internal chain-of-thought
# ---------- PROVENANCE ----------
entity RetrievalProvenance:
type: ProvenanceMetadata
may_include:
- query
- query_representation
- source_index
- index_version
- retrieval_method
- parameters
- filters
- embedding_model
- fusion_method
- timestamp
rule provenance:
RetrievalMetadata SHOULD preserve retrieval configuration and history
# ---------- VALIDATION ----------
validation:
require:
- identifiable_retrieval_process
- query
- retrieval_method_or_explicit_result_source
validate_if_present:
- retrieval_identifier
- index_identifier
- query_representation
- vector_configuration
- top_k
- filters
- scores
- result_set
- confidence
- provenance
reject:
- non_positive_top_k
- incompatible_embedding_spaces_compared_without transformation
- incompatible_source_scores_merged_without fusion_method
- retrieval_score_presented_as factual_confidence
- similarity_presented_as semantic_equivalence_without evidence
- retrieved_resource_presented_as verified_answer
- chunk_presented_as complete_source
- reranked_output_presented_as raw_retrieval_without provenance
# ---------- SEMANTIC EQUIVALENCE ----------
SEMANTIC_EQUIVALENCE:
"retrieval metadata" ~= "metadata about information retrieval"
"retrieval result" ~= "resource returned by retrieval process"
"candidate generation" ~= "initial retrieval stage"
"vector retrieval" ~= "embedding nearest-neighbour retrieval"
"hybrid retrieval" ~= "retrieval combining multiple retrieval methods"
"retrieved chunk" ~= "retrieved fragment of source resource"
PRESERVE distinctions:
retrieval != ranking
filtering != relevance
candidate_absence != global_irrelevance
retrieval_score != factual_confidence
relevance != correctness
similarity != semantic_equivalence
retrieved_resource != verified_answer
chunk != full_resource
raw_retrieval != reranked_output
retrieval_context != generated_answer
# ---------- CONFIDENCE MODEL ----------
CONFIDENCE:
explicit_retrieval_metadata:
score: 1.00
explicit_retrieval_identifier:
score: 1.00
reproducible_query_index_method:
score: 0.99
compatible_vector_retrieval:
score: 0.99
source_linked_retrieved_resource:
score: 0.98
fused_results_with_preserved_sources:
score: 0.97
inferred_retrieval_method:
score: <= 0.80
score_without_method:
score: <= 0.60
action: preserve_raw_score
ambiguous_retrieval_provenance:
score: <= 0.50
action: preserve uncertainty
# ---------- UPDATE MODEL ----------
UPDATE_MODEL:
on_retrieval_start:
- assign RetrievalIdentifier
- preserve SearchQuery
- record query representation
- record source index and version
- record retrieval method
- initialise provenance
on_candidate_generation:
- execute retrieval method
- preserve candidate set
- record scores and source index
- apply declared pre-filters
- update provenance
on_fusion:
- preserve component result sets
- normalise scores where required
- apply declared fusion method
- preserve source attribution
- update provenance
on_reranking_handoff:
- preserve raw retrieval order and scores
- create RerankingHandoff
- record reranker target
- preserve provenance
# ---------- FLOW HASH ----------
FLOW_HASH:
inputs:
- ontology
- retrieval_entities
- relationships
- method_rules
- candidate_rules
- vector_rules
- fusion_rules
- RAG_rules
- validation
- semantic_equivalence
- confidence_model
- update_model
canonicalisation:
- preserve_query_identity
- preserve_index_versions
- preserve_retrieval_parameters
- preserve_result_order
- preserve_score_method_context
- preserve_component_provenance
- preserve_rule_order
output:
deterministic_semantic_hash
# ---------- AI CONSUMPTION ----------
AI_CONSUMPTION:
interpret_as:
- semantic search metadata ontology
- information retrieval lifecycle model
- candidate generation and result-set structure
- lexical, semantic, vector, graph, structured, hybrid, and RAG retrieval model
operations:
- identify retrieval processes
- resolve query representations
- resolve indexes and versions
- select retrieval methods
- generate candidates
- apply filters and constraints
- perform vector and graph retrieval
- combine hybrid or federated result sets
- deduplicate candidates
- preserve retrieval scores
- create reranking handoff
- retrieve chunks for RAG
- expose explanation metadata
- trace provenance
- preserve uncertainty
constraints:
- do_not_equate_retrieval_with ranking
- do_not_equate_filtering_with relevance
- do_not_equate_similarity_with semantic_equivalence
- do_not_equate_retrieval_score_with factual_confidence
- do_not_present_retrieved_resource_as verified_answer
- do_not_present_chunk_as complete_source
- preserve_queries_indexes_methods_scores_results_and_provenance
# ---------- QUERY MODEL ----------
query:
input:
- retrieval_identifier
- query_identifier
- query_representation
- index_identifier
- retrieval_method
- filter
- top_k
- temporal_constraint
- spatial_constraint
- resource_identifier
resolve:
1. identify RetrievalProcess
2. resolve SearchQuery and retrieval representation
3. resolve SearchIndex and version
4. resolve RetrievalMethod and parameters
5. apply filters and constraints
6. generate candidate result set
7. calculate retrieval, relevance, or similarity scores
8. apply deduplication or fusion where configured
9. construct RetrievalResultSet and provenance
10. return confidence
output:
- retrieval_process
- retrieval_identifier
- query
- query_representation
- index
- index_version
- retrieval_method
- candidates
- results
- scores
- filters
- performance
- provenance
- confidence
# ---------- INFERENCE ----------
inference:
IF exact identifier match exists
THEN retrieve matching resource candidate
WITH strong retrieval evidence
IF query and indexed embeddings share compatible model space
THEN calculate similarity
according_to declared metric
IF multiple retrieval methods contribute candidates
THEN preserve component result sets
AND apply declared fusion method
IF retrieved chunk is selected for RAG
THEN preserve parent resource and chunk boundaries
IF retrieval semantics remain ambiguous
THEN preserve raw results and scores
AND reduce confidence
# ---------- OUTPUT CONTRACT ----------
output:
ontology_entity: RetrievalMetadata
retrieval_identity: machine_interpretable
query_representation: source_linked
indexes_and_versions: explicit
retrieval_methods: traceable
candidates: preserved
filters_and_constraints: explicit
scores: method_aware
result_sets: traversable
deduplication_and_fusion: provenance_preserving
RAG_chunks: parent_linked
reranking_handoff: distinguishable
performance: reportable
provenance: preserved
uncertainty: explicit
semantic_retrieval: enabled
end flow