Loading TOC...
XQuery vec:vector-score

MarkLogic 12 Product Documentation
vec.vectorScore

vec.vectorScore(
   score as Number,
   distance as Number,
   [distanceWeight as Number?],
   [weight as Number?]
) as (Number|String)

Summary

A helper function that returns a hybrid score using a cts score and a vector distance calculation result. You can tune the effect of the vector distance on the score using the distanceWeight option. The ideal value for distanceWeight depends on your application. The hybrid score is calculated using the formula: score = weight * annScore + (1 - weight) * ctsScore. - annScore is derived from the distance and distanceWeight, where a larger distanceWeight reduces the annScore for the same distance. - weight determines the contribution of the annScore and ctsScore to the final score. A weight of 0.5 balances both equally. This formula allows you to combine traditional cts scoring with vector-based distance scoring, providing a flexible way to rank results.

Parameters
score The cts.score of the matching document.
distance The distance between the vector in the matching document and the query vector. Examples, the result of a call to ovec:cosine-distance() or ovec:euclidean-distance().
distanceWeight The weight of the vector distance on the annScore. This value is a positive coefficient that scales the distance. A larger distanceWeight produces a lower annScore for the same distance. The default value is 1.
weight The weight of the annScore in the final hybrid score. This value is a coefficient between 0 and 1, where 0 gives full weight to the cts score and 1 gives full weight to the annScore. The default value is 0.5.

See Also

Example

const op = require('/MarkLogic/optic');

// grab a query vector from the document below at the array node named 'emb'
const qv = vec.vector(fn.head(fn.doc('embedding104206.json')).xpath("/array-node('emb')"))

// define a word query to find relevant documents
const query = cts.wordQuery('turtle')

// define a view named 'from_search' that contains our search results
const search = op.fromSearch(query,['fragmentId','score'],'from_search')

// join a TDE view named 'wiki_vectors' with which to calculate a hybrid score :)
const result = op.fromView('vecs','wiki_vectors',null,op.fragmentIdCol('view_frag'))
                 .joinInner(search,op.on(op.viewCol('wiki_vectors','view_frag'),op.viewCol('from_search','fragmentId')))
                 .joinDocUri(op.col('uri'),op.fragmentIdCol('view_frag'))
                 .bind(op.as('cosDistance',op.vec.cosineDistance(op.viewCol('wiki_vectors','embedding'),qv)))
                 .bind(op.as('hybridScore',op.vec.vectorScore(op.col('score'),op.col('cosDistance'),0.5,1.0)))
                 .select([op.col('score'),op.col('cosDistance'),op.col('hybridScore'),op.col('uri')])
                 .orderBy([op.desc(op.col('hybridScore')),op.col('uri')])
                 .result()
result;

=> A table with the result of scoring the document, combining the cts score with vector distance

Stack Overflow iconStack Overflow: Get the most useful answers to questions from the MarkLogic community, or ask your own question.