Towards Making Document Ranking Effective

While making the best use of Big Data, data science aims at avoiding an information overload. For instance, search engines mine information based on a user query. Essentially, recommendation engines, such as Amazon, help narrow down on the appropriate information that the user may be seeking.

Such recommendation systems are built based on broadly two types of filtering: collaborative filtering and content-based filtering.

(Image source: https://www.themarketingtechnologist.co/building-a-recommendation-engine-for-geek-setting-up-the-prerequisites-13/)

This blog focusses on the latter. Content-based recommenders make use of keywords associated with products or services and the user profile(including past likings by the user or current information/product/service being examined by the user). For instance, if a user follows news updates on Federer and Nadal, (s)he may be interested in news updates on these tennis players or on tennis. This is where techniques to process natural language may come to use. With a motley of information available, the TF-IDF term weighting scheme is one such technique to retrieve an appropriate set of information(documents) based on the presence of certain key terms. The steps in building a recommender involve obtaining score for the terms and deciding on a threshold score to assign certain terms as tags to a document.

This blog discusses a TF-IDF scheme as proposed by Paik [1]. In the context of a recommender, given an input document (which acts as a query Q) and information (set of documents D1, D2..DN) the task is to assign score to each of the documents(D1, D2..DN) with respect to Q, to quantify the saliency of the query terms in the information documents. Such a scoring can then be used to extract top relevant documents for recommendation.

Most information retrieval models make use of three factors to infer the importance of a term in a document:

1. Term frequency(tf) in a document
2. Document length
3. Document frequency of the term, which gives higher importance to documents that contain terms that are rare in the collection

Most term weighting models apply either term frequency based normalization or length based normalization to the terms. Both these models have their limitations. For instance, length based normalization may constrain the retrieval of long documents.

Paik proposes a two aspect term frequency normalization scheme, that is a weighted combination of

1. relative tf weighting within a document(RITF) and
2. the tf normalization based on document length(LRTF)

to compute a TF-IDF score for terms, resulting in the formulation:

TF(t, D)= w * RITF(t, D)+ (1 - w)*LRTF(t, D), where 0 < w < 1

Such a scheme maintains a balance in preferring long and short length documents. The task comes to determining the appropriate value of the weight w. Assigning the appropriate weight is dependent on the query length. As has been observed, for a long query, the RITF prefers long documents as the term matches is proportional to the length of the document [2]. On the other hand, as discussed before, LRTF, which is a length based normalization, prefers short documents. Hence, to balance out the effect, more weight should be assigned to the RITF factor in case of short queries. In case of long queries, LRTF should be given more weight.

While MATF is a TF-IDF based model, language and probabilistic models are other options also used in generating ranks of recommendations, based on the content.

References:

[1] Paik, Jiaul H. "A novel TF-IDF weighting scheme for effective ranking." Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval. ACM, 2013.

[2] Singhal, Amit, Chris Buckley, and Manclar Mitra. "Pivoted document length normalization." ACM SIGIR Forum. Vol. 51. No. 2. ACM, 2017.