WIKIFIER

Recent trends have shown text classification as important aspect of Natural language processing. Wikification is one such task of recognizing and associating words in text with their significant Wikipedia pages.

The blog describes about a software tool Wikifier by COGINITIVE COMPUTATION GROUP that takes text as input and returns text enhanced with links to encyclopedic resources such as Wikipedia. In doing so, the Wikifier can assist human readers and automated systems in three key ways:

• By resolving entities and clarifying ambiguous and variable text.
• By providing background knowledge about unfamiliar topics.
• By grounding controversial claims from partisan sources in impartial encyclopedic resources.

Snapshot of Wikifier with random example and wikified data is shown below:

 

In the wikified text, entities are highligted for instance Mubarak could be referred as either Ishq Mubarak song or Suzanne Mubarak, wife of former Egptian President Hosni Mubarak. The Wikifier, clearly disambiguates based on context and associates with Suzanne Mubarak.

APPROACH

The Wikifier takes the document as input, considering set of mentions (entities as highlighted), M = {m₁, m₂, …..., m_n}. The system maps the set of mentions to corresponding the set of Wikifipedia titles, W = {w₁, w₂, …..., w_k}. There could be certain cases where a mention does not have any Wikipedia page, considering this as exception NULL Wikipedia title is added.

Visualising the bipartite graph, with mentions as one set and Wikipedia titles as other. Approach perform two-level optimization for disambiguates:

• Local Disambiguation
• Global Disambiguation

The following graph illustrates the optimization, marking dark edges as correct titles for corresponding mentions.

In local approach, each mention m_i is disambiguated separately with score function f(m_i,t_j) that describes how likely title t_j correctly disambiguates mention m_i and assigns title with highest score with content similar to document:

Γ^*local = argmax _i=1∑^N f(m_i,t_j)

                                                 Γ

After getting all the titles, now considered all of them have same score with respect to the context. Global optimization is applied with coherence function ψ as:

Γ^* = argmax [ _i=1∑^N f(m_i,t_j) + ψ(Γ) ]

                                                 Γ

The global optimization is NP-Hard problem and thus approximation is done by estimating from pairwise ψ(t_i, t_j) to get optimal Γ^*_.

LIMITATION

Unlikely the mentions taken from general text, Wikipedia mentions are likely to have Wikipedia pages. Thus, primary challenge is when mentions does not have Wikipedia pages.

REFERNCES

1. L. Ratinov and D. Roth and D. Downey and M. Anderson, Local and Global Algorithms for Disambiguation to Wikipedia ACL (2011)

2. Xiao Cheng and Dan Roth, Relational Inference for Wikification EMNLP (2013)