|
Published Articles >> Table of Contents >> Abstract
2003 IEEE/WIC International Conference on Web Intelligence (WI'03)
p. 249
Detecting Near-Replicas on the Web by Content and Hyperlink Analysis
Ernesto Di Iorio, Università di Siena
Michelangelo Diligenti, Università di Siena
Marco Gori, Università di Siena
Marco Maggini, Università di Siena
Augusto Pucci, Università di Siena
Full Article Text:
  
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/WI.2003.1241201
Send link to a friend
| Abstract |
|
The presence of near-replicas of documents is very common on the Web. Documents may be replicated completely or partially for different reasons (versions, mirrors, etc.), or the same resource can be associated to different URLs (dynamically generated pages, etc.). Whilst replication can improve information accessibility by the users, the presence of near-replicated documents can hinder the effectiveness of search engines (for example, decreasing the coverage). We propose a method to detect similar pages, in particular replicas and near-replicas, which is based on a pair of signatures. The first signature is obtained by a random projection of the bag-of-words vector representing the page contents. The second signature is computed by a recursive equation which exploits the connectivity among the Web
pages to code the context of each page. The accuracy of the proposed approach is analyzed and validated by experimental results which show that on the given dataset near-replicas can be detected with a precision-recall of 93%.
|
 Additional Information
|
Citation:
Ernesto Di Iorio, Michelangelo Diligenti, Marco Gori, Marco Maggini, Augusto Pucci,
"Detecting Near-Replicas on the Web by Content and Hyperlink Analysis,"
wi,
p. 249,
2003 IEEE/WIC International Conference on Web Intelligence (WI'03),
2003
|
|
