An Ensemble Click Model for Web Document Ranking

Document Type : Original Article


Department of Artificial Intelligence, Faculty of Computer Engineering, K. N. Toosi University of Technology, Tehran, Iran


Annually, web search engine providers spend a lot of money on re-ranking documents in search engine result pages (SERP). Click models provide advantageous information for re-ranking documents in SERPs through modeling interactions among users and search engines. Here, three modules are employed to predict users' clicks on SERPs simultaneously, the first module tries to predict users' click behaviors using Probabilistic Graphical Models, the second module is a Time-series Deep Neural Click Model which predicts users' clicks on documents and finally, the third module is a similarity-based measure which creates a graph of document-query relations and uses SimRank Algorithm to predict the similarity. After running these three simultaneous processes, three click probability values are fed to an MLP classifier as inputs. The MLP classifier learns to decide on top of the three preceding modules, then it predicts a probability value which shows how probable a document is to be clicked by a user. The proposed system is evaluated on the Yandex dataset as a standard click log dataset. The results demonstrate the superiority of our model over the well-known click models in terms of perplexity.


1.     Ghose, A., Ipeirotis, P. G., and Li, B., “Examining the impact of ranking on consumer behavior and search engine revenue”, Management Science, Vol. 60, No. 7, (2014), 1632–1654.
2.     Borisov, A., Markov, I., De Rijke, M. and Serdyukov, P., “A neural click model for web search”, In Proceedings of the 25th International Conference on World Wide Web, (2016), 531–541.
3.     Grotov, A., Chuklin, A., Markov, I., Stout, L., Xumara, F. and de Rijke, M., “A comparative study of click models for web search”, In International Conference of the Cross-Language Evaluation Forum for European Languages, (2015), 78–90.
4.     Liu, Z., Mao, J., Wang, C., Ai, Q., Liu, Y. and Nie, J.Y., “Enhancing click models with mouse movement information”, Information Retrieval Journal, Vol. 20, No. 1, (2017), 53–80.
5.     Aouad, A., Feldman, J., Segev, D. and Zhang, D., “Click-based MNL: Algorithmic frameworks for modeling click data in assortment optimization”, Available at SSRN 3340620, (2019).
6.     Dupret, G. E. and Piwowarski, B., “A user browsing model to predict search engine click data from past observations”, In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, (2008), 331–338.
7.     Jeh, G. and Widom, J., “SimRank: a measure of structural-context similarity”, In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, (2002), 538–543.
8.     Guo, F., Liu, C., and Wang, Y. M., “Efficient multiple-click models in web search”, In Proceedings of the second acm international conference on web search and data mining, (2009), 124–131.
9.     Chapelle, O. and Zhang, Y., “A dynamic bayesian network click model for web search ranking”, In Proceedings of the 18th international conference on World wide web, (2009), 1–10.
10.   Borisov, A., Wardenaar, M., Markov, I. and de Rijke, M., “A click sequence model for web search”, In the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, (2018), 45–54.
11.   Liu, Q., Yu, F., Wu, S. and Wang, L., “A convolutional click prediction model”, In Proceedings of the 24th ACM international on conference on information and knowledge management, (2015), 1743–1746.
12.   Ni, Z., Ma, X., Sun, X. and Bian, L., “A Click Prediction Model Based on Residual Unit with Inception Module”, In Pacific Rim International Conference on Artificial Intelligence, (2019), 393–403.
13.   Zheng, Y., Mao, J., Liu, Y., Luo, C., Zhang, M. and Ma, S., “Constructing Click Model for Mobile Search with Viewport Time”, ACM Transactions on Information Systems (TOIS), Vol. 37, No. 4, (2019), 1–34.
14.   Zhang, Y., Dai, H., Xu, C., Feng, J., Wang, T., Bian, J., Wang, B. and Liu, T.Y., “Sequential click prediction for sponsored search with recurrent neural networks”, In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, (2014).
15.   Dietterich, T. G., “Ensemble methods in machine learning”, In International workshop on multiple classifier systems (pp. 1-15). Springer, Berlin, Heidelberg, (2000), 1–15.
16.   Zhang, Y., Chen, W., Wang, D. and Yang, Q., “User-click modeling for understanding and predicting search-behavior”, In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, (2011), 1388–1396.
17.   Sak, H., Senior, A. W., and Beaufays, F., “Long short-term memory recurrent neural network architectures for large scale acoustic modeling”, In 15th Annual Conference of the International Speech Communication Association, Singapore, (2014), 338-342.