Modeling Action-level Satisfaction for Search Task Satisfaction Prediction - Wang, Hongning; Song, Yang; Chang, Ming-Wei; He, Xiaodong; Awadallah, Ahmed; White, Ryen W. (ACM SIGIR 2014, 2014)
Search satisfaction is a property of a user’s search process. Understanding it is critical for search providers to evaluate the performance and improve the eﬀectiveness of search engines. Existing methods model search satisfaction holistically at the search-task level, ignoring important dependencies between action-level satisfaction and overall task satisfaction. We hypothesize that searchers’ latent action-level satisfaction (i.e., whether they believe they were satisﬁed with the results of a query or click) inﬂuences their observed search behaviors and contributes to overall search satisfaction. We conjecture that by modeling search satisfaction at the action level, we can build more complete and more accurate predictors of search-task satisfaction. To do this, we develop a latent structural learning method, whereby rich structured features and dependency relations unique to search satisfaction prediction are explored. Using insitu search satisfaction judgments provided by searchers, we show that there is signiﬁcant value in modeling action-level satisfaction in search-task satisfaction prediction. In addition, experimental results on large-scale logs from Bing.com demonstrate clear beneﬁt from using inferred action satisfaction labels for other applications such as document relevance estimation and query suggestion.