Making search engines smarter

A Penn State researcher analyzed nearly 1 million Web searches to detect patterns of query reformulation and create models to predict them -- models that may help create more advanced search engines.

"The key finding in the research is that we are moving from descriptive aspects to predictive models in Web searching," said Jim Jansen, associate professor of information sciences and technology and one author of the paper "Patterns of Query Reformulation during Web Searching," published in the online edition of the Journal of the American Society for Information Science and Technology and scheduled for the July issue.

Researchers found that the search terms in 22 percent of queries were reformulated or changed to more precisely convey the information for which the user was searching.

"They typically moved to narrow their query at the start of the session, moving to reformulation in the mid and latter portions of the sessions," Jansen said. "It appears that the assistance to narrow the query and alternate query terms would be most beneficial immediately after the initial query submission."

Researchers also found low rates of users asking for system assistance in helping to find the desired information -- perhaps because they are too focused on using their own search terms to find information. "The implication is that system assistance should be most specifically targeted when the user is making a cognitive shift because it appears users are open to system intervention," Jansen said.

Jansen said this research is a critical step in helping to design more advanced search engines.

"Given that one can predict future states of query formulation based on previous and present states with a reasonable degree of accuracy, one can design information systems that provide query reformulation assistance, automated searching assistance systems, recommender systems and others," Jansen said.

Penn State