Google and the Evolution of Search

I. Human Evaluators — Google Engineering director Scott Huffman

1. 구글 "검색 품질 평가 팀"에 관한 얘기
• 초반에 세르게이의 favorite 10쿼리 에서 출발하여 현재 검색 품질 관리에 이름
• 여기에서 평가된 검색 품질은 알고리즘 개선에 활용되긴 하지만 직접적으로 영향 미치지 않음.
• 검색품질이 좋다는 것은 결국 "사람"이 아는 것이므로.
• 아마도 머신러닝 데이터로 활용할 듯.
  알고리즘 팀이 하는 건.. 중요한 팩터는 뽑아야 할거고, 머신러닝을 통해 개선하지 않을까.

2. no longer solely and automatically determined by the company’s vaunted algorithms. Now they simply “relied heavily” on them
   
   So Google’s objective evaluation and ranking of Web sites is to some extent defined by subjective reasoning of a collective human intelligence
   
   Things like “tell us how good you think this result is for this query.” And then out of the data, we produce a set of aggregate metrics that we look at and that we can track over time.
   
   But ultimately, the real quality control is done by the folks who are working on ranking and search UI.
   
   The evaluation program really just gives our engineers an aggregate measure of how good their algorithms are so they can improve them.
   

   
   

II. Cheating the System — Google software engineer Matt Cutts

1. 어뷰징, 스팸 등 문서/사이트 퀄러티에 관한 얘기
2. 역시나 사람 손을 태우느냐에 대한 부분 질문 
   

   we are sometimes willing to take manual action on those sorts of policy violations. But Google’s philosophy is that wherever you can use machines and algorithms

   
   
3. 인상적인 건, 언젠가 사람이 개입하지 않을 날이 올까? 에 대한 Matt cutt의 답변
   

   our intent is always to try to make sure that we handle things efficiently with machines and algorithms. But I don’t know that we will ever get there completely.
   

4. 나도 매우 동감 최대한 효율적이어야 하지만 사람을 완전히 뺄 수 있을까. 

III. What’s Next in Search? Much, Much Better Search — Google Fellow Amit Singhal

• recall & precision
• recall => precision =>combination of recall and precision
• search evaluation
• human evaluator
• how fresh is the document? How relevant? How useful?
• Whatever type of content is relevant to you should just show up in your search results
• 이거야말로 통합검색 아닌가.