Some people even claim that Google result counts are a meaningless metric.

Well, “meaningless” might be an overstatement, but as a matter of fact, Google counts can be incorrect by the order of several digits or more. I think that many people trust Google counts without knowing their inaccuracy. Please think twice before using Google counts as an indicator of how common a certain expression is.

This confused some people on our site:

There is a post on meta.english.stackexchange.com on the same topic.


Please try by yourself. You can see the inaccuracy of counts by yourself. Here is what I tried, reproducing the example in the second link above. I searched "られてられて" (with quotation marks) on Google. It said that there were about 37,800,000 results. (This number seems to vary from time to time.) We may take this large number as an evidence that the form “られてられて” is fairly common. However, when I went forward in the search results, the list came to an end after only 417 results! At the end of the list, Google showed:

In order to show you the most relevant results, we have omitted some entries very similar to the 417 already displayed.
If you like, you can repeat the search with the omitted results included.

I clicked the link to repeat the search with the omitted results included, and did the same thing. This time it showed 500 results, which is larger than 417 but not even close to 37,800,000. This means that there is some problem with Google counts, and we should not believe them without thinking.

share
1  
Google-fundamentalists. –  user458 Oct 4 '11 at 22:12
1  
Because I was one of them before I learned about the inaccuracy, I cannot blame people who trust Google counts…. –  Tsuyoshi Ito Oct 4 '11 at 22:41
    
The count may be tricky. But I am amazed at the fact that there are so many people who thinks it academic to just adopt the result from google without any consultation to informants or fieldwork, and calling it a "research". –  user458 Oct 4 '11 at 22:50
3  
Even if the results are accurate, they do next to nothing to prove whether something is right or wrong. The most one can deduce is "It is a commonly encountered term". Just because it has a high amount of hits does not make it right. Probabilistic reasoning is not a good enough justification for something. –  Flaw Oct 5 '11 at 6:10
1  
@sawa: I see. There is no way to stop hobbyists from using inaccurate methods, but I hope that professional linguists and students in linguistics are not like that…. –  Tsuyoshi Ito Oct 5 '11 at 11:25
2  
@Flaw Absolutely. google.com/search?q="maintainence"; gives me 24.3m results. Even if we have a billion valid results, it just doesn't make "maintainence" the correct spelling for "maintenance" –  Pacerier Oct 7 '11 at 13:23
3  
@Flaw That certainly depends on your definition of "right". One thing is what textbooks/language-governing bodies/prescriptive grammarians prescribe, another thing is the language which is actually being used. Most serious linguists are at least as interested in the latter. –  dainichi May 7 '12 at 1:22

2 Answers 2

The Balanced Corpus of Contemporary Written Japanese (BCCWJ) is publicly available and can be searched for free using a site called 少納言. It consists of a hundred million words of published Japanese from 1971-2008. The result counts should be accurate, and in many situations it should be a better tool than Google for researching Japanese usage.

More information is available in both Japanese and English.

There's also a paper available with more details, titled "KOTONOHA and BCCWJ: Development of a Balanced Corpus of Contemporary Written Japanese" (PDF format).

I'm not sure if many people on JLU know that this resource exists, and I certainly haven't noticed many answers citing it. I hope people consider it and use it (or other corpora) when they feel it's appropriate to do so.

share
1  
I had tried it before, but as you can see, I recently started using it to answer some of the questions on the site. Thank you for reminding me of it! –  Tsuyoshi Ito Nov 4 '13 at 1:58

For reference. http://searchengineland.com/why-google-cant-count-results-properly-53559

share
    
Even after reading this, I don't really understand how/why the estimates are so inaccurate... –  atlantiza Jun 13 '12 at 20:01
    
Can you summarize what the article says, in case it suffers link rot? –  Andrew Grimm Jun 27 '12 at 22:08
    
@AndrewGrimm I do not want to spend time doing it. If the link becomes dead, I will rather delete this answer. –  user458 Jun 28 '12 at 1:31
    
To summarize briefly, computation time is valuable, so search engine algorithms try to perform the bare minimum amount of computation needed to deliver good results on the first page. If you signal that you want more in-depth results, such as by checking up to the 417th result, the algorithm is forced to do more computation, and only then does it give the better estimate. –  無色受想行識 Mar 28 '13 at 5:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .