Friday, December 17, 2010

Ngrams

Similar tothis post on Lexicon I was shown a new tool from Google today that can search the texts of over 5 million books going back centuries and in a number of languages. The New York Times covered it here. It goes much deeper than the Facebook tool to examine cultural tends and is pretty flexible for refining your query. The data is even available for download and use by anyone. Below are a couple of posts concerning the terminology of planning and some more recent trends in the field.




One of the key questions with this data is which books are likely to be skipped or prohibited in this cataloging and what are the trends among them? And then more importantly, what trends to these book represent or add to? That's all probably covered in this paper but I haven't read it yet.

Another important difference from the Lexicon tool is that you can not look at trends in seasonality. The data that comes on in books isn't as spontaneous as that captured in Google searches, Facebook wall posts (recently removed), or tweets.