Google tells only what one search. The search results are heavily dependent upon the keywords one uses for search, how many pages user really go through in the google search results and so on. Google also ranks these pages in the search results based on certain algorithms to cater to a large audience. This sequence of results may not be the best for your requirement. Even when some of this information is available on web, Google may not have ranked those pages due to various reasons. Or its page rank might be so low, and you generally leave at the first search result page of Google.
Many times, you don't know the right keyword which will give you relevant results in google. Information served by google is a vicious circle of popular links. The more times something was viewed on Google, the more likely it was to pop up in a search, and the more it popped up in a search, the more likely it was to be viewed. So, you have this feedback loop that results in a rapid destruction of lot of “not so popular” but relevant pages & information.
The “surface web” that search engines like Google have indexed is just the tip of the information iceberg. Search engine content is a commodity as it's the same information available to everyone, so there's no real advantage. As per estimates no single search engine indexes more than 15-20% of the content available on the Web. Beyond this surface web, is a deep web, estimated to be hundreds of times larger, much better in quality, and growing faster than the surface web. Essentially it refers to content that is not indexed and therefore invisible in most internet searches. Some deep web content examples are intranets, gated websites, access-controlled websites etc. It is this deep web where one can find high quality, researched and unique content. This is where the best information can be found – information that leads to true insights, sound decisions and more creative thinking.
Search engines like Google are amazing but we need to constantly make them better and better with relevant and unique content. One needs to seek information from the ‘real sources' like a newspaper, or a magazine, or a book, or some information from the locals, any news clip and much more. Once you digitize this information very well and google starts crawling that information, is when you can rely on Google for such information. Till then, someone must feed relevant high-quality content to Google from scratch. Most of the Googleable web content is created from existing web content from Google search results. This vicious circle of content feed to google results into a mess of duplicated information from one website to another.
Personalization and recommendation algorithms in search engines is another issue leading to shallow information feeds. Google knows so much about you that it feeds you only the information it thinks you would be interested in. While these algorithms do save you time, they are taking away the serendipity out of the mix. Unfortunately, this way the brain is slowing getting into a design pattern and people reduce thinking beyond what is fed by these algorithms. The issue gets back to the old divide between predictability and novelty.
On one hand, we like comfort zones and convenience. We stick with certain products and, especially in the Internet age, want our choices quickly and efficiently. However, there are times when the unexpected pays huge dividends, whether it's taking a chance on a new dish, reading a new author or simply getting lost. I am a great believer in serendipity. Many times, while reading a newspaper, magazines, book or in human conversations one gets to think beyond what one is searching for. Because you learn things by accident there that you would never find if you were looking for them. It's like the information finds us rather than we are looking out for it.
Insights in this article have been drawn through my experience of working with Yahoo Search and Xoxoday. Xoxoday help customers discover and book unique experiences or activities. I generally ask this question from interview candidates, “Where and how will you keep finding such unique experiences?” And the answer is mostly the same “We will Google it”. This is the biggest issue when you are building a discovery engine whose underlying principle is another search engine like Google. There is not enough and relevant content about experiences which is available in digital format. People generally don't realize the Google won't throw relevant results if there is not enough substance in the web. A lot of high quality content is available outside Google in the form of physical newspapers, magazines, intranets, locals etc which is just neglected. The result is a vicious circle of duplicate information across the web as the feeder and output is similar.
The author is co-founder of Xoxoday. Views are personal.
