The Irish newspaper industry seems to have chosen today for a bout of collective hyperbole about search engines. Here's what the Examiner had to say:
Incredible, isn't it, that the newspapers are powerless to defend themselves against this "theft" and "piracy". Oh, wait - they're not. Instead, they've deliberately chosen to allow in search engines and to profit from the traffic which they generate.
Here's a non-technical explanation. You don't have to allow your site to be indexed by search engines. If you don't want your site to appear on Google you can use a simple file, known as robots.txt, which tells search engines what they can and can't do. The Examiner has one, as does the Irish Times and the Independent. So do they tell these "pirates" and "thieves" to keep out? Absolutely not. In fact, all three provide sitemaps for search engines which summarise their sites and make them easier to index, while both the Irish Times and the Independent provide specific instructions for the "Mediapartners-Google" searcher. Why do they do this? Because of a business decision that they benefit from the readership which added visibility in search engines generates.
The tone of the piece in the Examiner is entirely deceptive: far from being the helpless victim of "theft" and "piracy", the newspaper has chosen, for its own commercial advantage, to allow its site to be indexed and to benefit from the resulting visitors. Should the newspaper object, it is free to opt-out at any point. But it is shoddy work to misrepresent the position to its readers in this way.
Work generated through effort, skill, imagination, professionalism, and usually considerable capital investment, is pirated by businesses with no connection to the creative process as a means to win revenue without risk or outlay. This process is hardly different to what we more commonly describe as theft. The scale of the piracy is astounding. In 2010, while every media company in the country shed jobs and cut costs to the bone, a single search engine operating in Ireland offered around 150,000 newspaper articles that cost publishers an estimated €46.5m to generate. Last year that site offered more than 350,000 articles at a cost equivalent to more than €110m. And all without paying one cent to those who created those articles.There's more in the same vein from the Irish Times and the Independent.
Incredible, isn't it, that the newspapers are powerless to defend themselves against this "theft" and "piracy". Oh, wait - they're not. Instead, they've deliberately chosen to allow in search engines and to profit from the traffic which they generate.
Here's a non-technical explanation. You don't have to allow your site to be indexed by search engines. If you don't want your site to appear on Google you can use a simple file, known as robots.txt, which tells search engines what they can and can't do. The Examiner has one, as does the Irish Times and the Independent. So do they tell these "pirates" and "thieves" to keep out? Absolutely not. In fact, all three provide sitemaps for search engines which summarise their sites and make them easier to index, while both the Irish Times and the Independent provide specific instructions for the "Mediapartners-Google" searcher. Why do they do this? Because of a business decision that they benefit from the readership which added visibility in search engines generates.
The tone of the piece in the Examiner is entirely deceptive: far from being the helpless victim of "theft" and "piracy", the newspaper has chosen, for its own commercial advantage, to allow its site to be indexed and to benefit from the resulting visitors. Should the newspaper object, it is free to opt-out at any point. But it is shoddy work to misrepresent the position to its readers in this way.

