We're not the media if we depend on Google (or Amazon)

Update: 25 January 2013. This morning when I typed Catalonia into the Google Search box, and then clicked the News tab, all of the stories about the Declaration of Sovereignty were hidden behind an even smaller "+ Show more" link below the generic AP post carried by the Huffington Post. If I click on "All 88 news sources" I get this complete fail:



Maybe there is a conspiracy!

I've also changed the headline of this article from "We're not the media if we don't appear on Google (or Amazon)" to "We're not the media if we depend on Google (or Amazon)". We need an alternative.

---

Yesterday there was a news story I was particularly interested in. But searching for it on the internets made me rethink a lot of assumptions about the idea that "we are the media". I first heard that expression used by Amanda Palmer, a singer/songwriter/artist who ran an incredibly successful crowdsourcing campaign a few months ago—raising over $1 million—to fund a new record. One of her most intoxicating claims was that she didn't need to sign with a studio, and she didn't need to follow a studio's rules and directives. With the support of individual people—me included—she could put out her record and go on tour. In short, she and "we are the media".

I've been writing books for the last 20 or so years with the same idea in mind. My HTML books helped people realize that they could publish their ideas, their writings, their photographs, their lives without having to ask for anyone's permission. At first, I published a gallery of these brand new websites, and I loved going through them and seeing democracy in action: all these people sharing their thoughts with the world, with no gatekeepers to stand in the way.

The gallery quick grew to multiple pages of lists, and the web to billions, and so I left the job of cataloging these new creations to the brand new Google. I'll get to that in a moment.

A few years ago, when I discovered EPUB, I had the same feeling of democratic excitement: here was a technology that could help people publish and otherwise share their writings. We humans yearn to share our stories with others. EPUB, an open format, facilitates such an interchange.

But yesterday, it really hit home how far we are from complete democracy of content. And that's because though we may have the technology and even free tools for creating webpages and ebooks, there are massive economic and political powers that stand between us and our readers. We cannot ignore them. Let me show you yesterday's example.

First, let me admit that the word I typed into Google was "Catalonia". But the ramifications go way beyond this particular news item. So, please bear with me as I use my favorite example to illustrate something much larger.

When I type in "Catalonia", Google says there are 83 news items. When I ask Google to show me "all 83 news sources", it gives me a list of only 16. You can try this at home.

Down at the bottom of the page, Google has a little disclaimer that says it omits some entries that are "very similar to those already displayed". And this is probably not the right time to discuss wire services and the sad fact that a single mediocre story with no byline about a small, new far away country might be reproduced verbatim, with no fact chacking, in 80 or so American newspapers. But I digress. But if you click "repeat the search with omitted results included" Google still only displays 25 items. There is no way to see all 83.



I would very much like to know which other 58 news sources are reporting on my topic. Why won't Google show me? And how does it decide which 25 items to show? And isn't it a problem that I am necessarily limited by Google's criteria?

Do I think there's a conspiracy to keep the other 58 news sources from being revealed? Probably not, but regardless, it bothers me quite a bit that there's no way to get to them. Like it or not, we are not the media, if we cannot be Googled.

And it doesn't stop there. Amazon recently decided that authors of the same genre books cannot review other authors' books. Amazon became successful because of its review system. I am not the only person who used to look at Amazon reviews in order to decide which book to find from the library. Amazon's reviews were an amazing crowdsourcing adventure, now run amok by enterprising and back-stabbing authors on the one hand, and Amazon wanting to watch the gates, on the other.

And don't even get me started on Amazon search. How much publisher money is behind Amazon's search results? I have no idea, but I'm guessing it's not insignificant. We are not the media if we don't come up on Amazon.

The Amazon review issue brings up yet another aspect of this whole story, which I touched on in an essay titled "If you know it already, don't tell". The basic gist of that article is that we believe people more when we don't think they have an axe to grind, even if, ironically, as in the case I describe, the person isn't particularly knowledgeable at all about the topic.

This is the same conundrum that Amazon's new policy throws at us, and that Porter Anderson confronted in his recent post in On the Ether. Where is the line between enough knowledge about a subject in order to give a reasoned opinion and so much knowledge about the subject that your opinion is biased? In this day and age when every one's past opinions are documented publicly, can there even be unbiased opinions? I think not. I believe that the important thing is to know what the opinion is, so we can use that information to assess the validity of the opinion. Suppressing all the opinions that Amazon has decided fall into a particular category of user solves nothing.

In these heady times in which self-publishers, and the web designers that came before them, are gleeful with democratic information sharing, we must be absolutely aware that the gatekeepers have simply moved the gates. Whereas in earlier eras we couldn't even publish without their help, now we can publish, only to find our sites and books hidden in a sea of information. Even if your book or website is in there, if Google or Amazon or whatever search product of the moment won't bring it up, it's as if it weren't there.

All is not lost. Twitter, and to some degree Facebook as well, are still amazingly democratic, allowing us to connect directly with kindred spirits all over the world. While we can't forget that social media still generates only a fraction of web traffic (on the order of 5-6%), I think its influence goes far beyond those numbers, if only in its ability to connect people and movements. But will they last? And Kickstarter (and for example, the Catalan crowdsourcing site Verkami) are great at helping us harness the power of hundreds (or thousands if you're Amanda Palmer) to offer alternatives to these gatekeepers or, when need be, attract their attention. Technology-wise, the use of accurate metadata can also help make a book or website "findable".

Ironically, it is the very birth of Google that gives me hope. Way back then, Altavista was the most popular search engine, and I remember being increasingly frustrated with its ads, its promoted links, and the seemingly deliberate obstacles that it placed between me and what I was looking for. This clean, lithe little ad-free Google came along and all of a sudden we could find things again. It felt like the whole world changed over to this little search engine that actually worked in a matter of weeks. Perhaps it's time for some new disruption.

And while we wait (or develop) the tools to maintain our democratic web, we must be careful to be aware that right now there is no unbiased opinion, and no unbiased search.

Links to Maps from iBooks

A reader asks what happens with the Map links in the Barcelona Beyond Gaudí book now that iOS no longer has the Google Maps app. It's a good question, so I thought I'd answer it here.

In the Barcelona Beyond Gaudí book, I targeted the map request to Google:

<p><a href="#left" id="caption">A spiral decoration on the gates to Sant Pau Hospital. </a><a href="http://maps.google.com/maps/?q=loc:41.411403,2.174297+(Spiral%20Decoration%20on%20Gate)" ><img src="images/map.gif" alt="map button" id="mapbutton" /></a></p>

And it looks like this:



When your reader clicks the Map link in iOS5, iBooks passes them to the Google Maps app. In iOS6, iBooks passes such links to the Google Maps page, in Safari:



And the reader can still click on the Directions or Street View buttons at the bottom. Here is Street View:



Then a quick four finger flick, and the reader can jump back to iBooks.

To send the reader to the Apple maps app instead of the Google Maps page, you just have to target the link to the Apple maps server:

<p><a href="#left" id="caption">A spiral decoration on the gates to Sant Pau Hospital. </a><a href="http://maps.apple.com/maps/?ll=41.411403,2.174297" ><img src="images/map.gif" alt="map button" id="mapbutton" /></a></p>

Then when your reader clicks the Map link, iBooks sends them to the Apple Maps app (without asking if they want to leave the iBooks app, one should note).

Note that the Apple Maps app doesn't mark where the location is on the map:



And there doesn't seem to be a street view, though you can click the page curl in the bottom-right corner to view Satellite view and then zoom in.



You can get more information about creating links to the Apple Maps app in the iOS Developer Library.

Google Alerts ≠ Google

For many years, the only news that ever made it into English-language papers about Catalonia had to do with food, travel, or on a lucky day, Catalan superstar architect Antoni Gaudí. I even started a blog, and got permission from a number of online news sites so that I could take matters into my own hands and translate the articles that I thought deserved wider attention in the non-Catalans speaking world. I remember being particularly outraged that Spain's Minister of Transportation had decided to build a high-speed rail line directly under (and I mean directly under), Barcelona's famed Sagrada Família, and it was barely covered in international papers (though today I did find this one article in the New York Times, Murphy's Law).

Part of the problem is that foreign correspondents have, until very recently, concentrated themselves in Madrid, thinking (erroneously in my opinion) that that was a reasonable vantage point for covering Barcelona. But, it's pretty obvious if you've read any of their articles that the city you live in makes a difference on the information and the opinions you collect and then relate. Imagine covering the US election from Moscow, or Havana. You just don't get the same story.

But in the last three weeks, since Catalonia's massive demonstration of a million and a half people, and President Mas' subsequent call for a referendum on a new sovereign state for Catalonia, I haven't been able to keep up. There have been hundreds, even thousands of articles in English-language newspapers.

And while trying to keep up with it all, I've learned a few things about Google and Google Alerts that you might find useful even if you're not so interested in my adopted country.

You don't need me to tell you what Google is. Google Alerts is less well known: it's a Google-based service that offers to "email updates of the latest relevant Google results (web, news, etc.) based on your queries" and at the intervals "daily, weekly, when it happens" that you specify.

The problem, however, is that Google Alerts does not produce the same results as Google, and in particular its results aren't nearly as good or complete. Since September 11, my Google Alert for "Catalonia, Catalunya and/or Catalans" has resulted in all of 15 results. 15. Part of the issue is that Google Alerts (like Google) groups articles into categories, and just shows the first of many. For example, here is the Google Alert that I received on September 12, the day after one and a half million people marched in Barcelona for independence:

There are three news entries, and each one has a "See all stories on this topic" link. And there's a bizarre rant from some website. But I thought Google Alerts was supposed to show me all the stories? It clearly does not.

If I go to Google and search for Catalonia (by itself), even today on October 1, I can travel back to the entry on September 11, and see that there were actually 240 articles, and Google itself shows me the summaries of seven or eight:

How are Google and Google Alerts grouping articles into categories? It seems a bit random. Syndicalization might be one criteria; when the exact same article appears in various news sources. I recognize that there's a lot of that. AP articles are obviously picked up by multiple newspapers, but a lot of other papers share articles as well. I found the same exact article in The Telegraph (in the UK) and in the Sydney Herald (in Australia).

But that's not the only criteria that Google Alerts uses, at least in my recent experience. It also seems to lump together articles that are completely different.

So, my question is if you're asking Google Alerts to give you all of the pages about a given topic, what good does it do for it to send you an email with a link to just one of them, saying you should search for the others? Isn't that what Google is for?

The second issue is that Google Alerts doesn't get them all by a long stretch. For example, I am writing this at 12:09pm on October 4th. At 10pm on October 2, a full 36 hours ago, Google showed me that the New York Times had just published a hugely influential, detailed op-ed about Catalonia, called Spanish Prisoners. I have yet to receive any word about it from Google Alerts. Today, there is a new article about Catalonia in Bloomberg Business Week. There has been nary a peep from Google Alerts, even though it promised to send me links to a news story "when it happens".

In short, Google Alerts doesn't do what it promises. It sends what it considers a few representative articles and a link to additional ones, and often misses the most important stories. If you want to find all the important articles on a topic, you're much better off going to Google yourself.

And if you're interested in Catalonia, I've got the best news roundup around, thanks to Google, but not Google Alerts :)

Google Maps invents own Spanish names of Catalan streets, rendering the program useless (and incredibly offensive)

This morning on Twitter, I saw that Google had decided to replace Catalan placenames with crazy, sometimes random, translations into Spanish. The hashtag #googlecat has quickly become a TT (trending topic) on Twitter as people around Catalonia are outraged that Google has decided to call Catalan places however they like.

Pretty incredible. You, dear non-Catalan reader, might think it's much ado about nothing, really, how much difference is there between "Plaça Catalunya" and "Plaza de Cataluña"? Who cares? But there are very large issues here, which I hope to address one by one.

First, the names of streets, squares, and roads are all in Catalan in Catalonia. If you're looking for a street, the only street sign you will find will be in Catalan. Google Maps won't do a fat lot of good to you if it's giving you a translation into Spanish. For example, one of the streets near my apartment is “Carrer de Sant Domènec" (or Saint Dominic street) but Google has it labeled as "Calle de San Domingo".



(Then, there's the added weirdness that you can't actually say "San Domingo", it would be "Santo Domingo".) There are many, many other bizarre translations. Things like "Verga María" [Cock Mary] (and I don't mean roosters) for "Verge Maria" [Virgin Mary]. Or changing the name of the town, “Sant Boi de Llobregat” into "San Baudillo de Llobregat".

Think it won't matter, that Catalan and Spanish are "close enough"? Try "Calle del Oxidado" instead of "Carrer Rovellat". Or "Rambla de la Colina" instead of "Rambla del Turó". Let's just say I wouldn't rely on Google Maps to get around Catalonia any more.



Vilaweb was finally able to get a first explanation from Google Spain about the problem. “The problem has to do with a database that was supplied by a third party and the technicians are working on updating it."

Ultimately, to Catalans who are continually feeling mistreated by the Spanish government, who will not stand up for the Catalan language in Spain, the European parliament or anywhere else, it feels like a crass political move by Google. Perhaps it's just a technical error, but I just don't have that faith anymore.

Interesting that names in English have not been translated into Spanish, as in Plaça John Lennon, which turned into "Plaza John Lennon" but not "Plaza Juan Lennon".

Check into Twitter and follow #googlecat to see egregious examples and anger. Follow Vilaweb for the latest news.