Tyler’s Toolbox

Avatar

“This isn’t exactly Rocket Surgery”

Social data for search giants

I had a good conversation this afternoon with a friend who works at Google, and we touched on ideas about how the world of search and the world of social might collide.

We agreed that no one’s going to beat the established players in search, and if/when an entrant does beat Google, it will be with a disruptive technology that uses a very different method of information discovery. Executives at Google have all read Innovator’s Dilemma and they are rightly concerned that real-time search and social search (and soon, geotagging) could represent this disruptive technology within search. The company is focused on how those movements might interact with search and have experimented with including real-time content in search results.

I’ve got several ideas on how real-time and social data could affect search, in particular, the conversation reminded of an email chain I had contributed to this summer; I’ve pasted my bit below:

Here’s a more complete description of the search idea I mentioned to you today.  Aside from some of the interesting strategies available to the big 3 search providers, I think Facebook has an opportunity to play an interesting role in search by offering a service adding social data to search results.

Facebook could offer a Facebook Connect implementation specifically for search engines that allowed them to check URLs against a database of friend’s posted links.  This would allow the search engine to enhance relevancy. Think of it like this (forgive the quick/ugly mockup):  http://skitch.com/tylerwillis/bswcr/presentation2

This could help Facebook move the needle on three strategic goals: increase engagement, increase ubiquity of graph availability (connect), and user growth.
- Offers a quick/easy way to gain influence in a new area of user’s habits, namely search.
- Easily the biggest Connect implementation to date if done with Bing, Yahoo, or Google. Huge legitimacy marker for Connect’s capabilities.
- It offers a large, visible step towards FB becoming ubiquitous, which would have a positive effect on new user signups.

It would be great to make a launch partner of a top-notch engine (Bing or Google). Biggest problem would probably be building something that served results fast enough for Search Engines to actually use.  Only real question I can come up with is whether Facebook would be willing to be part of a search solution of companies they see as competitive (Google specifically) – IMO, the benefits laid out above outweigh the costs of propping up a future competitor’s current solution.

Bottom line: John Borthwick sees an oncoming verticalization of search; my answer to Borthwick’s claim is the part of my argument I hold most tenatively, but, for most people, I believe that content isn’t interesting just because it is now, social, or nearby; our information discovery platforms must still apply good filtering and context to it’s content/results to meet a user’s needs. Anderson’s Vanishing Point Theory alone isn’t enough to build a universally interesting news application — one has to apply other metrics to judge it’s interest to a reader. Innovator’s Dilemma has shown us that the disruptive technology will get there soon enough (Individual services building upon their success with consumers who care a lot about now/social/nearby and getting better and better at relevance for the mass market), but it’s also shown that large organizations can adapt to disruptive technology by either buying a smaller organization that’s kept independent and encouraged to continue growth (Youtube, attempted with Twitter) or by pushing an corporate shift despite likely stiff internal/organizational resistance. Google failed to buy twitter, therefore it has to push forward with the latter option.

Using the type of UI displayed in the rough mockup I included in the email (link), Google could add social (or realtime or location) data as a contextual meta-layer. I can tell that Google is already thinking of this meta-layer, because it’s exactly the layer that sidewiki is writing to (conversations occuring about information found at a website address). But, Google has misstepped with Sidewiki by trying to own the input. Our conversations are fractured, and occur all across the web, Google should instead focus on indexing and storing data from other services (Twitter, Facebook, etc.) and using that information to display that content as a meta-layer on top of search results and/or to reorder search results. Luckily for the goog, they’ve got some experience with indexing and displaying content from disparate locations.

This idea is complimentary to my opinion about bit.ly being a great asset for a search engine to pickup.

Bit.ly’s relevance to Search Engines

Twitter’s foray into search (through last year’s acquisition of Summize) has been commented on by every pundit under the sun. Twitter Search is has proved the benefits of real-time search — namely quicker access to feedback, which gives the ability to respond and steer the person’s experience.

Businesses have found this useful, the limitations of real-time search (RTS) have kept it primarily out of consumer search habits. Twitter today could make a lot of money by sitting as the platform for feedback and service between businesses and consumers, but to justify their recently rumored $1B valuation, you can bet solving the limitations of real-time search (RTS) as it applies to consumer search is on their radar.

The big problems with RTS revolve around the inability to distinguish noise from signal in the short term. Just being the most recent doesn’t make you the most interesting, but it does make you more interesting. The hard problem to solve here is how to factor timeliness into the algorithm for search relevancy. Twitter Search currently understands timeliness, and understands the basics of how people are voting with their actions (although, trending topics is just scraping the top layer of something that needs rich click AND publish data to be interesting), but it doesn’t quite have the larger search algorithm game figured out. Twitter has to internally solve the problem of bringing in more standard search knowledge and expertise.

Bit.ly, a startup that shortens links for use in micro-updates, has developed an incredible (and timely) database on how many people are posting links, how many of those people are unique or simply reposting from someone else, and how many people are clicking them. That data could be very interesting to Twitter as a way to dig deeper on trending, but it could be even more interesting to someone who is trying to work real-time information into an existing search algorithm. Do we know any companies that are very concerned about making sure they are on top of the next innovation in search? Google comes to mind; Microsoft Bing should be paying attention too.

Twitter could benefit from bringing bit.ly in house, but I don’t think they’d benefit as immediately from acquiring bit.ly as Google or Bing would. Not that bit.ly has to sell, but, they’ve only raised 2M, could likely get a large sticker price, and could get to help reshape a search service relied on by hundreds of millions of people. If I were BD at GOOG or MSFT, I’d be starting conversations.

Bonus: Twitter, I’m not just dishing out free BD advice to the big guys, you get some too! Buy or build a service similar to Blippr and automatically give feedback on products based on what people are saying on your service, then, license this rich “micro-review” data to companies like Amazon and RichRelevance.

Come see me speak, Oct. 6th in Sunnyvale

Digesting Twitter: Short Tips for Morning Catch-Up

Often I’ll find myself spending 20-60 minutes a day in a situation where I have some time to kill and my phone. Given the value I get from Twitter (breaking news and friend updates mixxed together) and the way I use twitter (not reading every tweet, just catching it when I can), I’ll often use this time to twitter.

Twitter is famous for cell phone use, but more for SMS – which is not my style; I use three tools: m.twitter.com, TwitterBerry, and my mobile browser. Given that m.twitter doesn’t record where you where in the updates, visiting links as they look interesting and then going back and finding your place can be a real hassle.  I create a twitter digest. I go through and make an email draft of links that look good, notes i want to keep, or ideas I have. Here’s an example of yesterday morning’s digest:

http://tinyurl.com/34bwp4

http://tinyurl.com/2qy4sz

http://tinyurl.com/2qr6a8

http://tinyurl.com/399qn9

http://twitter.com/confession

jowyang: Quotable looks about half accurate for this thread http://tinyurl.com/3yzcol 3 minutes ago

http://tinyurl.com/3bkael

http://tinyurl.com/3xhmbq

http://mashable.com/2008/03…

http://tinyurl.com/38yh6c

http://tinyurl.com/2qmy3p

http://tinyurl.com/2zsd3f

http://tinyurl.com/2l9sns

http://tinyurl.com/3du44p

http://snurl.com/22gpo

apenny: Tyler Perry (20 Mill movie this weekend) is such a fabulous rags to riches tale.

http://www.google.com/searc…

http://tinyurl.com/3384fe (doesn’t display on blackberry? Wtf ?)

http://tinyurl.com/3dw4tu

rycaut: I wax a bit surprised to OH people complaining about the nudity which I think says more about Americans than the filmmakers about 11 hours ago
rycaut: btw The Bank Job was quite good if not particularly surprising (not The Usual Suspects but a fun period crime caper) about 11 hours ago

Follow @garyvee @warriors @Pistachio?

–8pm last night–
Sent via BlackBerry from T-Mobile

As you can see, it’s mostly links, mixed in with notes about two films I want to see and a service I wanted to try but had yet to. I wanted to reply to some messages that I saw while making this digest, when that happened I switched to TwitterBerry (so as not to lose my place) and posted the reply. Then it’s simply an action of visiting these sites in order until something else needs to be done – if I get a chance to do something more productive, I email the unfinished list to myself for checking when I get back to my machine.

Creating this digest takes me 10-20 minutes depending on volume and interestingness of my friends, and it’s easy to make sure I see everything interesting tweeted over a period of time. Also, in case you’re wondering, the above digest represents 13 hours of activity.

,










All your rights are belong to me.