Powerset and Powerlabs, roundup

In previous posts, I liveblogged about Powerset's presentation this evening of Powerlabs to a select group of journalists, technologists, and bloggers. The big idea is that Powerlabs (launching in September, ahead of Powerset's search engine) will be a Digg-like site where community members can suggest and vote on Powerset features. Powerset aims to be incredibly open - as Steve Newcomb joked, the only thing stealth about them is that they're in stealth mode.

What Powerset is shooting for is ambitious, and has the potential to greatly improve how we find information on-line. Google has a lot of talent and smarts, but all the major players are doing variants of the same thing - statistical analysis on top of keyword search, an idea that goes back decades to Salton's work on document indexing. Powerset's approach has its roots too - decades of linguistic research and development at Xerox PARC - but turning even the best research platform into an internet search engine requires a lot of work. One example - the core engine they licensed originally took over a minute per sentence to index Wikipedia entries - now, with optimizations, it's down to a second or less. Still pretty CPU intensive, but as Steve Newcomb pointed out, indexing costs are small compared to the normal runtime costs of a popular search engine. At Google scale, Powerset would be profitable even with the increased compute needs.

Steve also made a point of saying that Powerset has never called themselves a Google killer. Still, they're trying to do something that's very cool. If they can do what they demod tonight on a grand scale, I'll switch.

The Powerlabs Crew

The Powerlabs Crew
Originally uploaded by John Griffiths.

Developers from Powerset at the Powerlabs event tonight.

Scott Waterman and Ethan Stock at the Powerset demo

Sticker swap
Originally uploaded by onohoku.

Scott Waterman (Powerset) and Ethan Scott (zvents) at the Powerset dem. the Zvents office

Powerset demo

Part two of my liveblogging from Powerset. They're talking about how they differ with respect to indexing. This helps them with both matching documents and ranking them.

In indexing, they parse each sentence on the page. For example:
'Sir Edward Heath died of pneumonia.'

Here's how they index this sentence.

- extract entities and semantic relationships.
- - expand to find similar entities and abstractions

-In this phase, they understand that:
1. Sir Edward was a UK prime minister - a politician
2. pneumonia is a disease
3. if you died from something, you were killed by it

This is a big change from the search we know (Google). It lets the user phrase their query in a lot of different ways. For instance, Powerset can answer the following:

- 'what killed edward heath'
- 'which prime minister died of pneumonia'
- 'what was sir edward heath killed by'
- 'what politician died from pneumonia'
- 'politician died from disease'

Powerset has so far indexed NY Times corpus, Wikipedia, and is working with Freebase.

Liveblogging from Powerset's Powerlabs demo

Powerset is giving a demo for about 35 partners, journalists, and bloggers right now in San Francisco. I'll be liveblogging this event.

The core team members are introducing themselves now. John Lowe from AskJeeves, Tim Converse from Yahoo, Kevin Clark who's their lead Ruby developer, along with a lot of other Powerset linguists and developers in the room.

Steve Newcomb is about to go into the demo. More in a bit.

Powerset's tantalizing Powerlabs annoucement

Powerlabs Screenshot
Originally uploaded by official_powerset.

Powerset is about to make an announcement of their Powerlabs project. It's going to be a site where they solicit feedback from users in different search domains (travel, entertainment, even porn) about how they use the internet and how they'd like to see search work.

There's talk that they'll also provide access to Powerset search for the lucky users who get to be part of this test.

Keep an eye on Powerset founder Steve Newcomb's blog later today for the official announcement.

Update: the announcement just came out: Powerlabs: the first screenshot.