Nano2Sol by Marc Boucher

Next week I’m headed to New York for the O’Reilly Money:Tech Conference billed as “Where Web 2.0 Meets Wall Street.” The conference speaker list is impressive. I see this conference as a potential opportunity to extend Hyperix Search into the vertical finance segment.

Over at Mashable their reporting today that;

“Reuters will publicly discuss their new initiative around a social network for traders; LinkedIn will introduce a new way for members to create connections and networks with experts by industry; Stormwatch will come out of stealth mode to unveil a project in the financial service industry related to tracking sentiment indicators in real time; Eventvestor, a new ad-driven service for aggregating and tracking financial events, will launch; Motley Fool will present evidence that the collective analysis of the community, and their own analysis, is wisdom not to be dismissed.”

And if you want to go Mashable has a discount coupon to save you 20% off the registration.

Greg Linden has an excellent post about MapReduce and how some database gurus view at as “a giant step backwards”. Personally I don’t buy it and we’re implementing something similar for Hyperix. Here’s an excerpt of what Greg had to say.

“The comments on the post are enjoyable and useful. Many rightfully point out that it might not be fair to compare a system like MapReduce to a full database. DeWitt and Stonebraker do partially address this, though, by not just limiting their criticism to GFS, but also going after BigTable.

The most compelling part of the post for me is their argument that some algorithms require random access to data, something that is not well supported by GFS, and it is not always easy or efficient to restructure those algorithms primarily to do sequential scans.“

A lot of notable writers are touting 2008 as the year Vertical Search Engines really hit the mainstream. I’m a believer otherwise I wouldn’t be working on a vertical search platform.

Here’s some of the posts from the last couple of days;

From AltSearchEngines - 75 per cent of online publishers see vertical search as way to reclaim online community from Google

“Nearly three quarters of online publishers see the benefit of developing vertical search engines as a way to claw back online communities from Google, a study published last month has claimed.”

From John Battelle’s Blog - Blekko

“The web is big. Really, really big. It’s literally billions and billions of pages. It’s Carl Sagan big. And it’s doubling in size every year or two.

So the idea that what you can see in positions 1-3 above the fold on Google are the sum of what the web has to say about every possible query is crazy.

And yet they have 85%+ market share, and little effective competition. At the same time there is such a fabulous business in search. It’s the highest monetization service on the web, by far.”

(more…)

As some of you may know one of my other projects is Project Phoenix which is an effort of my newly created company Hyperix Search, Inc. We’re focusing on a vertical search platform. One important aspect of any serious search platform is the need for a distributed file system and it’s map and reduce operations.

MapReduce is a programming model and an associated implementation for processing and generating large data sets and was originally created by Google. For some time now other companies and efforts having trying to replicate and improve on the process that Google created. This video takes you through Microsoft’s version of MapReduce called Dryad.

Recently Read/WriteWeb started a new feature on alternative search engines. The editor is Charles Knight and his latest Top 100 list for August is out.

“Some of them did not even exist a year ago. One of my goals is to show my readers the “latest and the greatest” search engine innovations. The motto for the blog [ASE], after all, is “the most wonderful search engines you’ve never seen,” and my favorite comment of all is, “Wow! I didn’t even know that most of these existed!”

Once I have our first vertical search engine in production it will be fun to watch how it fares on the top 100 list.