January 2008 Archives

Next week I'm headed to New York for the O'Reilly Money:Tech Conference billed as "Where Web 2.0 Meets Wall Street.” The conference speaker list is impressive. I see this conference as a potential opportunity to extend Hyperix Search into the vertical finance segment.

Over at Mashable their reporting today that;

"Reuters will publicly discuss their new initiative around a social network for traders; LinkedIn will introduce a new way for members to create connections and networks with experts by industry; Stormwatch will come out of stealth mode to unveil a project in the financial service industry related to tracking sentiment indicators in real time; Eventvestor, a new ad-driven service for aggregating and tracking financial events, will launch; Motley Fool will present evidence that the collective analysis of the community, and their own analysis, is wisdom not to be dismissed."

And if you want to go Mashable has a discount coupon to save you 20% off the registration.

Greg Linden has an excellent post about MapReduce and how some database gurus view at as "a giant step backwards". Personally I don't buy it and we're implementing something similar for Hyperix. Here's an excerpt of what Greg had to say.

"The comments on the post are enjoyable and useful. Many rightfully point out that it might not be fair to compare a system like MapReduce to a full database. DeWitt and Stonebraker do partially address this, though, by not just limiting their criticism to GFS, but also going after BigTable.

The most compelling part of the post for me is their argument that some algorithms require random access to data, something that is not well supported by GFS, and it is not always easy or efficient to restructure those algorithms primarily to do sequential scans."

A day after the MDA deal to sell its space division to ATK we learn that the Canadian Space Agency president Laurier Boisvert resigned in December and was replaced on January 1st by Guy Bujold, an assistant deputy minister at Industry Canada.

The Canadian Press is reporting that Mr. Boisvert may have left because of the pending MDA deal.

"He was officially replaced by Bujold this past Jan. 1, but the resignation was communicated to CSA staff on Dec. 20.

There has been speculation Boisvert may have quit over the planned sale of MacDonald, Dettwiler and Associates Ltd.,a Vancouver-based company known as MDA, to Alliant Techsystems (NYSE:ATK) of Minneapolis."

Even if this was the case which in itself begs for more questions to be asked, why are we only learning of this now?

Yesterday's acquisition of Canada's MacDonald Dettwiler and Associates (MDA) by Alliant Techsystems (ATK) is bad for Canada for so many reasons. It's reminiscent of when Avro Canada closed down and most of their talent headed to the US and were instrumental in helping them with the Apollo program. While MDA is not shutting down operations in Canada it should be asked what the long term viability is for them in Canada. After all the Canadian dollar is stronger than the US dollar and the Canadian government seems indifferent to sustaining a space sector made in Canada.

MDA is the leading space company in Canada. Best known for the Canadarm which it acquired from another Canadian company, Spar Aerospace in 1999, MDA has led the way in Canada for the space commercialization sector. It's also known for its satellite systems which included the recently launched Radarsat-2. Their planetary robotics division had been growing and the Canadian Space Agency had been hoping that it would be a key contributor to the European Space Agencies ExoMars rover by having Canada build the rover through MDA. Even if the government had changed its mind and approved this plan it seems unlikely to do so now that MDA would be under foreign control.

Yesterday the Mars Institute began initial building on its island in Second Life. While public access is not available just yet, we plan on opening up the area soon. Details of what we plan for the island will be released as soon as possible. But we are looking at a presence for the Haughton-Mars Project and will be collaborating with other groups.

A lot of notable writers are touting 2008 as the year Vertical Search Engines really hit the mainstream. I'm a believer otherwise I wouldn't be working on a vertical search platform.

Here's some of the posts from the last couple of days;

From AltSearchEngines - 75 per cent of online publishers see vertical search as way to reclaim online community from Google

"Nearly three quarters of online publishers see the benefit of developing vertical search engines as a way to claw back online communities from Google, a study published last month has claimed."

From John Battelle's Blog - Blekko

"The web is big. Really, really big. It's literally billions and billions of pages. It's Carl Sagan big. And it's doubling in size every year or two.

So the idea that what you can see in positions 1-3 above the fold on Google are the sum of what the web has to say about every possible query is crazy.

And yet they have 85%+ market share, and little effective competition. At the same time there is such a fabulous business in search. It's the highest monetization service on the web, by far."

Since we're getting closer to the public unveiling of the first vertical search engine produced from the Hyperix search platform I think it's time to release a couple of tidbits about the platform. Hints have been out there for a while and some people within the search community are already aware of this, but one of the primary components of Hyperix is the use of Hadoop.

Here's the quick intro of what Hadoop does from the open source site:

The US National Institutes of Health (NIH) will now be required to make public all of the research it funds 12 months after it's published. For the vertical search space this provides a huge boost as we'll be able to crawl the complete text of papers and make useful searchable products from them.

ReadWriteWeb brought this to my attention in their post NIH: $29b in Health Science Set to Go Online for Free. Here's their take on it;

This should open up a whole world of new opportunities for online research. Readers outside of the academic world but aware of the financial future of health information online in the commercial sector can imagine the analogous excitement about this announcement for academic researchers.

Over at the Yahoo! Search blog Sharad Verma recaps WebmasterWorld's Pubcon. I could not attend but it sounds like I missed a good conference and relevant keynote for Hyperix. It's nothing new to me but it's nice to see other people talking about it.

Noteworthy Keynote

I thought that Richard Rosenblatt's keynote on Wednesday delivered sound insight. According to Richard, most content online today is about sports, politics, news, and other common topics, leaving long tail topics underserved. He emphasized that there is significant demand for quality content in the long tail and therefore an unaddressed opportunity to create content and capitalize on the monetization opportunities. Susan Esparza at Bruce Clay, Inc. lived blogged from the address, where Richard revealed, "The old model was about owning a generic domain name (pets.com). The new is that the search engines don't care where you are. Get a one or two word domain on a nontraditional domain. Target the wide body and the long tail."

For my first post in 2008 I've decided to write a little note about opportunities for vertical search platforms such as the one we're developing at Hyperix.

I've said for years now that while the big search engines do a great job for some things they do poorly at others and that many opportunities exist for those who go the vertical search route as a business model. There's a lot of data out there that can be made into a business product. Today Tim O'Reilly on the O'Reilly Radar wrote about one such opportunity for vertical search in the finance sector in his piece "Predicting Financial Market Performance with Real Time Web Data". To quote Tim;