I'm Feeling Lucky?

The original Google interface, back when they were just a search engine, had a text entry field and two buttons: Google Search and I'm Feeling Lucky. Most of us click on the Google Search button by instinct perhaps because we've been unsuccessful with the IFL button or perhaps because we just can't believe that it could actually work. The IFL button bypasses the results page and takes you directly to the first web site returned in your query. Could save a lot of time, right? If you are looking for a popular or unambiguous page, say "iPod Shuffle," you may expect to be taken right to Apple's iPod Shuffle page and you'd be right (as of the date of this blog). The one person I talked to who actually used this feature said "when I'm doing a search for http://www.nytimes.com/ it always gets me there." I'm sure people who are a bit more accomplished at using a browser use it too....

But any search that is reasonably ambiguous will require some manual filtering to get you to the site you want. Entering "Steve Connolly" will get you not the famous blog you see here, rather, the site of the well known Elvis impersonator (although we've never been seen in the same room together). In fact, since many of our queries are ambiguous, we prefer the option of manually filtering the results before plunging onto a web page. But what if your workflow required numerous nested queries? Requiring a hybrid automatic/manual process would improve accuracy but slow the process and consume valuable resources. One solution is to reduce ambiguity by standardizing the vocabulary; using the standard vocabularies of ICD-10, UMLS in the case of biomedical vocabularies. By agreeing to a standard vocabulary users can quickly determine whether the term "Steve Connolly" writes a blog or croons in a Vegas lounge. Would this put the I'm Feeling Lucky button out of business? Not yet. Although we've agreed to a single definition of the term, there may be many references to it, and these references would be returned in the query. We have however, significantly cut down on the inaccuracies in our results.

The next step is to standardize the associations to references. Google uses the sheer number of links connecting two terms which can be misleading. Google bombs are examples of how these associations can be manipulated; one well known example is the term "miserable failure" which has been linked by many individual Web sites to the biography of George W. Bush, thus, at the time of this blog a query of "miserable failure" will return the biography as the top ranked result. Semantic Web technologies seek to provide more explicit associations between terms, eliminating statistically based results with results that are definitive.

Once we've decided what a "Steve Connolly" is, next we can ask what a "Steve Connolly" does. We've established a subject, then we can query for associations, or predicates, and objects connected to those predicates. Implementing the subject, predicate, object format of the Semantic Web is neither easy nor straightforward and thus the original Web is in no danger of disappearing. But given time and effort the two original Google buttons: Google Search and I'm Feeling Lucky will gradually merge into one: I'm Feeling Google??

Commercialization

After explaining our new software platform to Dr. James Wasmuth, a post-doc at the Hospital for Sick Children in Toronto, he asked me how we could compete with a university that may write a similar software application and offer it free as open source software. His view of the situation was that people would rather use free software than pay for software with the same functionality. His is a common view of software that is based on the assumptions that:

1. the technology is the product
   
2. the value of software is in the code

If a university group, funded by a grant, developed code that could reproduce the feature set that we as a for-profit company had also developed, they would have technology that would be comparable to ours. But technology per se does not have value in the market. Only by commercializing that technology through a product development process can you deliver value to the market. Although the university group may have employed certain commercialization methods; a snazzy user interface, documentation, etc. it is doubtful that they would have employed enough of these methods to make the software truly commercial. Universities neither cultivate the skills nor fund the efforts of commercialization and thus any attempts at commercialization would tend to fall short.

With that in mind, the simple answer to his question was "we don't." Our intent is to provide a commercial software product to the market. A product that has the full suite of commercial features such as proper documentation, high reliability and customer support and training to name a few. If we came across a university group that was developing a similar technology our strategy would be to collaborate; there must be something that they have that our product does not which could lead to licensing their technology in our product.

The second assumption is the old "factory model" of software production; software has the value characteristics of a typical manufactured good. This concept has been thoroughly debunked by Eric S. Raymond in his seminal book on open source software: The Cathedral and the Bazaar. Because software is in constant flux; expanding, adding new features, fixing bugs and adapting to new environments, it is never really "finished," you simply use a single version of a process that will continue on into the future. The value of a single static version of that code is small, what is of much more value is the ability to have access to the continuing process of development.

Barriers to the Semantic Web

Although I'm a big fan of semantic web technology, we incorporate it in our product, it isn't the silver bullet some people think it is. Zack Rosen has written an excellent blog on some reasons why semantic web adoption hasn't taken off.

Applied Research

In contrast to basic research, applied research is directed toward a specific goal; a solution to a problem. Whereas basic research's mission is the advancement of knowledge, applied research is intended to answer more specific questions. Nevertheless, applied research is still "research" in that it is required to explore new areas of knowledge in order to fulfill its stated goal. Applied research does not imply that the target solution will be found; there are often unforeseen obstacles in reaching the target. However, with a specific target directing the effort there is a higher likelihood of obtaining the goal than with a basic research project.

Applied research fits between basic research and an engineered project. An engineered project has much of the obstacles well defined before the project begins. Although a bridge may never have been built in a certain area, the bridge building technologies and processes have been well developed and don't usually involve large amounts of unexplored territory. Drug development can be considered applied research because it starts with the target of creating a certain response in humans but may also have many unknown effects which are found along the development process.

Basic research is conducted primarily by non-profit research organizations, i.e. government labs and universities which do not require a commercial product as the end result of their efforts. Government agencies may fund basic research at for-profit companies, however, for-profit companies often find it difficult to fund basic research since the profit motive precludes the unfettered advancement of knowledge in favor of advancement of knowledge about an area that may discover a profitable product.

Non-profit research institutions often engage in applied research, sometimes as an extension of knowledge or experience gained through basic research. Because of the experience gained though their basic research, the institution is many times the best qualified to carry out the applied research. However, although they are both considered "research," the skills and processes involved in carrying out basic and applied research are quite different and an institution adept at one may be equally inept at the other. Institutions that are skilled at basic research can resolve this deficiency by transferring their technology to a commercial enterprise who will direct the knowledge obtained from the basic research toward a commercial product. Again using drug development as an example; a research institution may discovery a technology, biological target, novel compound or gene through basic research and transfer this knowledge to a company that will use this knowledge to develop it into a drug for treating disease. Although the discovery of the knowledge is a critical part of the process, it is by no means a certain commercial success. Conversely, a company adept in moving an existing discovery through the drug development process in an efficient, consistent manner may spend many years and millions of dollars trying to come up with a single new discovery. This points to one of the many instances of where creativity and productivity often clash.