| | |
|  Objectives | |
| | |
 |
To mine and organize vast amounts of unstructured data.
As the total amount of publicly available information continues to grow, the need for data mining techniques is growing as well. Search engines are currently meeting this need to a certain extent, but I believe we have only scratched the surface. The data mining applications of the future will become increasing sophisticated within, allowing for greater accuracy and better functionality for the user.
In my final project at the University of Bristol, "Improving Web Search with Reinforced Heuristics", I was able to increase search engine accuracy with three personal innovations. First, I applied an adjustment to word weights and PageRank link weights based on an estimate of the probability that a user would reach the part of the page in which a word or link appears. Then I combined the new PageRank values with values that were reinforced with rewards or punishments according to where they appeared in search results, in what order they were clicked, and how much time passed before a user returned. Finally, I created a new Levenshtein-like distance algorithm which applies phonetically-logical substitution/insertion/deletion weights, and I used it along with frequency data to build a spelling suggestion system that appears to have greater accuracy than some major search engines. Altogether, this project showed me that building a high-quality search engine is a worthy challenge, and I am determined to work for a major search engine company so that my efforts to meet meet this challenge can directly benefit the world of web search. |
|
|
To apply machine learning techniques to challenging problems.
Machine learning is a relatively new field, and the challenging problems of web search are even more recent than many machine learning algorithms. Furthermore, the number of applications for machine learning techniques will only increase as time goes on. Therefore, I am excited to find new ways to use machine learning algorithms, and I know that the sky is the limit.
In my final project at the University of Bristol, "Improving Web Search with Reinforced Heuristics", the method for adjusting search results with clickthrough information was based on the machine learning concepts of reinforcement learning, ant colony optimization, and collective intelligence. If I had more time to explore the problem, I would have found appropriate uses for even more machine learning algorithms. While working for a major search engine company, I hope to continue this type of work. |
|
|
To have a positive effect on some aspect of peoples' lives.
The goal of any good programmer is to create applications that will help users to perform tasks with greater ease. In the case of web search, the benefit to the user is quite extreme for two major reasons. The first reason is that it would be physically impossible for any single user to ever read a fraction of the information that is indexed by a major search engine, so there is no unaided human equivalent to the task. The second reason is that search engines are used more than most other types of applications.
In my final project at the University of Bristol, "Improving Web Search with Reinforced Heuristics", the engine I built was only released for 16 days, and it responded to 2,119 queries. The effect of my work on a major search engine can be far greater, so I look forward to being able to have that chance. |
|
|
