A Web Search Engine: General Information

Daily we use the Internet and search tools in particular when looking for information. The search results are commonly presented in the form of a list and are usually called hits. The information may comprise web pages, images, data and other types of files. Some search tools also gather information available in databases or open directories. Unlike Internet directories that are maintained by human editors, search engines operate algorithmically or are a mix of algorithmic and human input. Web search engines work by storing data about many web pages which they retrieve from the INTERNET. These pages are retrieved by a web crawler, also known as a spider. It is an automatically-controlled Web browser that follows every link it finds. Afterwards the content of each page is analyzed to decide how to index it. Words, for example, are removed from titles, headings or special fields called meta tags. Data about web pages are saved and stored in an index database for further use in queries. Some search engines, such as Google, save and store the whole or part of the source page (also called a cache) and information about web pages, while others, such as AltaVista, save and store every word of every page they discover. This cached page always contains the initial search text, as it is the one that was actually indexed. Consequently, it can be very helpful because it comprises information that can no longer be found anywhere else on the Internet. When an Internet user types key words in the search field, the engine carries out checks on its catalogue and shows a listing of best-matching web pages according to its parameters, normally with a short summary coupled with the title of the document and at times parts of the text. Some search engines have introduced an advanced option called proximity search which allows users to define the length between search terms. The usefulness of a search engine hinges on the relevancy of the results it provides. Since there can be millions of web pages that include a certain word or word combination, web pages can be divided into relevant and irrelevant ones. The majority of search engines employ methods to grade the results to feature the "best" results first. The way a search engine displays web pages is specific to a search engine. The techniques also alter with time, as the use of Internet services changes and new techniques appear.