3 Explanation Of Search Engines And Search Engine Spiders

A search engine is quite simply a large database of information. This information is stored in the form of records and is gathered from websites throughout the World Wide Web.

In order to gather this information, search engines use automated software programs known as ‘search engine spiders ‘, which are commonly referred to as ‘spiders’ or ‘bots’. Search engine spiders perform in the same way that web browsers (e.g. Internet Explorer and Firefox) would, except that they are much more limited in what they can do, i.e. they cannot accept Cookies or read JavaScript for example. The sole purpose of a search engine spider is to gather website information and send it back to the search engine database. Search engine spiders use the links between websites as a means of finding their way around the web; this is what is known as ‘crawling’. As a spider follows the links from website to website, it is crawling the web for new content and information.

New content collected from a spider is sent to, and analysed by the search engines ‘algorithm’, the algorithm determines what the web page is about and how it should be ranked in relation to other records (or websites) in its database. There are over 100 different factors that an algorithm will use to analyse a website, and the algorithm will vary from search engine to search engine. This is because different engines put more weighting (or importance) on different factors. Once the algorithm has finished, the new web page is then added to the search engines ‘index’. The index is a term commonly used when describing a search engines database.

When a user enters a query into a search engine such as Google, the query is then sent back to the search engine algorithm. The algorithm then pulls all the relevant records from its Index, and determines which records are the most relevant for that keyword and ranks them accordingly.