Search Engines

How a search engine like Google assembles its index (03/14/2011)
(...) The result is every link on every page until all the links happen to be followed. This is the way new pages are added to the site's database, by using those links the spider hasn't seen before. The pages discovered by the spider are copied verbatim to the search site's database and copied over each time they're updated. (...)

Google searches take place all over the Internet (10/03/2010)
(...) Keep in mind, Google also owns DoubleClick, the largest Internet ad serving technology. So even if you don't see a Google search box anywhere, the ads on that page could be served by Google. Part of the reason the Google habit is so hard to break is that you can't escape it. (...)

Discover the features of AltaVista and what it does best (08/28/2010)
(...) It digs down into the page and finds actual words and phrases from within the full-text of the website. As a result, if you can pin your search down to one word or a unique phrase, you can find it - quickly. It offers a broad range of search functionality and when it returns results, matching search term are highlighted - a time saver. (...)

What special features make Google unique among search engines (08/28/2010)
(...) For example, searching Adobe Acrobat PDF files and limiting the domain to the U.S. government, is an extremely effective way of pinpointing American government sites, since the U. (...)

Gigablast and Lycos are promising search engines (08/28/2010)
(...) It reads your results, then makes categorizes the common groupings it finds in its results - a sort of clustering. Unlike all the other search engines, the default search setting on Gigablast was an OR operator. Most search tools use an AND operator. (...)

Take a glimpse at these great meta search tools (08/28/2010)
(...) You can also build results by grading the sites you see into useful or garbage. This allows you to customize and filter future results. What remains unclear is if they have built their own database, or are they buying it from one of the other search firms? Regardless, this is a useful new search tool. (...)

Why search engines do not index all web pages (08/28/2010)
(...) Remember: it is costly to search, index, and store the results. So, many search engine crawlers limit the number of pages on a website they search, or limit the number of pages indexed, dumping older ones and replacing them with new ones, or restrict the kinds of pages they crawl by cataloging only certain types of domain names. Another problem is simple economics. (...)

What are search engine crawlers and what do they do (08/27/2010)
(...) This cuts down on the bogus URLs and helps the crawler be thorough. The search engine crawls and indexes information, but search engines do further refinement before the information is available to the public. The companies perform spam detection and removal, duplication detection and removal, and also do some database quality testing. (...)

Search engine algorithms and how websites rank (08/27/2010)
(...) Nobody does it the same, which is why you get different results when you do a search, even the exact same search on two different search engines. Some search engines like Google put extra emphasis on what sites are linked to the primary site ("link analysis"), while others, like Teoma, rank searches for "hubs" and "authorities" related to your query terms that form a "community" about the topic. Search engines have found some creative ways to deal with bad queries. (...)

Natural language search engines and when to use them (08/27/2010)
(...) The pages are isolated. Then, the engine summarizes and analyzes potentially relevant documents, finding commonality among relevant documents. Inktomi and many other search tools like Oingo, Lexiquest, Iphrase and Manning and Napier Information System's Map-It offer phrase detection and more advanced natural language products including concept mapping to find what you mean, not what you say. (...)

Comparison between search engines and subject directories (08/27/2010)
(...) As a rule, the more specific you are, the less likely you will be to find what you are looking for in a subject directory. Some subject directories do not have search capabilities. In those cases, your only choice is to browse through the categories and subcategories. (...)

What are meta search tools and what purpose do they have (08/27/2010)
(...) Most meta-search tools can conduct only initial, simple searches, meaning you cannot refine or limit your search. This is the big problem with meta-search tools: to get wide access to many search tools, you give up several things, including the ability to do any kind of sophisticated or advanced searching. Ultimately, the results from meta-search tools are less precise. (...)

Search engine positioning and the so called sponsored links (08/27/2010)
(...) To the credit of the major meta-search tools, when I conducted a similar search in March 2004, on "breast cancer" using Dogpile, thirty-one of the first sixty results were "sponsored" - in other words, paid for - but all of those results were clearly marked as sponsored results. A similar search on Overture, the paid-placement company that Yahoo! owns, all of the top forty results were sponsored but also clearly marked. Many of the meta-search engines still do not clearly mark the results. (...)