![[Home]](/wiki.gif)
AranhabotAranhabot|
Links:
|
Subjects > Computers > Internet > Web > Search Engine News > Web Robots And Spiders
Aranhabot is the spider that Amazon.com uses to check their affiliate sites.
Another robot name used by Amazon: amzn_assoc. It has been reported coming from these addresses:
Alexa.com is owned by Amazon.com, and their spider is related to Amazon. Amazon has been rumored to own Archive.org, but Amazon denies this, saying that Archive.org is a 501(c)(3) public nonprofit company. Amazon has claimed that the spidering that Alexa does, (their broken link crawl), will not be shared with Archive.org, or included in their database.
Amazon has said, In an effort to help you optimize your Web site(s), we're offering a new service in conjunction with our wholly-owned subsidiary, Alexa Internet, to identify broken Amazon.com Associates links. This service is offered to benefit both Amazon.com Associates and Amazon.com. By identifying broken links on your site(s), we make it easier for you to keep your links up-to-date, and our program as a whole operates more smoothly. We can assure you that this information is not shared with anyone but the Amazon.com Associate. Alexa does not use any information gathered by the broken link crawler for its own purposes. To ensure the performance of your server is not affected, Alexa Internet has configured their crawler to restrict page hits to 2 per second. To find out more information about this service, click here--http://forums.prospero.com/n/mb/message.asp?webtag=am-assosanbd&msg=25.1&ctx=0
This spider is mentioned occassionally as using very large amounts of bandwidth. Amazon's robots are frequently mentioned as not bothering to obey the robots.txt exclusion.
Several webmasters have complained about Archive.org continuing to present information that they do not want to be public, such as their 1-800 numbers that were on older web pages.
Amazon's Policy Requires Their Associates To Not Ban Amazon's Robots Many
2) Policy:
As a service to Associates, we will work with our corporate affiliate Alexa Internet, Inc. to crawl Associates web sites to locate broken Amazon.com links. This will enable us to e-mail reports to Associates that identify the broken Amazon.com links on their sites.
2.1) Official Language in Operating Agreement
“. In addition, you acknowledge that we (and our corporate affiliates, such as Alexa Internet, Inc.) may crawl or otherwise monitor your site for the purpose of ensuring the quality and reliability of Special Links on your site (for example, to detect links that are broken or non-functional, links to products that are out of stock or otherwise unavailable, etc.). Therefore, you agree that we and our corporate affiliates may take such actions and that you will not seek to block or otherwise interfere with such crawling or monitoring (and that we and our corporate affiliates may use technical means to overcome any methods used on your site to block or interfere with such crawling or monitoring).“
Other pages mentioning Aranhabot, and other Amazon spiders:
Other sites about internet spiders:
|
http://images.amazon.com/images/P/B0001XQNSE.01-A1KDZ23Y0QWKQ3.MZZZZZZZ.jpg
|
Search for books about:
|
Interested in Royalty Free Production Music?
