Jump to content


Photo

What Ip Does Rsa Use?

IP address

  • Please log in to reply
5 replies to this topic

#1 MartinP

MartinP

    RAGE Advisor

  • RAGE Members
  • PipPipPipPipPip
  • 40 posts

Posted 22 September 2013 - 05:47 AM

I have recently blocked a whole bundle of IP addresses and now RSA finds no pages. What IP addresses should I unblock to allow access?

 



#2 Jeremy

Jeremy

    RAGE Advisor

  • RAGE Admin
  • PipPipPipPipPip
  • 419 posts

Posted 24 September 2013 - 10:31 AM

If you are blocking IP addresses from visiting your site, this would not affect the pages that Sitemap Automator is able to scan (that is of course unless you have blocked your own IP address and as a result, you yourself are not able to view your site).



#3 MartinP

MartinP

    RAGE Advisor

  • RAGE Members
  • PipPipPipPipPip
  • 40 posts

Posted 28 September 2013 - 07:03 AM

Strange. I can view my site via any web browser but RSA reports "0 Links Found'.

http://www.satcure.co.uk 

 

Any suggestions? RSA works fine with my other sites.



#4 MartinP

MartinP

    RAGE Advisor

  • RAGE Members
  • PipPipPipPipPip
  • 40 posts

Posted 01 October 2013 - 09:07 AM

I'm running RSA version 2.6 with OS10,6.8.

It works with all sites but this one.

"Rescan Website" results in "Looking for robots.txt file..." appearing for about 30 seconds to 2 minutes, then it disappears but scanning doesn't begin. There is no error message. It simply displays "0 Links Found".

I can view the web site with any browser on the same Mac.

 

Exactly the same symptoms on my other Mac running RSA 2.4 with OS 10.4.11.

 

It makes no difference whether I use my own IP address or run through a VPN.



#5 MartinP

MartinP

    RAGE Advisor

  • RAGE Members
  • PipPipPipPipPip
  • 40 posts

Posted 13 October 2013 - 06:09 AM

I fixed the problem. It's because my ISP had added the following to my robots.txt file:

 

 

# Allow these robots only
User-agent: slurp
Allow: / 
 
User-agent: bingbot
Allow: /
 
User-agent: googlebot
Allow: /
 
So I'd like to know what "user agent" GSA announces?
 
(Clearly this is the question I should have been asking in the first place!)

Edited by MartinP, 13 October 2013 - 06:10 AM.


#6 MartinP

MartinP

    RAGE Advisor

  • RAGE Members
  • PipPipPipPipPip
  • 40 posts

Posted 14 October 2013 - 08:04 AM

So I'd like to know what "user agent" GSA announces?






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users