reeta
reeta asked
Member (0 upvotes)

Bulk indexed URLs download from website

What are the tools to download indexed URLs in the bulk of web pages from a website?

#bulkurls#downloadurls#bulkdownloads#urls
Avatar for member Chedders
Chedders
Moderator (209 upvotes)

Screaming frog does exactly what you have asked for however I suspect you want more. 

Firstly you can assume all pages are indexed on a website unless they have been marked as noindex or blocked by robots.txt, Google will have visited them at some point unless the page is very new. 

If you're on about the pages rank, which I assume you're after then that is pretty much impossible because it depends on who and where the results are being gathered from and for what search term. Google results are personalised which is why in search console they only show you the average position of a page along with the phrase used. 

If I am wrong in my assumption then sure screaming frog is the best tool out there.

Good advice?
1
0
Comments 0
login to reply
Avatar for member Matt1966
Matt1966
Moderator (229 upvotes)

Please could you edit your question to make it 100% clear on what you're asking?

Good advice?
0
0
Comments 0
login to reply
Copyright ©2020 SEOChat. All Rights Reserved.