Matt1966 asked
Member (243 upvotes)

ScreamingFrog vs DeepCrawl - what is your favourite spider?

I'm a massive fan of ScreamingFrog - primarily because it's free! But I've heard great things about DeepCrawl also.

What is your choice of Spider, and why? (doesn't necessarily have to be ScreamingFrog or DeepCrawl).

Chedders
Moderator (243 upvotes)

I do have my own but if I am honest screamingfrog does a better and faster job so I tend to use that for most of my crawling. Yes, I could spend the time on my crawler but what's the point when screamingfrog does it so well.

Good advice?

Comments 0

Adnan123 Member

What about with huge websites? Screaming Frog tends to struggle at XXX,XXX/X,XXX,XXX URI kind of range. Well, they do with my 16RAM PC (when executing Javascript). At that point I've got to say, DeepCrawl does seem appealing as they'll handle the crawling on their side.

Matt1966
Moderator (243 upvotes)

I'm a massive fan of ScreamingFrog - primarily because it's free! But I've heard great things about DeepCrawl also.

What is your choice of Spider, and why? (doesn't necessarily have to be ScreamingFrog or DeepCrawl).

Good advice?

Comments 0

ScreamingFrog vs DeepCrawl - what is your favourite spider?

Related Questions