MOZ team says, “When should you disallow search engines in your robots.txt file, and when should you use meta robots tags in a page header? What about nofollowing links? In today’s Whiteboard Friday, Rand covers these tools and their appropriate use in four situations that SEOs commonly find themselves facing“.

Controlling Search Engine Crawlers for Better Indexation and Rankings

MOZ Blog

Sharing is caring