182k views
1 vote
If your application URL is: www.example.org/new/ , which of the of following links will be crawled if the Crawl Scope is set to "Limit to content at or below URL subdirectory"? (select two) (Choose all that apply)

(A) www.example.org/existing
(B) www.example.org
(C)www.example.org/new/customers
(D) www.example.org/new

1 Answer

1 vote

Final Answer:

In the context of a crawl scope limited to content at or below the URL subdirectory www.example.org/new/, the crawler will only include URLs within or beneath this subdirectory. Therefore, the links that will be crawled are www.example.org/new/customers and www.example.org/new.

(C)www.example.org/new/customers

(D) www.example.org/new

Step-by-step explanation:

When the crawl scope is set to "Limit to content at or below URL subdirectory" with the base URL www.example.org/new, it signifies a restriction on the crawler's exploration to only consider content within or beneath the specified subdirectory, which in this case is "/new/". This directive filters out URLs that are not part of or below this subdirectory.

Firstly, (C) www.example.org/new/customers is within the specified subdirectory /new/, and thus, it satisfies the criteria for being crawled. The crawler will follow this link because it adheres to the limitation imposed by the crawl scope.

Similarly, (D) www.example.org/new is also within the specified subdirectory /new/, meeting the criteria for inclusion. The crawler will explore this link as it aligns with the defined crawl scope.

Conversely, options (A) www.example.org/existing and (B) www.example.org do not fall within or below the specified subdirectory (/new/), so they are excluded from the crawl. The crawler, following the defined limitations, will not traverse these URLs.

In essence, the crawl scope acts as a filter, guiding the crawler's exploration to specific subdirectories and ensuring that only relevant content within or beneath the specified subdirectory is included in the crawling process.

User NiceToMytyuk
by
8.4k points