{"id":3377,"date":"2024-03-14T12:24:55","date_gmt":"2024-03-14T12:24:55","guid":{"rendered":"http:\/\/screamingfrog.club\/custom-search\/"},"modified":"2024-07-03T07:28:31","modified_gmt":"2024-07-03T07:28:31","slug":"custom-search","status":"publish","type":"post","link":"https:\/\/screamingfrog.club\/en\/custom-search\/","title":{"rendered":"Custom Search"},"content":{"rendered":"

<\/div><\/div><\/span>

Scraping Source Code<\/h2><\/span>
<\/div><\/div><\/div>

The Seo Spider allows you to check for the presence of any element contained in the source code of a website’s pages through the “Custom Search” feature. By default, the system searches for raw HTML while switching to the “Javascript Rendered” scan mode allows you to extract the rendered data as well. The tool allows you to configure up to 100 custom searches that return you URLs that contain or do not contain the input you are looking for.<\/p>\n<\/div>

<\/a><\/span><\/div>

To use the “Custom Search” feature, simply click on the “Add” button on the screen, name the filter, choose the “Contains” or “Does Not Contains” option, choose the search type between text or RegEX, enter the input to be searched and define where to search (html, page text, a particular element, XPath etc.).<\/p>\n<\/div>

<\/a><\/span><\/div>

In the following example, we assumed a custom search with two objectives:<\/p>\n

    \n
  1. the identification of an ecommerce’s unavailable products by searching for the “Out of Stock” label found on the detail pages (Contains).<\/li>\n
  2. The definition of pages where the Google Tag Manager script is missing (Does Not Contain)<\/li>\n<\/ol>\n<\/div>
    <\/a><\/span><\/div>

    To collect the results simply give “ok” and crawl.<\/p>\n

    Once the scan is processed, the data will be populated in real time and divided into columns in the “Custom Search” tab.<\/p>\n

    If you wish, you can isolate one or the other “Custom Search” by using the drop-down filter on the left side of the upper window.<\/p>\n<\/div>

    <\/a><\/span><\/div>

    For convenience, the same results can also be accessed from the sidebar panel under “Custom Search.”<\/p>\n

    <\/div>\n<\/div>
    <\/a><\/span><\/div>

    To complete the analysis you can export the results by clicking directly on the “Export” button on the tab or use the “Bulk Export” function to get all the Inlinks of your custom search.<\/p>\n<\/div>

    <\/a><\/span><\/div>

    Now that you understand the versatility of “Custom Search” I propose some best practices that are very useful in daily Seo Audits.<\/p>\n

    A very interesting first use in “Text Only” mode is to unearth any misspellings, acronyms etc. considering the “case sensitive” aspect of the words (RegEXs are already “case sensitive” by default) by enabling the relevant option.<\/p>\n<\/div>

    <\/a><\/span><\/div>

    Another application of “Custom Search” in RegEX mode is searching for specific queries (exact match). This research allows you to uncover any cannibalization issues with strategic keywords found on different landing pages that could penalize Search Intent on Google.<\/p>\n<\/div>

    <\/span><\/div>

    The previous search can also be expanded to more words by using the same “Custom Search” and dividing the individual RegEXs with the “|” (Pipe Symbol). This search of allows us to evaluate the density of queries in the text and avoid possible overstuffing penalties.<\/p>\n<\/div>

    <\/span><\/div>

    In the following example, the queries were searched for: “search engine optimization”, “natural” and “free”.<\/p>\n

    <\/div>\n<\/div>
    <\/a><\/span><\/div>

    The scan result presents for the first page 5 instances (3 times “search engine optimization”, 1 time each “natural” and “free”).<\/p>\n<\/div>

    <\/div><\/div><\/span>

    Custom Search function options<\/h3><\/span>
    <\/div><\/div><\/div>

    When you use the “Custom Search” function, you have 7 options that allow you to limit the search area:<\/p>\n