Custom Search

Checking for the presence of any element in the Html code of a website with 'Custom Search'.

Scraping Source Code

The Seo Spider allows you to check for the presence of any element contained in the source code of a website’s pages through the “Custom Search” feature. By default, the system searches for raw HTML while switching to the “Javascript Rendered” scan mode allows you to extract the rendered data as well. The tool allows you to configure up to 100 custom searches that return you URLs that contain or do not contain the input you are looking for.

To use the “Custom Search” feature, simply click on the “Add” button on the screen, name the filter, choose the “Contains” or “Does Not Contains” option, choose the search type between text or RegEX, enter the input to be searched and define where to search (html, page text, a particular element, XPath etc.).

In the following example, we assumed a custom search with two objectives:

  1. the identification of an ecommerce’s unavailable products by searching for the “Out of Stock” label found on the detail pages (Contains).
  2. The definition of pages where the Google Tag Manager script is missing (Does Not Contain)

To collect the results simply give “ok” and crawl.

Once the scan is processed, the data will be populated in real time and divided into columns in the “Custom Search” tab.

If you wish, you can isolate one or the other “Custom Search” by using the drop-down filter on the left side of the upper window.

For convenience, the same results can also be accessed from the sidebar panel under “Custom Search.”

To complete the analysis you can export the results by clicking directly on the “Export” button on the tab or use the “Bulk Export” function to get all the Inlinks of your custom search.

Now that you understand the versatility of “Custom Search” I propose some best practices that are very useful in daily Seo Audits.

A very interesting first use in “Text Only” mode is to unearth any misspellings, acronyms etc. considering the “case sensitive” aspect of the words (RegEXs are already “case sensitive” by default) by enabling the relevant option.

Another application of “Custom Search” in RegEX mode is searching for specific queries (exact match). This research allows you to uncover any cannibalization issues with strategic keywords found on different landing pages that could penalize Search Intent on Google.

The previous search can also be expanded to more words by using the same “Custom Search” and dividing the individual RegEXs with the “|” (Pipe Symbol). This search of allows us to evaluate the density of queries in the text and avoid possible overstuffing penalties.

In the following example, the queries were searched for: “search engine optimization”, “natural” and “free”.

The scan result presents for the first page 5 instances (3 times “search engine optimization”, 1 time each “natural” and “free”).

Custom Search function options

When you use the “Custom Search” function, you have 7 options that allow you to limit the search area:

  • HTML: includes the entire HTML code.
  • Page Text: considers only the text of the web page but all HTML elements are excluded.
  • Page Text No Anchors: includes text on web pages, excluding any HTML or any text contained within HTML anchor tags (also known as A Elements). This can be useful when searching for words that are also included in the text of links within menus.
  • HTML Head: the search will be isolated to the html of the header.
  • HTML Body: the search examines the text and html elements of the html Body tag.
  • XPath: allows you to specify an XPath to define the location in the HTML where the search is to be performed. For example, if you want to limit the survey only to the text contained in h3 titles, you can provide //h3.
  • Content Area: the search is performed by default within the html Body element.

For example, we could research whether the name Screaming Frog was always spelled correctly or has “Misspellings” in blog articles. Specifically, we queried the Seo Spider to check whether the Seo Tool’s company name is spelled correctly or has the lowercase letter at the beginning of the second word: “Screaming frog.”

Please note: The custom searches setup screen presents a “singular-line” input field, but by clicking on the two arrows in the field, you have the option of expanding it to also enter multiple lines of code or entire sentences.

Custom Search Video

Seo Spider Tab