{"id":5399,"date":"2026-05-15T15:41:56","date_gmt":"2026-05-15T15:41:56","guid":{"rendered":"https:\/\/screamingfrog.club\/?p=5399"},"modified":"2026-05-15T15:41:56","modified_gmt":"2026-05-15T15:41:56","slug":"how-to-open-screaming-frog-crawls-in-python","status":"publish","type":"post","link":"https:\/\/screamingfrog.club\/en\/how-to-open-screaming-frog-crawls-in-python\/","title":{"rendered":"How to Open Screaming Frog Crawls Directly in Python"},"content":{"rendered":"
<\/div><\/div><\/span>

How to Open Screaming Frog Crawls Directly in Python with the screamingfrog Library<\/span><\/h2><\/span>
<\/div><\/div><\/div>

Anyone who works in Screaming Frog on a regular basis knows the rhythm of an audit.<\/p>\n

The crawl finishes, you open the GUI, click into one of the reports, export to CSV, then spend a few minutes cleaning columns<\/a> before the actual analysis can begin.<\/p>\n

Multiply this by titles, response codes, inlinks<\/a>, canonicals, structured data<\/a>, and the other recurring reports, and you have already burned a good portion of the afternoon on what is, fundamentally, preparatory work.<\/p>\n

The screamingfrog<\/strong><\/span> Python library was built precisely to skip this step. It opens a .dbseospider<\/span><\/strong> file directly inside Python and lets you query the underlying crawl through the same logical units that the GUI presents to the user: pages, links, and the individual report tabs.<\/p>\n

When the typed interface is not enough, the library also exposes raw SQL for those situations where the question is unusual. In my own workflow I have not exported a CSV from the GUI in several months, and I do not miss it.<\/p>\n

In this guide we will go through a series of operations against a real crawl file: loading the .dbseospider<\/span><\/strong>, querying pages and links, pulling familiar tab reports by name, scoping queries to a section of the site, and finally falling back on raw SQL when the typed API is not enough.<\/p>\n

The walkthrough closes with a short end-to-end example that ties the four interfaces together.
\nA clarification before we begin. The library is not a replacement for Screaming Frog itself, since the crawler remains the data source. It is a replacement for the export-clean-analyze loop that usually follows every crawl, which is a different proposition entirely.<\/p>\n<\/div>

<\/div><\/div><\/span>

What the screamingfrog library changes<\/h2><\/span>
<\/div><\/div><\/div>

Screaming Frog already stores the crawl in a database-backed format<\/a>. The application reads from it, the GUI displays from it, and the exports come out of it. The screamingfrog<\/span> Python library reads from that same crawl data and gives you a programmatic interface on top of it. Nothing is re-parsed from a CSV, and nothing is derived from the export pipeline.
\nThere are four main interfaces for asking questions of the crawl:<\/p>\n