Cookies Analysis<\/h2><\/span><\/div><\/div><\/div>Given the importance and evolution of privacy laws, it has become increasingly important to understand the presence and purpose of cookies on our website so that we are compliant and do not incur penalties. With Screaming Frog you have the ability to identify cookies that are set automatically without explicit user consent on your entire project.<\/p>\n
\n- 1. Enable the ‘Cookies’ option for extraction with Screaming Frog.<\/li>\n<\/ul>\n
The first action is to enable cookie extraction on your website through the “Config > Spider > Extraction” option and select “Cookies” under “URL Details.” In this way, the Seo Spider saves all cookies discovered during the scan.<\/p>\n<\/div>
![](\"http:\/\/screamingfrog.club\/wp-content\/uploads\/2024\/03\/Screaming-Frog-Cookies.png\")
<\/a><\/span><\/div>When you choose to store cookies, the automatic exclusion performed by the SEO Spider for Google Analytics tracking tags is disabled to provide an accurate view of all cookies issued while still going to dirty GA data.<\/p>\n<\/div>
<\/div><\/div><\/span>Rendering JS<\/h3><\/span><\/div><\/div><\/div>When you choose to store cookies, the automatic exclusion performed by the SEO Spider for Google Analytics tracking tags is disabled to provide an accurate view of all cookies issued while still going to dirty GA data.<\/p>\n<\/div>
![](\"http:\/\/screamingfrog.club\/wp-content\/uploads\/2024\/03\/Screaming-Frog-Rendering-JS.png\")
<\/a><\/span><\/div>The default window size is automatically set on Googlebot Smartphone, but this can be adjusted on desktop if there are differences in the way the site issues cookies.<\/p>\n<\/div>
<\/div><\/div><\/span>Set User-Agent<\/h3><\/span><\/div><\/div><\/div>To best simulate browsing a user and not a Bot edit the user-agent on Chrome via “Config > User-agent”.<\/p>\n
<\/div>\n<\/div>![](\"http:\/\/screamingfrog.club\/wp-content\/uploads\/2024\/03\/user-agent-chrome.png\" "\\"user")
<\/span><\/div><\/div><\/div><\/span>Robots.txt management<\/h3><\/span><\/div><\/div><\/div>\n- 4. Set the “Ignore robots.txt” option.<\/li>\n<\/ul>\n
Click on “Configuration > robots.txt ” and select “Ignore robots.txt” or “Ignore robots.txt but report status.” This operation allows you to load all resources and respective cookies and prevent some resources from being blocked during scanning.<\/p>\n<\/div>
![](\"http:\/\/screamingfrog.club\/wp-content\/uploads\/2024\/03\/Robots-configurazione-screaming-frog.png\")
<\/a><\/span><\/div>\n- Crawl the website<\/li>\n
- Displays the number of cookies for each URL<\/li>\n<\/ul>\n
After the scan you are able to see how many cookies were discovered for each URL scanned. This index can be found in the “Internal” tab in the “cookies” column.<\/p>\n<\/div>
![](\"http:\/\/screamingfrog.club\/wp-content\/uploads\/2024\/03\/cookie-audit-nocookies-seo-spider.jpeg\")
<\/a><\/span><\/div><\/div><\/div><\/span>Cookie Tab<\/h3><\/span><\/div><\/div><\/div>\n- 7. View Cookies Tab<\/li>\n<\/ul>\n
Now all you have to do is see what cookies are discovered during the crawl; you just click on one or more URLs in the upper window to find all the details in the lower panel of the “Cookies” tab.<\/p>\n<\/div>
![](\"http:\/\/screamingfrog.club\/wp-content\/uploads\/2024\/03\/cookie-audit-cookies-tab-seo-spider.jpeg\")
<\/a><\/span><\/div>Below is an example of the “Cookies” tab populated with details.<\/p>\n<\/div>
![](\"http:\/\/screamingfrog.club\/wp-content\/uploads\/2024\/03\/cookie-audit-cookies-tab-close-seo-spider.jpeg\")
<\/a><\/span><\/div>The columns available to you are as follows:<\/p>\n
\n- Cookie Type<\/strong>: identifies where the cookie was discovered (“HTTP”, “On-Page” etc).<\/li>\n
- Cookie<\/strong> Name: collects the name of the cookie.<\/li>\n
- Cookie<\/strong> Value: defines the value of the cookie.<\/li>\n
- Domain<\/strong>: shows the domain that issued the first- or third-party cookie.<\/li>\n
- Expiration<\/strong> Time: indicates the expiration time of the cookie.<\/li>\n
- Secure<\/strong>: displays the details of the security attribute of the cookie.<\/li>\n
- HttpOnly:<\/strong> shows the details of the HttpOnly attribute of the cookie.<\/li>\n
- Address<\/strong>: displays the URL on which the cookie was found.<\/li>\n<\/ul>\n
\n- 8. Export aggregate cookie report<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/span>
Cookie Report<\/h3><\/span><\/div><\/div><\/div>You are now able to export a very valuable aggregate summary of discovered cookies.<\/p>\n<\/div><\/div><\/div><\/div><\/div>
Report > Cookie > Cookie Summary<\/span><\/p>\n<\/div><\/div><\/div><\/div><\/div>
Given the importance and evolution of privacy laws, it has become increasingly important to understand the presence and purpose of cookies on our website so that we are compliant and do not incur penalties. With Screaming Frog you have the ability to identify cookies that are set automatically without explicit user consent on your entire project.<\/p>\n
- \n
- 1. Enable the ‘Cookies’ option for extraction with Screaming Frog.<\/li>\n<\/ul>\n
The first action is to enable cookie extraction on your website through the “Config > Spider > Extraction” option and select “Cookies” under “URL Details.” In this way, the Seo Spider saves all cookies discovered during the scan.<\/p>\n<\/div>
<\/a><\/span><\/div>When you choose to store cookies, the automatic exclusion performed by the SEO Spider for Google Analytics tracking tags is disabled to provide an accurate view of all cookies issued while still going to dirty GA data.<\/p>\n<\/div>
<\/div><\/div><\/span>Rendering JS<\/h3><\/span>
<\/div><\/div><\/div>When you choose to store cookies, the automatic exclusion performed by the SEO Spider for Google Analytics tracking tags is disabled to provide an accurate view of all cookies issued while still going to dirty GA data.<\/p>\n<\/div>
<\/a><\/span><\/div>The default window size is automatically set on Googlebot Smartphone, but this can be adjusted on desktop if there are differences in the way the site issues cookies.<\/p>\n<\/div>
<\/div><\/div><\/span>Set User-Agent<\/h3><\/span>
<\/div><\/div><\/div>To best simulate browsing a user and not a Bot edit the user-agent on Chrome via “Config > User-agent”.<\/p>\n
<\/div>\n<\/div><\/span><\/div><\/div><\/div><\/span>Robots.txt management<\/h3><\/span>
<\/div><\/div><\/div>- \n
- 4. Set the “Ignore robots.txt” option.<\/li>\n<\/ul>\n
Click on “Configuration > robots.txt ” and select “Ignore robots.txt” or “Ignore robots.txt but report status.” This operation allows you to load all resources and respective cookies and prevent some resources from being blocked during scanning.<\/p>\n<\/div>
<\/a><\/span><\/div>- \n
- Crawl the website<\/li>\n
- Displays the number of cookies for each URL<\/li>\n<\/ul>\n
After the scan you are able to see how many cookies were discovered for each URL scanned. This index can be found in the “Internal” tab in the “cookies” column.<\/p>\n<\/div>
<\/a><\/span><\/div><\/div><\/div><\/span>Cookie Tab<\/h3><\/span>
<\/div><\/div><\/div>- \n
- 7. View Cookies Tab<\/li>\n<\/ul>\n
Now all you have to do is see what cookies are discovered during the crawl; you just click on one or more URLs in the upper window to find all the details in the lower panel of the “Cookies” tab.<\/p>\n<\/div>
<\/a><\/span><\/div>Below is an example of the “Cookies” tab populated with details.<\/p>\n<\/div>
<\/a><\/span><\/div>The columns available to you are as follows:<\/p>\n
- \n
- Cookie Type<\/strong>: identifies where the cookie was discovered (“HTTP”, “On-Page” etc).<\/li>\n
- Cookie<\/strong> Name: collects the name of the cookie.<\/li>\n
- Cookie<\/strong> Value: defines the value of the cookie.<\/li>\n
- Domain<\/strong>: shows the domain that issued the first- or third-party cookie.<\/li>\n
- Expiration<\/strong> Time: indicates the expiration time of the cookie.<\/li>\n
- Secure<\/strong>: displays the details of the security attribute of the cookie.<\/li>\n
- HttpOnly:<\/strong> shows the details of the HttpOnly attribute of the cookie.<\/li>\n
- Address<\/strong>: displays the URL on which the cookie was found.<\/li>\n<\/ul>\n
- \n
- 8. Export aggregate cookie report<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/span>
Cookie Report<\/h3><\/span>
<\/div><\/div><\/div>You are now able to export a very valuable aggregate summary of discovered cookies.<\/p>\n<\/div><\/div><\/div><\/div><\/div>
Report > Cookie > Cookie Summary<\/span><\/p>\n<\/div><\/div><\/div><\/div><\/div>
- Cookie<\/strong> Name: collects the name of the cookie.<\/li>\n
- Cookie Type<\/strong>: identifies where the cookie was discovered (“HTTP”, “On-Page” etc).<\/li>\n
- 7. View Cookies Tab<\/li>\n<\/ul>\n
- 4. Set the “Ignore robots.txt” option.<\/li>\n<\/ul>\n