You want to use the terminal to bulk export csv files from Screaming Frog. Here's the complete list of Bulk Export for you to use. Grab the list for free.
As you might have read in several of my blog posts (like: 5 reasons to stop with WordPress, or Medium stopped offering custom domains), I could not really get used to some blog frameworks. And by switching a lot my sites tend to fluctuate in Google Search a lot.
So every now and then I use Screaming Frog to test my sites and see what I can do to improve them. Now, I’m not using the GUI that Screaming Frog has, but I use the command line.
With this command line I export the Bulk Export and the Export Tabs and process them with Python to an actually readable Word Document. I build that in the past just for fun.
The ‘Bulk Export’ in the Screaming Frog GUI is located under the top level menu and allows bulk exporting of all data. You can export all instances of a link found in a crawl via the ‘All inlinks’ option, or export all inlinks to URLs with specific status codes such as 2XX, 3XX, 4XX or 5XX responses.
As I’m using it in Python I wanted to know the names of all the Bulk Export and the filenames that Screaming Frog would spit out.
And Guess what…. there is no complete list! Not even the Screaming Frog people could tell me what Bulk Export items there are.
I would really like an over view (for command line) with -bulk-export and -export-tabs.
We don’t have one I am afraid.
We may look at writing them all out at some stage, but we don’t have one now, or one I can offer you at a set timescale unfortunately.
Bummerrrrrrr
Python OCR the Bulk Export tab images
So….. no way to get the Screaming Frog Bulk Export List. I can tell you I did not want to type 60 Bulk Export names and file names.
What I could do was to make a copy of the names in the GUI overview and retrieve the text. So, I did some coding in python to actually get the text from the overview. (yes… I might have as well typed the names myself. But I just love coding 🙂
So I made screenshots, saved the screen shots in a folder and ran my python script
Screaming Frog Bulk Export List
So here is the list. Where name is the actual name you can use in the command line option from screaming frog “–bulk-export” and file is the filename coming out of the command line processing.
YML output can be found here.
file: all_inlinks.csv
name: All Inlinks
file: all_outlinks.csv
name: All Outlinks
file: queued_urls.csv
name: Queued URLs
file: all_anchor_text.csv
name: All Anchor Text
file: all_images.csv
name: All Images
file: null
name: Screenshots
file: null
name: All Page Source
file: external_links.csv
name: External Links
file: response_codes_blocked_by_robots_txt.csv
name: Response Codes:Blocked by Robots.txt Inlinks
file: response_codes_blocked_resource.csv
name: Response Codes:Blocked Resource Inlinks
file: response_codes_no_response.csv
name: Response Codes:No Response Inlinks
file: response_codes_success_(2xx).csv
name: Response Codes:Success (2xx) Inlinks
file: response_codes_redirection_(3xx).csv
name: Response Codes:Redirection (3xx) Inlinks
file: response_codes_redirection_(javascript).csv
name: Response Codes:Redirection (JavaScript) Inlinks
file: response_codes_redirection_(meta_refresh).csv
name: Response Codes:Redirection (Meta Refresh) Inlinks
file: response_codes_client_error_(4xx).csv
name: Response Codes:Client Error (4xx) Inlinks
file: response_codes_server_error_(5xx).csv
name: Response Codes:Server Error (5xx) Inlinks
file: directives_index.csv
name: Directives:Index Inlinks
file: directives_noindex.csv
name: Directives:Noindex Inlinks
file: directives_follow.csv
name: Directives:Follow Inlinks
file: directives_nofollow.csv
name: Directives:Nofollow Inlinks
file: directives_none.csv
name: Directives:None Inlinks
file: directives_noarchive.csv
name: Directives:NoArchive Inlinks
file: directives_nosnippet.csv
name: Directives:NoSnippet Inlinks
file: directives_maxsnippet.csv
name: Directives:Max-Snippet Inlinks
file: directives_maximagepreview.csv
name: Directives:Max-Image-Preview Inlinks
file: directives_maxvnameeopreview.csv
name: Directives:Max-Vnameeo-Preview Inlinks
file: directives_noodp.csv
name: Directives:NoODP Inlinks
file: directives_noydir.csv
name: Directives:NoYDIR Inlinks
file: directives_noimageindex.csv
name: Directives:NoImageIndex Inlinks
file: directives_notranslate.csv
name: Directives:NoTranslate Inlinks
file: directives_unavailable_after.csv
name: Directives:Unavailable_After Inlinks
file: directives_refresh.csv
name: Directives:Refresh Inlinks
file: canonicals_contains_canonical.csv
name: Canonicals:Contains Canonical Inlinks
file: canonicals_self_referencing.csv
name: Canonicals:Self Referencing Inlinks
file: canonicals_canonicalised.csv
name: Canonicals:Canonicalised Inlinks
file: canonicals_missing.csv
name: Canonicals:Missing Inlinks
file: canonicals_multiple.csv
name: Canonicals:Multiple Inlinks
file: canonicals_nonindexable_canonical.csv
name: Canonicals:Non-Indexable Canonical Inlinks
file: amp_all.csv
name: AMP:All Inlinks
file: amp_non200_response.csv
name: AMP:Non-200 Response Inlinks
file: amp_nonconfirming_canonical.csv
name: AMP:Non-Confirming Canonical Inlinks
file: amp_missing_nonamp_canonical.csv
name: AMP:Missing Non-AMP Canonical Inlinks
file: amp_nonindexable_canonical.csv
name: AMP:Non-Indexable Canonical Inlinks
file: amp_indexable.csv
name: AMP:Indexable Inlinks
file: amp_nonindexable.csv
name: AMP:Non-Indexable Inlinks
file: structured_data_contains_structured_data.csv
name: Structured Data:Contains Structured Data
file: structured_data_valnameation_errors.csv
name: Structured Data:Valnameation Errors
file: structured_data_valnameation_warnings.csv
name: Structured Data:Valnameation Warnings
file: structured_data_jsonld_urls.csv
name: Structured Data:JSON-LD URLs
file: structured_data_microdata_urls.csv
name: Structured Data:Microdata URLs
file: structured_data_rdfa_urls.csv
name: Structured Data:RDFa URLs
file: images_missing_alt_text_inlinks.csv
name: Images:Images Missing Alt Text Inlinks
file: images_over_100_kb.csv
name: Images:Images over X KB Inlinks
file: sitemaps_urls_in_sitemap.csv
name: Sitemaps:URLs in Sitemap Inlinks
file: search_console_orphan_urls.csv
name: Sitemaps:Orphan URLs Inlinks
file: sitemaps_nonindexable_urls_in_sitemap.csv
name: Sitemaps:Non-Indexable URLs in Sitemap Inlinks
file: sitemaps_urls_in_multiple_sitemaps.csv
name: Sitemaps:URLs in Multiple Sitemaps Inlinks
file: custom_search_all.csv
name: Custom Search:All Inlinks
file: custom_extraction_all.csv
name: Custom Extraction:All Inlinks
Like it? Don’t forget to buy me a Coffee!!