gildas-lormeau/single-file-cli

SingleFile CLI (Command Line Interface)

Introduction

SingleFile can be launched from the command line by running it into a (headless) browser.

It runs through Deno as a standalone script injected into the web page via the Chrome DevTools Protocol, or via WebDriver BiDi when using Firefox, instead of being embedded into a WebExtension.

Installation

SingleFile can be run without installing it, just download the executable file and save it in the directory of your choice here: https://github.com/gildas-lormeau/single-file-cli/releases

Make sure Chrome or a Chromium-based browser is installed in the default folder. Installations for the current user are also found (%LOCALAPPDATA% on Windows, ~/Applications on macOS, snap and the PATH on Linux). Otherwise you might need to set the --browser-executable-path option to help SingleFile locating the path of the executable file. Firefox can be used instead by setting the --browser-engine firefox option, with some limitations described in the help. The Flatpak version of Firefox needs access to /tmp, where the temporary profile is created. Allow it with flatpak override --user --filesystem=/tmp org.mozilla.firefox and set --browser-executable-path to /var/lib/flatpak/exports/bin/org.mozilla.firefox, or to ~/.local/share/flatpak/exports/bin/org.mozilla.firefox for a per-user installation.

Installation with Docker

• Installation from Docker Hub

  docker pull capsulecode/singlefile

  docker tag capsulecode/singlefile singlefile

• Manual installation

  git clone --depth 1 --recursive https://github.com/gildas-lormeau/single-file-cli.git

  cd single-file-cli

  docker build --no-cache -t singlefile .

• Run

  docker run singlefile "https://www.wikipedia.org"

• Run with Firefox

  docker run singlefile "https://www.wikipedia.org" --browser-engine firefox

• Run and redirect the result into a file

  docker run singlefile "https://www.wikipedia.org" > wikipedia.html

• Run and mount a volume to get the saved file in the current directory

  • Save one page

    docker run -v %cd%:/usr/src/app/out singlefile "https://www.wikipedia.org" wikipedia.html (Windows)

    docker run -v $(pwd):/usr/src/app/out singlefile "https://www.wikipedia.org" wikipedia.html (Linux/UNIX)

  • Save one or multiple pages by using the filename template (see --filename-template option)

    docker run -v %cd%:/usr/src/app/out singlefile "https://www.wikipedia.org" --dump-content=false (Windows)

    docker run -v $(pwd):/usr/src/app/out singlefile "https://www.wikipedia.org" --dump-content=false (Linux/UNIX)

• An alternative docker file can be found here https://github.com/screenbreak/SingleFile-dockerized. It allows you to save pages from the command line interface or through an HTTP server.

Manual installation

• Install Deno

• There are 3 ways to download the code of SingleFile, choose the one you prefer:

  • Install with npm and run single-file via npx (npm and npx are installed with Node.js)

    npm install "single-file-cli"
    npx single-file ...
    

    You can also install SingleFile globally with -g when running npm install.

  • Download and unzip manually the main archive provided by Github

    unzip main.zip .
    cd single-file-cli-main
    

  • Download with git

    git clone --depth 1 --recursive https://github.com/gildas-lormeau/single-file-cli.git
    cd single-file-cli
    

• Make single-file executable (Linux/Unix/BSD etc.).

  chmod +x single-file
  

Run

• Syntax

  single-file <url> [output] [options ...]
  

• Display help

  single-file --help
  

• Record what was observed while saving, for later verification: see MANIFEST.md

  single-file https://www.wikipedia.org wikipedia.html --manifest
  

• Examples

  • Dump the HTML content of https://www.wikipedia.org into the console

  single-file https://www.wikipedia.org --dump-content
  

  • Save https://www.wikipedia.org into wikipedia.html in the current folder

  single-file https://www.wikipedia.org wikipedia.html
  

  • Save a page needing a logged-in session. Create the profile first, log in to the website in the browser window, and quit the browser to save it. Then reuse the profile as many times as needed, it is copied and left unmodified.

  single-file --create-browser-profile ./profiles/example https://www.example.com/login
  single-file https://www.example.com/account account.html --browser-profile=./profiles/example
  

  • Save https://www.wikipedia.org with Firefox

  single-file https://www.wikipedia.org wikipedia.html --browser-engine firefox
  

  • Save a list of URLs stored into list-urls.txt in the current folder

  single-file --urls-file=list-urls.txt
  

  • Save https://www.wikipedia.org and crawl its internal links with the query parameters removed from the URL

  single-file https://www.wikipedia.org --crawl-links=true --crawl-inner-links-only=true --crawl-max-depth=1 --crawl-rewrite-rule="^(.*)\\?.*$ $1"
  

  • Save https://www.wikipedia.org and external links only

  single-file https://www.wikipedia.org --crawl-links=true --crawl-inner-links-only=false --crawl-external-links-max-depth=1 --crawl-rewrite-rule="^.*wikipedia.*$"
  

  • Save http://info.cern.ch (i.e. the first website) and its internal links into a single self-extracting ZIP file, and mark the links to pages missing from the archive

  single-file http://info.cern.ch first-website.html --crawl-links=true --crawl-max-depth=2 --crawl-save-archive=true --crawl-save-archive-mark-unarchived-links=true --compress-content=true
  

• Exit code

  The exit code is 0 when all the pages have been saved, 1 when at least one capture failed (the other pages of the batch or the crawl are still saved), and 255 when an error prevented the process from running.

Compile executables

• Compile executables into /dist

./compile.sh

License

SingleFile and SingleFile CLI are licensed under AGPL. Code derived from third-party projects is licensed under MIT. Please contact me at gildas.lormeau <at> gmail.com if you are interested in licensing the SingleFile code for a commercial service or product.