Images missing when saving login-protected pages / Saved as HTML in browser

[uqs]

Environment

• Operating System: Arch Linux
• node --version: v22.7.0
• npm --version: 10.8.2
• yarn --version, if using Yarn:
• percollate --version: 2.1.0

Description

I couldn't find a way to have percollate see the full article on, say a paywalled Substack. So I get Chrome to save the page as HTML (with a foo_files directory to have the images).

This renders fine in a browser, but when directing percollate to bundle that into an epub, it complains and the images don't come out right.

Example:

% percollate epub -o test.epub foo.html
Fetching: foo.html(node:12768) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
 ✓
Enhancing web page: foo.html ✓
Saving EPUB...
TypeError: Only absolute URLs are supported
    at getNodeRequestOptions (/usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1327:9)
    at /usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1450:19
    at new Promise (<anonymous>)
    at fetch (/usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1447:9)
    at file:///usr/lib/node_modules/percollate/index.js:689:12
TypeError: Only absolute URLs are supported
    at getNodeRequestOptions (/usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1327:9)
    at /usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1450:19
    at new Promise (<anonymous>)
    at fetch (/usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1447:9)
    at file:///usr/lib/node_modules/percollate/index.js:689:12
TypeError: Only absolute URLs are supported
    at getNodeRequestOptions (/usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1327:9)
    at /usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1450:19
    at new Promise (<anonymous>)
    at fetch (/usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1447:9)
    at file:///usr/lib/node_modules/percollate/index.js:689:12
TypeError: Only absolute URLs are supported
    at getNodeRequestOptions (/usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1327:9)
    at /usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1450:19
    at new Promise (<anonymous>)
    at fetch (/usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1447:9)
    at file:///usr/lib/node_modules/percollate/index.js:689:12
TypeError: Only absolute URLs are supported
    at getNodeRequestOptions (/usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1327:9)
    at /usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1450:19
    at new Promise (<anonymous>)
    at fetch (/usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1447:9)
    at file:///usr/lib/node_modules/percollate/index.js:689:12
TypeError: Only absolute URLs are supported
    at getNodeRequestOptions (/usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1327:9)
    at /usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1450:19
    at new Promise (<anonymous>)
    at fetch (/usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1447:9)
    at file:///usr/lib/node_modules/percollate/index.js:689:12
TypeError: Only absolute URLs are supported
    at getNodeRequestOptions (/usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1327:9)
    at /usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1450:19
    at new Promise (<anonymous>)
    at fetch (/usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1447:9)
    at file:///usr/lib/node_modules/percollate/index.js:689:12
TypeError: Only absolute URLs are supported
    at getNodeRequestOptions (/usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1327:9)
    at /usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1450:19
    at new Promise (<anonymous>)
    at fetch (/usr/lib/node_modules/percollate/node_modules/node-fetch/lib/index.js:1447:9)
    at file:///usr/lib/node_modules/percollate/index.js:689:12
1647716 total bytes, archive closed
Saved EPUB: test.epub

Oddly, the links inside the HTML file are not file:/// links, but I guess no absolute path can be given there, so yeah.

The "saved webpage" HTML instead has: <img src="./foo_files/somehash.jpg"> as the tag

The resulting epub has <img src="./rr-47d0c990-66f9-11ef-a3b5-398245b910c0.jpg" /> so should roughly work, but that file is not present in the epub zip :/

[danburzo]

In the latest Percollate version (4.2.0), the error is:

TypeError: node-fetch cannot load file:///path/to/image.jpg. URL scheme "file" is not supported.

When bundling into an EPUB, images must be fetch()ed locally and the file:// scheme is not supported. I suggest you save just the HTML content (Save Page As… > Web Page, HTML Only), and use the --url option to provide the original URL:

percollate epub -o test.epub local.html --url=https://substack/original/url

[uqs]

Hmm, I'll have to try this later on, but this assumes that the images are not required to have the same authentication as the paywalled article ...