edward/goodreads-backup
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
Code to backup my data from Goodreads. Daily cronjob to download CSV of books I've read. Also grab book recommendations, possibly using Python Playwright.
13 commits 1 branch 0 tags 58 KiB
Find a file
Edward Betts 56dff875e7 Fix bot detection causing blank page from Goodreads 
Goodreads returns an empty HTML page to headless browsers it identifies
as automated (navigator.webdriver=true, AutomationControlled feature).
This caused the Export Library button to never be found.

- Launch Chromium with --disable-blink-features=AutomationControlled
- Set a realistic user agent and viewport on the browser context
- Strip navigator.webdriver via an init script
- Guard the logged_in check so a blank page isn't mistaken for a session
- Wait for networkidle before querying the export button, and use
  wait_for(visible) instead of is_visible() to tolerate JS render delay
- Add CSS class fallback (button.js-LibraryExport) if role lookup fails
- Add AGENTS.md documenting the bot detection pattern and export flow

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 11:53:42 +00:00
templates Add old code for downloading recommendations 2024-04-17 09:11:41 +01:00
.gitignore Add gitignore 2023-12-22 10:56:05 +00:00
AGENTS.md Fix bot detection causing blank page from Goodreads 2026-02-18 11:53:42 +00:00
backup.py Fix bot detection causing blank page from Goodreads 2026-02-18 11:53:42 +00:00
get.py Update code style 2024-04-17 10:02:26 +01:00
LICENSE Initial commit 2023-12-19 10:14:38 +00:00
parse.py Add docstring 2024-04-17 10:05:05 +01:00
README.md Fix line endings 2023-12-22 10:55:55 +00:00

README.md

goodreads-backup

Code to backup my data from Goodreads.

Daily cronjob to download CSV of books I've read.

Also grab book recommendations, possibly using Python Playwright.