Show recent Commons uploads obtained via Flickr mail

Display recent Wikimedia Commons uploads on the home page, filtered to
only show images that were obtained by contacting creators via Flickr
mail. Each upload shows:
- Thumbnail linking to Commons
- Creator name linking to their Flickr profile
- Link to the illustrated Wikipedia article (or Wikidata item)

Features:
- Parse sent mail messages to extract Flickr and Wikipedia URLs
- Match Commons uploads with sent mail by normalized Flickr URL
- Cache Commons API thumbnail responses and sent mail index
- Handle Wikidata item URLs (Q-numbers) with correct links
- Add update_flickr_uploads.py script to find uploads from UploadWizard
  contributions by checking Commons API metadata

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Edward Betts 2026-02-06 10:42:34 +00:00
parent 0062de8ede
commit a2d29d7937
6 changed files with 562 additions and 3 deletions

View file

@ -66,6 +66,34 @@ static image servers:
Converts a Flickr username/path alias to the NSID (internal user ID) needed
for the Flickr mail URL. Scrapes the user's profile page for embedded params.
### Commons Uploads Display
Shows recent Wikimedia Commons uploads on the home page, filtered to only
those obtained via Flickr mail requests.
**Data files** (in `commons_contributions/`):
- `flickr_uploads.json`: List of Commons uploads from Flickr with metadata
- `thumbnail_cache.json`: Cached Commons API thumbnail URLs (7-day TTL)
- `sent_mail_index.json`: Index of sent mail messages (flickr_url → wikipedia_url)
**Key functions**:
- `build_sent_mail_index()`: Parses sent mail JSON files, extracts Flickr and
Wikipedia URLs from message bodies, caches the index
- `get_recent_commons_uploads()`: Loads uploads, filters by sent mail match,
fetches thumbnails from Commons API
- `normalize_flickr_url()`: Normalizes URLs for matching (removes protocol, www, trailing slash)
**CommonsUpload dataclass**:
- `title`, `thumb_url`, `commons_url`, `flickr_url`, `creator`, `timestamp`
- `wikipedia_url`, `creator_profile_url`: Extracted from sent mail
- `is_wikidata_item` property: Detects Q-number URLs
- `wiki_link_url`, `wiki_link_label`: Handles Wikidata vs Wikipedia links
**Maintenance script** (`update_flickr_uploads.py`):
Run to find Flickr uploads from UploadWizard contributions that don't have
the Flickr URL in the edit comment. Queries Commons API for image metadata
and checks the Credit field for Flickr URLs.
## Request Flow
1. User enters Wikipedia article title/URL → `start()` extracts article name