Limit order discovery and Open Food Facts searches

This commit is contained in:
Edward Betts 2026-10-07 11:16:50 +01:00
parent 628e5c3823
commit 41772b4d1a
5 changed files with 185 additions and 78 deletions

View file

@ -62,11 +62,11 @@ Playwright launches Chromium with `headless=False`. Complete sign-in and any CAP
`--profile`, `--archive`, `--config`, and `--login-timeout` customize locations and login waiting time. Profile/archive paths are relative to the working directory. Do not run two browser processes with the same profile.
The command checks the delivered-order list through Ocado's explicit end-of-list marker, but filters out completed orders before loading their receipts. `--limit` counts new orders after this filtering. `--order-id` explicitly selects an old order for retry, refreshing, or importing previously excluded items; `--refresh` alone does not reselect completed orders. It reads each order's authenticated `/api/order/v6/orders/ID/decorated` JSON response as the browser loads it. It does not parse prices or quantities from rendered HTML or use the live trolley. Accepted substitutions are included; unavailable originals and rejected replacements are excluded. Discontinued products work without a product-page link.
The command checks the newest orders first and stops loading history once a batch contains a completed order. On the first run, it continues through Ocado's explicit end-of-list marker. Previously discovered incomplete orders remain eligible for retry, and completed orders are filtered out before loading receipts. Explicit `--order-id` selections continue searching until all requested orders are found or the list ends. `--limit` counts new orders after this filtering. `--order-id` explicitly selects an old order for retry, refreshing, or importing previously excluded items; `--refresh` alone does not reselect completed orders. It reads each order's authenticated `/api/order/v6/orders/ID/decorated` JSON response as the browser loads it. It does not parse prices or quantities from rendered HTML or use the live trolley. Accepted substitutions are included; unavailable originals and rejected replacements are excluded. Discontinued products work without a product-page link.
Delivered item quantities and paid line prices must reconcile with the API order summary before importing. Delivery, bag charges and order-level credits are not stock items. Exact expiry dates and delivery dates come from the API, so historical orders do not depend on relative UI labels such as “Expired” or “Tuesday”.
Normalized receipts are saved to `history/ORDER_ID.json`, excluding account/payment details. Successful downloads are reused unless `--refresh` is specified. `history/orders.json` keeps the full manifest even when you use `--limit` or `--order-id`. Failed orders are listed in `history/download-errors.json` and retried on the next browser run.
Normalized receipts are saved to `history/ORDER_ID.json`, excluding account/payment details. Successful downloads are reused unless `--refresh` is specified. `history/orders.json` retains all previously discovered orders and merges in each new batch, including when you use `--limit` or `--order-id`. Failed orders are listed in `history/download-errors.json` and retried on the next browser run.
## Shelf-stable selection
@ -146,16 +146,16 @@ Tests use sanitized real API data and a fake Grocy API. They cover delivered ord
.venv/bin/ocado-grocy-off # preview, no Grocy writes
.venv/bin/ocado-grocy-off --apply # apply clear matches
.venv/bin/ocado-grocy-off --offline # repeat a preview from cached data
.venv/bin/ocado-grocy-off --refresh-cache # download a fresh catalog
.venv/bin/ocado-grocy-off --refresh-cache # download fresh search responses
```
Set `[openfoodfacts].contact` in `config.toml` (or pass `--contact`) for the identifying User-Agent required by [Open Food Facts](https://openfoodfacts.github.io/openfoodfacts-server/api/). Requests are spaced at least 6.2 seconds apart. Run one enrichment process at a time.
The script reads the import journal for this Grocy server and matches live imported products against brand catalogs from the official [Search-a-licious API](https://openfoodfacts.github.io/search-a-licious/users/ref-openapi/). Large searches are split to avoid truncated results. Recognized household and personal-care categories are skipped. Missing data, different pack sizes, name/variant differences and multiple matching barcodes require review. Automatic matching requires the same brand, meaningful name words and metric pack size, including multipack structure, plus a valid GTIN check digit. Previously established Open Food Facts barcode matches are retained on reruns, including manually reviewed matches. Current product details are fetched again and checked before applying a catalog match.
The script reads the import journal for this Grocy server and searches by brand and product-name words for each eligible imported product using the official [Search-a-licious API](https://openfoodfacts.github.io/search-a-licious/users/ref-openapi/). Normal imports search only products from the orders being processed; queued failed enrichments are retried separately. Each product search reads at most 100 candidates, without paginating through whole brand catalogs. Truncated results require review and cannot produce automatic matches. `--limit` applies before searches, and non-food products do not trigger searches. Recognized household and personal-care categories are skipped. Missing data, different pack sizes, name/variant differences and multiple matching barcodes require review. Automatic matching requires the same brand, meaningful name words and metric pack size, including multipack structure, plus a valid GTIN check digit. Previously established Open Food Facts barcode matches are retained on reruns, including manually reviewed matches. Current product details are fetched again and checked before applying a search match.
Writes add a barcode to Grocy's barcode table and an `openfoodfacts` section to the product description. Available nutrition, ingredient/allergen information, labels, packaging, serving size and image/source links are retained with attribution. Nutrition values keep their source units and basis; they are not copied into Grocy's calories-per-stock-unit field. Product titles, stock, purchase dates and stock notes are never edited. Deleted stock is never recreated. Product descriptions and barcodes are snapshotted before applying, and reruns are idempotent. Future Ocado metadata refreshes preserve the enrichment even after Grocy sanitizes the description HTML.
Private caches, snapshots and reports live in `history/openfoodfacts/`. `report.json` contains the preview and up to ten ranked candidates per product; `applied-report.json` records writes and failures. The catalog is only a source of candidates: Open Food Facts is crowdsourced, and matching by name cannot prove which physical barcode was on a previously purchased pack. Ambiguous matches are left unchanged. Check the current package for authoritative allergen information.
Private caches, snapshots and reports live in `history/openfoodfacts/`. `report.json` contains the preview and up to ten ranked candidates per product; `applied-report.json` records writes and failures. Search results are only candidates: Open Food Facts is crowdsourced, and matching by name cannot prove which physical barcode was on a previously purchased pack. Ambiguous matches are left unchanged. Check the current package for authoritative allergen information.
For a manually verified match, create a JSON mapping of **Grocy product ID** to **barcode string** (retain leading zeroes), then run: