Prepare public release with MIT license and private user storage

This commit is contained in:
Edward Betts 2026-10-07 11:32:59 +01:00
parent 41772b4d1a
commit eb8932134a
12 changed files with 175 additions and 26 deletions

21
LICENSE Normal file
View file

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Edward Betts
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View file

@ -12,7 +12,27 @@ python3 -m venv .venv
.venv/bin/python -m playwright install chromium
```
Copy `config.example.toml` to `config.toml` if you do not already have a config, then set the Grocy URL and API key. `GROCY_URL` and `GROCY_API_KEY` override the config.
Create a private user configuration, then set your Grocy URL, API key and Open Food Facts contact:
```sh
install -d -m 700 "${XDG_CONFIG_HOME:-$HOME/.config}/ocado-grocy"
install -m 600 config.example.toml "${XDG_CONFIG_HOME:-$HOME/.config}/ocado-grocy/config.toml"
```
Run the `install` command only for a new configuration: it replaces an existing file. `GROCY_URL` and `GROCY_API_KEY` override the config for the main importer. Ocado sign-in happens in the browser; no Ocado password is stored in the TOML file.
Default private locations (the corresponding `XDG_CONFIG_HOME`, `XDG_STATE_HOME` and `XDG_DATA_HOME` variables are supported):
| Contents | Location |
| --- | --- |
| Configuration and Grocy API key | `~/.config/ocado-grocy/config.toml` |
| Import journal | `~/.local/state/ocado-grocy/imports.sqlite3` |
| Browser profile and login cookies | `~/.local/state/ocado-grocy/browser/` |
| Receipts, images, OFF responses and reports | `~/.local/share/ocado-grocy/history/` |
Below, `history/` means the configured archive directory. The journal prevents duplicate stock imports: keep it together with the receipts when moving installations.
Existing checkouts containing `./config.toml` keep the old local paths until migrated. To migrate, stop the importer and browser, move the configuration, journal, `.ocado-browser/` and `history/` to the locations above, and update `[import].state_file` to the journal's absolute path (or remove it to use the new default). Do not overwrite an existing destination. All three commands use the same journal resolver. Explicit custom configs retain a journal next to that config unless `state_file` overrides it; relative overrides are resolved against the config directory.
## Run
@ -58,9 +78,9 @@ When an order is explicitly selected with `--order-id`, enrichment also runs for
## Authentication and downloads
Playwright launches Chromium with `headless=False`. Complete sign-in and any CAPTCHA in that window. The private `.ocado-browser/` profile saves cookies and local storage for reuse. `.ocado-browser/auth.json` is an additional cookie/storage backup. The application does not store your password. Both the profile and the receipt archive are excluded from Git.
Playwright launches Chromium with `headless=False`. Complete sign-in and any CAPTCHA in that window. The private browser profile saves cookies and local storage for reuse. Its `auth.json` is an additional cookie/storage backup. The application does not store your password. Both the profile and the receipt archive are excluded from Git.
`--profile`, `--archive`, `--config`, and `--login-timeout` customize locations and login waiting time. Profile/archive paths are relative to the working directory. Do not run two browser processes with the same profile.
`--profile`, `--archive`, `--config`, and `--login-timeout` customize locations and login waiting time. Explicit relative profile/archive paths are relative to the working directory; the defaults are user-specific locations outside the checkout. Do not run two browser processes with the same profile.
The command checks the newest orders first and stops loading history once a batch contains a completed order. On the first run, it continues through Ocado's explicit end-of-list marker. Previously discovered incomplete orders remain eligible for retry, and completed orders are filtered out before loading receipts. Explicit `--order-id` selections continue searching until all requested orders are found or the list ends. `--limit` counts new orders after this filtering. `--order-id` explicitly selects an old order for retry, refreshing, or importing previously excluded items; `--refresh` alone does not reselect completed orders. It reads each order's authenticated `/api/order/v6/orders/ID/decorated` JSON response as the browser loads it. It does not parse prices or quantities from rendered HTML or use the live trolley. Accepted substitutions are included; unavailable originals and rejected replacements are excluded. Discontinued products work without a product-page link.
@ -176,3 +196,7 @@ Copy the saved Ocado image URLs into native Grocy product pictures:
```
This covers all existing journal-linked imported products, including non-food products and products whose stock has been used up. It keeps existing pictures and skips deleted products. The main import command runs this step automatically; the standalone command can also be rerun at any time, optionally with `--order-id ORDER_ID`. Downloads use a separate session so Grocy credentials never go to Ocado. Each uploaded file is read back and verified before assigning it to the product, using the [Grocy files API](https://github.com/grocy/grocy/blob/master/grocy.openapi.json). Only the picture field is changed; titles, descriptions and stock remain untouched. Results are saved in `history/images/report.json`. Failed updates can be retried safely.
## License
The project code is available under the [MIT License](LICENSE). Dependencies, Ocado content and downloaded Open Food Facts data and images retain their respective licences; the MIT licence does not relicense that material.

View file

@ -6,8 +6,9 @@ api_key = "replace-with-your-key"
# Used only for newly created products. Existing products keep their location.
location = "Ocado imports"
quantity_unit = "Pack"
# Relative to this config file. Keep this journal to prevent duplicate imports.
state_file = "imports.sqlite3"
# Default journal: ~/.local/state/ocado-grocy/imports.sqlite3 (XDG_STATE_HOME supported).
# Keep this journal to prevent duplicate imports. Optional override:
# state_file = "/absolute/path/to/imports.sqlite3"
# Optional: map an Ocado retailer product ID (or exact receipt name if no ID)
# to an existing Grocy product. Stock units per purchased pack can be overridden.

View file

@ -61,7 +61,7 @@ class Grocy:
class Journal:
"""Commit intent before each write; ambiguous writes require explicit reconciliation."""
def __init__(self, path: Path):
path.parent.mkdir(parents=True, exist_ok=True)
path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
self.db = sqlite3.connect(path)
self.db.execute("""CREATE TABLE IF NOT EXISTS imports (
server TEXT, order_id TEXT, line INTEGER, fingerprint TEXT,

View file

@ -14,6 +14,7 @@ import click
from lxml import html
from .grocy import Grocy, Journal, import_receipt, refresh_imported_products, align_recorded_lines
from .paths import config_file, archive_directory, browser_directory, journal_file
from .pantry import classify
from .receipt import ImportError, parse_document, parse_ocado_order, receipt_date
@ -228,9 +229,7 @@ def run_import(receipts, config, config_path, archive, dry_run, failures, echo=p
if not url or not key:
raise ImportError("Configure Grocy URL and API key before importing")
api = Grocy(url,key)
state = Path(settings.get("state_file", "imports.sqlite3"))
if not state.is_absolute():
state = config_path.resolve().parent / state
state = journal_file(config, config_path)
journal = Journal(state)
for receipt in receipts:
if not dry_run:
@ -266,9 +265,9 @@ def run_import(receipts, config, config_path, archive, dry_run, failures, echo=p
@click.command(context_settings={"help_option_names":["-h","--help"]})
@click.option("--config", "config_path", type=click.Path(path_type=Path,dir_okay=False),default="config.toml",show_default=True)
@click.option("--profile", type=click.Path(path_type=Path,file_okay=False),default=".ocado-browser",show_default=True)
@click.option("--archive", type=click.Path(path_type=Path,file_okay=False),default="history",show_default=True)
@click.option("--config", "config_path", type=click.Path(path_type=Path,dir_okay=False),default=config_file,show_default="~/.config/ocado-grocy/config.toml")
@click.option("--profile", type=click.Path(path_type=Path,file_okay=False),default=browser_directory,show_default="~/.local/state/ocado-grocy/browser")
@click.option("--archive", type=click.Path(path_type=Path,file_okay=False),default=archive_directory,show_default="~/.local/share/ocado-grocy/history")
@click.option("--since", help="Only orders delivered on or after YYYY-MM-DD.")
@click.option("--limit", type=click.IntRange(min=1),help="Process at most this many new delivered orders (or explicitly selected orders).")
@click.option("--dry-run", is_flag=True,help="Download and report selections without writing to Grocy.")
@ -296,9 +295,7 @@ def main(config_path,profile,archive,since,limit,dry_run,cached,login_only,login
connection = config.get('grocy', {})
server = Grocy(os.environ.get('GROCY_URL', connection.get('url', '')),
os.environ.get('GROCY_API_KEY', connection.get('api_key', ''))).url
state = Path(config.get('import', {}).get('state_file', 'imports.sqlite3'))
if not state.is_absolute():
state = config_path.resolve().parent / state
state = journal_file(config, config_path)
completed = completed_orders(state, server, archive, config)
if not dry_run and not login_only:
mark_orders(state, server, completed, 'complete')

View file

@ -10,6 +10,7 @@ import click
import requests
from .grocy import Grocy, imported_products
from .paths import config_file, archive_directory, journal_file
from .history import private_json
from .product_metadata import read_metadata
@ -72,15 +73,15 @@ def add_picture(client, downloader, product):
@click.command()
@click.option('--config', type=click.Path(path_type=Path), default=Path('config.toml'))
@click.option('--report', type=click.Path(path_type=Path), default=Path('history/images/report.json'))
@click.option('--config', type=click.Path(path_type=Path), default=config_file, show_default='~/.config/ocado-grocy/config.toml')
@click.option('--report', type=click.Path(path_type=Path), default=lambda: archive_directory() / 'images/report.json', show_default='archive/images/report.json')
@click.option('--apply', is_flag=True, help='Upload and assign missing product pictures.')
@click.option('--order-id', 'order_ids', multiple=True, help='Limit to products imported from these orders.')
def main(config, report, apply, order_ids):
"""Add Ocado images to journal-linked products; keep existing pictures."""
settings = tomllib.loads(config.read_text())
client = Grocy(**settings['grocy'])
state = config.parent / settings.get('import', {}).get('state_file', 'imports.sqlite3')
state = journal_file(settings, config)
products = imported_products(client, state, order_ids)
return run_images(client, products, report, apply=apply)

View file

@ -12,6 +12,7 @@ import click
import requests
from .grocy import Grocy, imported_products
from .paths import config_file, archive_directory, journal_file
from .history import private_json
from .product_metadata import read_metadata, replace_metadata
@ -166,8 +167,8 @@ def apply_match(client, product, candidate, barcodes):
@click.command()
@click.option('--config', type=click.Path(path_type=Path), default=Path('config.toml'))
@click.option('--cache', type=click.Path(path_type=Path), default=Path('history/openfoodfacts'))
@click.option('--config', type=click.Path(path_type=Path), default=config_file, show_default='~/.config/ocado-grocy/config.toml')
@click.option('--cache', type=click.Path(path_type=Path), default=lambda: archive_directory() / 'openfoodfacts', show_default='archive/openfoodfacts')
@click.option('--contact', help='Contact for the Open Food Facts User-Agent; defaults to config [openfoodfacts].contact.')
@click.option('--refresh-cache', is_flag=True, help='Fetch fresh OFF responses, replacing cached responses.')
@click.option('--apply', is_flag=True, help='Apply unambiguous matches and explicitly reviewed mappings.')
@ -184,7 +185,7 @@ def main(config, cache, contact, refresh_cache, apply, offline, mapping, limit,
if refresh_cache and offline:
raise click.UsageError('--refresh-cache cannot be combined with --offline')
client = Grocy(**settings['grocy'])
state = config.parent / settings.get('import', {}).get('state_file', 'imports.sqlite3')
state = journal_file(settings, config)
products = imported_products(client, state, order_ids)
mappings = json.loads(mapping.read_text()) if mapping else {}
return run_openfoodfacts(client, products, cache, contact, apply=apply, offline=offline,

View file

@ -45,7 +45,7 @@ def completed_orders(state, server, archive, config):
def mark_orders(state, server, orders, status):
if not orders:
return
state.parent.mkdir(parents=True, exist_ok=True)
state.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
with sqlite3.connect(state) as db:
db.execute('''CREATE TABLE IF NOT EXISTS loaded_orders (
server TEXT NOT NULL, order_id TEXT NOT NULL, status TEXT NOT NULL,

44
ocado_grocy/paths.py Normal file
View file

@ -0,0 +1,44 @@
"""User-specific paths, with compatibility for existing checkout installations."""
import os
from pathlib import Path
def user_directory(variable, fallback):
value = os.environ.get(variable, '')
# XDG paths must be absolute; ignore empty or relative values.
base = Path(value) if value and Path(value).is_absolute() else Path.home() / fallback
return base / 'ocado-grocy'
def config_file():
# Keep an existing installation on its original journal until migrated.
legacy = Path('config.toml')
return legacy if legacy.exists() else user_directory('XDG_CONFIG_HOME', '.config') / 'config.toml'
def state_directory():
return user_directory('XDG_STATE_HOME', '.local/state')
def archive_directory():
if Path('config.toml').exists():
return Path('history')
return user_directory('XDG_DATA_HOME', '.local/share') / 'history'
def browser_directory():
if Path('config.toml').exists():
return Path('.ocado-browser')
return state_directory() / 'browser'
def journal_file(settings, config_path):
configured = settings.get('import', {}).get('state_file')
if configured:
path = Path(configured).expanduser()
return path if path.is_absolute() else config_path.resolve().parent / path
user_config = user_directory('XDG_CONFIG_HOME', '.config') / 'config.toml'
if config_path.resolve() == user_config.resolve():
return state_directory() / 'imports.sqlite3'
# Explicit custom/legacy configs retain the historical relative default.
return config_path.resolve().parent / 'imports.sqlite3'

View file

@ -6,6 +6,7 @@ from pathlib import Path
import click
from .grocy import Grocy, imported_products
from .paths import journal_file
from .history import private_json
from .images import run_images
from .openfoodfacts import run_openfoodfacts
@ -18,9 +19,7 @@ def enrich_orders(config, config_path, archive, order_ids, *, images=True, openf
connection = config.get('grocy', {})
client = Grocy(os.environ.get('GROCY_URL', connection.get('url', '')),
os.environ.get('GROCY_API_KEY', connection.get('api_key', '')))
state = Path(config.get('import', {}).get('state_file', 'imports.sqlite3'))
if not state.is_absolute():
state = config_path.resolve().parent / state
state = journal_file(config, config_path)
products = imported_products(client, state, order_ids)
key = order_ids[0] if len(order_ids) == 1 and order_ids[0].isdigit() else hashlib.sha256(','.join(sorted(order_ids)).encode()).hexdigest()[:16]
output = archive / 'enrichment' / key
@ -65,7 +64,7 @@ def retry_enrichment(config, config_path, archive, *, images=True, openfoodfacts
connection = config.get('grocy', {})
client = Grocy(os.environ.get('GROCY_URL', connection.get('url','')),
os.environ.get('GROCY_API_KEY', connection.get('api_key','')))
state = config_path.resolve().parent / config.get('import',{}).get('state_file','imports.sqlite3')
state = journal_file(config, config_path)
if not state.exists():
return {'errors':[]}
queue = RetryQueue(state, client.url)

View file

@ -7,6 +7,7 @@ name = "ocado-grocy"
version = "0.1.0"
description = "Import Ocado orders into Grocy using Playwright"
requires-python = ">=3.11"
license = {file = "LICENSE"}
dependencies = ["lxml>=5,<7", "click>=8.1,<9", "requests>=2.31,<3", "playwright>=1.50,<2"]
[project.optional-dependencies]

60
tests/test_paths.py Normal file
View file

@ -0,0 +1,60 @@
from pathlib import Path
from click.testing import CliRunner
from ocado_grocy import paths
from ocado_grocy.history import main
def isolate(tmp_path, monkeypatch):
monkeypatch.chdir(tmp_path)
for name, directory in [('XDG_CONFIG_HOME', 'config'), ('XDG_STATE_HOME', 'state'), ('XDG_DATA_HOME', 'data')]:
monkeypatch.setenv(name, str(tmp_path / directory))
def test_xdg_paths_and_shared_journal(tmp_path, monkeypatch):
isolate(tmp_path, monkeypatch)
assert paths.config_file() == tmp_path / 'config/ocado-grocy/config.toml'
assert paths.browser_directory() == tmp_path / 'state/ocado-grocy/browser'
assert paths.archive_directory() == tmp_path / 'data/ocado-grocy/history'
assert paths.journal_file({}, paths.config_file()) == tmp_path / 'state/ocado-grocy/imports.sqlite3'
def test_legacy_installation_keeps_its_journal_and_archive(tmp_path, monkeypatch):
isolate(tmp_path, monkeypatch)
(tmp_path / 'config.toml').write_text('')
assert paths.config_file() == Path('config.toml')
assert paths.browser_directory() == Path('.ocado-browser')
assert paths.archive_directory() == Path('history')
assert paths.journal_file({}, paths.config_file()) == tmp_path / 'imports.sqlite3'
def test_custom_journal_paths(tmp_path, monkeypatch):
isolate(tmp_path, monkeypatch)
config = tmp_path / 'custom/config.toml'
assert paths.journal_file({}, config) == config.parent / 'imports.sqlite3'
assert paths.journal_file({'import': {'state_file': 'journal.db'}}, config) == config.parent / 'journal.db'
target = tmp_path / 'journal.db'
assert paths.journal_file({'import': {'state_file': str(target)}}, config) == target
def test_invalid_xdg_values_use_home(tmp_path, monkeypatch):
isolate(tmp_path, monkeypatch)
monkeypatch.setattr(Path, 'home', classmethod(lambda cls: tmp_path))
monkeypatch.setenv('XDG_CONFIG_HOME', 'relative')
monkeypatch.setenv('XDG_STATE_HOME', '')
assert paths.config_file() == tmp_path / '.config/ocado-grocy/config.toml'
assert paths.state_directory() == tmp_path / '.local/state/ocado-grocy'
def test_cli_defaults_are_resolved_at_invocation(tmp_path, monkeypatch):
isolate(tmp_path, monkeypatch)
calls = []
def download(profile, archive, **kwargs):
calls.append((profile, archive))
return [], []
monkeypatch.setattr('ocado_grocy.history.download_orders', download)
result = CliRunner().invoke(main, ['--no-images', '--no-openfoodfacts'])
assert result.exit_code == 0, result.output
assert calls == [(paths.browser_directory(), paths.archive_directory())]
assert not (tmp_path / 'imports.sqlite3').exists()