Prepare public release with MIT license and private user storage
This commit is contained in:
parent
41772b4d1a
commit
eb8932134a
12 changed files with 175 additions and 26 deletions
21
LICENSE
Normal file
21
LICENSE
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
MIT License
|
||||
|
||||
Copyright (c) 2026 Edward Betts
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
30
README.md
30
README.md
|
|
@ -12,7 +12,27 @@ python3 -m venv .venv
|
|||
.venv/bin/python -m playwright install chromium
|
||||
```
|
||||
|
||||
Copy `config.example.toml` to `config.toml` if you do not already have a config, then set the Grocy URL and API key. `GROCY_URL` and `GROCY_API_KEY` override the config.
|
||||
Create a private user configuration, then set your Grocy URL, API key and Open Food Facts contact:
|
||||
|
||||
```sh
|
||||
install -d -m 700 "${XDG_CONFIG_HOME:-$HOME/.config}/ocado-grocy"
|
||||
install -m 600 config.example.toml "${XDG_CONFIG_HOME:-$HOME/.config}/ocado-grocy/config.toml"
|
||||
```
|
||||
|
||||
Run the `install` command only for a new configuration: it replaces an existing file. `GROCY_URL` and `GROCY_API_KEY` override the config for the main importer. Ocado sign-in happens in the browser; no Ocado password is stored in the TOML file.
|
||||
|
||||
Default private locations (the corresponding `XDG_CONFIG_HOME`, `XDG_STATE_HOME` and `XDG_DATA_HOME` variables are supported):
|
||||
|
||||
| Contents | Location |
|
||||
| --- | --- |
|
||||
| Configuration and Grocy API key | `~/.config/ocado-grocy/config.toml` |
|
||||
| Import journal | `~/.local/state/ocado-grocy/imports.sqlite3` |
|
||||
| Browser profile and login cookies | `~/.local/state/ocado-grocy/browser/` |
|
||||
| Receipts, images, OFF responses and reports | `~/.local/share/ocado-grocy/history/` |
|
||||
|
||||
Below, `history/` means the configured archive directory. The journal prevents duplicate stock imports: keep it together with the receipts when moving installations.
|
||||
|
||||
Existing checkouts containing `./config.toml` keep the old local paths until migrated. To migrate, stop the importer and browser, move the configuration, journal, `.ocado-browser/` and `history/` to the locations above, and update `[import].state_file` to the journal's absolute path (or remove it to use the new default). Do not overwrite an existing destination. All three commands use the same journal resolver. Explicit custom configs retain a journal next to that config unless `state_file` overrides it; relative overrides are resolved against the config directory.
|
||||
|
||||
## Run
|
||||
|
||||
|
|
@ -58,9 +78,9 @@ When an order is explicitly selected with `--order-id`, enrichment also runs for
|
|||
|
||||
## Authentication and downloads
|
||||
|
||||
Playwright launches Chromium with `headless=False`. Complete sign-in and any CAPTCHA in that window. The private `.ocado-browser/` profile saves cookies and local storage for reuse. `.ocado-browser/auth.json` is an additional cookie/storage backup. The application does not store your password. Both the profile and the receipt archive are excluded from Git.
|
||||
Playwright launches Chromium with `headless=False`. Complete sign-in and any CAPTCHA in that window. The private browser profile saves cookies and local storage for reuse. Its `auth.json` is an additional cookie/storage backup. The application does not store your password. Both the profile and the receipt archive are excluded from Git.
|
||||
|
||||
`--profile`, `--archive`, `--config`, and `--login-timeout` customize locations and login waiting time. Profile/archive paths are relative to the working directory. Do not run two browser processes with the same profile.
|
||||
`--profile`, `--archive`, `--config`, and `--login-timeout` customize locations and login waiting time. Explicit relative profile/archive paths are relative to the working directory; the defaults are user-specific locations outside the checkout. Do not run two browser processes with the same profile.
|
||||
|
||||
The command checks the newest orders first and stops loading history once a batch contains a completed order. On the first run, it continues through Ocado's explicit end-of-list marker. Previously discovered incomplete orders remain eligible for retry, and completed orders are filtered out before loading receipts. Explicit `--order-id` selections continue searching until all requested orders are found or the list ends. `--limit` counts new orders after this filtering. `--order-id` explicitly selects an old order for retry, refreshing, or importing previously excluded items; `--refresh` alone does not reselect completed orders. It reads each order's authenticated `/api/order/v6/orders/ID/decorated` JSON response as the browser loads it. It does not parse prices or quantities from rendered HTML or use the live trolley. Accepted substitutions are included; unavailable originals and rejected replacements are excluded. Discontinued products work without a product-page link.
|
||||
|
||||
|
|
@ -176,3 +196,7 @@ Copy the saved Ocado image URLs into native Grocy product pictures:
|
|||
```
|
||||
|
||||
This covers all existing journal-linked imported products, including non-food products and products whose stock has been used up. It keeps existing pictures and skips deleted products. The main import command runs this step automatically; the standalone command can also be rerun at any time, optionally with `--order-id ORDER_ID`. Downloads use a separate session so Grocy credentials never go to Ocado. Each uploaded file is read back and verified before assigning it to the product, using the [Grocy files API](https://github.com/grocy/grocy/blob/master/grocy.openapi.json). Only the picture field is changed; titles, descriptions and stock remain untouched. Results are saved in `history/images/report.json`. Failed updates can be retried safely.
|
||||
|
||||
## License
|
||||
|
||||
The project code is available under the [MIT License](LICENSE). Dependencies, Ocado content and downloaded Open Food Facts data and images retain their respective licences; the MIT licence does not relicense that material.
|
||||
|
|
|
|||
|
|
@ -6,8 +6,9 @@ api_key = "replace-with-your-key"
|
|||
# Used only for newly created products. Existing products keep their location.
|
||||
location = "Ocado imports"
|
||||
quantity_unit = "Pack"
|
||||
# Relative to this config file. Keep this journal to prevent duplicate imports.
|
||||
state_file = "imports.sqlite3"
|
||||
# Default journal: ~/.local/state/ocado-grocy/imports.sqlite3 (XDG_STATE_HOME supported).
|
||||
# Keep this journal to prevent duplicate imports. Optional override:
|
||||
# state_file = "/absolute/path/to/imports.sqlite3"
|
||||
|
||||
# Optional: map an Ocado retailer product ID (or exact receipt name if no ID)
|
||||
# to an existing Grocy product. Stock units per purchased pack can be overridden.
|
||||
|
|
|
|||
|
|
@ -61,7 +61,7 @@ class Grocy:
|
|||
class Journal:
|
||||
"""Commit intent before each write; ambiguous writes require explicit reconciliation."""
|
||||
def __init__(self, path: Path):
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
|
||||
self.db = sqlite3.connect(path)
|
||||
self.db.execute("""CREATE TABLE IF NOT EXISTS imports (
|
||||
server TEXT, order_id TEXT, line INTEGER, fingerprint TEXT,
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ import click
|
|||
from lxml import html
|
||||
|
||||
from .grocy import Grocy, Journal, import_receipt, refresh_imported_products, align_recorded_lines
|
||||
from .paths import config_file, archive_directory, browser_directory, journal_file
|
||||
from .pantry import classify
|
||||
from .receipt import ImportError, parse_document, parse_ocado_order, receipt_date
|
||||
|
||||
|
|
@ -228,9 +229,7 @@ def run_import(receipts, config, config_path, archive, dry_run, failures, echo=p
|
|||
if not url or not key:
|
||||
raise ImportError("Configure Grocy URL and API key before importing")
|
||||
api = Grocy(url,key)
|
||||
state = Path(settings.get("state_file", "imports.sqlite3"))
|
||||
if not state.is_absolute():
|
||||
state = config_path.resolve().parent / state
|
||||
state = journal_file(config, config_path)
|
||||
journal = Journal(state)
|
||||
for receipt in receipts:
|
||||
if not dry_run:
|
||||
|
|
@ -266,9 +265,9 @@ def run_import(receipts, config, config_path, archive, dry_run, failures, echo=p
|
|||
|
||||
|
||||
@click.command(context_settings={"help_option_names":["-h","--help"]})
|
||||
@click.option("--config", "config_path", type=click.Path(path_type=Path,dir_okay=False),default="config.toml",show_default=True)
|
||||
@click.option("--profile", type=click.Path(path_type=Path,file_okay=False),default=".ocado-browser",show_default=True)
|
||||
@click.option("--archive", type=click.Path(path_type=Path,file_okay=False),default="history",show_default=True)
|
||||
@click.option("--config", "config_path", type=click.Path(path_type=Path,dir_okay=False),default=config_file,show_default="~/.config/ocado-grocy/config.toml")
|
||||
@click.option("--profile", type=click.Path(path_type=Path,file_okay=False),default=browser_directory,show_default="~/.local/state/ocado-grocy/browser")
|
||||
@click.option("--archive", type=click.Path(path_type=Path,file_okay=False),default=archive_directory,show_default="~/.local/share/ocado-grocy/history")
|
||||
@click.option("--since", help="Only orders delivered on or after YYYY-MM-DD.")
|
||||
@click.option("--limit", type=click.IntRange(min=1),help="Process at most this many new delivered orders (or explicitly selected orders).")
|
||||
@click.option("--dry-run", is_flag=True,help="Download and report selections without writing to Grocy.")
|
||||
|
|
@ -296,9 +295,7 @@ def main(config_path,profile,archive,since,limit,dry_run,cached,login_only,login
|
|||
connection = config.get('grocy', {})
|
||||
server = Grocy(os.environ.get('GROCY_URL', connection.get('url', '')),
|
||||
os.environ.get('GROCY_API_KEY', connection.get('api_key', ''))).url
|
||||
state = Path(config.get('import', {}).get('state_file', 'imports.sqlite3'))
|
||||
if not state.is_absolute():
|
||||
state = config_path.resolve().parent / state
|
||||
state = journal_file(config, config_path)
|
||||
completed = completed_orders(state, server, archive, config)
|
||||
if not dry_run and not login_only:
|
||||
mark_orders(state, server, completed, 'complete')
|
||||
|
|
|
|||
|
|
@ -10,6 +10,7 @@ import click
|
|||
import requests
|
||||
|
||||
from .grocy import Grocy, imported_products
|
||||
from .paths import config_file, archive_directory, journal_file
|
||||
from .history import private_json
|
||||
from .product_metadata import read_metadata
|
||||
|
||||
|
|
@ -72,15 +73,15 @@ def add_picture(client, downloader, product):
|
|||
|
||||
|
||||
@click.command()
|
||||
@click.option('--config', type=click.Path(path_type=Path), default=Path('config.toml'))
|
||||
@click.option('--report', type=click.Path(path_type=Path), default=Path('history/images/report.json'))
|
||||
@click.option('--config', type=click.Path(path_type=Path), default=config_file, show_default='~/.config/ocado-grocy/config.toml')
|
||||
@click.option('--report', type=click.Path(path_type=Path), default=lambda: archive_directory() / 'images/report.json', show_default='archive/images/report.json')
|
||||
@click.option('--apply', is_flag=True, help='Upload and assign missing product pictures.')
|
||||
@click.option('--order-id', 'order_ids', multiple=True, help='Limit to products imported from these orders.')
|
||||
def main(config, report, apply, order_ids):
|
||||
"""Add Ocado images to journal-linked products; keep existing pictures."""
|
||||
settings = tomllib.loads(config.read_text())
|
||||
client = Grocy(**settings['grocy'])
|
||||
state = config.parent / settings.get('import', {}).get('state_file', 'imports.sqlite3')
|
||||
state = journal_file(settings, config)
|
||||
products = imported_products(client, state, order_ids)
|
||||
return run_images(client, products, report, apply=apply)
|
||||
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ import click
|
|||
import requests
|
||||
|
||||
from .grocy import Grocy, imported_products
|
||||
from .paths import config_file, archive_directory, journal_file
|
||||
from .history import private_json
|
||||
from .product_metadata import read_metadata, replace_metadata
|
||||
|
||||
|
|
@ -166,8 +167,8 @@ def apply_match(client, product, candidate, barcodes):
|
|||
|
||||
|
||||
@click.command()
|
||||
@click.option('--config', type=click.Path(path_type=Path), default=Path('config.toml'))
|
||||
@click.option('--cache', type=click.Path(path_type=Path), default=Path('history/openfoodfacts'))
|
||||
@click.option('--config', type=click.Path(path_type=Path), default=config_file, show_default='~/.config/ocado-grocy/config.toml')
|
||||
@click.option('--cache', type=click.Path(path_type=Path), default=lambda: archive_directory() / 'openfoodfacts', show_default='archive/openfoodfacts')
|
||||
@click.option('--contact', help='Contact for the Open Food Facts User-Agent; defaults to config [openfoodfacts].contact.')
|
||||
@click.option('--refresh-cache', is_flag=True, help='Fetch fresh OFF responses, replacing cached responses.')
|
||||
@click.option('--apply', is_flag=True, help='Apply unambiguous matches and explicitly reviewed mappings.')
|
||||
|
|
@ -184,7 +185,7 @@ def main(config, cache, contact, refresh_cache, apply, offline, mapping, limit,
|
|||
if refresh_cache and offline:
|
||||
raise click.UsageError('--refresh-cache cannot be combined with --offline')
|
||||
client = Grocy(**settings['grocy'])
|
||||
state = config.parent / settings.get('import', {}).get('state_file', 'imports.sqlite3')
|
||||
state = journal_file(settings, config)
|
||||
products = imported_products(client, state, order_ids)
|
||||
mappings = json.loads(mapping.read_text()) if mapping else {}
|
||||
return run_openfoodfacts(client, products, cache, contact, apply=apply, offline=offline,
|
||||
|
|
|
|||
|
|
@ -45,7 +45,7 @@ def completed_orders(state, server, archive, config):
|
|||
def mark_orders(state, server, orders, status):
|
||||
if not orders:
|
||||
return
|
||||
state.parent.mkdir(parents=True, exist_ok=True)
|
||||
state.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
|
||||
with sqlite3.connect(state) as db:
|
||||
db.execute('''CREATE TABLE IF NOT EXISTS loaded_orders (
|
||||
server TEXT NOT NULL, order_id TEXT NOT NULL, status TEXT NOT NULL,
|
||||
|
|
|
|||
44
ocado_grocy/paths.py
Normal file
44
ocado_grocy/paths.py
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
"""User-specific paths, with compatibility for existing checkout installations."""
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def user_directory(variable, fallback):
|
||||
value = os.environ.get(variable, '')
|
||||
# XDG paths must be absolute; ignore empty or relative values.
|
||||
base = Path(value) if value and Path(value).is_absolute() else Path.home() / fallback
|
||||
return base / 'ocado-grocy'
|
||||
|
||||
|
||||
def config_file():
|
||||
# Keep an existing installation on its original journal until migrated.
|
||||
legacy = Path('config.toml')
|
||||
return legacy if legacy.exists() else user_directory('XDG_CONFIG_HOME', '.config') / 'config.toml'
|
||||
|
||||
|
||||
def state_directory():
|
||||
return user_directory('XDG_STATE_HOME', '.local/state')
|
||||
|
||||
|
||||
def archive_directory():
|
||||
if Path('config.toml').exists():
|
||||
return Path('history')
|
||||
return user_directory('XDG_DATA_HOME', '.local/share') / 'history'
|
||||
|
||||
|
||||
def browser_directory():
|
||||
if Path('config.toml').exists():
|
||||
return Path('.ocado-browser')
|
||||
return state_directory() / 'browser'
|
||||
|
||||
|
||||
def journal_file(settings, config_path):
|
||||
configured = settings.get('import', {}).get('state_file')
|
||||
if configured:
|
||||
path = Path(configured).expanduser()
|
||||
return path if path.is_absolute() else config_path.resolve().parent / path
|
||||
user_config = user_directory('XDG_CONFIG_HOME', '.config') / 'config.toml'
|
||||
if config_path.resolve() == user_config.resolve():
|
||||
return state_directory() / 'imports.sqlite3'
|
||||
# Explicit custom/legacy configs retain the historical relative default.
|
||||
return config_path.resolve().parent / 'imports.sqlite3'
|
||||
|
|
@ -6,6 +6,7 @@ from pathlib import Path
|
|||
import click
|
||||
|
||||
from .grocy import Grocy, imported_products
|
||||
from .paths import journal_file
|
||||
from .history import private_json
|
||||
from .images import run_images
|
||||
from .openfoodfacts import run_openfoodfacts
|
||||
|
|
@ -18,9 +19,7 @@ def enrich_orders(config, config_path, archive, order_ids, *, images=True, openf
|
|||
connection = config.get('grocy', {})
|
||||
client = Grocy(os.environ.get('GROCY_URL', connection.get('url', '')),
|
||||
os.environ.get('GROCY_API_KEY', connection.get('api_key', '')))
|
||||
state = Path(config.get('import', {}).get('state_file', 'imports.sqlite3'))
|
||||
if not state.is_absolute():
|
||||
state = config_path.resolve().parent / state
|
||||
state = journal_file(config, config_path)
|
||||
products = imported_products(client, state, order_ids)
|
||||
key = order_ids[0] if len(order_ids) == 1 and order_ids[0].isdigit() else hashlib.sha256(','.join(sorted(order_ids)).encode()).hexdigest()[:16]
|
||||
output = archive / 'enrichment' / key
|
||||
|
|
@ -65,7 +64,7 @@ def retry_enrichment(config, config_path, archive, *, images=True, openfoodfacts
|
|||
connection = config.get('grocy', {})
|
||||
client = Grocy(os.environ.get('GROCY_URL', connection.get('url','')),
|
||||
os.environ.get('GROCY_API_KEY', connection.get('api_key','')))
|
||||
state = config_path.resolve().parent / config.get('import',{}).get('state_file','imports.sqlite3')
|
||||
state = journal_file(config, config_path)
|
||||
if not state.exists():
|
||||
return {'errors':[]}
|
||||
queue = RetryQueue(state, client.url)
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ name = "ocado-grocy"
|
|||
version = "0.1.0"
|
||||
description = "Import Ocado orders into Grocy using Playwright"
|
||||
requires-python = ">=3.11"
|
||||
license = {file = "LICENSE"}
|
||||
dependencies = ["lxml>=5,<7", "click>=8.1,<9", "requests>=2.31,<3", "playwright>=1.50,<2"]
|
||||
|
||||
[project.optional-dependencies]
|
||||
|
|
|
|||
60
tests/test_paths.py
Normal file
60
tests/test_paths.py
Normal file
|
|
@ -0,0 +1,60 @@
|
|||
from pathlib import Path
|
||||
|
||||
from click.testing import CliRunner
|
||||
|
||||
from ocado_grocy import paths
|
||||
from ocado_grocy.history import main
|
||||
|
||||
|
||||
def isolate(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
for name, directory in [('XDG_CONFIG_HOME', 'config'), ('XDG_STATE_HOME', 'state'), ('XDG_DATA_HOME', 'data')]:
|
||||
monkeypatch.setenv(name, str(tmp_path / directory))
|
||||
|
||||
|
||||
def test_xdg_paths_and_shared_journal(tmp_path, monkeypatch):
|
||||
isolate(tmp_path, monkeypatch)
|
||||
assert paths.config_file() == tmp_path / 'config/ocado-grocy/config.toml'
|
||||
assert paths.browser_directory() == tmp_path / 'state/ocado-grocy/browser'
|
||||
assert paths.archive_directory() == tmp_path / 'data/ocado-grocy/history'
|
||||
assert paths.journal_file({}, paths.config_file()) == tmp_path / 'state/ocado-grocy/imports.sqlite3'
|
||||
|
||||
|
||||
def test_legacy_installation_keeps_its_journal_and_archive(tmp_path, monkeypatch):
|
||||
isolate(tmp_path, monkeypatch)
|
||||
(tmp_path / 'config.toml').write_text('')
|
||||
assert paths.config_file() == Path('config.toml')
|
||||
assert paths.browser_directory() == Path('.ocado-browser')
|
||||
assert paths.archive_directory() == Path('history')
|
||||
assert paths.journal_file({}, paths.config_file()) == tmp_path / 'imports.sqlite3'
|
||||
|
||||
|
||||
def test_custom_journal_paths(tmp_path, monkeypatch):
|
||||
isolate(tmp_path, monkeypatch)
|
||||
config = tmp_path / 'custom/config.toml'
|
||||
assert paths.journal_file({}, config) == config.parent / 'imports.sqlite3'
|
||||
assert paths.journal_file({'import': {'state_file': 'journal.db'}}, config) == config.parent / 'journal.db'
|
||||
target = tmp_path / 'journal.db'
|
||||
assert paths.journal_file({'import': {'state_file': str(target)}}, config) == target
|
||||
|
||||
|
||||
def test_invalid_xdg_values_use_home(tmp_path, monkeypatch):
|
||||
isolate(tmp_path, monkeypatch)
|
||||
monkeypatch.setattr(Path, 'home', classmethod(lambda cls: tmp_path))
|
||||
monkeypatch.setenv('XDG_CONFIG_HOME', 'relative')
|
||||
monkeypatch.setenv('XDG_STATE_HOME', '')
|
||||
assert paths.config_file() == tmp_path / '.config/ocado-grocy/config.toml'
|
||||
assert paths.state_directory() == tmp_path / '.local/state/ocado-grocy'
|
||||
|
||||
|
||||
def test_cli_defaults_are_resolved_at_invocation(tmp_path, monkeypatch):
|
||||
isolate(tmp_path, monkeypatch)
|
||||
calls = []
|
||||
def download(profile, archive, **kwargs):
|
||||
calls.append((profile, archive))
|
||||
return [], []
|
||||
monkeypatch.setattr('ocado_grocy.history.download_orders', download)
|
||||
result = CliRunner().invoke(main, ['--no-images', '--no-openfoodfacts'])
|
||||
assert result.exit_code == 0, result.output
|
||||
assert calls == [(paths.browser_directory(), paths.archive_directory())]
|
||||
assert not (tmp_path / 'imports.sqlite3').exists()
|
||||
Loading…
Add table
Add a link
Reference in a new issue