edward/agenda
1
0
Fork 0
Code Issues 9 Pull requests Projects Releases Packages Wiki Activity

Compare commits

base: edward:main
Branches Tags
edward:main
edward:launch-page
edward:navbar
..
compare: edward:launch-page
Branches Tags
edward:main
edward:launch-page
edward:navbar

No commits in common. "main" and "launch-page" have entirely different histories.
main ... launch-pag

149 changed files with 1160 additions and 37446 deletions
Download patch file Download diff file Expand all files Collapse all files

17
.eslintrc.js
View file

@ -1,17 +0,0 @@
module.exports = {
"env": {
"browser": true,
"es6": true
},
"extends": "eslint:recommended",
"globals": {
"Atomics": "readonly",
"SharedArrayBuffer": "readonly"
},
"parserOptions": {
"ecmaVersion": 14,
"sourceType": "module"
},
"rules": {
}
};

6
.gitignore vendored
View file

@ -3,8 +3,4 @@ __pycache__/
__pycache__
.mypy_cache
config
.hypothesis
personal-data
static/bootstrap5
static/leaflet*
static/es-module-shims
.hypothesis/

67
AGENTS.md
View file

@ -1,67 +0,0 @@
# Development Guidelines
## Project Overview
This is a personal agenda web application built with Flask that tracks various events and important dates:
- Events: birthdays, holidays, travel itineraries, conferences, waste collection schedules
- Space launches, meteor showers, astronomical events
- Financial information (FX rates, stock market)
- UK-specific features (holidays, waste collection, railway schedules)
- Authentication via UniAuth
- Frontend uses Bootstrap 5, Leaflet for maps, FullCalendar for calendar views
## Python Environment
- Always use `python3` directly, never `python`
- All Python code should include type annotations
- Use `typing.Any` instead of `Any` in type hints (import from typing module)
- Run `mypy --strict` (fix any type errors in the file) and `black` on modified code after creating or modifying Python files
- Avoid running `black .`
- Main entry point: `python3 web_view.py` (Flask app on port 5000)
- Tests: Use `pytest` (tests in `/tests/` directory)
## Project Structure
- `agenda/` - Main Python package with modules for different event types
- `web_view.py` - Flask web application entry point
- `templates/` - Jinja2 HTML templates
- `static/` - CSS, JS, and frontend assets
- `config/` - Configuration files
- `personal-data/` - User's personal data (not in git)
## Git Workflow
- Avoid committing unrelated untracked files (e.g., `node_modules/`, build artifacts)
- Only commit relevant project files
- Personal data directory (`personal-data/`) is excluded from git
## Conference attendance fields
Conferences in `conferences.yaml` support optional `attend_start` and `attend_end` fields for when you arrive late or leave early. Both accept a plain date or a datetime with time and timezone (YAML datetime syntax). When present, the trip page shows the attendance dates instead of the official conference dates. The official `start`/`end` fields are always kept for context.
```yaml
- name: FOSDEM
start: 2023-02-04
end: 2023-02-05
attend_end: 2023-02-04 # left after day 1
attend_start: 2023-02-04 14:00:00+00:00 # or with time
```
## Notes
- Trip stats new-country badges come from `agenda.stats.calculate_yearly_stats` via `year_stats.new_countries` (first-visit year, excluding `PREVIOUSLY_VISITED`).
- Trip stats are calculated in `agenda/stats.py`:
- `travel_legs()` extracts airlines, airports, and stations from individual trip travel legs
- `calculate_yearly_stats()` aggregates stats per year including flight/train counts, airlines, airports, stations
- `calculate_overall_stats()` aggregates yearly stats into overall totals for the summary section
## Travel type patterns
Transport types: `flight`, `train`, `ferry`, `coach`, `bus`.
**Road transport (bus and coach)** share a common loader `load_road_transport()` in `trip.py`. `load_coaches` and `load_buses` are thin wrappers that pass type name, YAML filenames, and CO2 factor (coach: 0.027 kg/km, bus: 0.1 kg/km). Both use `from_station`/`to_station` fields and support scalar or dict-keyed GeoJSON route filenames in the stop/station data.
**Ferry** is loaded separately: uses `from_terminal`/`to_terminal` fields and GeoJSON routes come from the terminal's `routes` dict (always a dict, not scalar).
**Route rendering** (`get_trip_routes`): bus and coach are handled in a single combined block (they use the same pattern — `from_station`/`to_station`, `{type}_routes/` folder). Ferry always has a geojson file and renders as type `"train"` for the map renderer.
**Trip elements** (`Trip.elements()` in `types.py`): bus and coach are handled in a single combined block using `item["type"] in ("coach", "bus")`. Ferry is separate because it uses `from_terminal`/`to_terminal` and always requires `arrive` (not optional).
**Location collection** (`get_locations`): bus → `bus_stop`, coach → `coach_station`, ferry → `ferry_terminal` (all separate map pin types).
**CO2 factors** (kg CO2e per passenger per km): train 0.037, coach 0.027, ferry 0.02254, bus 0.1.
**Schengen tracking**: ferry journeys are tracked for Schengen compliance; bus and coach are not.

32
agenda/__init__.py
View file

@ -2,7 +2,6 @@
from datetime import date, datetime, time
import pycountry
import pytz
uk_tz = pytz.timezone("Europe/London")
@ -11,34 +10,3 @@ uk_tz = pytz.timezone("Europe/London")
def uk_time(d: date, t: time) -> datetime:
"""Combine time and date for UK timezone."""
return uk_tz.localize(datetime.combine(d, t))
def format_list_with_ampersand(items: list[str]) -> str:
"""Join a list of strings with commas and an ampersand."""
if len(items) > 1:
return ", ".join(items[:-1]) + " & " + items[-1]
elif items:
return items[0]
return ""
def get_country(alpha_2: str | None) -> pycountry.db.Country | None:
"""Lookup country by alpha-2 country code."""
if not alpha_2:
return None
if alpha_2.count(",") > 3: # ESA
return pycountry.db.Country(flag="🇪🇺", name="ESA")
if not alpha_2:
return None
if alpha_2 == "xk":
return pycountry.db.Country(
flag="\U0001f1fd\U0001f1f0", name="Kosovo", alpha_2="xk"
)
country: pycountry.db.Country | None = None
if len(alpha_2) == 2:
country = pycountry.countries.get(alpha_2=alpha_2.upper())
elif len(alpha_2) == 3:
country = pycountry.countries.get(alpha_3=alpha_2.upper())
return country

62
agenda/accommodation.py
View file

@ -1,23 +1,20 @@
"""Accommodation."""
from collections import defaultdict
from datetime import datetime, timedelta
"""Accomodation"""
import yaml
from .event import Event
from .types import StrDict, Trip
from .types import Event
def get_events(filepath: str) -> list[Event]:
"""Get accommodation from YAML."""
"""Get accomodation from YAML."""
with open(filepath) as f:
return [
Event(
date=item["from"],
end_date=item["to"],
name="accommodation",
title=(
title="🧳"
+ (
f'{item["location"]} Airbnb'
if item.get("operator") == "airbnb"
else item["name"]
@ -26,52 +23,3 @@ def get_events(filepath: str) -> list[Event]:
)
for item in yaml.safe_load(f)
]
def prepare_accommodation_list(
items: list[StrDict], trips: list[Trip], now: datetime
) -> StrDict:
"""Link stays to trips and prepare date groups and annual night counts."""
# Create a dictionary to hold stats for each year
year_stats: defaultdict[int, dict[str, int]] = defaultdict(
lambda: {"total_nights": 0, "nights_abroad": 0}
)
# Calculate stats for each year
for stay in items:
current_date = stay["from"].date()
end_date = stay["to"].date()
while current_date < end_date:
year = current_date.year
year_stats[year]["total_nights"] += 1
if stay.get("country") != "gb":
year_stats[year]["nights_abroad"] += 1
current_date += timedelta(days=1)
# Sort the stats by year in descending order
sorted_year_stats = sorted(
year_stats.items(), key=lambda item: item[0], reverse=True
)
trip_lookup = {}
for trip in trips:
for trip_stay in trip.accommodation:
key = (trip_stay["from"], trip_stay["name"])
trip_lookup[key] = trip
for item in items:
key = (item["from"], item["name"])
if this_trip := trip_lookup.get(key):
item["linked_trip"] = this_trip
past = [conf for conf in items if conf["to"] < now]
current = [conf for conf in items if conf["from"] <= now and conf["to"] >= now]
future = [conf for conf in items if conf["from"] > now]
return {
"past": past,
"current": current,
"future": future,
"year_stats": sorted_year_stats,
}

801
agenda/add_new_conference.py
View file

@ -1,801 +0,0 @@
"""Helpers for adding conferences to the YAML data file."""
import configparser
import json
import os
import re
import sys
import typing
from datetime import date, datetime, time, timezone
from urllib.parse import parse_qs, urlparse
import html2text
import lxml.html
import openai
import pycountry
import requests
import yaml
from agenda.conference import ConferenceSeries, conference_date_fields, load_series
USER_AGENT = "add-new-conference/0.1"
COORDINATE_PATTERNS = (
re.compile(r"@(-?\d+(?:\.\d+)?),(-?\d+(?:\.\d+)?)"),
re.compile(r"[?&]q=(-?\d+(?:\.\d+)?),(-?\d+(?:\.\d+)?)"),
re.compile(r"[?&](?:ll|center)=(-?\d+(?:\.\d+)?),(-?\d+(?:\.\d+)?)"),
re.compile(r"!3d(-?\d+(?:\.\d+)?)!4d(-?\d+(?:\.\d+)?)"),
re.compile(r"[?&]destination=(-?\d+(?:\.\d+)?),(-?\d+(?:\.\d+)?)"),
)
def read_api_key() -> str:
"""Read API key from ~/.config/openai/config."""
config_path = os.path.expanduser("~/.config/openai/config")
parser = configparser.ConfigParser()
parser.read(config_path)
return parser["openai"]["api_key"]
def conference_yaml_format_description() -> str:
"""Return the conference YAML format description for LLM prompts."""
return """
Use this YAML format for one conference entry.
Required fields:
- `name`: event name.
- `topic`: topic/category.
- `location`: city or location label. Use `TBC` if the page confirms a future
event but not a city.
- Date information in nested `dates`.
Event name should have any four digit year removed, so "BSDCan 2027" becomes "BSDCan".
Preferred date shape:
- `dates.status`: one of `exact`, `tentative`, or `approximate`.
- For `exact`: use when the page confirms specific dates/times. Include
`dates.start` and `dates.end` as YAML dates or timezone-aware datetimes.
- For `tentative`: use when specific dates are guessed or explicitly
unconfirmed. Include `dates.start`, `dates.end`, and preferably `dates.label`
and `dates.basis`.
- For `approximate`: use when only a broad date phrase is known. Include
`dates.label`, `dates.earliest`, and `dates.latest`. Examples: `March 2027`
should become earliest `2027-03-01`, latest `2027-03-31`; `mid-April 2027`
should become a sensible bounded range such as `2027-04-11` to `2027-04-20`.
Important date rule:
- If the source page contains exact dates, output `dates.status: exact` even if
the existing agenda entry or conference announcement previously had only
approximate dates.
- Always include an end date for `exact` and `tentative`. For a single-day
event, `dates.end` can be the same as `dates.start`.
- Do not output legacy top-level `start`, `end`, or `date_status`.
Common optional fields:
- `series`: a key from the known conference series list, when this event belongs
to a listed series.
- `country`: valid ISO 3166-1 alpha-2 country code in lowercase, for example
`ca`, `gb`, `us`. Do not output country names.
- `venue`, `address`, `latitude`, `longitude`, `url`, `cfp_url`, `cfp_end`,
`hashtag`, `description`.
- For an address written in a non-Latin script, output a conventional
Latin-script rendering only when it can be derived confidently. Do not invent
or loosely translate an address. If no reliable rendering is available, omit
`address`; coordinates are preferable to an unreadable or inaccurate address.
- `free`, `price`, `currency`, `hackathon`, `online`, `attendees`.
- When `free: true`, omit `price` and `currency` because they are redundant.
- Do not include `going`, `registered`, `accommodation_booked`,
`transport_booked`, or `trip` unless the source explicitly says they apply to
my attendance.
"""
def yaml_example_text() -> str:
"""Return examples of the conference YAML format."""
return """
- name: Geomob London
series: geomob-london
topic: Maps
location: London
country: gb
dates:
status: exact
start: 2026-01-28 18:00:00+00:00
end: 2026-01-28 22:00:00+00:00
url: https://thegeomob.com/post/jan-28th-2026-geomoblon-details
venue: Geovation Hub
address: Sutton Yard, 65 Goswell Rd, London EC1V 7EN
latitude: 51.5242464
longitude: -0.0997024
free: true
hashtag: '#geomobLON'
- name: DebConf 25
series: debconf
topic: Debian
location: Plouzane
country: fr
dates:
status: exact
start: 2025-07-07
end: 2025-07-20
url: https://wiki.debian.org/DebConf/25
cfp_url: https://debconf25.debconf.org/talks/new/
venue: Ecole nationale superieure Mines-Telecom Atlantique Bretagne Pays de la Loire
campus de Brest
latitude: 48.35934
longitude: -4.569889
- name: Wikimedia Hackathon
series: wikimedia-hackathon
topic: Wikimedia
location: Albania
country: al
dates:
status: approximate
label: mid-April 2027
earliest: 2027-04-11
latest: 2027-04-20
url: https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2027
hackathon: true
- name: PyCascades
series: pycascades
topic: Python
location: Seattle, Washington
country: us
dates:
status: approximate
label: March 2027
earliest: 2027-03-01
latest: 2027-03-31
"""
def series_prompt_text(series: dict[str, ConferenceSeries]) -> str:
"""Return compact known series text for the LLM prompt."""
if not series:
return "No known conference series loaded."
lines = ["Known conference series IDs:"]
for series_id, item in sorted(series.items()):
details = [item["name"]]
if topic := item.get("topic"):
details.append(f"topic: {topic}")
if location := item.get("usual_location"):
details.append(f"usual location: {location}")
if country := item.get("country"):
details.append(f"country: {country}")
lines.append(f"- {series_id}: " + "; ".join(details))
return "\n".join(lines)
def build_prompt(
url: str,
source_text: str,
detected_coordinates: tuple[float, float] | None,
series: dict[str, ConferenceSeries] | None = None,
) -> str:
"""Build prompt with embedded YAML format details and examples."""
coordinate_note = ""
if detected_coordinates is not None:
coordinate_note = (
"\nDetected venue coordinates from a map link on the page:\n"
f"latitude: {detected_coordinates[0]}\n"
f"longitude: {detected_coordinates[1]}\n"
)
prompt = f"""
I keep a record of interesting conferences in a YAML file.
Format rules:
{conference_yaml_format_description()}
{series_prompt_text(series or {})}
Here are some examples of the format I use:
{yaml_example_text()}
Now here is a new conference of interest:
Conference URL: {url}
Return the YAML representation for this conference following the same style and
keys as the examples. Only include keys if the information is available. Do not
invent details.
Important: if this is a Geomob event, use a `dates.end` datetime of 22:00 local
time on the event date unless the page explicitly provides a different end time.
{coordinate_note}
Wrap your answer in a JSON object with a single key "yaml".
===
{source_text}
"""
return prompt
def get_from_open_ai(prompt: str, model: str = "gpt-5.4") -> dict[str, str]:
"""Pass prompt to OpenAI and get reply."""
client = openai.OpenAI(api_key=read_api_key())
response = client.chat.completions.create(
messages=[{"role": "user", "content": prompt}],
model=model,
response_format={"type": "json_object"},
)
reply = response.choices[0].message.content
assert isinstance(reply, str)
return typing.cast(dict[str, str], json.loads(reply))
def fetch_webpage(url: str) -> lxml.html.HtmlElement:
"""Fetch webpage HTML and parse it."""
response = requests.get(url, headers={"User-Agent": USER_AGENT})
response.raise_for_status()
return lxml.html.fromstring(response.content)
def webpage_to_text(root: lxml.html.HtmlElement) -> str:
"""Convert parsed HTML into readable text content."""
metadata = extract_event_metadata(root)
root_copy = lxml.html.fromstring(lxml.html.tostring(root))
for script_or_style in root_copy.xpath("//script|//style"):
script_or_style.drop_tree()
text_maker = html2text.HTML2Text()
text_maker.ignore_links = True
text_maker.ignore_images = True
page_text = text_maker.handle(lxml.html.tostring(root_copy, encoding="unicode"))
if not metadata:
return page_text
metadata_text = "\n".join(f"- {key}: {value}" for key, value in metadata.items())
return f"Structured event metadata:\n{metadata_text}\n\n{page_text}"
def extract_event_metadata(root: lxml.html.HtmlElement) -> dict[str, str]:
"""Extract Schema.org Event microdata, including visually hidden values."""
event_nodes = root.xpath(
'//*[@itemscope and contains(@itemtype, "schema.org/Event")]'
)
if not event_nodes:
return {}
metadata: dict[str, str] = {}
for element in event_nodes[0].xpath(".//*[@itemprop]"):
key = str(element.get("itemprop", "")).strip()
value = str(element.get("content", "")).strip()
if key and value and key not in metadata:
metadata[key] = value
return metadata
def apply_event_metadata_dates(
conf: dict[str, typing.Any], root: lxml.html.HtmlElement
) -> None:
"""Fill missing generated dates from Schema.org Event microdata."""
dates = conf.get("dates")
if isinstance(dates, dict) and dates.get("start") is not None:
return
if conf.get("start") is not None:
return
metadata = extract_event_metadata(root)
start = parse_yaml_date_value(metadata.get("startDate"))
if start is None:
return
end = parse_yaml_date_value(metadata.get("endDate")) or start
conf["dates"] = {"status": "exact", "start": start, "end": end}
def parse_osm_url(url: str) -> tuple[float, float] | None:
"""Extract latitude/longitude from an OpenStreetMap URL."""
parsed = urlparse(url)
query = parse_qs(parsed.query)
mlat = query.get("mlat")
mlon = query.get("mlon")
if mlat and mlon:
return float(mlat[0]), float(mlon[0])
if parsed.fragment.startswith("map="):
parts = parsed.fragment.split("/")
if len(parts) >= 3:
return float(parts[-2]), float(parts[-1])
return None
def extract_google_maps_latlon(url: str) -> tuple[float, float] | None:
"""Extract latitude/longitude from a Google Maps URL."""
for pattern in COORDINATE_PATTERNS:
match = pattern.search(url)
if match:
return float(match.group(1)), float(match.group(2))
return None
def latlon_from_google_maps_url(
url: str, timeout: int = 10
) -> tuple[float, float] | None:
"""Resolve a Google Maps URL and extract latitude/longitude."""
response = requests.get(
url,
allow_redirects=True,
timeout=timeout,
headers={"User-Agent": "lookup.py/1.0"},
)
response.raise_for_status()
coordinates = extract_google_maps_latlon(response.url)
if coordinates is not None:
return coordinates
return extract_google_maps_latlon(response.text)
def parse_coordinates_from_url(url: str) -> tuple[float, float] | None:
"""Extract latitude/longitude from a supported map URL."""
lower_url = url.lower()
if "openstreetmap.org" in lower_url:
return parse_osm_url(url)
if "google." in lower_url or "maps.app.goo.gl" in lower_url:
coordinates = extract_google_maps_latlon(url)
if coordinates is not None:
return coordinates
try:
return latlon_from_google_maps_url(url)
except requests.RequestException:
return None
return None
def detect_page_coordinates(root: lxml.html.HtmlElement) -> tuple[float, float] | None:
"""Detect venue coordinates from Google Maps or OSM links."""
for link in root.xpath("//a[@href]"):
href = str(link.get("href", "")).strip()
if not href:
continue
coordinates = parse_coordinates_from_url(href)
if coordinates is not None:
return coordinates
return None
def parse_date(date_str: str) -> datetime:
"""Parse ISO date or datetime into a naive datetime (UTC if tz-aware)."""
try:
dt = datetime.fromisoformat(date_str)
except ValueError:
dt = datetime.fromisoformat(date_str.split("T")[0])
if dt.tzinfo is not None:
dt = dt.astimezone(timezone.utc).replace(tzinfo=None)
return dt
def data_dir_from_conferences_path(yaml_path: str) -> str:
"""Return personal-data directory from a conferences.yaml path."""
return os.path.dirname(os.path.abspath(yaml_path))
def url_has_year_component(url: str) -> bool:
"""Return True if the URL contains a year or edition path component."""
parsed = urlparse(url)
components = [part for part in parsed.path.split("/") if part]
if parsed.netloc:
components.extend(part for part in parsed.netloc.split(".") if part)
for component in components:
if re.fullmatch(r"20\d{2}", component):
return True
if re.search(r"(?:^|[-_/])20\d{2}(?:$|[-_/])", component):
return True
if re.fullmatch(r"\d{1,2}x", component, flags=re.IGNORECASE):
return True
return False
def insert_sorted(
conferences: list[dict[str, typing.Any]], new_conf: dict[str, typing.Any]
) -> list[dict[str, typing.Any]]:
"""Insert a conference sorted by start date and skip duplicate URLs."""
new_url = new_conf.get("url")
new_start = conference_sort_datetime(new_conf)
new_year = new_start.year
update_idx = find_inexact_existing_conference(conferences, new_conf)
if update_idx is not None:
existing = conferences.pop(update_idx)
merged = dict(existing)
merged.update(new_conf)
print(f"Updating inexact conference entry: {existing.get('name')}")
return insert_sorted(conferences, merged)
if new_url:
for conf in conferences:
if conf.get("url") == new_url:
existing_start = conference_sort_datetime(conf)
existing_year = existing_start.year
if url_has_year_component(new_url):
print(f"⚠️ Conference with URL {new_url} already exists, skipping.")
return conferences
if existing_year == new_year:
print(
f"⚠️ Conference already exists in YAML "
f"(url={new_url}, year={existing_year}), skipping."
)
return conferences
for idx, conf in enumerate(conferences):
existing_start = conference_sort_datetime(conf)
if new_start < existing_start:
conferences.insert(idx, new_conf)
return conferences
conferences.append(new_conf)
return conferences
def date_ranges_overlap(
first: dict[str, typing.Any], second: dict[str, typing.Any]
) -> bool:
"""Return True if two conference date ranges overlap."""
first_fields = conference_date_fields(first)
second_fields = conference_date_fields(second)
return typing.cast(date, first_fields["start_date"]) <= typing.cast(
date, second_fields["end_date"]
) and typing.cast(date, second_fields["start_date"]) <= typing.cast(
date, first_fields["end_date"]
)
def same_conference_identity(
existing: dict[str, typing.Any], new_conf: dict[str, typing.Any]
) -> bool:
"""Return True if two entries appear to represent the same conference."""
existing_url = existing.get("url")
new_url = new_conf.get("url")
if existing_url and new_url and existing_url == new_url:
return True
existing_series = existing.get("series")
new_series = new_conf.get("series")
if existing_series and new_series and existing_series == new_series:
return date_ranges_overlap(existing, new_conf)
return str(existing.get("name", "")).casefold() == str(
new_conf.get("name", "")
).casefold() and date_ranges_overlap(existing, new_conf)
def find_inexact_existing_conference(
conferences: list[dict[str, typing.Any]], new_conf: dict[str, typing.Any]
) -> int | None:
"""Return index of an inexact existing entry that exact new data can update."""
new_fields = conference_date_fields(new_conf)
if new_fields["date_status"] != "exact":
return None
for idx, existing in enumerate(conferences):
existing_fields = conference_date_fields(existing)
if existing_fields["date_status"] == "exact":
continue
if same_conference_identity(existing, new_conf):
return idx
return None
def conference_sort_datetime(conf: dict[str, typing.Any]) -> datetime:
"""Return conference sort date as a datetime."""
sort_date = conference_date_fields(conf)["sort_date"]
if isinstance(sort_date, datetime):
return sort_date
return datetime.combine(sort_date, time())
def validate_country(conf: dict[str, typing.Any]) -> None:
"""Ensure country is a valid ISO 3166-1 alpha-2 code, normalise if possible."""
country = conf.get("country")
if not country:
return
country = country.strip()
if len(country) == 2:
if pycountry.countries.get(alpha_2=country.upper()):
conf["country"] = country.lower()
return
raise ValueError(f"❌ Invalid ISO 3166-1 code '{country}'")
match = pycountry.countries.get(name=country)
if not match:
try:
match = pycountry.countries.search_fuzzy(country)[0]
except LookupError as exc:
raise ValueError(
f"❌ Country '{country}' not recognised as ISO 3166-1"
) from exc
conf["country"] = match.alpha_2.lower()
def validate_series(
conf: dict[str, typing.Any], series: dict[str, ConferenceSeries]
) -> None:
"""Ensure a generated series ID exists in conference_series.yaml."""
series_id = conf.get("series")
if series_id is None:
return
if not isinstance(series_id, str) or series_id not in series:
raise ValueError(
f"Generated conference uses unknown series {series_id!r}. "
"Add it to conference_series.yaml first or remove the series field."
)
def normalize_free_event_fields(conf: dict[str, typing.Any]) -> None:
"""Remove redundant pricing fields from free conferences."""
if conf.get("free") is True:
conf.pop("price", None)
conf.pop("currency", None)
def parse_yaml_datetime(value: typing.Any) -> datetime | None:
"""Convert YAML date/datetime values to a datetime."""
if isinstance(value, datetime):
return value
if isinstance(value, date):
return datetime.combine(value, time())
if isinstance(value, str):
try:
return datetime.fromisoformat(value)
except ValueError:
return datetime.combine(date.fromisoformat(value.split("T")[0]), time())
return None
def parse_yaml_date_value(value: typing.Any) -> date | datetime | None:
"""Convert YAML date/datetime strings to date-like values."""
if isinstance(value, datetime):
return value
if isinstance(value, date):
return value
if not isinstance(value, str):
return None
try:
if " " in value or "T" in value:
return datetime.fromisoformat(value)
return date.fromisoformat(value)
except ValueError:
return None
def normalize_date_values(conf: dict[str, typing.Any]) -> None:
"""Normalize quoted ISO date/datetime values produced by the LLM."""
dates = conf.get("dates")
if isinstance(dates, dict):
for field in ("start", "end", "earliest", "latest"):
if field in dates:
parsed = parse_yaml_date_value(dates[field])
if parsed is not None:
dates[field] = parsed
for field in ("start", "end"):
if field in conf:
parsed = parse_yaml_date_value(conf[field])
if parsed is not None:
conf[field] = parsed
def same_type_as_start(
start_value: typing.Any,
new_dt: datetime,
keep_timezone: bool = True,
prefer_datetime: bool = False,
) -> typing.Any:
"""Return end value shaped like the start value when possible."""
if isinstance(start_value, datetime):
if keep_timezone:
return new_dt
return new_dt.replace(tzinfo=None)
if isinstance(start_value, date):
if prefer_datetime:
return new_dt
return new_dt.date()
if isinstance(start_value, str):
if prefer_datetime or " " in start_value or "T" in start_value:
return new_dt.isoformat(sep=" ")
return new_dt.date().isoformat()
return new_dt
def normalize_dates_field(conf: dict[str, typing.Any]) -> None:
"""Move legacy top-level date fields into the nested dates mapping."""
normalize_date_values(conf)
raw_dates = conf.get("dates")
dates = raw_dates if isinstance(raw_dates, dict) else None
if dates is None and ("start" in conf or "end" in conf):
start = conf.pop("start", None)
end = conf.pop("end", start)
status = str(conf.pop("date_status", "exact"))
conf["dates"] = {"status": status, "start": start, "end": end}
return
if dates is not None:
if "start" in conf and "start" not in dates:
dates["start"] = conf["start"]
if "end" in conf and "end" not in dates:
dates["end"] = conf["end"]
if "date_status" in conf and "status" not in dates:
dates["status"] = conf["date_status"]
conf.pop("start", None)
conf.pop("end", None)
conf.pop("date_status", None)
normalize_date_values(conf)
def validate_generated_conference(conf: dict[str, typing.Any]) -> None:
"""Validate generated conference YAML before inserting it."""
try:
conference_date_fields(conf)
except ValueError as exc:
generated_yaml = yaml.dump(conf, sort_keys=False, allow_unicode=True).strip()
raise ValueError(
"Generated conference YAML is missing valid date information. "
"Expected nested `dates:` with exact/tentative start/end or "
f"approximate earliest/latest.\n\nGenerated YAML:\n{generated_yaml}"
) from exc
def maybe_extract_explicit_end_time(source_text: str) -> int | None:
"""Extract an explicit 12-hour clock end time for Geomob-style pages."""
lowered = source_text.lower()
if "10pm" in lowered or "10 pm" in lowered or "22:00" in lowered:
return 22
if "11pm" in lowered or "11 pm" in lowered or "23:00" in lowered:
return 23
return None
def normalise_end_field(new_conf: dict[str, typing.Any], source_text: str) -> None:
"""Ensure an end value exists, with a Geomob-specific fallback."""
dates = new_conf.get("dates")
nested_dates = dates if isinstance(dates, dict) else None
start_value = (
nested_dates.get("start") if nested_dates is not None else new_conf.get("start")
)
if start_value is None:
return
start_dt = parse_yaml_datetime(start_value)
if start_dt is None:
return
name = str(new_conf.get("name", ""))
url = str(new_conf.get("url", ""))
is_geomob = "geomob" in name.lower() or "thegeomob.com" in url.lower()
if is_geomob:
end_hour = maybe_extract_explicit_end_time(source_text)
if end_hour is None:
end_hour = 22
geomob_end = start_dt.replace(hour=end_hour, minute=0, second=0, microsecond=0)
end_value = same_type_as_start(start_value, geomob_end, prefer_datetime=True)
if nested_dates is not None:
nested_dates["end"] = end_value
else:
new_conf["end"] = end_value
return
if nested_dates is not None:
if "end" not in nested_dates:
nested_dates["end"] = same_type_as_start(start_value, start_dt)
return
if "end" not in new_conf:
new_conf["end"] = same_type_as_start(start_value, start_dt)
def load_conferences(yaml_path: str) -> list[dict[str, typing.Any]]:
"""Load conference YAML."""
with open(yaml_path) as file:
loaded = yaml.safe_load(file)
assert isinstance(loaded, list)
return typing.cast(list[dict[str, typing.Any]], loaded)
def load_conference_series_for_path(yaml_path: str) -> dict[str, ConferenceSeries]:
"""Load conference series next to the target conferences YAML file."""
return load_series(data_dir_from_conferences_path(yaml_path))
def dump_conferences(yaml_path: str, conferences: list[dict[str, typing.Any]]) -> None:
"""Write conference YAML."""
with open(yaml_path, "w") as file:
text = yaml.dump(conferences, sort_keys=False, allow_unicode=True)
text = text.replace("\n- name:", "\n\n- name:")
file.write(text.lstrip())
def add_new_conference(url: str, yaml_path: str) -> bool:
"""Fetch, generate and insert a conference into the YAML file."""
conferences = load_conferences(yaml_path)
if url_has_year_component(url):
for conf in conferences:
if conf.get("url") == url:
fields = conference_date_fields(conf)
if fields["date_status"] != "exact":
continue
print(
"⚠️ Conference already exists in YAML "
+ f"(url={url}), skipping before API call."
)
return False
soup = fetch_webpage(url)
source_text = webpage_to_text(soup)
detected_coordinates = detect_page_coordinates(soup)
series = load_conference_series_for_path(yaml_path)
prompt = build_prompt(url, source_text, detected_coordinates, series)
new_yaml_text = get_from_open_ai(prompt)["yaml"]
new_conf = yaml.safe_load(new_yaml_text)
if isinstance(new_conf, list):
new_conf = new_conf[0]
assert isinstance(new_conf, dict)
validate_country(new_conf)
validate_series(new_conf, series)
normalize_free_event_fields(new_conf)
apply_event_metadata_dates(new_conf, soup)
normalize_dates_field(new_conf)
normalise_end_field(new_conf, source_text)
normalize_dates_field(new_conf)
validate_generated_conference(new_conf)
if detected_coordinates is not None:
new_conf["latitude"] = detected_coordinates[0]
new_conf["longitude"] = detected_coordinates[1]
updated = insert_sorted(conferences, new_conf)
dump_conferences(yaml_path, updated)
return True
def main(argv: list[str] | None = None) -> int:
"""CLI entrypoint."""
args = argv if argv is not None else sys.argv[1:]
if not args:
raise SystemExit("Usage: add-new-conference URL")
yaml_path = os.path.expanduser("~/src/personal-data/conferences.yaml")
add_new_conference(args[0], yaml_path)
return 0

190
agenda/airbnb.py
View file

@ -1,190 +0,0 @@
"""Library for parsing Airbnb booking HTML files."""
import json
import re
import typing
from datetime import datetime
from typing import Any
from zoneinfo import ZoneInfo
import lxml.html
import pycountry
StrDict = dict[str, typing.Any]
def build_datetime(date_str: str, time_str: str, tz_name: str) -> datetime:
"""
Combine an ISO date string, HH:MM time string, and a timezone name
into a timezone-aware datetime in the specified timezone.
"""
dt_str = f"{date_str}T{time_str}"
naive_dt = datetime.fromisoformat(dt_str)
return naive_dt.replace(tzinfo=ZoneInfo(tz_name))
def list_to_dict(items: list[typing.Any]) -> dict[str, typing.Any]:
"""Convert a flat list to a dict, assuming alternating keys and values."""
return {items[i]: items[i + 1] for i in range(0, len(items), 2)}
def extract_country_code(address: str) -> str | None:
"""Return ISO 3166-1 alpha-2 country code from a free-text address."""
address_lower = address.lower()
for country in pycountry.countries:
if country.name.lower() in address_lower:
return str(country.alpha_2.lower())
if (
hasattr(country, "official_name")
and country.official_name.lower() in address_lower
):
return str(country.alpha_2.lower())
return None
def get_json_blob(tree: Any) -> str:
data_id = "data-injector-instances"
js_string = tree.xpath(f'//*[@id="{data_id}"]/text()')[0]
return str(js_string)
def get_ui_state(tree: Any) -> StrDict:
data_id = "data-injector-instances"
js_string = tree.xpath(f'//*[@id="{data_id}"]/text()')[0]
big_blob = json.loads(str(js_string))
ui_state = walk_tree(big_blob, "uiState")
return list_to_dict(ui_state[0])
def get_reservation_data(ui_state: StrDict) -> StrDict:
return {
row["id"]: row for row in ui_state["reservation"]["scheduled_event"]["rows"]
}
def get_room_url(tree: Any) -> str | None:
for e in tree.xpath('//a[@data-testid="reservation-destination-link"]'):
href = e.get("href")
assert isinstance(href, str)
if not href.startswith("/room"):
continue
return "https://www.airbnb.co.uk" + href
return None
def get_price_from_reservation(reservation: StrDict) -> str:
price = reservation["payment_summary"]["subtitle"]
assert isinstance(price, str)
tc = "Total cost: "
if price.startswith(tc):
price = price[len(tc) :]
assert price[0] == "£"
return price[1:]
def extract_booking_from_html(html_file: str) -> StrDict:
"""Extract booking information from Airbnb HTML file."""
with open(html_file, "r", encoding="utf-8") as f:
text_content = f.read()
confirmation_match = re.search(
r"/trips/v1/reservation-details/ro/RESERVATION2_CHECKIN/([A-Z0-9]+)",
text_content,
)
if confirmation_match is None:
raise ValueError("Could not find confirmation code in HTML")
confirmation_code = confirmation_match.group(1)
tree = lxml.html.parse(html_file)
root = tree.getroot()
try:
ui_state = get_ui_state(tree)
except Exception:
print(html_file)
raise
reservation = get_reservation_data(ui_state)
m_guests = re.match(r"^(\d+) guests?$", reservation["guests"]["subtitle"])
if m_guests is None:
raise ValueError("Could not parse number of guests")
number_of_adults = int(m_guests.group(1))
price = get_price_from_reservation(reservation)
metadata = ui_state["reservation"]["metadata"]
country_code = metadata["country"].lower()
title = reservation["dynamic_marquee_title_image_v3"]["title"]
location = title.rpartition(" in ")[2]
checkin_checkout = reservation["checkin_checkout_arrival_guide"]
check_in_time = checkin_checkout["leading_subtitle"]
check_out_time = checkin_checkout["trailing_subtitle"]
check_in = build_datetime(
metadata["check_in_date"], check_in_time, metadata["timezone"]
)
check_out = build_datetime(
metadata["check_out_date"], check_out_time, metadata["timezone"]
)
address = reservation["map"]["address"] if "map" in reservation else None
if "header_action.pdp" in reservation:
name = reservation["header_action.pdp"]["subtitle"]
else:
name = root.findtext(".//h1")
booking = {
"type": "apartment",
"operator": "airbnb",
"name": name,
"location": location,
"booking_reference": confirmation_code,
"booking_url": f"https://www.airbnb.co.uk/trips/v1/reservation-details/ro/RESERVATION2_CHECKIN/{confirmation_code}",
"country": country_code,
"latitude": metadata["lat"],
"longitude": metadata["lng"],
"timezone": metadata["timezone"],
"from": check_in,
"to": check_out,
"price": price,
"currency": "GBP",
"number_of_adults": number_of_adults,
}
if address:
booking["address"] = address
room_url = get_room_url(tree)
if room_url is not None:
booking["url"] = room_url
return booking
def walk_tree(data: Any, want_key: str) -> Any:
"""Recursively search for a dict containing 'reservation' and return its value."""
if isinstance(data, dict):
if want_key in data:
return data[want_key]
for key, value in data.items():
result = walk_tree(value, want_key)
if result is not None:
return result
elif isinstance(data, list):
for item in data:
result = walk_tree(item, want_key)
if result is not None:
return result
return None
def parse_multiple_files(filenames: list[str]) -> list[StrDict]:
"""Parse multiple Airbnb HTML files and return a list of booking dictionaries."""
bookings = []
for html_file in sorted(filenames):
booking = extract_booking_from_html(html_file)
assert booking
bookings.append(booking)
return bookings

9053
agenda/airport_index.json
View file

File diff suppressed because it is too large Load diff

147
agenda/airport_lookup.py
View file

@ -1,147 +0,0 @@
"""Local airport names from OurAirports' public-domain data (1 October 2026)."""
import json
import typing
import unicodedata
from functools import lru_cache
from pathlib import Path
import pycountry
import yaml
from agenda.types import StrDict
def normalized(value: str) -> str:
"""Match names regardless of case or accents."""
return "".join(
c
for c in unicodedata.normalize("NFKD", value.casefold())
if not unicodedata.combining(c)
).strip()
@lru_cache(maxsize=1)
def airports() -> dict[str, list[typing.Any]]:
"""Load the bundled index once; no network access during autocomplete."""
return typing.cast(
dict[str, list[typing.Any]],
json.loads(Path(__file__).with_name("airport_index.json").read_text()),
)
@lru_cache(maxsize=1)
def airport_groups() -> dict[tuple[str, str], list[str]]:
"""Named cross-border groups from the conference location mappings."""
mapping: dict[str, str | list[str]] = json.loads(
Path(__file__).with_name("conference_airports.json").read_text()
)
return {
(country.upper(), normalized(city)): codes
for key, codes in mapping.items()
for country, city in [key.split(":", 1)]
if isinstance(codes, list)
}
def code_list(query: str) -> tuple[str, ...]:
"""Recognize one or more exact IATA codes, without fuzzy matching."""
codes = tuple(dict.fromkeys(part.strip().upper() for part in query.split(",")))
return (
codes
if all(len(code) == 3 and code.isascii() and code.isalpha() for code in codes)
else ()
)
def preferred_names(personal_data: str | None) -> dict[str, str]:
"""Read personal airport names afresh so edits appear without a reload."""
if personal_data is None:
return {}
path = Path(personal_data) / "airports.yaml"
if not path.exists():
return {}
records: dict[str, StrDict] = yaml.safe_load(path.read_text()) or {}
names = {}
for key, record in records.items():
code = str(record.get("iata") or key).upper()
name = record.get("name")
if isinstance(name, str) and name.strip():
names[code] = name.strip()
return names
def group_match(
codes: tuple[str, ...], preferred: dict[str, str] | None = None
) -> StrDict | None:
"""Represent multiple airports as one selectable destination."""
names = []
for code in codes:
record = airports().get(code)
if code == "LON":
names.append("London (all airports)")
elif record:
names.append((preferred or {}).get(code, str(record[0])))
else:
return None
return {"code": ",".join(codes), "name": " / ".join(names)}
def matches(query: str, personal_data: str | None = None) -> list[StrDict]:
"""Prefer scheduled airports serving an exact city; accept country suffixes."""
query = query.strip()
preferred = preferred_names(personal_data)
codes = code_list(query)
if codes:
match = group_match(codes, preferred)
return [match] if match else []
city, _, suffix = query.partition(",")
needle = normalized(city)
if not needle:
return []
country: str | None = None
if suffix.strip():
try:
subdivision = typing.cast(typing.Any, pycountry.subdivisions).get(
code="US-" + suffix.strip().upper()
)
country = (
"US"
if subdivision
else str(pycountry.countries.lookup(suffix.strip()).alpha_2)
)
except LookupError:
pass
for (group_country, group_city), group_codes in airport_groups().items():
if needle == group_city and country in (None, group_country):
match = group_match(tuple(group_codes), preferred)
if match:
return [match]
candidates: list[tuple[tuple[int, int, int, str], StrDict]] = []
for code, record in airports().items():
name, municipality, iso_country, scheduled = record
if country and iso_country != country:
continue
name_text, city_text = normalized(name), normalized(municipality)
display_name = preferred.get(code, str(name))
preferred_text = normalized(display_name)
if (
needle not in name_text
and needle not in city_text
and needle not in preferred_text
):
continue
rank = (
int(not scheduled),
int(city_text != needle),
int(
not (name_text.startswith(needle) or preferred_text.startswith(needle))
),
code,
)
candidates.append((rank, {"code": code, "name": display_name}))
candidates.sort(key=lambda item: item[0])
result = [record for _, record in candidates[:10]]
if needle == "london" and country in (None, "GB"):
result.insert(0, {"code": "LON", "name": "London (all airports)"})
return result[:10]

4
agenda/birthday.py
View file

@ -4,7 +4,7 @@ from datetime import date
import yaml
from .event import Event
from .types import Event
YEAR_NOT_KNOWN = 1900
@ -42,7 +42,7 @@ def get_birthdays(from_date: date, filepath: str) -> list[Event]:
Event(
date=bday.replace(year=bday.year + offset),
name="birthday",
title=f'{entity["label"]} ({display_age})',
title=f'🎈 {entity["label"]} ({display_age})',
)
)

131
agenda/bristol_waste.py
View file

@ -1,131 +0,0 @@
"""Waste collection schedules."""
import json
import os
import typing
from collections import defaultdict
from datetime import date, datetime, timedelta
import httpx
from .event import Event
from .utils import make_waste_dir
ttl_hours = 12
BristolSchedule = list[dict[str, typing.Any]]
async def get(start_date: date, data_dir: str, uprn: str, cache: str) -> list[Event]:
"""Get waste collection schedule from Bristol City Council."""
by_date: defaultdict[date, list[str]] = defaultdict(list)
for item in await get_data(data_dir, uprn, cache):
service = get_service(item)
for d in collections(item):
if d < start_date and service not in by_date[d]:
by_date[d].append(service)
return [
Event(name="waste_schedule", date=d, title="Bristol: " + ", ".join(services))
for d, services in by_date.items()
]
async def get_data(data_dir: str, uprn: str, cache: str) -> BristolSchedule:
"""Get Bristol Waste schedule, with cache."""
now = datetime.now()
waste_dir = os.path.join(data_dir, "waste")
make_waste_dir(data_dir)
existing_data = os.listdir(waste_dir)
existing = [f for f in existing_data if f.endswith(f"_{uprn}.json")]
if existing:
recent_filename = max(existing)
recent = datetime.strptime(recent_filename, f"%Y-%m-%d_%H:%M_{uprn}.json")
delta = now - recent
def get_from_recent() -> BristolSchedule:
json_data = json.load(open(os.path.join(waste_dir, recent_filename)))
return typing.cast(BristolSchedule, json_data["data"])
if (
cache != "refresh"
and existing
and (cache == "force" or delta < timedelta(hours=ttl_hours))
):
return get_from_recent()
try:
r = await get_web_data(uprn)
except httpx.ReadTimeout:
return get_from_recent()
with open(f'{waste_dir}/{now.strftime("%Y-%m-%d_%H:%M")}_{uprn}.json', "wb") as out:
out.write(r.content)
return typing.cast(BristolSchedule, r.json()["data"])
async def get_web_data(uprn: str) -> httpx.Response:
"""Get JSON from Bristol City Council."""
UA = "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"
HEADERS = {
"Accept": "*/*",
"Accept-Language": "en-GB,en;q=0.9",
"Connection": "keep-alive",
"Ocp-Apim-Subscription-Key": "47ffd667d69c4a858f92fc38dc24b150",
"Ocp-Apim-Trace": "true",
"Origin": "https://bristolcouncil.powerappsportals.com",
"Referer": "https://bristolcouncil.powerappsportals.com/",
"Sec-Fetch-Dest": "empty",
"Sec-Fetch-Mode": "cors",
"Sec-Fetch-Site": "cross-site",
"Sec-GPC": "1",
"User-Agent": UA,
}
_uprn = str(uprn).zfill(12)
async with httpx.AsyncClient(timeout=20) as client:
# Initialise form
payload = {"servicetypeid": "7dce896c-b3ba-ea11-a812-000d3a7f1cdc"}
response = await client.get(
"https://bristolcouncil.powerappsportals.com/completedynamicformunauth/",
headers=HEADERS,
params=payload,
)
host = "bcprdapidyna002.azure-api.net"
# Set the search criteria
payload = {"Uprn": "UPRN" + _uprn}
response = await client.post(
f"https://{host}/bcprdfundyna001-llpg/DetailedLLPG",
headers=HEADERS,
json=payload,
)
# Retrieve the schedule
payload = {"uprn": _uprn}
response = await client.post(
f"https://{host}/bcprdfundyna001-alloy/NextCollectionDates",
headers=HEADERS,
json=payload,
)
return response
def get_service(item: dict[str, typing.Any]) -> str:
"""Bristol waste service name."""
service: str = item["containerName"]
return "Recycling" if "Recycling" in service else service.partition(" ")[2]
def collections(item: dict[str, typing.Any]) -> typing.Iterable[date]:
"""Bristol dates from collections."""
for collection in item["collection"]:
for collection_date_key in ["nextCollectionDate", "lastCollectionDate"]:
yield date.fromisoformat(collection[collection_date_key][:10])

257
agenda/build_place_yaml.py
View file

@ -1,257 +0,0 @@
"""Build airport and station YAML entries from Wikidata."""
import argparse
import typing
from pathlib import Path
import requests
import yaml
API_URL = "https://www.wikidata.org/w/api.php"
PERSONAL_DATA_DIR = Path("~/src/personal-data").expanduser()
USER_AGENT = "agenda-build-place-yaml/0.1"
Entity = dict[str, typing.Any]
Entities = dict[str, Entity]
class WikidataClient:
"""Small Wikidata API client for place lookups."""
def __init__(self) -> None:
"""Create a Wikidata API session."""
self.session = requests.Session()
self.session.headers.update({"User-Agent": USER_AGENT})
def get_json(self, params: dict[str, str]) -> dict[str, typing.Any]:
"""Fetch JSON from Wikidata."""
response = self.session.get(API_URL, params=params)
response.raise_for_status()
return typing.cast(dict[str, typing.Any], response.json())
def get_alpha2_country_code(self, qid: str) -> str:
"""Query the Wikidata API for alpha-2 country code."""
data = self.get_json(
{
"action": "wbgetclaims",
"entity": qid,
"property": "P297",
"format": "json",
}
)
p297 = data["claims"]["P297"]
return typing.cast(str, p297[0]["mainsnak"]["datavalue"]["value"]).lower()
def search_entities(self, query: str) -> Entities:
"""Search Wikidata and return detailed entities."""
search_data = self.get_json(
{
"action": "query",
"list": "search",
"format": "json",
"srsearch": query,
}
)
search_results = search_data["query"]["search"]
if not search_results:
return {}
ids = [result["title"] for result in search_results]
entity_data = self.get_json(
{
"action": "wbgetentities",
"format": "json",
"ids": "|".join(ids),
}
)
return typing.cast(Entities, entity_data["entities"])
def entity_names(entity: Entity) -> set[str]:
"""Return labels and aliases for a Wikidata entity."""
names: set[str] = {lang["value"] for lang in entity.get("labels", {}).values()}
for alias_list in entity.get("aliases", {}).values():
for alias in alias_list:
names.add(alias["value"])
return names
def exact_station_match(station_name: str, entity: Entity) -> bool:
"""Return whether the entity names match the requested station name."""
names = entity_names(entity)
return station_name in names or f"{station_name} station" in names
def entity_claim_value(
entity: Entity, property_id: str, default: typing.Any = None
) -> typing.Any:
"""Return the first Wikidata claim value for a property."""
claims = entity.get("claims", {})
if property_id not in claims:
return default
return claims[property_id][0]["mainsnak"]["datavalue"]["value"]
def build_station_info(
client: WikidataClient, station_name: str, entity_id: str, entity: Entity
) -> dict[str, typing.Any]:
"""Build a stations.yaml entry."""
coords = entity_claim_value(entity, "P625")
country_value = entity_claim_value(entity, "P17")
uic = entity_claim_value(entity, "P722")
uk_station_code = entity_claim_value(entity, "P4755")
station_info: dict[str, typing.Any] = {
"name": station_name,
"latitude": coords["latitude"],
"longitude": coords["longitude"],
"country": client.get_alpha2_country_code(country_value["id"]),
"wikidata": entity_id,
"routes": {},
}
if uic is not None:
station_info["uic"] = uic
if uk_station_code is not None:
station_info["alpha3"] = uk_station_code
return station_info
def search_for_station(
client: WikidataClient, station_name: str
) -> dict[str, typing.Any]:
"""Search for a station and return a stations.yaml entry."""
haswbstatement = "P31=Q55488|P31=Q18543139|P31=Q1147171"
entities = client.search_entities(f"{station_name} haswbstatement:{haswbstatement}")
for entity_id, entity in entities.items():
if exact_station_match(station_name, entity):
return build_station_info(client, station_name, entity_id, entity)
if entities:
entity_id, entity = next(iter(entities.items()))
return build_station_info(client, station_name, entity_id, entity)
raise ValueError(f"No Wikidata station found for {station_name!r}")
def build_airport_info(
client: WikidataClient, iata: str, entity_id: str, entity: Entity
) -> dict[str, typing.Any]:
"""Build an airports.yaml entry."""
label = entity["labels"]["en"]["value"]
claims = entity["claims"]
coords = claims["P625"][0]["mainsnak"]["datavalue"]["value"]
country_qid = claims["P17"][0]["mainsnak"]["datavalue"]["value"]["id"]
info: dict[str, typing.Any] = {
"iata": iata,
"name": label,
"city": label,
"country": client.get_alpha2_country_code(country_qid),
"latitude": coords["latitude"],
"longitude": coords["longitude"],
"qid": entity_id,
}
website = entity_claim_value(entity, "P856")
if website is not None:
info["website"] = website
return info
def search_for_airport(client: WikidataClient, iata: str) -> dict[str, typing.Any]:
"""Search for an airport by IATA code and return an airports.yaml entry."""
entities = client.search_entities(f"haswbstatement:P238={iata.upper()}")
if not entities:
raise ValueError(f"No Wikidata airport found for IATA code {iata!r}")
entity_id, entity = next(iter(entities.items()))
return build_airport_info(client, iata.upper(), entity_id, entity)
def load_yaml(path: Path) -> typing.Any:
"""Load a YAML file."""
return yaml.safe_load(path.read_text())
def dump_yaml(data: typing.Any) -> str:
"""Dump YAML using the local personal-data style."""
return yaml.dump(data, sort_keys=False, allow_unicode=True)
def dump_yaml_list_with_blank_lines(items: list[dict[str, typing.Any]]) -> str:
"""Dump a YAML list with a blank line between top-level items."""
text = dump_yaml(items).lstrip()
return text.replace("\n- ", "\n\n- ")
def upsert_station(data_dir: Path, station_info: dict[str, typing.Any]) -> bool:
"""Add or replace a station entry. Return True when an existing entry changed."""
path = data_dir / "stations.yaml"
stations = typing.cast(list[dict[str, typing.Any]], load_yaml(path))
for index, station in enumerate(stations):
if station.get("name") == station_info["name"]:
stations[index] = station_info
path.write_text(dump_yaml_list_with_blank_lines(stations))
return True
stations.append(station_info)
path.write_text(dump_yaml_list_with_blank_lines(stations))
return False
def upsert_airport(data_dir: Path, airport_info: dict[str, typing.Any]) -> bool:
"""Add or replace an airport entry. Return True when an existing entry changed."""
path = data_dir / "airports.yaml"
airports = typing.cast(dict[str, dict[str, typing.Any]], load_yaml(path))
iata = typing.cast(str, airport_info["iata"])
existed = iata in airports
airports[iata] = airport_info
path.write_text(dump_yaml(airports))
return existed
def station_main(argv: list[str] | None = None) -> int:
"""CLI entrypoint for building and importing a station YAML entry."""
parser = argparse.ArgumentParser(
description="Add or update a station in personal-data/stations.yaml."
)
parser.add_argument("station_name")
parser.add_argument("--data-dir", default=str(PERSONAL_DATA_DIR))
parser.add_argument("--print-only", action="store_true")
args = parser.parse_args(argv)
station_info = search_for_station(WikidataClient(), args.station_name)
if args.print_only:
print(dump_yaml_list_with_blank_lines([station_info]).strip())
return 0
replaced = upsert_station(Path(args.data_dir), station_info)
action = "Updated" if replaced else "Added"
print(f"{action} station {station_info['name']} in {args.data_dir}/stations.yaml")
return 0
def airport_main(argv: list[str] | None = None) -> int:
"""CLI entrypoint for building and importing an airport YAML entry."""
parser = argparse.ArgumentParser(
description="Add or update an airport in personal-data/airports.yaml."
)
parser.add_argument("iata")
parser.add_argument("--data-dir", default=str(PERSONAL_DATA_DIR))
parser.add_argument("--print-only", action="store_true")
args = parser.parse_args(argv)
airport_info = search_for_airport(WikidataClient(), args.iata)
if args.print_only:
print(dump_yaml({airport_info["iata"]: airport_info}).strip())
return 0
replaced = upsert_airport(Path(args.data_dir), airport_info)
action = "Updated" if replaced else "Added"
print(f"{action} airport {airport_info['iata']} in {args.data_dir}/airports.yaml")
return 0

668
agenda/busy.py
View file

@ -1,668 +0,0 @@
"""Identify busy events and gaps when nothing is scheduled."""
import itertools
import typing
from datetime import date, datetime, timedelta, timezone
import flask
import pycountry
from . import events_yaml, get_country, travel
from .event import Event
from .types import StrDict, Trip
def busy_event(e: Event) -> bool:
"""Busy."""
if e.name not in {
"event",
"accommodation",
"conference",
"transport",
"meetup",
"party",
"trip",
"hackathon",
}:
return False
if e.title in ("IA UK board meeting", "Mill Road Winter Fair"):
return False
if e.name == "conference" and not e.going:
return False
if not e.title:
return True
if e.title == "LHG Run Club" or "Third Thursday Social" in e.title:
return False
lc_title = e.title.lower()
return (
"rebels" not in lc_title
and "south west data social" not in lc_title
and "dorkbot" not in lc_title
)
def get_busy_events(
start: date, config: flask.config.Config, trips: list[Trip]
) -> list[Event]:
"""Find busy events from a year ago to two years in the future."""
last_year = start - timedelta(days=365)
next_year = start + timedelta(days=2 * 365)
my_data = config["PERSONAL_DATA"]
events = events_yaml.read(my_data, last_year, next_year, skip_trips=True)
for trip in trips:
event_type = "trip"
if trip.events and not trip.conferences:
event_type = trip.events[0]["name"]
elif len(trip.conferences) == 1 and trip.conferences[0].get("hackathon"):
event_type = "hackathon"
events.append(
Event(
name=event_type,
title=trip.title + " " + trip.country_flags,
date=trip.start,
end_date=trip.end,
url=flask.url_for("trip_page", start=trip.start.isoformat()),
)
)
busy_events = [
e
for e in sorted(events, key=lambda e: e.as_date)
if (e.as_date >= start or (e.end_date and e.end_as_date >= start))
and e.as_date < next_year
and busy_event(e)
]
return busy_events
def _parse_datetime_field(datetime_obj: datetime | date | str) -> tuple[datetime, date]:
"""Parse a datetime field that could be datetime object or string."""
if isinstance(datetime_obj, datetime):
dt = datetime_obj
elif isinstance(datetime_obj, date):
dt = datetime.combine(datetime_obj, datetime.min.time(), tzinfo=timezone.utc)
elif isinstance(datetime_obj, str):
dt = datetime.fromisoformat(datetime_obj.replace("Z", "+00:00"))
else:
raise ValueError(f"Invalid datetime format: {datetime_obj}")
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
else:
dt = dt.astimezone(timezone.utc)
return dt, dt.date()
def _get_accommodation_location(
acc: StrDict, on_trip: bool = False
) -> tuple[str | None, pycountry.db.Country]:
"""Get location from accommodation data."""
c = get_country(acc["country"])
assert c
assert isinstance(acc["location"], str)
return (acc["location"] if on_trip else None, c)
def _find_most_recent_travel_within_trip(
trip: Trip,
target_date: date,
) -> tuple[str | None, pycountry.db.Country | None] | None:
"""Find the most recent travel location within a trip."""
uk_airports = {"LHR", "LGW", "STN", "LTN", "BRS", "BHX", "MAN", "EDI", "GLA"}
trip_most_recent_date: date | None = None
trip_most_recent_location: tuple[str | None, pycountry.db.Country | None] | None = (
None
)
trip_most_recent_datetime: datetime | None = None
# Check flights within trip period
for travel_item in trip.travel:
if travel_item["type"] == "flight" and "arrive" in travel_item:
arrive_datetime, arrive_date = _parse_datetime_field(travel_item["arrive"])
# Only consider flights within this trip and before target date
if not (trip.start <= arrive_date <= target_date):
continue
# Compare both date and time to handle same-day flights correctly
if (
trip_most_recent_date is None
or arrive_date > trip_most_recent_date
or (
arrive_date == trip_most_recent_date
and (
trip_most_recent_datetime is None
or arrive_datetime > trip_most_recent_datetime
)
)
):
trip_most_recent_date = arrive_date
trip_most_recent_datetime = arrive_datetime
destination_airport = travel_item["to"]
assert "to_airport" in travel_item
airport_info = travel_item["to_airport"]
airport_country = airport_info["country"]
if airport_country == "gb":
if destination_airport in uk_airports:
# UK airport while on trip - show actual location
location_name = airport_info.get(
"city", airport_info.get("name", "London")
)
trip_most_recent_location = (
location_name,
get_country("gb"),
)
else:
trip_most_recent_location = (None, get_country("gb"))
else:
location_name = airport_info.get(
"city", airport_info.get("name", destination_airport)
)
trip_most_recent_location = (
location_name,
get_country(airport_country),
)
# Check accommodations within trip period
for acc in trip.accommodation:
if "from" in acc:
try:
_, acc_date = _parse_datetime_field(acc["from"])
except ValueError:
continue
# Only consider accommodations within this trip and before/on target date
if trip.start <= acc_date <= target_date:
# Accommodation takes precedence over flights on the same date
# or if it's genuinely more recent
if (
trip_most_recent_date is None
or acc_date > trip_most_recent_date
or acc_date == trip_most_recent_date
):
trip_most_recent_date = acc_date
trip_most_recent_location = _get_accommodation_location(
acc, on_trip=True
)
# Check trains within trip period
for travel_item in trip.travel:
if travel_item["type"] == "train":
for leg in travel_item.get("legs", []):
if "arrive" in leg:
try:
arrive_datetime, arrive_date = _parse_datetime_field(
leg["arrive"]
)
except ValueError:
continue
# Only consider trains within this trip and before target date
if trip.start <= arrive_date <= target_date:
# Compare both date and time to handle same-day arrivals correctly
if (
trip_most_recent_date is None
or arrive_date > trip_most_recent_date
or (
arrive_date == trip_most_recent_date
and (
trip_most_recent_datetime is None
or arrive_datetime > trip_most_recent_datetime
)
)
):
trip_most_recent_date = arrive_date
trip_most_recent_datetime = arrive_datetime
# For trains, we can get station info from to_station if available
destination = leg.get("to")
assert "to_station" in leg
station_info = leg["to_station"]
station_country = station_info["country"]
if station_country == "gb":
trip_most_recent_location = (
destination,
get_country("gb"),
)
else:
trip_most_recent_location = (
destination,
get_country(station_country),
)
# Check ferries within trip period
for travel_item in trip.travel:
if travel_item["type"] == "ferry" and "arrive" in travel_item:
try:
arrive_datetime, arrive_date = _parse_datetime_field(
travel_item["arrive"]
)
except ValueError:
continue
# Only consider ferries within this trip and before target date
if trip.start <= arrive_date <= target_date:
# Compare both date and time to handle same-day arrivals correctly
if (
trip_most_recent_date is None
or arrive_date > trip_most_recent_date
or (
arrive_date == trip_most_recent_date
and (
trip_most_recent_datetime is None
or arrive_datetime > trip_most_recent_datetime
)
)
):
trip_most_recent_date = arrive_date
trip_most_recent_datetime = arrive_datetime
# For ferries, we can get terminal info from to_terminal if available
destination = travel_item.get("to")
assert "to_terminal" in travel_item
terminal_info = travel_item["to_terminal"]
terminal_country = terminal_info.get("country", "gb")
terminal_city = terminal_info.get("city", destination)
if terminal_country == "gb":
trip_most_recent_location = (
terminal_city,
get_country("gb"),
)
else:
trip_most_recent_location = (
terminal_city,
get_country(terminal_country),
)
return trip_most_recent_location
def _get_trip_location_by_progression(
trip: Trip, target_date: date
) -> tuple[str | None, pycountry.db.Country | None] | None:
"""Determine location based on trip progression and date."""
locations = trip.locations()
if not locations:
return None
# If only one location, use it (when on a trip, always show the location)
if len(locations) == 1:
city, country = locations[0]
return (city, country)
# Multiple locations: use progression through the trip
if not trip.end:
city, country = locations[-1]
return (city, country)
trip_duration = (trip.end - trip.start).days + 1
days_into_trip = (target_date - trip.start).days
# Simple progression: first half at first location, second half at last location
if days_into_trip <= trip_duration // 2:
city, country = locations[0]
else:
city, country = locations[-1]
return (city, country)
def _find_most_recent_travel_before_date(
target_date: date,
trips: list[Trip],
) -> tuple[str | None, pycountry.db.Country | None] | None:
"""Find the most recent travel location before a given date."""
uk_airports = {"LHR", "LGW", "STN", "LTN", "BRS", "BHX", "MAN", "EDI", "GLA"}
most_recent_location: tuple[str | None, pycountry.db.Country | None] | None = None
most_recent_date: date | None = None
most_recent_datetime: datetime | None = None
# Check all travel across all trips
for trip in trips:
# Check flights
for travel_item in trip.travel:
if travel_item["type"] == "flight" and "arrive" in travel_item:
try:
arrive_datetime, arrive_date = _parse_datetime_field(
travel_item["arrive"]
)
except ValueError:
continue
if arrive_date <= target_date:
# Compare both date and time to handle same-day flights correctly
if (
most_recent_date is None
or arrive_date > most_recent_date
or (
arrive_date == most_recent_date
and (
most_recent_datetime is None
or arrive_datetime > most_recent_datetime
)
)
):
most_recent_date = arrive_date
most_recent_datetime = arrive_datetime
destination_airport = travel_item["to"]
# For flights, determine if we're "on trip" based on whether this is within any trip period
on_trip = any(
t.start <= arrive_date <= (t.end or t.start) for t in trips
)
if "to_airport" in travel_item:
airport_info = travel_item["to_airport"]
airport_country = airport_info.get("country", "gb")
if airport_country == "gb":
if not on_trip:
# When not on a trip, UK airports mean home
most_recent_location = (None, get_country("gb"))
else:
# When on a trip, show the actual location even for UK airports
location_name = airport_info.get(
"city", airport_info.get("name", "London")
)
most_recent_location = (
location_name,
get_country("gb"),
)
else:
location_name = airport_info.get(
"city",
airport_info.get("name", destination_airport),
)
most_recent_location = (
location_name,
get_country(airport_country),
)
else:
most_recent_location = (
destination_airport,
get_country("gb"),
)
# Check trains
elif travel_item["type"] == "train":
for leg in travel_item.get("legs", []):
if "arrive" in leg:
try:
arrive_datetime, arrive_date = _parse_datetime_field(
leg["arrive"]
)
except ValueError:
continue
if arrive_date <= target_date:
# Compare both date and time to handle same-day arrivals correctly
if (
most_recent_date is None
or arrive_date > most_recent_date
or (
arrive_date == most_recent_date
and (
most_recent_datetime is None
or arrive_datetime > most_recent_datetime
)
)
):
most_recent_date = arrive_date
most_recent_datetime = arrive_datetime
destination = leg.get("to")
on_trip = any(
t.start <= arrive_date <= (t.end or t.start)
for t in trips
)
if "to_station" in leg:
station_info = leg["to_station"]
station_country = station_info.get("country", "gb")
if station_country == "gb":
if not on_trip:
most_recent_location = (
None,
get_country("gb"),
)
else:
most_recent_location = (
destination,
get_country("gb"),
)
else:
most_recent_location = (
destination,
get_country(station_country),
)
else:
most_recent_location = (
destination,
get_country("gb"),
)
# Check ferries
elif travel_item["type"] == "ferry" and "arrive" in travel_item:
try:
arrive_datetime, arrive_date = _parse_datetime_field(
travel_item["arrive"]
)
except ValueError:
continue
if arrive_date <= target_date:
# Compare both date and time to handle same-day arrivals correctly
if (
most_recent_date is None
or arrive_date > most_recent_date
or (
arrive_date == most_recent_date
and (
most_recent_datetime is None
or arrive_datetime > most_recent_datetime
)
)
):
most_recent_date = arrive_date
most_recent_datetime = arrive_datetime
destination = travel_item.get("to")
on_trip = any(
t.start <= arrive_date <= (t.end or t.start) for t in trips
)
if "to_terminal" in travel_item:
terminal_info = travel_item["to_terminal"]
terminal_country = terminal_info.get("country", "gb")
terminal_city = terminal_info.get("city", destination)
if terminal_country == "gb":
if not on_trip:
most_recent_location = (None, get_country("gb"))
else:
most_recent_location = (
terminal_city,
get_country("gb"),
)
else:
most_recent_location = (
terminal_city,
get_country(terminal_country),
)
else:
most_recent_location = (destination, get_country("gb"))
# Check accommodation - only override if accommodation is more recent
for acc in trip.accommodation:
if "from" in acc:
try:
_, acc_date = _parse_datetime_field(acc["from"])
except ValueError:
continue
if acc_date <= target_date:
# Only update if this accommodation is more recent than existing result
if most_recent_date is None or acc_date > most_recent_date:
most_recent_date = acc_date
on_trip = any(
t.start <= acc_date <= (t.end or t.start) for t in trips
)
most_recent_location = _get_accommodation_location(
acc, on_trip=on_trip
)
return most_recent_location
def _check_return_home_heuristic(
target_date: date, trips: list[Trip]
) -> tuple[str | None, pycountry.db.Country | None] | None:
"""Check if should return home based on recent trips that have ended."""
for trip in trips:
if trip.end and trip.end < target_date:
locations = trip.locations()
if locations:
final_city, final_country = locations[-1]
final_alpha_2 = final_country.alpha_2
days_since_trip = (target_date - trip.end).days
# If trip ended in UK, you should be home now
if hasattr(final_country, "alpha_2") and final_country.alpha_2 == "GB":
return (None, get_country("gb"))
return None
def get_location_for_date(
target_date: date,
trips: list[Trip],
) -> tuple[str | None, pycountry.db.Country | None]:
"""Get location (city, country) for a specific date using travel history."""
# First check if currently on a trip
for trip in trips:
if not (trip.start <= target_date <= (trip.end or trip.start)):
continue
# For trips, find the most recent travel within the trip period
trip_location = _find_most_recent_travel_within_trip(
trip,
target_date,
)
if trip_location:
return trip_location
# Fallback: determine location based on trip progression and date
progression_location = _get_trip_location_by_progression(trip, target_date)
if progression_location:
return progression_location
# Find most recent travel before this date
recent_travel = _find_most_recent_travel_before_date(target_date, trips)
# Check for recent trips that have ended - prioritize this over individual travel data
# This handles cases where you're traveling home after a trip (e.g. stopovers, connections)
return_home = _check_return_home_heuristic(target_date, trips)
if return_home:
return return_home
# Return most recent location or default to home
if recent_travel:
return recent_travel
return (None, get_country("gb"))
def weekends(
start: date, busy_events: list[Event], trips: list[Trip], data_dir: str
) -> typing.Sequence[StrDict]:
"""Next ten weekends."""
weekday = start.weekday()
# Calculate the difference to the next or previous Saturday
if weekday == 6: # Sunday
start_date = start - timedelta(days=1)
else:
start_date = start + timedelta(days=(5 - weekday))
weekends_info = []
for i in range(52):
saturday = start_date + timedelta(weeks=i)
sunday = saturday + timedelta(days=1)
saturday_events = [
event
for event in busy_events
if event.end_date and event.as_date <= saturday <= event.end_as_date
]
sunday_events = [
event
for event in busy_events
if event.end_date and event.as_date <= sunday <= event.end_as_date
]
saturday_location = get_location_for_date(
saturday,
trips,
)
sunday_location = get_location_for_date(
sunday,
trips,
)
weekends_info.append(
{
"date": saturday,
"saturday": saturday_events,
"sunday": sunday_events,
"saturday_location": saturday_location,
"sunday_location": sunday_location,
}
)
return weekends_info
def find_gaps(events: list[Event], min_gap_days: int = 3) -> list[StrDict]:
"""Gaps of at least `min_gap_days` between events in a list of events."""
# Sort events by start date
gaps: list[tuple[date, date]] = []
previous_event_end = None
by_start_date = {
d: list(on_day)
for d, on_day in itertools.groupby(events, key=lambda e: e.as_date)
}
by_end_date = {
d: list(on_day)
for d, on_day in itertools.groupby(events, key=lambda e: e.end_as_date)
}
for event in events:
# Use start date for current event
start_date = event.as_date
# If previous event exists, calculate the gap
if previous_event_end:
gap_days = (start_date - previous_event_end).days
if gap_days >= (min_gap_days + 2):
start_end = (
previous_event_end + timedelta(days=1),
start_date - timedelta(days=1),
)
gaps.append(start_end)
# Update previous event end date
end = event.end_as_date
if not previous_event_end or end > previous_event_end:
previous_event_end = end
return [
{
"start": gap_start,
"end": gap_end,
"after": by_start_date[gap_end + timedelta(days=1)],
"before": by_end_date[gap_start - timedelta(days=1)],
}
for gap_start, gap_end in gaps
]

142
agenda/calendar.py
View file

@ -1,115 +1,79 @@
"""Calendar."""
import typing
import uuid
from datetime import timedelta
from .event import Event
from .types import Event
# A map to associate event types with a specific calendar ID
event_type_calendar_map = {
"bank_holiday": "uk_holidays",
"conference": "conferences",
"us_holiday": "us_holidays",
"birthday": "birthdays",
"waste_schedule": "home",
"accommodation": "travel",
"market": "markets",
event_type_color_map = {
"bank_holiday": "success-subtle",
"conference": "primary-subtle",
"us_holiday": "secondary-subtle",
"birthday": "info-subtle",
"waste_schedule": "danger-subtle",
}
# Define the calendars (categories) for TOAST UI
# These will be passed to the frontend to configure colors and names
toastui_calendars = [
{"id": "default", "name": "General", "backgroundColor": "#00a9ff"},
{"id": "uk_holidays", "name": "UK Bank Holiday", "backgroundColor": "#28a745"},
{"id": "us_holidays", "name": "US Holiday", "backgroundColor": "#6c757d"},
{"id": "conferences", "name": "Conference", "backgroundColor": "#007bff"},
{"id": "birthdays", "name": "Birthday", "backgroundColor": "#17a2b8"},
{"id": "home", "name": "Home", "backgroundColor": "#dc3545"},
{"id": "travel", "name": "Travel", "backgroundColor": "#ffc107", "color": "#000"},
{"id": "markets", "name": "Markets", "backgroundColor": "#e2e3e5", "color": "#000"},
]
colors = {
"primary-subtle": "#cfe2ff",
"secondary-subtle": "#e2e3e5",
"success-subtle": "#d1e7dd",
"info-subtle": "#cff4fc",
"warning-subtle": "#fff3cd",
"danger-subtle": "#f8d7da",
}
def build_toastui_events(events: list[Event]) -> list[dict[str, typing.Any]]:
"""Build a list of event objects for TOAST UI Calendar."""
def build_events(events: list[Event]) -> list[dict[str, typing.Any]]:
"""Build list of events for FullCalendar."""
items: list[dict[str, typing.Any]] = []
one_day = timedelta(days=1)
for e in events:
if e.name == "today":
continue
# Determine the calendar ID for the event, defaulting if not mapped
calendar_id = event_type_calendar_map.get(e.name, "default")
# Handle special case for 'accommodation'
if e.name == "accommodation":
assert e.title and e.end_date
# All-day event for the duration of the stay
items.append(
{
"id": str(uuid.uuid4()),
"calendarId": calendar_id,
"title": e.title_with_emoji,
"start": e.as_date.isoformat(),
"end": (e.end_as_date + one_day).isoformat(),
"isAllday": True,
"raw": {"url": e.url},
}
)
# Timed event for check-in
items.append(
{
"id": str(uuid.uuid4()),
"calendarId": calendar_id,
"title": f"Check-in: {e.title}",
"start": e.date.isoformat(),
"end": (e.date + timedelta(hours=1)).isoformat(),
"isAllday": False,
"raw": {"url": e.url},
}
)
# Timed event for check-out
items.append(
{
"id": str(uuid.uuid4()),
"calendarId": calendar_id,
"title": f"Checkout: {e.title}",
"start": e.end_date.isoformat(),
"end": (e.end_date + timedelta(hours=1)).isoformat(),
"isAllday": False,
"raw": {"url": e.url},
}
)
item = {
"allDay": True,
"title": e.display_title,
"start": e.as_date.isoformat(),
"end": (e.end_as_date + one_day).isoformat(),
"url": e.url,
}
items.append(item)
item = {
"allDay": False,
"title": "checkin: " + e.title,
"start": e.date.isoformat(),
"url": e.url,
}
items.append(item)
item = {
"allDay": False,
"title": "checkout: " + e.title,
"start": e.end_date.isoformat(),
"url": e.url,
}
items.append(item)
continue
# Handle all other events
start_iso = e.date.isoformat()
if e.has_time:
end_iso = (e.end_date or e.date + timedelta(minutes=30)).isoformat()
end = e.end_date or e.date + timedelta(hours=1)
else:
# For all-day events, the end date is exclusive.
# For a single-day event, the end date should be the same as the start date.
# For a multi-day event, a day is added to make the range inclusive.
if e.end_date:
# This is a multi-day event, so add one day to the end.
end_date = e.end_as_date + one_day
else:
# This is a single-day event. The end date is the same as the start date.
end_date = e.as_date
end_iso = end_date.isoformat()
item: dict[str, typing.Any] = {
"id": str(uuid.uuid4()),
"calendarId": calendar_id,
"title": e.title_with_emoji,
"start": start_iso,
"end": end_iso,
"isAllday": not e.has_time,
end = (e.end_as_date if e.end_date else e.as_date) + one_day
item = {
"allDay": not e.has_time,
"title": e.display_title,
"start": e.date.isoformat(),
"end": end.isoformat(),
}
if e.name in event_type_color_map:
item["color"] = colors[event_type_color_map[e.name]]
item["textColor"] = "black"
if e.url:
item["raw"] = {"url": e.url}
item["url"] = e.url
items.append(item)
return items

474
agenda/car_journey_timeline.py
View file

@ -1,474 +0,0 @@
"""Import recorded car journeys from a Google Timeline export."""
from __future__ import annotations
import argparse
import json
import typing
from collections import defaultdict
from dataclasses import dataclass
from datetime import date, datetime, timedelta
from pathlib import Path
from geopy.distance import geodesic # type: ignore
from . import car_journey_yaml, trip as trip_module, utils
PERSONAL_DATA_DIR = Path("~/src/personal-data").expanduser()
VEHICLE_ACTIVITY_TYPES = frozenset({"IN_PASSENGER_VEHICLE", "IN_VEHICLE"})
ACTIVITY_SPLIT_GAP = timedelta(hours=2)
LatLon = tuple[float, float]
LonLat = tuple[float, float]
StrDict = dict[str, typing.Any]
@dataclass(frozen=True)
class TimelinePoint:
"""A single Google Timeline breadcrumb point."""
time: datetime
location: LatLon
@property
def lonlat(self) -> LonLat:
"""Return point as GeoJSON lon/lat."""
return (self.location[1], self.location[0])
@dataclass(frozen=True)
class VehicleActivity:
"""A vehicle activity segment plus its breadcrumb points."""
index: int
mode: str
start_time: datetime
end_time: datetime
start: LatLon
end: LatLon
points: tuple[TimelinePoint, ...]
@property
def path(self) -> tuple[LatLon, ...]:
"""Return a deduplicated path."""
coords = [point.location for point in self.points]
if not coords:
coords = [self.start, self.end]
elif coords[0] != self.start:
coords.insert(0, self.start)
if coords[-1] != self.end:
coords.append(self.end)
return tuple(dedupe_consecutive(coords))
@property
def depart_time(self) -> datetime:
"""Prefer the first breadcrumb time when available."""
return self.points[0].time if self.points else self.start_time
@property
def arrive_time(self) -> datetime:
"""Prefer the last breadcrumb time when available."""
return self.points[-1].time if self.points else self.end_time
@property
def leading_path(self) -> tuple[LatLon, ...]:
"""Return the first few coordinates for start matching."""
return self.path[: min(3, len(self.path))]
@property
def trailing_path(self) -> tuple[LatLon, ...]:
"""Return the last few coordinates for end matching."""
return self.path[-min(3, len(self.path)) :]
@dataclass(frozen=True)
class TimelineCarJourney:
"""A YAML-ready car journey row with a GeoJSON path."""
trip: date
depart: datetime
arrive: datetime
route: str
from_label: str
to_label: str
path: tuple[LatLon, ...]
mode: str
def as_yaml_item(self) -> StrDict:
"""Return the YAML mapping."""
return {
"trip": self.trip,
"depart": self.depart,
"arrive": self.arrive,
"route": self.route,
"from": self.from_label,
"to": self.to_label,
}
def as_geojson(self) -> StrDict:
"""Return a GeoJSON feature for this journey."""
return {
"type": "Feature",
"properties": {
"source": "google_timeline",
"mode": self.mode,
"start_time": self.depart.isoformat(),
"end_time": self.arrive.isoformat(),
},
"geometry": {
"type": "LineString",
"coordinates": [[lon, lat] for lat, lon in self.path],
},
}
def parse_latlng(value: str) -> LatLon:
"""Parse Google's ``lat°, lon°`` format."""
latitude, longitude = value.replace("°", "").split(",", 1)
return (float(latitude.strip()), float(longitude.strip()))
def days_spanned(start: datetime, end: datetime) -> list[date]:
"""Return all dates touched by a datetime range."""
day = start.date()
days = [day]
while day < end.date():
day += timedelta(days=1)
days.append(day)
return days
def dedupe_consecutive(coords: list[LatLon]) -> list[LatLon]:
"""Drop repeated adjacent coordinates."""
deduped: list[LatLon] = []
for coord in coords:
if deduped and deduped[-1] == coord:
continue
deduped.append(coord)
return deduped
def load_timeline_path_points(data: StrDict) -> dict[date, list[TimelinePoint]]:
"""Group all timelinePath points by local date."""
points_by_day: dict[date, list[TimelinePoint]] = defaultdict(list)
for segment in data.get("semanticSegments", []):
timeline_path = segment.get("timelinePath")
if not isinstance(timeline_path, list):
continue
for item in timeline_path:
if not isinstance(item, dict):
continue
point = item.get("point")
time_value = item.get("time")
if not isinstance(point, str) or not isinstance(time_value, str):
continue
breadcrumb = TimelinePoint(
time=datetime.fromisoformat(time_value),
location=parse_latlng(point),
)
points_by_day[breadcrumb.time.date()].append(breadcrumb)
for breadcrumbs in points_by_day.values():
breadcrumbs.sort(key=lambda item: item.time)
return points_by_day
def activity_points(
start: datetime,
end: datetime,
points_by_day: dict[date, list[TimelinePoint]],
) -> tuple[TimelinePoint, ...]:
"""Return breadcrumb points within an activity window."""
selected: list[TimelinePoint] = []
for day in days_spanned(start, end):
for point in points_by_day.get(day, []):
if start <= point.time <= end:
selected.append(point)
selected.sort(key=lambda item: item.time)
return tuple(selected)
def load_vehicle_activities(timeline_path: Path) -> list[VehicleActivity]:
"""Load vehicle activities and attach breadcrumb points."""
data = typing.cast(StrDict, json.loads(timeline_path.read_text()))
points_by_day = load_timeline_path_points(data)
activities: list[VehicleActivity] = []
next_index = 0
for index, segment in enumerate(data.get("semanticSegments", [])):
activity = segment.get("activity")
if not isinstance(activity, dict):
continue
mode = activity.get("topCandidate", {}).get("type")
start_location = activity.get("start", {}).get("latLng")
end_location = activity.get("end", {}).get("latLng")
start_time = segment.get("startTime")
end_time = segment.get("endTime")
if (
mode not in VEHICLE_ACTIVITY_TYPES
or not isinstance(start_location, str)
or not isinstance(end_location, str)
or not isinstance(start_time, str)
or not isinstance(end_time, str)
):
continue
depart = datetime.fromisoformat(start_time)
arrive = datetime.fromisoformat(end_time)
start_latlon = parse_latlng(start_location)
end_latlon = parse_latlng(end_location)
activity = VehicleActivity(
index=index,
mode=typing.cast(str, mode),
start_time=depart,
end_time=arrive,
start=start_latlon,
end=end_latlon,
points=activity_points(
depart,
arrive,
points_by_day,
),
)
for split_activity in split_activity_on_gaps(activity):
activities.append(
VehicleActivity(
index=next_index,
mode=split_activity.mode,
start_time=split_activity.start_time,
end_time=split_activity.end_time,
start=split_activity.start,
end=split_activity.end,
points=split_activity.points,
)
)
next_index += 1
return activities
def activity_matches_date(activity: VehicleActivity, target: date) -> bool:
"""Return whether an activity overlaps a date."""
return target in days_spanned(activity.start_time, activity.end_time)
def split_activity_on_gaps(activity: VehicleActivity) -> list[VehicleActivity]:
"""Split an activity when breadcrumbs contain long pauses."""
if len(activity.points) < 2:
return [activity]
chunks: list[list[TimelinePoint]] = [[]]
for point in activity.points:
current = chunks[-1]
if current and point.time - current[-1].time > ACTIVITY_SPLIT_GAP:
chunks.append([])
current = chunks[-1]
current.append(point)
if len(chunks) == 1:
return [activity]
split: list[VehicleActivity] = []
for chunk in chunks:
split.append(
VehicleActivity(
index=activity.index,
mode=activity.mode,
start_time=chunk[0].time,
end_time=chunk[-1].time,
start=chunk[0].location,
end=chunk[-1].location,
points=tuple(chunk),
)
)
return split
def distance_to_coords_km(coords: tuple[LatLon, ...], target: LatLon) -> float:
"""Return the closest coordinate distance to a target coordinate."""
return min(float(geodesic(coord, target).km) for coord in coords)
def select_activity_slice(
activities: list[VehicleActivity],
start_target: LatLon,
end_target: LatLon,
) -> list[VehicleActivity]:
"""Select the contiguous activity slice that best matches a planned leg."""
if not activities:
raise ValueError("no vehicle activities available for route matching")
best_slice: tuple[float, int, int] | None = None
for start_index, activity in enumerate(activities):
start_distance = distance_to_coords_km(activity.leading_path, start_target)
for end_index in range(start_index, len(activities)):
end_distance = distance_to_coords_km(
activities[end_index].trailing_path,
end_target,
)
score = start_distance + end_distance
if (
best_slice is None
or score < best_slice[0]
or (
score == best_slice[0]
and end_index - start_index < best_slice[2] - best_slice[1]
)
):
best_slice = (score, start_index, end_index)
assert best_slice is not None
return activities[best_slice[1] : best_slice[2] + 1]
def stop_labels(
from_label: str,
to_label: str,
count: int,
) -> list[tuple[str, str]]:
"""Return start/end labels for a split route."""
if count == 1:
return [(from_label, to_label)]
labels: list[tuple[str, str]] = []
for index in range(count):
start = from_label if index == 0 else f"Drive stop {index}"
end = to_label if index == count - 1 else f"Drive stop {index + 1}"
labels.append((start, end))
return labels
def route_name(activity: VehicleActivity) -> str:
"""Return a stable route filename stem for an imported activity."""
return "timeline_" + activity.depart_time.strftime("%Y%m%d_%H%M%S")
def build_timeline_journeys(
timeline_path: Path,
data_dir: Path,
today: date | None = None,
) -> list[TimelineCarJourney]:
"""Generate split past car journeys from timeline data."""
if today is None:
today = date.today()
activities = load_vehicle_activities(timeline_path)
existing = car_journey_yaml.load_yaml_list(data_dir / "car_journeys.yaml")
generated: list[TimelineCarJourney] = []
for item in existing:
trip_date = utils.as_date(item["trip"])
if trip_date >= today:
continue
route = item.get("route")
if not isinstance(route, str):
raise ValueError(f"car journey route must be a string: {item!r}")
route_path = car_journey_yaml.car_route_path(data_dir, route)
existing_geojson = car_journey_yaml.read_geojson(route_path)
endpoints = trip_module.geojson_route_endpoints(existing_geojson)
if endpoints is None:
raise ValueError(f"could not read route endpoints from {route_path}")
from_label, to_label = trip_module.car_route_labels(item)
if not isinstance(from_label, str) or not isinstance(to_label, str):
raise ValueError(f"could not determine car route labels for {item!r}")
journey_date = utils.as_date(item["depart"])
matches = [
activity
for activity in activities
if activity_matches_date(activity, journey_date)
]
selected = select_activity_slice(matches, endpoints[0], endpoints[1])
labels = stop_labels(from_label, to_label, len(selected))
for activity, (start_label, end_label) in zip(selected, labels, strict=True):
generated.append(
TimelineCarJourney(
trip=trip_date,
depart=activity.depart_time,
arrive=activity.arrive_time,
route=route_name(activity),
from_label=start_label,
to_label=end_label,
path=activity.path,
mode=activity.mode,
)
)
return generated
def write_timeline_routes(
data_dir: Path,
journeys: list[TimelineCarJourney],
) -> int:
"""Write GeoJSON files for generated journeys."""
route_dir = data_dir / "car_routes"
route_dir.mkdir(parents=True, exist_ok=True)
written = 0
for journey in journeys:
path = route_dir / f"{journey.route}.geojson"
path.write_text(json.dumps(journey.as_geojson(), separators=(",", ":")) + "\n")
written += 1
return written
def rewrite_car_journeys_yaml(
data_dir: Path,
journeys: list[TimelineCarJourney],
today: date | None = None,
) -> tuple[int, int]:
"""Replace past YAML rows with timeline-derived journeys."""
if today is None:
today = date.today()
path = data_dir / "car_journeys.yaml"
existing = car_journey_yaml.load_yaml_list(path)
future_rows = [item for item in existing if utils.as_date(item["trip"]) >= today]
generated_rows = [journey.as_yaml_item() for journey in journeys]
combined = generated_rows + future_rows
combined.sort(key=car_journey_yaml.journey_key)
path.write_text(car_journey_yaml.dump_yaml_list_with_blank_lines(combined))
replaced = len(existing) - len(future_rows)
return (replaced, len(generated_rows))
def import_from_timeline(
timeline_path: Path,
data_dir: Path = PERSONAL_DATA_DIR,
today: date | None = None,
) -> tuple[int, int, int]:
"""Write route files and replace past YAML rows."""
journeys = build_timeline_journeys(timeline_path, data_dir, today=today)
routes_written = write_timeline_routes(data_dir, journeys)
rows_replaced, rows_written = rewrite_car_journeys_yaml(
data_dir, journeys, today=today
)
return (routes_written, rows_replaced, rows_written)
def parse_date(value: str) -> date:
"""Parse an ISO date."""
return date.fromisoformat(value)
def main(argv: list[str] | None = None) -> int:
"""CLI entrypoint."""
parser = argparse.ArgumentParser(
description=(
"Replace past car_journeys.yaml rows with routes split from a "
"Google Timeline export."
)
)
parser.add_argument("timeline", type=Path)
parser.add_argument("--data-dir", type=Path, default=PERSONAL_DATA_DIR)
parser.add_argument("--today", type=parse_date, default=None)
args = parser.parse_args(argv)
routes_written, rows_replaced, rows_written = import_from_timeline(
args.timeline,
data_dir=args.data_dir,
today=args.today,
)
print(f"Wrote {routes_written} route GeoJSON file(s)")
print(f"Replaced {rows_replaced} past car journey row(s)")
print(f"Wrote {rows_written} split timeline journey row(s)")
return 0

762
agenda/car_journey_yaml.py
View file

@ -1,762 +0,0 @@
"""Generate car journey YAML and routes for trips."""
from __future__ import annotations
import argparse
import json
import os
import re
import typing
import unicodedata
from dataclasses import dataclass
from datetime import date, datetime
from pathlib import Path
import yaml
try:
import openrouteservice # type: ignore[import-not-found]
except ImportError: # pragma: no cover - optional dependency in tests
openrouteservice = None
from . import trip as trip_module
from . import utils
PERSONAL_DATA_DIR = Path("~/src/personal-data").expanduser()
DEFAULT_HOME_LABEL = "PCH"
DEFAULT_PROFILE = "driving-car"
DEFAULT_SNAP_RADIUS_METRES = 5000
LatLon = tuple[float, float]
LonLat = tuple[float, float]
GeoJSON = dict[str, typing.Any]
StrDict = dict[str, typing.Any]
RouteFetcher = typing.Callable[[LonLat, LonLat], GeoJSON]
class NoAliasDumper(yaml.SafeDumper):
"""YAML dumper that avoids anchors for repeated scalar values."""
def ignore_aliases(self, data: typing.Any) -> bool:
"""Disable aliases for all values."""
return True
class TripLike(typing.Protocol):
"""Trip fields needed for car journey generation."""
start: date
accommodation: list[StrDict]
travel: list[StrDict]
@dataclass(frozen=True)
class CarJourney:
"""A car journey ready to write to YAML and GeoJSON."""
trip: date
depart: date
arrive: date
route: str
start: LonLat
end: LonLat
def as_yaml_item(self) -> StrDict:
"""Return the YAML mapping for this journey."""
return {
"trip": self.trip,
"depart": self.depart,
"arrive": self.arrive,
"route": self.route,
}
def latlon_to_lonlat(coord: LatLon) -> LonLat:
"""Convert a lat/lon pair to lon/lat for routing APIs."""
return (coord[1], coord[0])
def route_label(value: str) -> str:
"""Return a filesystem-safe route label."""
ascii_value = (
unicodedata.normalize(
"NFKD",
value.strip()
.replace("æ", "ae")
.replace("Æ", "AE")
.replace("ð", "d")
.replace("Ð", "D")
.replace("þ", "th")
.replace("Þ", "Th"),
)
.encode("ascii", "ignore")
.decode("ascii")
)
label = re.sub(r"[^A-Za-z0-9]+", "_", ascii_value).strip("_")
if not label:
raise ValueError(f"empty route label from {value!r}")
return label
def load_yaml_list(path: Path) -> list[StrDict]:
"""Load a YAML list, returning an empty list when the file does not exist."""
if not path.exists():
return []
loaded = yaml.safe_load(path.read_text())
if loaded is None:
return []
if isinstance(loaded, list) and all(isinstance(item, dict) for item in loaded):
return typing.cast(list[StrDict], loaded)
raise ValueError(f"{path} must contain a YAML list")
def dump_yaml_list_with_blank_lines(items: list[StrDict]) -> str:
"""Dump a top-level YAML list using the personal-data spacing style."""
if not items:
return "---\n"
text = yaml.dump(
items, sort_keys=False, allow_unicode=True, Dumper=NoAliasDumper
).lstrip()
return "---\n" + text.replace("\n- ", "\n\n- ")
def car_route_path(data_dir: Path, route: str) -> Path:
"""Return the GeoJSON path for a car route name."""
route_filename = trip_module.route_filename_without_extension(route)
return data_dir / "car_routes" / f"{route_filename}.geojson"
def read_geojson(path: Path) -> GeoJSON:
"""Read a GeoJSON file."""
return typing.cast(GeoJSON, json.loads(path.read_text()))
def home_latlon_from_existing_routes(data_dir: Path, home_label: str) -> LatLon:
"""Find the home coordinate from existing car routes."""
route_dir = data_dir / "car_routes"
if not route_dir.exists():
raise ValueError(f"missing car route directory: {route_dir}")
home_route_prefix = f"{home_label}_to_"
home_route_suffix = f"_to_{home_label}"
for path in sorted(route_dir.glob("*.geojson")):
geojson_data = read_geojson(path)
endpoints = trip_module.geojson_route_endpoints(geojson_data)
if endpoints is None:
continue
route_name = path.stem
if route_name.startswith(home_route_prefix):
return endpoints[0]
if route_name.endswith(home_route_suffix):
return endpoints[1]
raise ValueError(
f"could not find home coordinate for {home_label!r} in {route_dir}"
)
def trip_for_date(data_dir: Path, trip_date: date) -> TripLike:
"""Return the trip with the given start date."""
trips = typing.cast(
list[TripLike], trip_module.build_trip_list(data_dir=str(data_dir))
)
for item in trips:
if item.start == trip_date:
return item
raise ValueError(f"could not find trip starting {trip_date}")
def primary_accommodation(trip: TripLike) -> StrDict:
"""Return the first accommodation for a trip."""
if not trip.accommodation:
raise ValueError(f"trip {trip.start} has no accommodation")
return min(trip.accommodation, key=lambda item: utils.as_datetime(item["from"]))
def accommodation_label(accommodation: StrDict) -> str:
"""Return the destination label for an accommodation."""
label = accommodation.get("location") or accommodation.get("name")
if not isinstance(label, str):
raise ValueError("accommodation has no location or name")
return route_label(label)
def accommodation_latlon(accommodation: StrDict) -> LatLon:
"""Return accommodation coordinates."""
latitude = accommodation.get("latitude")
longitude = accommodation.get("longitude")
if not isinstance(latitude, (int, float)) or not isinstance(
longitude, (int, float)
):
raise ValueError("accommodation is missing latitude/longitude")
return (float(latitude), float(longitude))
def sorted_accommodation(trip: TripLike) -> list[StrDict]:
"""Return trip accommodation sorted by check-in time."""
return sorted(trip.accommodation, key=lambda item: utils.as_datetime(item["from"]))
def airport_latlon(airport: StrDict) -> LatLon:
"""Return airport coordinates."""
latitude = airport.get("latitude")
longitude = airport.get("longitude")
if not isinstance(latitude, (int, float)) or not isinstance(
longitude, (int, float)
):
raise ValueError("airport is missing latitude/longitude")
return (float(latitude), float(longitude))
def flight_departure_datetime(item: StrDict) -> typing.Any:
"""Return flight departure value for sorting."""
return item["depart"]
def trip_flights(trip: TripLike) -> list[StrDict]:
"""Return trip flights sorted by departure time."""
flights = [item for item in trip.travel if item.get("type") == "flight"]
if not flights:
raise ValueError(f"trip {trip.start} has no flights")
return sorted(
flights, key=lambda item: utils.as_datetime(flight_departure_datetime(item))
)
def trip_ferries(trip: TripLike) -> list[StrDict]:
"""Return trip ferries sorted by departure time."""
ferries = [item for item in trip.travel if item.get("type") == "ferry"]
if not ferries:
raise ValueError(f"trip {trip.start} has no ferries")
return sorted(ferries, key=lambda item: utils.as_datetime(item["depart"]))
def location_latlon(location: StrDict, label: str) -> LatLon:
"""Return coordinates from a loaded travel location."""
latitude = location.get("latitude")
longitude = location.get("longitude")
if not isinstance(latitude, (int, float)) or not isinstance(
longitude, (int, float)
):
raise ValueError(f"{label} is missing latitude/longitude")
return (float(latitude), float(longitude))
def location_name(location: StrDict, fallback: str) -> str:
"""Return a route label source for a loaded travel location."""
name = location.get("name") or fallback
if not isinstance(name, str):
raise ValueError(f"location name must be a string: {location!r}")
return name
def car_journeys_for_accommodation(
trip: TripLike,
data_dir: Path,
home_label: str = DEFAULT_HOME_LABEL,
destination_label: str | None = None,
) -> list[CarJourney]:
"""Build outbound and return car journeys for a trip's primary accommodation."""
accommodation = primary_accommodation(trip)
home = latlon_to_lonlat(home_latlon_from_existing_routes(data_dir, home_label))
destination = latlon_to_lonlat(accommodation_latlon(accommodation))
destination_route_label = destination_label or accommodation_label(accommodation)
outbound_route = f"{route_label(home_label)}_to_{destination_route_label}"
return_route = f"{destination_route_label}_to_{route_label(home_label)}"
check_in = utils.as_date(accommodation["from"])
check_out = utils.as_date(accommodation["to"])
return [
CarJourney(
trip=trip.start,
depart=check_in,
arrive=check_in,
route=outbound_route,
start=home,
end=destination,
),
CarJourney(
trip=trip.start,
depart=check_out,
arrive=check_out,
route=return_route,
start=destination,
end=home,
),
]
def car_journeys_for_ferry(
trip: TripLike,
data_dir: Path,
home_label: str = DEFAULT_HOME_LABEL,
destination_label: str | None = None,
) -> list[CarJourney]:
"""Build car journeys around an outbound and return car ferry crossing."""
ferries = trip_ferries(trip)
outbound_ferry = ferries[0]
return_ferry = ferries[-1]
accommodation = primary_accommodation(trip)
home = latlon_to_lonlat(home_latlon_from_existing_routes(data_dir, home_label))
accommodation_coord = latlon_to_lonlat(accommodation_latlon(accommodation))
accommodation_route_label = destination_label or accommodation_label(accommodation)
outbound_from_terminal = outbound_ferry.get("from_terminal")
outbound_to_terminal = outbound_ferry.get("to_terminal")
return_from_terminal = return_ferry.get("from_terminal")
return_to_terminal = return_ferry.get("to_terminal")
terminals = (
outbound_from_terminal,
outbound_to_terminal,
return_from_terminal,
return_to_terminal,
)
if not all(isinstance(terminal, dict) for terminal in terminals):
raise ValueError("ferry is missing terminal details")
outbound_from = typing.cast(StrDict, outbound_from_terminal)
outbound_to = typing.cast(StrDict, outbound_to_terminal)
return_from = typing.cast(StrDict, return_from_terminal)
return_to = typing.cast(StrDict, return_to_terminal)
home_route_label = route_label(home_label)
outbound_from_label = route_label(
location_name(outbound_from, typing.cast(str, outbound_ferry.get("from", "")))
)
outbound_to_label = route_label(
location_name(outbound_to, typing.cast(str, outbound_ferry.get("to", "")))
)
return_from_label = route_label(
location_name(return_from, typing.cast(str, return_ferry.get("from", "")))
)
return_to_label = route_label(
location_name(return_to, typing.cast(str, return_ferry.get("to", "")))
)
outbound_from_coord = latlon_to_lonlat(
location_latlon(outbound_from, outbound_from_label)
)
outbound_to_coord = latlon_to_lonlat(
location_latlon(outbound_to, outbound_to_label)
)
return_from_coord = latlon_to_lonlat(
location_latlon(return_from, return_from_label)
)
return_to_coord = latlon_to_lonlat(location_latlon(return_to, return_to_label))
return [
CarJourney(
trip=trip.start,
depart=utils.as_date(outbound_ferry["depart"]),
arrive=utils.as_date(outbound_ferry["depart"]),
route=f"{home_route_label}_to_{outbound_from_label}",
start=home,
end=outbound_from_coord,
),
CarJourney(
trip=trip.start,
depart=utils.as_date(outbound_ferry["arrive"]),
arrive=utils.as_date(outbound_ferry["arrive"]),
route=f"{outbound_to_label}_to_{accommodation_route_label}",
start=outbound_to_coord,
end=accommodation_coord,
),
CarJourney(
trip=trip.start,
depart=utils.as_date(return_ferry["depart"]),
arrive=utils.as_date(return_ferry["depart"]),
route=f"{accommodation_route_label}_to_{return_from_label}",
start=accommodation_coord,
end=return_from_coord,
),
CarJourney(
trip=trip.start,
depart=utils.as_date(return_ferry["arrive"]),
arrive=utils.as_date(return_ferry["arrive"]),
route=f"{return_to_label}_to_{home_route_label}",
start=return_to_coord,
end=home,
),
]
def car_journeys_for_airport(
trip: TripLike,
data_dir: Path,
home_label: str = DEFAULT_HOME_LABEL,
) -> list[CarJourney]:
"""Build outbound and return car journeys for the trip's UK airport."""
flights = trip_flights(trip)
outbound_flight = flights[0]
return_flight = flights[-1]
outbound_airport = outbound_flight.get("from_airport")
return_airport = return_flight.get("to_airport")
if not isinstance(outbound_airport, dict) or not isinstance(return_airport, dict):
raise ValueError("flight is missing airport details")
outbound_iata = outbound_airport.get("iata")
return_iata = return_airport.get("iata")
if not isinstance(outbound_iata, str) or not isinstance(return_iata, str):
raise ValueError("flight airport is missing IATA code")
home = latlon_to_lonlat(home_latlon_from_existing_routes(data_dir, home_label))
outbound_destination = latlon_to_lonlat(airport_latlon(outbound_airport))
return_start = latlon_to_lonlat(airport_latlon(return_airport))
home_route_label = route_label(home_label)
outbound_route_label = route_label(outbound_iata)
return_route_label = route_label(return_iata)
return [
CarJourney(
trip=trip.start,
depart=utils.as_date(outbound_flight["depart"]),
arrive=utils.as_date(outbound_flight["depart"]),
route=f"{home_route_label}_to_{outbound_route_label}",
start=home,
end=outbound_destination,
),
CarJourney(
trip=trip.start,
depart=utils.as_date(return_flight["arrive"]),
arrive=utils.as_date(return_flight["arrive"]),
route=f"{return_route_label}_to_{home_route_label}",
start=return_start,
end=home,
),
]
def country_code(item: StrDict) -> str | None:
"""Return a lower-case country code from a YAML item."""
country = item.get("country")
return country.casefold() if isinstance(country, str) else None
def accommodation_route_label(accommodation: StrDict) -> str:
"""Return the route label for an accommodation."""
return accommodation_label(accommodation)
def airport_iata(airport: StrDict) -> str:
"""Return an airport IATA code."""
iata = airport.get("iata")
if not isinstance(iata, str):
raise ValueError(f"airport missing IATA code: {airport!r}")
return iata
def flight_airport(flight: StrDict, field: str) -> StrDict:
"""Return airport detail for a flight field."""
airport = flight.get(field)
if not isinstance(airport, dict):
raise ValueError(f"flight missing {field}")
return typing.cast(StrDict, airport)
def car_journeys_for_fly_drive(
trip: TripLike,
data_dir: Path,
home_label: str = DEFAULT_HOME_LABEL,
) -> list[CarJourney]:
"""Build car journeys for home-airport-hotel plus destination rental car legs."""
flights = trip_flights(trip)
accommodations = sorted_accommodation(trip)
if not accommodations:
raise ValueError(f"trip {trip.start} has no accommodation")
outbound_flight = flights[0]
return_flight = flights[-1]
outbound_destination_airport = flight_airport(outbound_flight, "to_airport")
return_origin_airport = flight_airport(return_flight, "from_airport")
return_destination_airport = flight_airport(return_flight, "to_airport")
home = latlon_to_lonlat(home_latlon_from_existing_routes(data_dir, home_label))
home_route_label = route_label(home_label)
preflight_accommodation = next(
(
accommodation
for accommodation in accommodations
if country_code(accommodation) == "gb"
and utils.as_date(accommodation["from"])
<= utils.as_date(outbound_flight["depart"])
),
None,
)
destination_country = country_code(outbound_destination_airport)
rental_accommodations = [
accommodation
for accommodation in accommodations
if country_code(accommodation) == destination_country
]
if not rental_accommodations:
raise ValueError("trip has no destination-country accommodation")
journeys: list[CarJourney] = []
if preflight_accommodation is not None:
preflight_label = accommodation_route_label(preflight_accommodation)
preflight_coord = latlon_to_lonlat(
accommodation_latlon(preflight_accommodation)
)
journeys.append(
CarJourney(
trip=trip.start,
depart=utils.as_date(preflight_accommodation["from"]),
arrive=utils.as_date(preflight_accommodation["from"]),
route=f"{home_route_label}_to_{preflight_label}",
start=home,
end=preflight_coord,
)
)
destination_airport_label = route_label(airport_iata(outbound_destination_airport))
destination_airport_coord = latlon_to_lonlat(
airport_latlon(outbound_destination_airport)
)
first_accommodation = rental_accommodations[0]
first_label = accommodation_route_label(first_accommodation)
first_coord = latlon_to_lonlat(accommodation_latlon(first_accommodation))
journeys.append(
CarJourney(
trip=trip.start,
depart=utils.as_date(outbound_flight["arrive"]),
arrive=utils.as_date(outbound_flight["arrive"]),
route=f"{destination_airport_label}_to_{first_label}",
start=destination_airport_coord,
end=first_coord,
)
)
for previous, current in zip(rental_accommodations, rental_accommodations[1:]):
previous_label = accommodation_route_label(previous)
current_label = accommodation_route_label(current)
journeys.append(
CarJourney(
trip=trip.start,
depart=utils.as_date(previous["to"]),
arrive=utils.as_date(previous["to"]),
route=f"{previous_label}_to_{current_label}",
start=latlon_to_lonlat(accommodation_latlon(previous)),
end=latlon_to_lonlat(accommodation_latlon(current)),
)
)
last_accommodation = rental_accommodations[-1]
return_origin_airport_label = route_label(airport_iata(return_origin_airport))
journeys.append(
CarJourney(
trip=trip.start,
depart=utils.as_date(return_flight["depart"]),
arrive=utils.as_date(return_flight["depart"]),
route=f"{accommodation_route_label(last_accommodation)}_to_{return_origin_airport_label}",
start=latlon_to_lonlat(accommodation_latlon(last_accommodation)),
end=latlon_to_lonlat(airport_latlon(return_origin_airport)),
)
)
return_destination_airport_label = route_label(
airport_iata(return_destination_airport)
)
journeys.append(
CarJourney(
trip=trip.start,
depart=utils.as_date(return_flight["arrive"]),
arrive=utils.as_date(return_flight["arrive"]),
route=f"{return_destination_airport_label}_to_{home_route_label}",
start=latlon_to_lonlat(airport_latlon(return_destination_airport)),
end=home,
)
)
return journeys
def car_journeys_for_trip(
trip: TripLike,
data_dir: Path,
destination: str,
home_label: str = DEFAULT_HOME_LABEL,
destination_label: str | None = None,
) -> list[CarJourney]:
"""Build outbound and return car journeys for a trip."""
if destination == "accommodation":
return car_journeys_for_accommodation(
trip,
data_dir,
home_label=home_label,
destination_label=destination_label,
)
if destination == "airport":
return car_journeys_for_airport(trip, data_dir, home_label=home_label)
if destination == "ferry":
return car_journeys_for_ferry(
trip,
data_dir,
home_label=home_label,
destination_label=destination_label,
)
if destination == "fly-drive":
return car_journeys_for_fly_drive(trip, data_dir, home_label=home_label)
raise ValueError(f"unknown car journey destination {destination!r}")
def openrouteservice_fetcher(api_key: str) -> RouteFetcher:
"""Return a route fetcher backed by openrouteservice."""
if openrouteservice is None:
raise ValueError("openrouteservice must be installed")
client = openrouteservice.Client(key=api_key)
def fetch(start: LonLat, end: LonLat) -> GeoJSON:
return typing.cast(
GeoJSON,
client.directions(
[start, end],
profile=DEFAULT_PROFILE,
format="geojson",
instructions=False,
radiuses=[DEFAULT_SNAP_RADIUS_METRES, DEFAULT_SNAP_RADIUS_METRES],
),
)
return fetch
def write_route_files(
data_dir: Path,
journeys: list[CarJourney],
fetch_route: RouteFetcher,
overwrite_routes: bool = False,
) -> int:
"""Write missing route GeoJSON files and return count written."""
route_dir = data_dir / "car_routes"
route_dir.mkdir(parents=True, exist_ok=True)
written = 0
for journey in journeys:
path = car_route_path(data_dir, journey.route)
if path.exists() and not overwrite_routes:
continue
route_geojson = fetch_route(journey.start, journey.end)
path.write_text(json.dumps(route_geojson, separators=(",", ":")) + "\n")
written += 1
return written
def journey_key(item: StrDict) -> tuple[date, datetime, datetime, str]:
"""Return a stable key for duplicate detection and sorting."""
route = item.get("route")
if not isinstance(route, str):
raise ValueError(f"car journey route must be a string: {item!r}")
return (
utils.as_date(item["trip"]),
utils.as_datetime(item["depart"]),
utils.as_datetime(item["arrive"]),
trip_module.route_filename_without_extension(route),
)
def import_car_journeys(data_dir: Path, journeys: list[CarJourney]) -> int:
"""Insert car journeys into car_journeys.yaml and return count added."""
path = data_dir / "car_journeys.yaml"
existing = load_yaml_list(path)
existing_keys = {journey_key(item) for item in existing}
new_items = [
journey.as_yaml_item()
for journey in journeys
if journey_key(journey.as_yaml_item()) not in existing_keys
]
if not new_items:
return 0
combined = existing + new_items
combined.sort(key=journey_key)
path.write_text(dump_yaml_list_with_blank_lines(combined))
return len(new_items)
def add_car_journeys_for_trip(
trip_date: date,
data_dir: Path = PERSONAL_DATA_DIR,
destination: str = "accommodation",
home_label: str = DEFAULT_HOME_LABEL,
destination_label: str | None = None,
fetch_route: RouteFetcher | None = None,
overwrite_routes: bool = False,
dry_run: bool = False,
) -> tuple[list[CarJourney], int, int]:
"""Add car journeys and routes for a trip."""
trip = trip_for_date(data_dir, trip_date)
journeys = car_journeys_for_trip(
trip,
data_dir,
destination=destination,
home_label=home_label,
destination_label=destination_label,
)
if dry_run:
return (journeys, 0, 0)
if fetch_route is None:
api_key = os.environ.get("ORS_API_KEY")
if not api_key:
raise ValueError("ORS_API_KEY must be set")
fetch_route = openrouteservice_fetcher(api_key)
routes_written = write_route_files(
data_dir, journeys, fetch_route, overwrite_routes=overwrite_routes
)
journeys_added = import_car_journeys(data_dir, journeys)
return (journeys, routes_written, journeys_added)
def parse_date(value: str) -> date:
"""Parse an ISO date."""
return date.fromisoformat(value)
def main(argv: list[str] | None = None) -> int:
"""CLI entrypoint."""
parser = argparse.ArgumentParser(
description=(
"Add car journeys from home to trip accommodation and back, "
"using openrouteservice for GeoJSON routes."
)
)
parser.add_argument("trip_date", type=parse_date)
parser.add_argument(
"--destination",
choices=("accommodation", "airport", "ferry", "fly-drive"),
default="accommodation",
help=(
"Route to accommodation check-in/out, the trip airport, "
"around an outbound/return car ferry, or a fly-drive chain."
),
)
parser.add_argument("--data-dir", default=str(PERSONAL_DATA_DIR))
parser.add_argument("--home-label", default=DEFAULT_HOME_LABEL)
parser.add_argument("--destination-label")
parser.add_argument("--overwrite-routes", action="store_true")
parser.add_argument("--dry-run", action="store_true")
args = parser.parse_args(argv)
journeys, routes_written, journeys_added = add_car_journeys_for_trip(
args.trip_date,
data_dir=Path(args.data_dir).expanduser(),
destination=args.destination,
home_label=args.home_label,
destination_label=args.destination_label,
overwrite_routes=args.overwrite_routes,
dry_run=args.dry_run,
)
for journey in journeys:
print(f"{journey.depart}: {journey.route}")
if args.dry_run:
return 0
print(f"Wrote {routes_written} route file(s)")
print(f"Added {journeys_added} car journey YAML item(s)")
return 0

33
agenda/carnival.py
View file

@ -1,33 +0,0 @@
"""Calculate the date for carnival."""
from datetime import date, timedelta
from dateutil.easter import easter
from .event import Event
def rio_carnival_events(start_date: date, end_date: date) -> list[Event]:
"""List of events for Rio Carnival for each year between start_date and end_date."""
events = []
for year in range(start_date.year, end_date.year + 1):
easter_date = easter(year)
carnival_start = easter_date - timedelta(days=51)
carnival_end = easter_date - timedelta(days=46)
# Only include the carnival if it falls within the specified date range
if (
start_date <= carnival_start <= end_date
or start_date <= carnival_end <= end_date
):
events.append(
Event(
name="carnival",
title="Rio Carnival",
date=carnival_start,
end_date=carnival_end,
url="https://en.wikipedia.org/wiki/Rio_Carnival",
)
)
return events

206
agenda/conference.py
View file

@ -2,31 +2,11 @@
import dataclasses
import decimal
import os
import typing
from datetime import date, datetime
import yaml
from . import utils
from .event import Event
from .types import DateOrDateTime, StrDict
MAX_CONF_DAYS = 20
DATE_STATUSES = {"exact", "approximate", "tentative"}
DATED_STATUSES = {"exact", "tentative"}
class ConferenceSeries(typing.TypedDict, total=False):
"""Conference series metadata."""
name: str
topic: str
url: str
notes: str
cadence: str
usual_location: str
country: str
from .types import Event
@dataclasses.dataclass
@ -36,11 +16,8 @@ class Conference:
name: str
topic: str
location: str
series: str | None = None
start: date | datetime | None = None
end: date | datetime | None = None
trip: date | None = None
country: str | None = None
start: date | datetime
end: date | datetime
venue: str | None = None
address: str | None = None
url: str | None = None
@ -52,19 +29,6 @@ class Conference:
online: bool = False
price: decimal.Decimal | None = None
currency: str | None = None
latitude: float | None = None
longitude: float | None = None
attend_start: date | datetime | None = None
attend_end: date | datetime | None = None
cfp_end: date | None = None
cfp_url: str | None = None
free: bool | None = None
hackathon: bool | None = None
ticket_type: str | None = None
attendees: int | None = None
hashtag: str | None = None
description: str | None = None
dates: StrDict | None = None
@property
def display_name(self) -> str:
@ -76,157 +40,19 @@ class Conference:
)
def _date_range_label(start: date, end: date) -> str:
"""Format conference date range for display."""
if start == end:
return start.strftime("%a %-d %b %Y")
if start.year == end.year and start.month == end.month:
return f"{start.strftime('%a %-d')}-{end.strftime('%-d %b %Y')}"
return f"{start.strftime('%a %-d %b')}-{end.strftime('%-d %b %Y')}"
def _require_date_value(value: typing.Any, field_name: str) -> DateOrDateTime:
"""Return date-like field value or raise ValueError."""
if isinstance(value, (date, datetime)):
return value
raise ValueError(f"conference dates field {field_name!r} must be a date/datetime")
def _require_date_only(value: typing.Any, field_name: str) -> date:
"""Return field value as a date or raise ValueError."""
return utils.as_date(_require_date_value(value, field_name))
def conference_date_fields(item: StrDict) -> StrDict:
"""Return derived date fields for a conference YAML item."""
raw_dates = item.get("dates")
if raw_dates is None:
status = typing.cast(str, item.get("date_status", "exact"))
if status not in DATE_STATUSES:
raise ValueError(f"unknown conference date status {status!r}")
start = _require_date_value(item.get("start"), "start")
end = _require_date_value(item.get("end", start), "end")
start_date = utils.as_date(start)
end_date = utils.as_date(end)
return {
"date_status": status,
"start": start,
"end": end,
"start_date": start_date,
"end_date": end_date,
"sort_date": start_date,
"latest_date": end_date,
"display_date": _date_range_label(start_date, end_date),
"has_exact_dates": status == "exact",
}
if not isinstance(raw_dates, dict):
raise ValueError("conference dates must be a mapping")
status_value = raw_dates.get("status", "exact")
if not isinstance(status_value, str) or status_value not in DATE_STATUSES:
raise ValueError(f"unknown conference date status {status_value!r}")
if status_value in DATED_STATUSES:
start = _require_date_value(raw_dates.get("start", item.get("start")), "start")
end = _require_date_value(raw_dates.get("end", item.get("end", start)), "end")
start_date = utils.as_date(start)
end_date = utils.as_date(end)
label = raw_dates.get("label")
display_date = (
label if isinstance(label, str) else _date_range_label(start_date, end_date)
)
return {
"date_status": status_value,
"start": start,
"end": end,
"start_date": start_date,
"end_date": end_date,
"sort_date": start_date,
"latest_date": end_date,
"display_date": display_date,
"has_exact_dates": status_value == "exact",
}
earliest = _require_date_only(raw_dates.get("earliest"), "earliest")
latest = _require_date_only(raw_dates.get("latest"), "latest")
label = raw_dates.get("label")
display_date = (
label if isinstance(label, str) else _date_range_label(earliest, latest)
)
return {
"date_status": status_value,
"start_date": earliest,
"end_date": latest,
"sort_date": earliest,
"latest_date": latest,
"display_date": display_date,
"has_exact_dates": False,
}
def validate_conference_date_fields(item: StrDict) -> StrDict:
"""Validate conference date fields and return derived values."""
fields = conference_date_fields(item)
if fields["start_date"] > fields["end_date"]:
raise ValueError("conference ends before it starts")
if fields["date_status"] in DATED_STATUSES:
duration = (fields["end_date"] - fields["start_date"]).days
if duration >= MAX_CONF_DAYS:
raise ValueError(
f"conference is {duration} days; maximum is {MAX_CONF_DAYS - 1}"
)
return fields
def load_series(data_dir: str) -> dict[str, ConferenceSeries]:
"""Load conference series metadata."""
filepath = os.path.join(data_dir, "conference_series.yaml")
if not os.path.exists(filepath):
return {}
loaded = yaml.safe_load(open(filepath, "r"))
if loaded is None:
return {}
if not isinstance(loaded, dict):
raise ValueError("conference_series.yaml must be a mapping")
return typing.cast(dict[str, ConferenceSeries], loaded)
def get_list(filepath: str) -> list[Event]:
"""Read conferences from a YAML file and return a list of Event objects."""
events: list[Event] = []
for item in yaml.safe_load(open(filepath, "r")):
try:
fields = validate_conference_date_fields(item)
except ValueError as exc:
raise AssertionError(str(exc)) from exc
normalized_item = dict(item)
if "start" in fields:
normalized_item["start"] = fields["start"]
normalized_item["end"] = fields["end"]
conf = Conference(**normalized_item)
if fields["has_exact_dates"]:
assert conf.start is not None and conf.end is not None
event = Event(
name="conference",
date=conf.start,
end_date=conf.end,
title=conf.display_name,
url=conf.url,
going=conf.going,
)
events.append(event)
if not conf.cfp_end:
continue
cfp_end_event = Event(
name="cfp_end",
date=conf.cfp_end,
title="CFP end: " + conf.display_name,
url=conf.cfp_url or conf.url,
return [
Event(
name="conference",
date=conf.start,
end_date=conf.end,
title=f"🎤 {conf.display_name}",
url=conf.url,
going=conf.going,
)
events.append(cfp_end_event)
return events
for conf in (
Conference(**conf)
for conf in yaml.safe_load(open(filepath, "r"))["conferences"]
)
]

46
agenda/conference_airports.json
View file

@ -1,46 +0,0 @@
{
"be:brussels": "BRU",
"de:berlin": "BER",
"de:hamburg": "HAM",
"de:heidelberg": "FRA",
"de:karlsruhe": "STR",
"de:chemnitz": "DRS",
"de:mildenberg": "BER",
"es:barcelona": "BCN",
"it:bologna": [
"BLQ",
"VRN",
"VCE"
],
"cy:limassol": "LCA",
"al:albania": "TIA",
"dk:funen": [
"BLL",
"CPH"
],
"ca:vancouver": "YVR",
"co:bogotá": "BOG",
"cl:santiago": "SCL",
"us:pasadena, california": "LAX",
"us:petaluma, california": "SFO",
"us:st. louis, mo": "STL",
"us:seattle": "SEA",
"us:doe bay resort & spa, orcas island, wa": "SEA",
"jp:asahikawa, hokkaido": "AKJ",
"se:malmö": [
"MMX",
"CPH"
],
"ch:bern": [
"BRN",
"BSL"
],
"gb:belfast": [
"BFS",
"BHD"
],
"de:bonn": [
"CGN",
"DUS"
]
}

110
agenda/conference_detail.py
View file

@ -1,110 +0,0 @@
"""Conference identities and airport suggestions, without network lookups."""
import json
import math
import re
import unicodedata
from pathlib import Path
import yaml
from agenda import airport_lookup
from agenda.types import StrDict
EUROPE = frozenset(
"al ad at be ba bg hr cy cz dk ee fi fr de gr hu is ie it lv li lt lu mt "
"md mc me nl mk no pl pt ro sm rs sk si es se ch tr ua gb va".split()
)
def conference_slug(conf: StrDict) -> str:
"""Use a readable title or series and location; the URL also includes a date."""
title = str(conf.get("series") or conf["name"])
ascii_title = (
unicodedata.normalize("NFKD", title).encode("ascii", "ignore").decode()
)
slug = re.sub(r"[^a-z0-9]+", "-", ascii_title.lower()).strip("-") or "conference"
location = (
unicodedata.normalize("NFKD", str(conf.get("location", "")))
.encode("ascii", "ignore")
.decode()
)
location_slug = re.sub(r"[^a-z0-9]+", "-", location.lower()).strip("-")
return f"{slug}-{location_slug}" if location_slug else slug
def distance_km(lat: float, lon: float, other_lat: float, other_lon: float) -> float:
"""Great-circle distance."""
a, b = math.radians(lat), math.radians(other_lat)
dlat, dlon = b - a, math.radians(other_lon - lon)
hav = math.sin(dlat / 2) ** 2 + math.cos(a) * math.cos(b) * math.sin(dlon / 2) ** 2
return 6371 * 2 * math.asin(math.sqrt(min(1, hav)))
def suggest_airport(conf: StrDict, personal_data: str) -> tuple[str | None, str]:
"""Prefer explicit mappings, then city matches, then nearby known airports."""
mapping_path = Path(__file__).with_name("conference_airports.json")
mapping: dict[str, str | list[str]] = json.loads(mapping_path.read_text())
overrides = Path(personal_data) / "conference_airports.yaml"
if overrides.exists():
mapping.update(yaml.safe_load(overrides.read_text()) or {})
location = airport_lookup.normalized(str(conf.get("location", "")))
country = str(conf.get("country", "")).lower()
mapped = {
airport_lookup.normalized(key): value for key, value in mapping.items()
}.get(f"{country}:{location}")
if mapped:
return (
",".join(mapped) if isinstance(mapped, list) else mapped
), "location mapping"
airports_path = Path(personal_data) / "airports.yaml"
airports: dict[str, StrDict] = (
yaml.safe_load(airports_path.read_text()) or {}
if airports_path.exists()
else {}
)
candidates = {
code: airport
for code, airport in airports.items()
if not country or str(airport.get("country", "")).lower() == country
}
for code, airport in candidates.items():
if location and location == airport_lookup.normalized(
str(airport.get("city", ""))
):
return code, "airport city matches conference location"
if conf.get("latitude") is not None and conf.get("longitude") is not None:
distances = [
(
distance_km(
float(conf["latitude"]),
float(conf["longitude"]),
float(airport["latitude"]),
float(airport["longitude"]),
),
code,
)
for code, airport in candidates.items()
if airport.get("latitude") is not None
and airport.get("longitude") is not None
]
if distances:
distance, code = min(distances)
if distance <= 150:
return (
code,
f"nearest airport in your data ({distance:.0f} km from venue)",
)
return None, "No airport mapping yet; enter a destination airport code."
def is_short_haul(conf: StrDict) -> bool:
"""Treat known European destinations within 3,500 km as short haul."""
if str(conf.get("country", "")).lower() not in EUROPE:
return False
if conf.get("latitude") is not None and conf.get("longitude") is not None:
return (
distance_km(51.47, -0.45, float(conf["latitude"]), float(conf["longitude"]))
<= 3500
)
return True

343
agenda/conference_flights.py
View file

@ -1,343 +0,0 @@
"""On-demand flight searches with shared disk caching and refresh throttling."""
import fcntl
import hashlib
import json
import logging
import os
import re
import tempfile
import typing
from datetime import date, datetime, timedelta, timezone
from pathlib import Path
import flask
from agenda import airport_lookup, flight_search_cache, google_flights
from agenda.types import StrDict
LONDON_AIRPORTS = ("LHR", "LGW", "STN", "LTN", "LCY", "SEN")
CACHE_VERSION = 4
logger = logging.getLogger(__name__)
REFRESH_INTERVAL = timedelta(hours=6)
ERROR_INTERVAL = timedelta(minutes=15)
def cache_path(
data_dir: str, start: date, end: date, destination: str, short_haul: bool
) -> Path:
"""Cache by all search inputs, independently of conference names."""
key = json.dumps(
[
CACHE_VERSION,
start.isoformat(),
end.isoformat(),
destination,
short_haul,
"GBP",
"en-GB",
"GB",
]
)
return (
Path(data_dir)
/ "conference-flights"
/ (hashlib.sha256(key.encode()).hexdigest() + ".json")
)
def read_cache(path: Path) -> StrDict | None:
"""Read cached results; a missing or damaged cache is a cache miss."""
try:
value = json.loads(path.read_text())
if not isinstance(value, dict) or not isinstance(value.get("updated_at"), str):
return None
for field in ("updated_at", "attempted_at"):
if field in value and datetime.fromisoformat(value[field]).tzinfo is None:
return None
if not isinstance(value.get("searches"), list):
return None
return typing.cast(StrDict, value)
except (OSError, ValueError, TypeError):
return None
def write_cache(path: Path, value: StrDict) -> None:
"""Replace JSON atomically so concurrent readers never see partial results."""
with tempfile.NamedTemporaryFile(mode="w", dir=path.parent, delete=False) as output:
temporary = output.name
json.dump(value, output)
try:
os.replace(temporary, path)
finally:
if os.path.exists(temporary):
os.unlink(temporary)
def airport_matches(query: str) -> list[StrDict]:
"""Look up locally, preferring airport names in the app's personal data."""
personal_data = (
str(flask.current_app.config["PERSONAL_DATA"])
if flask.has_app_context()
else None
)
return airport_lookup.matches(query, personal_data)
def resolve_airport(query: str) -> tuple[str, str]:
"""Resolve a code or the best name match, and expose its name for confirmation."""
codes = airport_lookup.code_list(query)
if len(codes) > 1:
for code in codes:
if not valid_airport(code):
raise ValueError(f"Unknown IATA airport code {code!r}.")
matches = airport_matches(",".join(codes))
if not matches:
raise ValueError("Unknown airport in destination group")
return str(matches[0]["code"]), str(matches[0]["name"])
code = query.strip().upper()
if valid_airport(code):
matches = airport_matches(code)
return code, str(matches[0]["name"]) if matches else code
if len(code) == 3 and code.isascii() and code.isalpha():
raise ValueError(
f"Unknown IATA airport code {code!r}. Enter a valid code or a longer city/airport name."
)
matches = airport_matches(query)
if not matches:
raise ValueError(
f"No airport found for {query!r}. Try a city, airport name or IATA code."
)
return str(matches[0]["code"]), str(matches[0]["name"])
def error_detail(exc: Exception) -> str:
"""Describe retry wrappers and underlying causes without losing the useful error."""
messages: list[str] = []
current: BaseException | None = exc
seen: set[int] = set()
while current is not None and id(current) not in seen and len(messages) < 3:
seen.add(id(current))
last_attempt = getattr(current, "last_attempt", None)
if last_attempt is not None:
underlying = last_attempt.exception()
if underlying is not None and id(underlying) not in seen:
current = underlying
continue
raw_message = str(current)
if "Browser logs:" in raw_message:
# Playwright repeats its long command before the useful stderr.
diagnostic = list(dict.fromkeys(re.findall(r"\[err\] (.+)", raw_message)))
if diagnostic:
raw_message = (
raw_message.split("Browser logs:", 1)[0]
+ " "
+ " ".join(diagnostic)
)
message = " ".join(raw_message.split())[:1000]
messages.append(
f"{type(current).__name__}: {message or 'No error message provided'}"
)
current = current.__cause__ or current.__context__
return " — ".join(messages)
def date_unavailable_message(exc: BaseException) -> str | None:
"""Use Google's date-range rejection as the banner through route wrappers."""
current: BaseException | None = exc
seen: set[int] = set()
while current is not None and id(current) not in seen:
seen.add(id(current))
if isinstance(current, google_flights.FlightDateUnavailableError):
return str(current)
current = current.__cause__ or current.__context__
return None
def flight_rank(result: StrDict) -> tuple[int, int]:
"""Prefer fewer stops, then BA-operated legs, preserving Google's ranking."""
ba_legs = [
leg.get("operating_airline_code", leg.get("airline_code")) == "BA"
for leg in result["legs"]
]
ba_preference = 0 if ba_legs and all(ba_legs) else 1 if any(ba_legs) else 2
return int(result["stops"]), ba_preference
def search_day(origin: str, destination: str, day: date, direct: bool) -> list[StrDict]:
"""Reuse a route/date cache before making any paced provider requests."""
directory = flight_search_cache.search_directory.get()
if directory is None:
raise RuntimeError(
"Flight searches must run inside a cached conference lookup."
)
return flight_search_cache.cached_day(
directory,
origin,
destination,
day.isoformat(),
direct,
lambda: fetch_day(origin, destination, day, direct, directory),
)
def fetch_day(
origin: str, destination: str, day: date, direct: bool, directory: Path
) -> list[StrDict]:
"""Read the browser search page, preferring direct and one-stop flights."""
max_stops = 0 if direct else 1
rows = google_flights.search(origin, destination, day, max_stops)
rows = [row for row in rows if row["stops"] <= max_stops]
if not rows and not direct:
rows = google_flights.search(origin, destination, day, 2)
rows = [row for row in rows if row["stops"] <= 2]
rows.sort(key=flight_rank)
return rows[:5]
def valid_airport(code: str) -> bool:
"""Accept known IATA codes and London's metropolitan code."""
return code == "LON" or code in airport_lookup.airports()
def search_origin(
origin: str, destination: str, start: date, end: date, flexible: bool
) -> StrDict:
"""Expand each direction from the closest date, stopping when flights are found."""
days = range(1, 5) if flexible else range(1, 3)
outbound: list[StrDict] = []
inbound: list[StrDict] = []
outbound_dates: list[str] = []
inbound_dates: list[str] = []
for offset in days:
departure = start - timedelta(days=offset if flexible else offset + 1)
if not outbound and departure >= date.today():
outbound_dates.append(departure.isoformat())
# Only stop expanding when flights arrive before the opening date.
outbound = [
row
for row in search_day(origin, destination, departure, origin == "BRS")
if date.fromisoformat(row["arrival"][:10]) < start
]
if not inbound and (flexible or offset == 1):
return_day = end + timedelta(days=offset)
inbound_dates.append(return_day.isoformat())
inbound = search_day(destination, origin, return_day, origin == "BRS")
if outbound and inbound:
break
return {
"origin": origin,
"outbound": outbound,
"inbound": inbound,
"outbound_dates": outbound_dates,
"inbound_dates": inbound_dates,
}
def lookup(
path: Path, start: date, end: date, destination: str, short_haul: bool
) -> StrDict:
"""Serialize searches across workers, reuse recent results, and cache errors."""
path.parent.mkdir(parents=True, exist_ok=True)
# A shared lock limits all conference requests, not just identical searches.
with (path.parent / "search.lock").open(
"a+"
) as lock, flight_search_cache.use_directory(
path.parent
), google_flights.use_browser(
path.parent
):
try:
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
raise ValueError(
"Another flight lookup is running. Please try again shortly."
) from None
cached = read_cache(path)
now = datetime.now(timezone.utc)
legacy_empty_or_failed = (
cached
and cached.get("transport") != flight_search_cache.TRANSPORT
and (
cached.get("error")
or not any(
search.get("outbound") or search.get("inbound")
for search in cached.get("searches", [])
)
)
)
retry_old_parse_failure = (
cached
and cached.get("error")
and "Unrecognized Google Flights result format"
in cached.get("error_details", "")
and cached.get("parser_version") != google_flights.PARSER_VERSION
)
needs_earlier_departure = (
cached
and not cached.get("error")
and any(
search.get("origin") == "LON"
and not search.get("outbound")
and "outbound_dates" not in search
for search in cached.get("searches", [])
)
)
if (
cached
and not needs_earlier_departure
and not retry_old_parse_failure
and not legacy_empty_or_failed
and (not cached.get("error") or cached.get("error_details"))
):
timestamp = str(cached.get("attempted_at", cached["updated_at"]))
interval = ERROR_INTERVAL if cached.get("error") else REFRESH_INTERVAL
if now - datetime.fromisoformat(timestamp) < interval:
return cached
try:
searches = []
if short_haul:
bristol = search_origin("BRS", destination, start, end, flexible=True)
searches.append(bristol)
if not bristol["outbound"] or not bristol["inbound"]:
searches.append(
search_origin("LON", destination, start, end, flexible=False)
)
else:
searches.append(
search_origin("LON", destination, start, end, flexible=False)
)
value: StrDict = {
"updated_at": now.isoformat(),
"destination": destination,
"searches": searches,
"transport": flight_search_cache.TRANSPORT,
"parser_version": google_flights.PARSER_VERSION,
}
except Exception as exc:
logger.exception(
"Conference flight lookup failed for %s (%s to %s)",
destination,
start,
end,
)
# Preserve previously successful data and its original timestamp.
value = dict(
cached
or {
"updated_at": now.isoformat(),
"destination": destination,
"searches": [],
}
)
value.update(
error=flight_search_cache.cooldown_message(path.parent)
or date_unavailable_message(exc)
or "Flight lookup failed. Please try again in 15 minutes.",
error_details=error_detail(exc),
attempted_at=now.isoformat(),
transport=flight_search_cache.TRANSPORT,
parser_version=google_flights.PARSER_VERSION,
)
write_cache(path, value)
return value

90
agenda/conference_ical.py
View file

@ -1,90 +0,0 @@
"""Generate iCalendar feeds for conferences."""
import hashlib
from datetime import datetime, timedelta, timezone
import agenda
import agenda.utils
from agenda import ical
from agenda.types import StrDict
def _conference_uid(conf: StrDict) -> str:
"""Generate deterministic UID for conference events."""
start = agenda.utils.as_date(conf["start"])
raw = f"conference|{start.isoformat()}|{conf.get('name','unknown')}"
digest = hashlib.sha1(raw.encode("utf-8")).hexdigest()
return f"conference-{digest}@agenda-codex"
def _conference_location(conf: StrDict) -> str | None:
"""Build conference location string."""
parts: list[str] = []
venue = conf.get("venue")
location = conf.get("location")
if isinstance(venue, str) and venue.strip():
parts.append(venue.strip())
if isinstance(location, str) and location.strip():
parts.append(location.strip())
if country_code := conf.get("country"):
country = agenda.get_country(country_code)
if country:
parts.append(country.name)
return ", ".join(parts) if parts else None
def _conference_description(conf: StrDict) -> str:
"""Build textual description for conferences."""
lines: list[str] = []
if topic := conf.get("topic"):
lines.append(f"Topic: {topic}")
if venue := conf.get("venue"):
lines.append(f"Venue: {venue}")
if address := conf.get("address"):
lines.append(f"Address: {address}")
if url := conf.get("url"):
lines.append(f"URL: {url}")
status_bits: list[str] = []
if conf.get("going"):
status_bits.append("attending")
if conf.get("speaking"):
status_bits.append("speaking")
if status_bits:
lines.append(f"Status: {', '.join(status_bits)}")
return "\n".join(lines) if lines else "Conference"
def build_conference_ical(items: list[StrDict]) -> bytes:
"""Build iCalendar feed for all conferences."""
lines = [
"BEGIN:VCALENDAR",
"VERSION:2.0",
"PRODID:-//Agenda Codex//Conferences//EN",
"CALSCALE:GREGORIAN",
"METHOD:PUBLISH",
"X-WR-CALNAME:Conferences",
]
generated = datetime.now(tz=timezone.utc)
for conf in items:
if not conf["has_exact_dates"]:
continue
start_date = agenda.utils.as_date(conf["start"])
end_date = agenda.utils.as_date(conf["end"])
end_exclusive = end_date + timedelta(days=1)
lines.append("BEGIN:VEVENT")
ical.append_property(lines, "UID", _conference_uid(conf))
ical.append_property(lines, "DTSTAMP", ical.format_datetime_utc(generated))
ical.append_property(lines, "DTSTART;VALUE=DATE", ical.format_date(start_date))
ical.append_property(lines, "DTEND;VALUE=DATE", ical.format_date(end_exclusive))
ical.append_property(lines, "SUMMARY", ical.escape_text(conf["name"]))
description = ical.escape_text(_conference_description(conf))
ical.append_property(lines, "DESCRIPTION", description)
if location := _conference_location(conf):
ical.append_property(lines, "LOCATION", ical.escape_text(location))
lines.append("END:VEVENT")
lines.append("END:VCALENDAR")
ical_text = "\r\n".join(lines) + "\r\n"
return ical_text.encode("utf-8")

221
agenda/conference_list.py
View file

@ -1,221 +0,0 @@
"""Prepare conference lists, country filters, series summaries, and timelines."""
import decimal
import hashlib
import os.path
import typing
from collections import defaultdict
from datetime import date, timedelta
import yaml
import agenda.conference
import agenda.conference_detail
from agenda.types import StrDict, Trip
def build_conference_list(data_dir: str, trips: list[Trip]) -> list[StrDict]:
"""Build conference list."""
filepath = os.path.join(data_dir, "conferences.yaml")
items: list[StrDict] = yaml.safe_load(open(filepath))
series_lookup = agenda.conference.load_series(data_dir)
conference_trip_lookup = {}
for trip in trips:
for trip_conf in trip.conferences:
key = (trip_conf["start"], trip_conf["name"])
conference_trip_lookup[key] = trip
for conf in items:
conf.update(agenda.conference.validate_conference_date_fields(conf))
conf["slug"] = agenda.conference_detail.conference_slug(conf)
price = conf.get("price")
if price:
conf["price"] = decimal.Decimal(price)
series_id = conf.get("series")
if isinstance(series_id, str):
conf["series_detail"] = series_lookup.get(series_id)
if "start" in conf:
key = (conf["start"], conf["name"])
if this_trip := conference_trip_lookup.get(key):
conf["linked_trip"] = this_trip
identities: defaultdict[tuple[date, str], list[StrDict]] = defaultdict(list)
for conf in items:
identities[(conf["sort_date"], conf["slug"])].append(conf)
for group in identities.values():
if len(group) > 1:
for conf in group:
identity = str((conf["name"], conf.get("url")))
conf["slug"] += "-" + hashlib.sha256(identity.encode()).hexdigest()[:8]
items.sort(key=lambda item: item["sort_date"])
return items
def conference_country_code(conf: StrDict) -> str | None:
"""Return normalized alpha-2 country code for a conference."""
country = conf.get("country")
if not isinstance(country, str):
return None
code = country.strip().lower()
return code if len(code) == 2 else None
def normalize_country_filter(value: str | None) -> str | None:
"""Normalize and validate a country query parameter."""
if not value:
return None
code = value.strip().lower()
if len(code) != 2:
return None
return code if agenda.get_country(code) else None
def filter_conferences_by_country(
items: list[StrDict], country_code: str | None
) -> list[StrDict]:
"""Filter conferences by country when a country code is provided."""
if not country_code:
return items
return [item for item in items if conference_country_code(item) == country_code]
def conference_country_options(items: list[StrDict]) -> list[StrDict]:
"""Return country options for the conference country filter."""
counts: defaultdict[str, int] = defaultdict(int)
for item in items:
code = conference_country_code(item)
if code and agenda.get_country(code):
counts[code] += 1
options: list[StrDict] = []
for code, count in counts.items():
country = agenda.get_country(code)
if not country:
continue
options.append(
{
"code": code,
"name": country.name,
"flag": country.flag,
"count": count,
}
)
options.sort(key=lambda item: str(item["name"]))
return options
def build_conference_series_list(
series_lookup: dict[str, agenda.conference.ConferenceSeries],
conferences: list[StrDict],
today: date,
) -> list[StrDict]:
"""Build conference series list with conference counts."""
series_items: list[StrDict] = []
for series_id, series in series_lookup.items():
linked = [conf for conf in conferences if conf.get("series") == series_id]
latest = max((conf["sort_date"] for conf in linked), default=None)
next_conf = next(
(conf for conf in linked if conf["latest_date"] >= today), None
)
attended = any(conf.get("going") or conf.get("linked_trip") for conf in linked)
item: StrDict = {
"id": series_id,
**series,
"count": len(linked),
"latest": latest,
"next_conf": next_conf,
"attended": attended,
}
series_items.append(item)
series_items.sort(key=lambda item: str(item["name"]).lower())
return series_items
def build_conference_timeline(
current: list[StrDict], future: list[StrDict], today: date, days: int = 90
) -> dict[str, typing.Any] | None:
"""Build data for a Gantt-style timeline of upcoming conferences."""
timeline_start = today
timeline_end = today + timedelta(days=days)
visible = [
c
for c in (current + future)
if c["has_exact_dates"]
and c["start_date"] <= timeline_end
and c["end_date"] >= today
]
if not visible:
return None
visible.sort(key=lambda c: c["start_date"])
# Greedy interval-coloring: assign each conference a lane (row)
lane_ends: list[date] = []
conf_data = []
for conf in visible:
lane = next(
(i for i, end in enumerate(lane_ends) if end < conf["start_date"]),
len(lane_ends),
)
if lane == len(lane_ends):
lane_ends.append(conf["end_date"])
else:
lane_ends[lane] = conf["end_date"]
start_off = max((conf["start_date"] - timeline_start).days, 0)
end_off = min((conf["end_date"] - timeline_start).days + 1, days)
left_pct = round(start_off / days * 100, 2)
width_pct = max(round((end_off - start_off) / days * 100, 2), 0.5)
conf_data.append(
{
"name": conf["name"],
"url": conf.get("url"),
"slug": conf.get("slug"),
"event_date": conf["sort_date"].isoformat(),
"lane": lane,
"left_pct": left_pct,
"width_pct": width_pct,
"key": f"{conf['start_date'].isoformat()}|{conf['name']}",
"label": (
f"{conf['name']} ({conf['start_date'].strftime('%-d %b')})"
if conf["start_date"] == conf["end_date"]
else (
f"{conf['name']}"
f" ({conf['start_date'].strftime('%-d %b')}–"
f"{conf['end_date'].strftime('%-d %b')})"
)
),
}
)
# Month markers for x-axis labels
months = []
d = today.replace(day=1)
while d <= timeline_end:
off = max((d - timeline_start).days, 0)
months.append(
{"label": d.strftime("%b %Y"), "left_pct": round(off / days * 100, 2)}
)
# advance to next month
d = (d.replace(day=28) + timedelta(days=4)).replace(day=1)
return {
"confs": conf_data,
"lane_count": len(lane_ends),
"months": months,
"days": days,
}

281
agenda/conference_page.py
View file

@ -1,281 +0,0 @@
"""Conference detail page and explicit flight lookup action."""
import hmac
import secrets
import typing
from datetime import date, datetime, timedelta
import flask
import werkzeug
import agenda
import agenda.conference
import agenda.conference_detail as detail
import agenda.conference_flights as flights
import agenda.conference_list
import agenda.flight_search_cache
import agenda.google_flights
import agenda.trip
from agenda.types import StrDict
blueprint = flask.Blueprint("conference", __name__)
@blueprint.app_template_filter("conference_date")
def format_date(value: typing.Any, part: str = "full") -> str:
"""Render dates without converting flight times out of the airport's local time."""
parsed = value
if isinstance(value, str):
try:
parsed = (
datetime.fromisoformat(value)
if "T" in value or " " in value
else date.fromisoformat(value)
)
except ValueError:
return value
if isinstance(parsed, datetime):
if part == "time":
return parsed.strftime("%H:%M")
if part == "date":
return parsed.strftime("%a %-d %b %Y")
return parsed.strftime("%a %-d %b %Y at %H:%M")
if isinstance(parsed, date):
return parsed.strftime("%a %-d %b %Y")
return str(value)
def searched_dates(
search: StrDict, direction: str, start: date, end: date
) -> list[str]:
"""Read checked dates; reconstruct the old policy for legacy cached results."""
field = direction + "_dates"
if field in search:
return typing.cast(list[str], search[field])
rows = search.get(direction, [])
if rows:
return sorted({str(row["departure"])[:10] for row in rows})
offsets = (
range(1, 5)
if search["origin"] == "BRS"
else (2 if direction == "outbound" else 1,)
)
return [
(
start - timedelta(days=offset)
if direction == "outbound"
else end + timedelta(days=offset)
).isoformat()
for offset in offsets
if direction != "outbound" or start - timedelta(days=offset) >= date.today()
]
def google_flights_link(
origin: str, destination: str, start: date, end: date, search: StrDict | None = None
) -> StrDict:
"""Open a round trip with the displayed route, dates and stop preference."""
departure = start - timedelta(days=1 if origin == "BRS" else 2)
returning = end + timedelta(days=1)
if search:
for direction in ("outbound", "inbound"):
rows = search.get(direction, [])
checked = search.get(direction + "_dates", [])
selected = (
str(rows[0]["departure"])[:10]
if rows
else checked[-1] if checked else None
)
if selected:
if direction == "outbound":
departure = date.fromisoformat(selected)
else:
returning = date.fromisoformat(selected)
max_stops = 0 if origin == "BRS" else 1
if (
search
and origin != "BRS"
and any(
row.get("stops", 0) > 1
for direction in ("outbound", "inbound")
for row in search.get(direction, [])
)
):
max_stops = 2
return {
"url": agenda.google_flights.search_url(
origin, destination, departure, max_stops, returning
),
"departure": departure,
"return": returning,
}
def find_conference(event_date: str, slug: str) -> StrDict:
"""Resolve a dated conference identity, including approximate dates."""
for conf in agenda.conference_list.build_conference_list(
flask.current_app.config["PERSONAL_DATA"], agenda.trip.build_trip_list()
):
if conf["sort_date"].isoformat() == event_date and conf["slug"] == slug:
return conf
flask.abort(404)
@blueprint.route("/conference/<event_date>/<slug>", methods=["GET", "POST"])
def page(event_date: str, slug: str) -> str | werkzeug.Response:
"""GET reads only cached data; POST explicitly requests a flight search."""
conf = find_conference(event_date, slug)
config = flask.current_app.config
suggested, airport_reason = detail.suggest_airport(conf, config["PERSONAL_DATA"])
parameters = (
flask.request.form if flask.request.method == "POST" else flask.request.args
)
default_location = str(conf.get("location") or "")
airport_input = parameters.get("airport", default_location).strip()[:200]
destination = airport_input.upper()
resolved_name = ""
resolution_error = ""
if airport_input:
try:
lookup_query = (
suggested
if suggested and airport_input == default_location
else airport_input
)
destination, resolved_name = flights.resolve_airport(lookup_query)
except ValueError as exc:
resolution_error = str(exc)
eligible = (
conf["date_status"] in agenda.conference.DATED_STATUSES
and conf["start_date"] > date.today()
and not conf.get("online")
and (
str(conf.get("country", "")).lower() != "gb"
or str(conf.get("location", "")).partition(",")[0].strip().casefold()
== "belfast"
)
)
cache = flights.cache_path(
config["DATA_DIR"],
conf["start_date"],
conf["end_date"],
destination,
detail.is_short_haul(conf),
)
token = flask.session.setdefault(
"conference_flight_token", secrets.token_urlsafe(32)
)
if flask.request.method == "POST":
if not hmac.compare_digest(
str(token), flask.request.form.get("csrf_token", "")
):
flask.abort(400)
if not eligible:
flask.abort(400)
if resolution_error or destination in {"BRS", "LON"}:
flask.flash(
resolution_error or "Choose a destination outside Bristol or London.",
"warning",
)
else:
try:
flights.lookup(
cache,
conf["start_date"],
conf["end_date"],
destination,
detail.is_short_haul(conf),
)
except ValueError as exc:
flask.flash(str(exc), "warning")
return flask.redirect(
flask.url_for(
"conference.page",
event_date=event_date,
slug=slug,
airport=airport_input if resolution_error else destination,
)
)
excluded = {
"name",
"location",
"country",
"venue",
"address",
"latitude",
"longitude",
"url",
"dates",
"date_status",
"start",
"end",
"start_date",
"end_date",
"sort_date",
"latest_date",
"display_date",
"has_exact_dates",
"series",
"series_detail",
"linked_trip",
"slug",
}
fields = [
(key.replace("_", " ").capitalize(), value)
for key, value in conf.items()
if key not in excluded and value is not None
]
cached = flights.read_cache(cache) if destination and not resolution_error else None
if cached:
for search in cached.get("searches", []):
for direction in ("outbound", "inbound"):
search[direction + "_dates"] = searched_dates(
search, direction, conf["start_date"], conf["end_date"]
)
search["google_flights"] = google_flights_link(
search["origin"],
destination,
conf["start_date"],
conf["end_date"],
search,
)
default_link = (
google_flights_link(
"BRS" if detail.is_short_haul(conf) else "LON",
destination,
conf["start_date"],
conf["end_date"],
)
if eligible and not resolution_error and destination
else None
)
return flask.render_template(
"conference_detail.html",
conf=conf,
fields=fields,
get_country=agenda.get_country,
airport=airport_input,
resolved_airport=destination if not resolution_error else "",
resolved_name=resolved_name,
resolution_error=resolution_error,
airport_reason=airport_reason,
suggested=suggested,
short_haul=detail.is_short_haul(conf),
eligible=eligible,
cached=cached,
default_google_flights=default_link,
csrf_token=token,
cooldown=agenda.flight_search_cache.cooldown_message(cache.parent),
)
@blueprint.get("/conference/airports")
def airport_search() -> flask.Response:
"""Autocomplete airport names locally; this endpoint never searches Google."""
query = flask.request.args.get("q", "").strip()[:200]
if len(query) < 2:
return flask.jsonify([])
try:
return flask.jsonify(flights.airport_matches(query))
except ValueError as exc:
return flask.jsonify(error=str(exc))

526
agenda/data.py
View file

@ -1,50 +1,51 @@
"""Agenda data."""
import asyncio
import collections
import itertools
import os
import typing
from datetime import date, datetime, timedelta, timezone
from time import time
from datetime import date, datetime, timedelta
import dateutil.rrule
import dateutil.tz
import flask
import lxml # type: ignore[import-untyped]
import holidays
import isodate # type: ignore
import lxml
import pytz
import yaml
from . import (
accommodation,
birthday,
bristol_waste,
busy,
carnival,
calendar,
conference,
domains,
economist,
events_yaml,
gandi,
fx,
gwr,
hn,
holidays,
meetup,
n_somerset_waste,
stock_market,
subscription,
sun,
thespacedevs,
travel,
uk_holiday,
uk_tz,
waste_schedule,
)
from .event import Event
from .types import StrDict
from .utils import time_function
from .types import Event, Holiday
StrDict = dict[str, typing.Any]
here = dateutil.tz.tzlocal()
# deadline to file tax return
# credit card expiry dates
# morzine ski lifts
# chalet availability calendar
# chalet availablity calendar
# starlink visible
@ -61,124 +62,298 @@ def timezone_transition(
]
async def n_somerset_waste_collection_events(
data_dir: str, postcode: str, uprn: str, force_cache: bool = False
def us_holidays(start_date: date, end_date: date) -> list[Holiday]:
"""Get US holidays."""
found: list[Holiday] = []
for year in range(start_date.year, end_date.year + 1):
hols = holidays.country_holidays("US", years=year, language="en")
found += [
Holiday(date=hol_date, name=title, country="us")
for hol_date, title in hols.items()
if start_date < hol_date < end_date
]
extra = []
for h in found:
if h.name != "Thanksgiving":
continue
extra += [
Holiday(date=h.date + timedelta(days=1), name="Black Friday", country="us"),
Holiday(date=h.date + timedelta(days=4), name="Cyber Monday", country="us"),
]
return found + extra
def get_nyse_holidays(
start_date: date, end_date: date, us_hols: list[Holiday]
) -> list[Event]:
"""NYSE holidays."""
known_us_hols = {(h.date, h.name) for h in us_hols}
found: list[Event] = []
rename = {"Thanksgiving Day": "Thanksgiving"}
for year in range(start_date.year, end_date.year + 1):
hols = holidays.financial_holidays("NYSE", years=year)
found += [
Event(
name="holiday",
date=hol_date,
title=rename.get(title, title),
)
for hol_date, title in hols.items()
if start_date < hol_date < end_date
]
found = [hol for hol in found if (hol.date, hol.title) not in known_us_hols]
for hol in found:
assert hol.title
hol.title += " (NYSE)"
return found
def get_holidays(country: str, start_date: date, end_date: date) -> list[Holiday]:
"""Get holidays."""
found: list[Holiday] = []
for year in range(start_date.year, end_date.year + 1):
hols = holidays.country_holidays(country.upper(), years=year, language="en_US")
found += [
Holiday(
date=hol_date,
name=title,
country=country.lower(),
)
for hol_date, title in hols.items()
if start_date < hol_date < end_date
]
return found
def midnight(d: date) -> datetime:
"""Convert from date to midnight on that day."""
return datetime.combine(d, datetime.min.time())
def dates_from_rrule(
rrule: str, start: date, end: date
) -> typing.Sequence[datetime | date]:
"""Generate events from an RRULE between start_date and end_date."""
all_day = not any(param in rrule for param in ["BYHOUR", "BYMINUTE", "BYSECOND"])
return [
i.date() if all_day else uk_tz.localize(i)
for i in dateutil.rrule.rrulestr(rrule, dtstart=midnight(start)).between(
midnight(start), midnight(end)
)
]
async def waste_collection_events(data_dir: str) -> list[Event]:
"""Waste colllection events."""
html = await n_somerset_waste.get_html(data_dir, postcode, uprn, force_cache)
postcode = "BS48 3HG"
uprn = "24071046"
html = await waste_schedule.get_html(data_dir, postcode, uprn)
root = lxml.html.fromstring(html)
events = n_somerset_waste.parse(root)
return typing.cast(list[Event], events)
events = waste_schedule.parse(root)
return events
async def bristol_waste_collection_events(
data_dir: str, start_date: date, uprn: str, force_cache: bool = False
data_dir: str, start_date: date
) -> list[Event]:
"""Waste colllection events."""
cache = "force" if force_cache else "recent"
return typing.cast(
list[Event], await bristol_waste.get(start_date, data_dir, uprn, cache)
)
uprn = "358335"
return await waste_schedule.get_bristol_gov_uk(start_date, data_dir, uprn)
def find_events_during_stay(
accommodation_events: list[Event], markets: list[Event]
) -> list[Event]:
"""Market events that happen during accommodation stays."""
overlapping_markets = []
for market in markets:
market_date = market.as_date
assert isinstance(market_date, date)
for e in accommodation_events:
start, end = e.as_date, e.end_as_date
assert start and end and all(isinstance(i, date) for i in (start, end))
# Check if the market date is within the accommodation dates.
if start <= market_date <= end:
overlapping_markets.append(market)
break # Breaks the inner loop if overlap is found.
return overlapping_markets
def combine_holidays(holidays: list[Holiday]) -> list[Event]:
"""Combine UK and US holidays with the same date and title."""
all_countries = {h.country for h in holidays}
def hide_markets_while_away(
events: list[Event], accommodation_events: list[Event]
) -> None:
"""Hide markets that happen while away."""
optional = [
e
for e in events
if e.name == "market" or (e.title and "LHG Run Club" in e.title)
]
going = [e for e in events if e.going]
standard_name = {
(1, 1): "New Year's Day",
(1, 6): "Epiphany",
(5, 1): "Labour Day",
(8, 15): "Assumption Day",
(12, 8): "Immaculate conception",
(12, 25): "Christmas Day",
(12, 26): "Boxing Day",
}
overlapping_markets = find_events_during_stay(
accommodation_events + going, optional
)
for market in overlapping_markets:
events.remove(market)
combined: collections.defaultdict[
tuple[date, str], set[str]
] = collections.defaultdict(set)
for h in holidays:
assert isinstance(h.name, str) and isinstance(h.date, date)
class AgendaData(typing.TypedDict, total=False):
"""Agenda Data."""
event_key = (h.date, standard_name.get((h.date.month, h.date.day), h.name))
combined[event_key].add(h.country)
now: datetime
stock_markets: list[str]
rockets: list[thespacedevs.Summary]
gwr_advance_tickets: date | None
data_gather_seconds: float
stock_market_times_seconds: float
timings: list[tuple[str, float]]
events: list[Event]
accommodation_events: list[Event]
gaps: list[StrDict]
sunrise: datetime
sunset: datetime
last_week: date
two_weeks_ago: date
errors: list[tuple[str, Exception]]
def rocket_launch_events(rockets: list[thespacedevs.Summary]) -> list[Event]:
"""Rocket launch events."""
events: list[Event] = []
for launch in rockets:
dt = None
net_precision = launch["net_precision"]
skip = {"Year", "Month", "Quarter", "Fiscal Year"}
if net_precision == "Day":
dt = datetime.strptime(launch["net"], "%Y-%m-%dT%H:%M:%SZ").date()
elif (
net_precision
and net_precision not in skip
and "Year" not in net_precision
and launch["t0_time"]
):
dt = pytz.utc.localize(
datetime.strptime(launch["net"], "%Y-%m-%dT%H:%M:%SZ")
for (d, name), countries in combined.items():
if len(countries) == len(all_countries):
country_list = ""
elif len(countries) < len(all_countries) / 2:
country_list = ", ".join(sorted(country.upper() for country in countries))
else:
country_list = "not " + ", ".join(
sorted(country.upper() for country in all_countries - set(countries))
)
if not dt:
continue
rocket_name = (
f'{launch["rocket"]["full_name"]}: '
+ f'{launch["mission_name"] or "[no mission]"}'
e = Event(
name="holiday",
date=d,
title=f"{name} ({country_list})" if country_list else name,
)
e = Event(name="rocket", date=dt, title=rocket_name)
events.append(e)
return events
def event_sort_datetime(event: Event) -> datetime:
"""Return a timezone-normalized datetime suitable for sorting events."""
dt = typing.cast(datetime, event.as_datetime)
if dt.tzinfo is None or dt.utcoffset() is None:
return dt
return dt.astimezone(timezone.utc).replace(tzinfo=None)
def get_yaml_event_date_field(item: dict[str, str]) -> str:
"""Event date field name."""
return (
"end_date"
if item["name"] == "travel_insurance"
else ("start_date" if "start_date" in item else "date")
)
async def get_data(now: datetime, config: flask.config.Config) -> AgendaData:
def get_yaml_event_end_date_field(item: dict[str, str]) -> str:
"""Event date field name."""
return (
"end_date"
if item["name"] == "travel_insurance"
else ("start_date" if "start_date" in item else "date")
)
def read_events_yaml(data_dir: str, start: date, end: date) -> list[Event]:
"""Read eventes from YAML file."""
events: list[Event] = []
for item in yaml.safe_load(open(os.path.join(data_dir, "events.yaml"))):
duration = (
isodate.parse_duration(item["duration"]) if "duration" in item else None
)
dates = (
dates_from_rrule(item["rrule"], start, end)
if "rrule" in item
else [item[get_yaml_event_date_field(item)]]
)
for dt in dates:
e = Event(
name=item["name"],
date=dt,
end_date=(
dt + duration
if duration
else (
item.get("end_date")
if item["name"] != "travel_insurance"
else None
)
),
title=item.get("title"),
url=item.get("url"),
)
events.append(e)
return events
def find_markets_during_stay(
accommodation_events: list[Event], markets: list[Event]
) -> list[Event]:
"""Market events that happen during accommodation stays."""
overlapping_markets = []
for market in markets:
for e in accommodation_events:
# Check if the market date is within the accommodation dates.
if e.as_date <= market.as_date <= e.end_as_date:
overlapping_markets.append(market)
break # Breaks the inner loop if overlap is found.
return overlapping_markets
def find_gaps(events: list[Event], min_gap_days: int = 3) -> list[StrDict]:
"""Gaps of at least `min_gap_days` between events in a list of events."""
# Sort events by start date
gaps: list[tuple[date, date]] = []
previous_event_end = None
by_start_date = {
d: list(on_day)
for d, on_day in itertools.groupby(events, key=lambda e: e.as_date)
}
by_end_date = {
d: list(on_day)
for d, on_day in itertools.groupby(events, key=lambda e: e.end_as_date)
}
for event in events:
# Use start date for current event
start_date = event.as_date
# If previous event exists, calculate the gap
if previous_event_end:
gap_days = (start_date - previous_event_end).days
if gap_days >= (min_gap_days + 2):
start_end = (
previous_event_end + timedelta(days=1),
start_date - timedelta(days=1),
)
gaps.append(start_end)
# Update previous event end date
end = event.end_as_date
if not previous_event_end or end > previous_event_end:
previous_event_end = end
return [
{
"start": gap_start,
"end": gap_end,
"after": by_start_date[gap_end + timedelta(days=1)],
"before": by_end_date[gap_start - timedelta(days=1)],
}
for gap_start, gap_end in gaps
]
def busy_event(e: Event) -> bool:
"""Busy."""
if e.name not in {
"event",
"accommodation",
"conference",
"dodainville",
"transport",
"meetup",
"party",
}:
return False
if e.title in ("IA UK board meeting", "Mill Road Winter Fair"):
return False
if e.name == "conference" and not e.going:
return False
if not e.title:
return True
if e.title == "LHG Run Club" or "Third Thursday Social" in e.title:
return False
lc_title = e.title.lower()
return "rebels" not in lc_title and "south west data social" not in lc_title
async def get_data(
now: datetime, config: flask.config.Config
) -> typing.Mapping[str, str | object]:
"""Get data to display on agenda dashboard."""
data_dir = config["DATA_DIR"]
@ -192,51 +367,28 @@ async def get_data(now: datetime, config: flask.config.Config) -> AgendaData:
minus_365 = now - timedelta(days=365)
plus_365 = now + timedelta(days=365)
t0 = time()
offline_mode = bool(config.get("OFFLINE_MODE"))
result_list = await asyncio.gather(
time_function(
"gwr_advance_tickets", gwr.advance_ticket_date, data_dir, offline_mode
),
time_function(
"backwell_bins",
n_somerset_waste_collection_events,
data_dir,
config["BACKWELL_POSTCODE"],
config["BACKWELL_UPRN"],
offline_mode,
),
time_function(
"bristol_bins",
bristol_waste_collection_events,
data_dir,
today,
config["BRISTOL_UPRN"],
offline_mode,
),
(
gbpusd,
gwr_advance_tickets,
bank_holiday,
rockets,
backwell_bins,
bristol_bins,
) = await asyncio.gather(
fx.get_gbpusd(config),
gwr.advance_ticket_date(data_dir),
uk_holiday.bank_holiday_list(last_year, next_year, data_dir),
thespacedevs.get_launches(rocket_dir, limit=40),
waste_collection_events(data_dir),
bristol_waste_collection_events(data_dir, today),
)
rockets = thespacedevs.read_cached_launches(rocket_dir)
results = {call[0]: call[1] for call in result_list}
errors = [(call[0], call[3]) for call in result_list if call[3]]
gwr_advance_tickets = results["gwr_advance_tickets"]
data_gather_seconds = time() - t0
t0 = time()
stock_market_times = stock_market.open_and_close()
stock_market_times_seconds = time() - t0
reply: AgendaData = {
reply: dict[str, typing.Any] = {
"now": now,
"stock_markets": stock_market_times,
"gbpusd": gbpusd,
"stock_markets": stock_market.open_and_close(),
"rockets": rockets,
"gwr_advance_tickets": gwr_advance_tickets,
"data_gather_seconds": data_gather_seconds,
"stock_market_times_seconds": stock_market_times_seconds,
"timings": [(call[0], call[2]) for call in result_list],
}
my_data = config["PERSONAL_DATA"]
@ -253,44 +405,85 @@ async def get_data(now: datetime, config: flask.config.Config) -> AgendaData:
if gwr_advance_tickets:
events.append(Event(name="gwr_advance_tickets", date=gwr_advance_tickets))
us_hols = holidays.us_holidays(last_year, next_year)
events += holidays.get_nyse_holidays(last_year, next_year, us_hols)
us_hols = us_holidays(last_year, next_year)
holidays: list[Holiday] = bank_holiday + us_hols
for country in (
"at",
"be",
"br",
"ch",
"cz",
"de",
"dk",
"ee",
"es",
"fi",
"fr",
"gr",
"it",
"ke",
"nl",
"pl",
):
holidays += get_holidays(country, last_year, next_year)
events += get_nyse_holidays(last_year, next_year, us_hols)
accommodation_events = accommodation.get_events(
os.path.join(my_data, "accommodation.yaml")
)
holiday_list = holidays.get_all(last_year, next_year, data_dir)
events += holidays.combine_holidays(holiday_list)
events += holidays.get_school_holidays(last_year, next_year, data_dir)
if flask.g.user.is_authenticated:
events += birthday.get_birthdays(
last_year, os.path.join(my_data, "entities.yaml")
)
events += domains.renewal_dates(my_data)
events += combine_holidays(holidays)
events += birthday.get_birthdays(last_year, os.path.join(my_data, "entities.yaml"))
events += accommodation_events
events += travel.all_events(my_data)
events += conference.get_list(os.path.join(my_data, "conferences.yaml"))
for key in "backwell_bins", "bristol_bins":
if results[key]:
events += results[key]
events += events_yaml.read(my_data, last_year, next_year)
events += backwell_bins + bristol_bins
events += read_events_yaml(my_data, last_year, next_year)
events += subscription.get_events(os.path.join(my_data, "subscriptions.yaml"))
events += gandi.get_events(data_dir)
events += economist.publication_dates(last_week, next_year)
events += meetup.get_events(my_data)
events += hn.whoishiring(last_year, next_year)
events += carnival.rio_carnival_events(last_year, next_year)
events += rocket_launch_events(rockets)
events += domains.renewal_dates(my_data)
# hide markets that happen while away
markets = [e for e in events if e.name == "market"]
going = [e for e in events if e.going]
overlapping_markets = find_markets_during_stay(
accommodation_events + going, markets
)
for market in overlapping_markets:
events.remove(market)
for launch in rockets:
dt = None
if launch["net_precision"] == "Day":
dt = datetime.strptime(launch["net"], "%Y-%m-%dT00:00:00Z").date()
elif launch["t0_time"]:
dt = pytz.utc.localize(
datetime.strptime(launch["net"], "%Y-%m-%dT%H:%M:%SZ")
)
if not dt:
continue
rocket_name = f'🚀{launch["rocket"]}: {launch["mission_name"] or "[no mission]"}'
e = Event(name="rocket", date=dt, title=rocket_name)
events.append(e)
events += [Event(name="today", date=today)]
busy_events = [
e
for e in sorted(events, key=lambda e: e.as_date)
if e.as_date > today and e.as_date < next_year and busy.busy_event(e)
if e.as_date > today and e.as_date < next_year and busy_event(e)
]
gaps = busy.find_gaps(busy_events)
gaps = find_gaps(busy_events)
events += [
Event(name="gap", date=gap["start"], end_date=gap["end"]) for gap in gaps
@ -300,7 +493,7 @@ async def get_data(now: datetime, config: flask.config.Config) -> AgendaData:
# at the top of the list for today. This is achieved by sorting first by
# the datetime attribute, and then ensuring that events with the name
# "today" are ordered before others on the same date.
events.sort(key=lambda e: (event_sort_datetime(e), e.name != "today"))
events.sort(key=lambda e: (e.as_datetime, e.name != "today"))
reply["gaps"] = gaps
@ -308,10 +501,9 @@ async def get_data(now: datetime, config: flask.config.Config) -> AgendaData:
reply["sunrise"] = sun.sunrise(observer)
reply["sunset"] = sun.sunset(observer)
reply["events"] = events
reply["accommodation_events"] = accommodation_events
reply["last_week"] = last_week
reply["two_weeks_ago"] = two_weeks_ago
reply["errors"] = errors
reply["fullcalendar_events"] = calendar.build_events(events)
return reply

4
agenda/domains.py
View file

@ -1,10 +1,10 @@
"""Domain renewal dates."""
"""Accomodation."""
import csv
import os
from datetime import datetime
from .event import Event
from .types import Event
url = "https://admin.gandi.net/domain/01578ef0-a84b-11e7-bdf3-00163e6dc886/"

2
agenda/economist.py
View file

@ -5,7 +5,7 @@ from datetime import date, time, timedelta
from dateutil.relativedelta import TH, relativedelta
from . import uk_time
from .event import Event
from .types import Event
def publication_dates(start_date: date, end_date: date) -> list[Event]:

149
agenda/event.py
View file

@ -1,149 +0,0 @@
"""Types."""
import datetime
from dataclasses import dataclass
from . import utils
from .types import DateOrDateTime, StrDict
emojis = {
"market": "🧺",
"us_presidential_election": "🗳️🇺🇸",
"bus_route_closure": "🚌❌",
"meetup": "👥",
"dinner": "🍷",
"party": "🍷",
"ba_voucher": "✈️",
"accommodation": "🏨", # alternative: 🧳
"flight": "✈️",
"conference": "🎤",
"rocket": "🚀",
"birthday": "🎈",
"waste_schedule": "🗑️",
"economist": "📰",
"running": "🏃",
"critical_mass": "🚴",
"trip": "🧳",
"hackathon": "💻",
}
@dataclass
class Event:
"""Event."""
name: str
date: DateOrDateTime
end_date: DateOrDateTime | None = None
title: str | None = None
url: str | None = None
going: bool | None = None
@property
def as_datetime(self) -> datetime.datetime:
"""Date/time of event."""
return utils.as_datetime(self.date)
@property
def has_time(self) -> bool:
"""Event has a time associated with it."""
return isinstance(self.date, datetime.datetime)
@property
def as_date(self) -> datetime.date:
"""Date of event."""
return (
self.date.date() if isinstance(self.date, datetime.datetime) else self.date
)
@property
def end_as_date(self) -> datetime.date:
"""Date of event."""
return (
(
self.end_date.date()
if isinstance(self.end_date, datetime.datetime)
else self.end_date
)
if self.end_date
else self.as_date
)
@property
def display_time(self) -> str | None:
"""Time for display on web page."""
return (
self.date.strftime("%H:%M")
if isinstance(self.date, datetime.datetime)
else None
)
@property
def display_timezone(self) -> str | None:
"""Timezone for display on web page."""
return (
self.date.strftime("%z")
if isinstance(self.date, datetime.datetime)
else None
)
def display_duration(self) -> str | None:
"""Duration for display."""
if self.end_as_date != self.as_date or not self.has_time:
return None
assert isinstance(self.date, datetime.datetime)
assert isinstance(self.end_date, datetime.datetime)
secs: int = int((self.end_date - self.date).total_seconds())
hours: int = secs // 3600
mins: int = (secs % 3600) // 60
if mins == 0:
return f"{hours:d}h"
if hours == 0:
return f"{mins:d} mins"
return f"{hours:d}h {mins:02d} mins"
def delta_days(self, today: datetime.date) -> str:
"""Return number of days from today as a string."""
delta = (self.as_date - today).days
match delta:
case 0:
return "today"
case 1:
return "1 day"
case _:
return f"{delta:,d} days"
@property
def display_date(self) -> str:
"""Date for display on web page."""
if isinstance(self.date, datetime.datetime):
return self.date.strftime("%a, %d, %b %Y %H:%M %z")
else:
return self.date.strftime("%a, %d, %b %Y")
@property
def display_title(self) -> str:
"""Name for display."""
return self.title or self.name
@property
def emoji(self) -> str | None:
"""Emoji."""
if self.title == "LHG Run Club":
return "🏃🍻"
return emojis.get(self.name)
@property
def title_with_emoji(self) -> str | None:
"""Title with optional emoji at the start."""
title = self.title or self.name
if title is None:
return None
emoji = self.emoji
return f"{emoji} {title}" if emoji else title

85
agenda/events_yaml.py
View file

@ -1,85 +0,0 @@
"""Read events from YAML."""
import os
import typing
from datetime import date, datetime
import dateutil.rrule
import isodate # type: ignore
import yaml
from . import uk_tz
from .event import Event
def midnight(d: date) -> datetime:
"""Convert from date to midnight on that day."""
return datetime.combine(d, datetime.min.time())
def dates_from_rrule(
rrule: str, start: date, end: date
) -> typing.Sequence[datetime | date]:
"""Generate events from an RRULE between start_date and end_date."""
all_day = not any(param in rrule for param in ["BYHOUR", "BYMINUTE", "BYSECOND"])
return [
i.date() if all_day else uk_tz.localize(i)
for i in dateutil.rrule.rrulestr(rrule, dtstart=midnight(start)).between(
midnight(start), midnight(end)
)
]
def get_yaml_event_date_field(item: dict[str, str]) -> str:
"""Event date field name."""
return (
"end_date"
if item["name"] == "travel_insurance"
else ("start_date" if "start_date" in item else "date")
)
def get_yaml_event_end_date_field(item: dict[str, str]) -> str:
"""Event date field name."""
return (
"end_date"
if item["name"] == "travel_insurance"
else ("start_date" if "start_date" in item else "date")
)
def read(
data_dir: str, start: date, end: date, skip_trips: bool = False
) -> list[Event]:
"""Read eventes from YAML file."""
events: list[Event] = []
for item in yaml.safe_load(open(os.path.join(data_dir, "events.yaml"))):
if "trip" in item and skip_trips:
continue
duration = (
isodate.parse_duration(item["duration"]) if "duration" in item else None
)
dates = (
dates_from_rrule(item["rrule"], start, end)
if "rrule" in item
else [item[get_yaml_event_date_field(item)]]
)
for dt in dates:
e = Event(
name=item["name"],
date=dt,
end_date=(
dt + duration
if duration
else (
item.get("end_date")
if item["name"] != "travel_insurance"
else None
)
),
title=item.get("title"),
url=item.get("url"),
)
events.append(e)
return events

87
agenda/flight_diagnostics.py
View file

@ -1,87 +0,0 @@
"""Bounded, best-effort diagnostic snapshots for failed flight searches."""
import gzip
import json
import logging
import os
import tempfile
import time
from datetime import datetime, timezone
from pathlib import Path
from agenda.types import StrDict
MAX_RECORDS = 100
RETENTION_SECONDS = 30 * 24 * 60 * 60
logger = logging.getLogger(__name__)
def write_failure(directory: Path, details: StrDict) -> Path | None:
"""Save one compressed JSON snapshot; logging failures never break lookups."""
temporary: str | None = None
try:
folder = directory / "errors"
folder.mkdir(parents=True, exist_ok=True)
truncated: list[str] = []
def limit(value: str, size: int, field: str) -> str:
data = value.encode("utf-8")
if len(data) > size:
truncated.append(field)
return data[:size].decode("utf-8", errors="ignore")
return value
snapshot = dict(details)
snapshot["format_version"] = 1
for field, size in (
("page_html", 2 * 1024 * 1024),
("flight_script", 1024 * 1024),
("page_text", 64 * 1024),
):
if isinstance(snapshot.get(field), str):
snapshot[field] = limit(snapshot[field], size, field)
responses = []
for index, response in enumerate(snapshot.get("http_errors", [])[:5]):
item = dict(response)
if isinstance(item.get("body"), str):
item["body"] = limit(
item["body"], 256 * 1024, f"http_errors[{index}].body"
)
responses.append(item)
snapshot["http_errors"] = responses
snapshot["truncated_fields"] = truncated
snapshot["recorded_at"] = datetime.now(timezone.utc).isoformat()
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%S%fZ")
with tempfile.NamedTemporaryFile(
dir=folder, prefix=".snapshot-", delete=False
) as temporary_output:
temporary = temporary_output.name
path = (
folder
/ f"{stamp}-{Path(temporary).name.removeprefix('.snapshot-')}.json.gz"
)
with gzip.open(temporary, "wt", encoding="utf-8") as compressed_output:
json.dump(snapshot, compressed_output, ensure_ascii=False)
os.chmod(temporary, 0o640)
os.replace(temporary, path)
temporary = None
records = sorted(
folder.glob("*.json.gz"),
key=lambda record: record.stat().st_mtime,
reverse=True,
)
cutoff = time.time() - RETENTION_SECONDS
for index, record in enumerate(records):
if index >= MAX_RECORDS or record.stat().st_mtime < cutoff:
record.unlink()
logger.info("Saved flight failure diagnostics to %s", path)
return path
except Exception:
logger.warning("Unable to save flight failure diagnostics", exc_info=True)
return None
finally:
if temporary is not None:
try:
Path(temporary).unlink(missing_ok=True)
except OSError:
pass

141
agenda/flight_search_cache.py
View file

@ -1,141 +0,0 @@
"""Shared pacing, HTTP 429 cooldowns and caches for individual flight searches."""
import contextvars
import fcntl
import hashlib
import json
import time
import typing
from contextlib import contextmanager
from datetime import datetime, timezone
from pathlib import Path
from agenda.types import StrDict
REQUEST_INTERVAL = 3.0
RATE_LIMIT_COOLDOWN = 30 * 60
DAY_CACHE_SECONDS = 6 * 60 * 60
DAY_CACHE_VERSION = 1
TRANSPORT = "playwright-chromium"
search_directory: contextvars.ContextVar[Path | None] = contextvars.ContextVar(
"conference_flight_search_directory", default=None
)
class RateLimitCooldownError(RuntimeError):
"""Google flight requests are temporarily paused across all workers."""
@contextmanager
def use_directory(directory: Path) -> typing.Iterator[None]:
"""Share a lookup's configured cache directory with its individual searches."""
token = search_directory.set(directory)
try:
yield
finally:
search_directory.reset(token)
def read_state(path: Path) -> StrDict:
"""Read shared JSON state, treating damaged files as a cache miss."""
try:
state = json.loads(path.read_text())
return typing.cast(StrDict, state) if isinstance(state, dict) else {}
except (OSError, ValueError):
return {}
def cooldown_message(directory: Path) -> str | None:
"""Describe an active shared cooldown without making any requests."""
state = read_state(directory / "rate-limit.json")
until = state.get("cooldown_until")
if not isinstance(until, (float, int)) or until <= time.time():
return None
when = datetime.fromtimestamp(until, timezone.utc).strftime(
"%a %-d %b at %H:%M UTC"
)
return f"Google returned HTTP 429. All flight searches are paused until {when}. Cached results remain available."
def write_state(path: Path, state: StrDict) -> None:
"""Use the agenda's atomic JSON writer for state and day caches."""
# Imported lazily to avoid a module cycle.
from agenda.conference_flights import write_cache
write_cache(path, state)
def wait_for_request(directory: Path) -> None:
"""Space browser searches across processes."""
directory.mkdir(parents=True, exist_ok=True)
with (directory / "request.lock").open("a+") as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
message = cooldown_message(directory)
if message:
raise RateLimitCooldownError(message)
state_path = directory / "rate-limit.json"
state = read_state(state_path)
last = state.get("last_request_at", 0)
last = float(last) if isinstance(last, (float, int)) else 0.0
delay = min(REQUEST_INTERVAL, max(0.0, last + REQUEST_INTERVAL - time.time()))
if delay:
time.sleep(delay)
state["last_request_at"] = time.time()
write_state(state_path, state)
def block_requests(directory: Path) -> None:
"""Set a shared cooldown immediately after the first HTTP 429."""
directory.mkdir(parents=True, exist_ok=True)
with (directory / "request.lock").open("a+") as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
path = directory / "rate-limit.json"
state = read_state(path)
state["cooldown_until"] = time.time() + RATE_LIMIT_COOLDOWN
state["transport"] = TRANSPORT
write_state(path, state)
def cached_day(
directory: Path,
origin: str,
destination: str,
day: str,
direct: bool,
fetch: typing.Callable[[], list[StrDict]],
) -> list[StrDict]:
"""Reuse a route/date's results across conferences, including empty results."""
key = json.dumps(
[
DAY_CACHE_VERSION,
origin,
destination,
day,
direct,
"BEST",
"BA",
"GBP",
"en-GB",
"GB",
]
)
folder = directory / "days"
folder.mkdir(parents=True, exist_ok=True)
path = folder / (hashlib.sha256(key.encode()).hexdigest() + ".json")
# Recheck under the lock so concurrent lookups cannot duplicate a fetch.
with path.with_suffix(".lock").open("a+") as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
state = read_state(path)
timestamp = state.get("fetched_at")
if (
isinstance(timestamp, (float, int))
and time.time() - timestamp < DAY_CACHE_SECONDS
and isinstance(state.get("flights"), list)
and (state["flights"] or state.get("transport") == TRANSPORT)
):
return typing.cast(list[StrDict], state["flights"])
rows = fetch()
write_state(
path, {"fetched_at": time.time(), "flights": rows, "transport": TRANSPORT}
)
return rows

244
agenda/fx.py
View file

@ -9,10 +9,6 @@ from decimal import Decimal
import flask
import httpx
DEFAULT_FX_CACHE_TTL_HOURS = 24
DEFAULT_FX_FAILURE_RETRY_HOURS = 24
FRANKFURTER_CACHE_PREFIX = "frankfurter"
async def get_gbpusd(config: flask.config.Config) -> Decimal:
"""Get the current value for GBPUSD, with caching."""
@ -46,243 +42,3 @@ async def get_gbpusd(config: flask.config.Config) -> Decimal:
data = json.loads(r.text, parse_float=Decimal)
return typing.cast(Decimal, 1 / data["quotes"]["USDGBP"])
def read_cached_rates(
filename: str | None, currencies: list[str]
) -> dict[str, Decimal]:
"""Read FX rates from cache."""
if filename is None:
return {}
with open(filename) as file:
data = json.load(file, parse_float=Decimal)
frankfurter_rates = _frankfurter_rates(data, currencies)
if frankfurter_rates:
return frankfurter_rates
if isinstance(data, list):
return {}
if not isinstance(data, dict):
return {}
rates = data.get("rates")
if isinstance(rates, dict):
return {cur: Decimal(rates[cur]) for cur in currencies if cur in rates}
quotes = data.get("quotes")
if isinstance(quotes, dict):
return {
cur: Decimal(quotes[f"GBP{cur}"])
for cur in currencies
if f"GBP{cur}" in quotes
}
return {}
def _frankfurter_rates(data: typing.Any, currencies: list[str]) -> dict[str, Decimal]:
"""Extract rates from Frankfurter's response format."""
if not isinstance(data, list):
return {}
rates: dict[str, Decimal] = {}
for item in data:
if not isinstance(item, dict):
continue
quote = item.get("quote")
rate = item.get("rate")
if isinstance(quote, str) and quote in currencies and rate is not None:
rates[quote] = Decimal(rate)
return rates
def _fx_cache_datetime(filename: str) -> datetime:
"""Extract the cache timestamp from an FX cache filename."""
return datetime.strptime(filename[:16], "%Y-%m-%d_%H:%M")
def _has_required_quotes(filename: str, currencies: list[str]) -> bool:
"""Return true if the cache file contains the requested GBP quotes."""
try:
return len(read_cached_rates(filename, currencies)) == len(currencies)
except (OSError, json.JSONDecodeError):
return False
def _latest_file(
fx_dir: str, filenames: list[str], currencies: list[str], *, valid_only: bool
) -> str | None:
"""Return the newest matching FX cache filename."""
matching_files = [
filename
for filename in filenames
if not valid_only
or _has_required_quotes(os.path.join(fx_dir, filename), currencies)
]
return max(matching_files) if matching_files else None
def get_rates_exchangerate_host(config: flask.config.Config) -> dict[str, Decimal]:
"""Get exchange rates from exchangerate.host.
Kept as a fallback implementation in case we decide to switch back from
Frankfurter.
"""
currencies = config["CURRENCIES"]
access_key = config["EXCHANGERATE_ACCESS_KEY"]
data_dir = config["DATA_DIR"]
cache_ttl_hours = int(config.get("FX_CACHE_TTL_HOURS", DEFAULT_FX_CACHE_TTL_HOURS))
failure_retry_hours = int(
config.get("FX_FAILURE_RETRY_HOURS", DEFAULT_FX_FAILURE_RETRY_HOURS)
)
now = datetime.now()
now_str = now.strftime("%Y-%m-%d_%H:%M")
fx_dir = os.path.join(data_dir, "fx")
os.makedirs(fx_dir, exist_ok=True) # Ensure the directory exists
currency_string = ",".join(sorted(currencies))
file_suffix = f"{currency_string}_to_GBP.json"
existing_data = os.listdir(fx_dir)
existing_files = [f for f in existing_data if f[17:] == file_suffix]
latest_attempt = _latest_file(fx_dir, existing_files, currencies, valid_only=False)
latest_valid = _latest_file(fx_dir, existing_files, currencies, valid_only=True)
latest_valid_path = (
os.path.join(fx_dir, latest_valid) if latest_valid is not None else None
)
if latest_valid is not None:
recent = _fx_cache_datetime(latest_valid)
delta = now - recent
if delta < timedelta(hours=cache_ttl_hours) or config["OFFLINE_MODE"]:
return read_cached_rates(latest_valid_path, currencies)
if latest_attempt is not None:
recent_attempt = _fx_cache_datetime(latest_attempt)
attempt_delta = now - recent_attempt
if attempt_delta < timedelta(hours=failure_retry_hours):
return read_cached_rates(latest_valid_path, currencies)
url = "http://api.exchangerate.host/live"
params = {"currencies": currency_string, "source": "GBP", "access_key": access_key}
filename = f"{now_str}_{file_suffix}"
try:
with httpx.Client() as client:
response = client.get(url, params=params, timeout=10)
except (httpx.ConnectError, httpx.ReadTimeout):
return read_cached_rates(latest_valid_path, currencies)
try:
data = json.loads(response.text, parse_float=Decimal)
except json.decoder.JSONDecodeError:
return read_cached_rates(latest_valid_path, currencies)
if not data.get("success", True) or not isinstance(data.get("quotes"), dict):
with open(os.path.join(fx_dir, filename), "w") as file:
file.write(response.text)
return read_cached_rates(latest_valid_path, currencies)
with open(os.path.join(fx_dir, filename), "w") as file:
file.write(response.text)
return {
cur: Decimal(data["quotes"][f"GBP{cur}"])
for cur in currencies
if f"GBP{cur}" in data["quotes"]
}
def get_rates(config: flask.config.Config) -> dict[str, Decimal]:
"""Get current values of exchange rates for a list of currencies against GBP."""
currencies = config["CURRENCIES"]
data_dir = config["DATA_DIR"]
cache_ttl_hours = int(config.get("FX_CACHE_TTL_HOURS", DEFAULT_FX_CACHE_TTL_HOURS))
failure_retry_hours = int(
config.get("FX_FAILURE_RETRY_HOURS", DEFAULT_FX_FAILURE_RETRY_HOURS)
)
now = datetime.now()
now_str = now.strftime("%Y-%m-%d_%H:%M")
fx_dir = os.path.join(data_dir, "fx")
os.makedirs(fx_dir, exist_ok=True) # Ensure the directory exists
currency_string = ",".join(sorted(currencies))
legacy_file_suffix = f"{currency_string}_to_GBP.json"
frankfurter_file_suffix = f"{FRANKFURTER_CACHE_PREFIX}_{legacy_file_suffix}"
existing_data = os.listdir(fx_dir)
valid_cache_files = [
f
for f in existing_data
if f[17:] in (legacy_file_suffix, frankfurter_file_suffix)
]
attempt_files = [f for f in existing_data if f[17:] == frankfurter_file_suffix]
# Cache filenames already record the requested currencies. Compare against
# the newest request, including failed attempts, so failures retain backoff.
all_rate_files = [f for f in existing_data if f.endswith("_to_GBP.json")]
latest_request = max(all_rate_files) if all_rate_files else None
currencies_changed = latest_request is not None and latest_request[17:] not in (
legacy_file_suffix,
frankfurter_file_suffix,
)
latest_attempt = _latest_file(fx_dir, attempt_files, currencies, valid_only=False)
latest_source_valid = _latest_file(
fx_dir, attempt_files, currencies, valid_only=True
)
latest_valid = _latest_file(fx_dir, valid_cache_files, currencies, valid_only=True)
latest_valid_path = (
os.path.join(fx_dir, latest_valid) if latest_valid is not None else None
)
if config["OFFLINE_MODE"]:
return read_cached_rates(latest_valid_path, currencies)
if latest_source_valid is not None and not currencies_changed:
recent = _fx_cache_datetime(latest_source_valid)
delta = now - recent
if delta < timedelta(hours=cache_ttl_hours):
return read_cached_rates(
os.path.join(fx_dir, latest_source_valid), currencies
)
if latest_attempt is not None and not currencies_changed:
recent_attempt = _fx_cache_datetime(latest_attempt)
attempt_delta = now - recent_attempt
if attempt_delta < timedelta(hours=failure_retry_hours):
return read_cached_rates(latest_valid_path, currencies)
url = "https://api.frankfurter.dev/v2/rates"
params = {"base": "GBP", "quotes": currency_string}
filename = f"{now_str}_{frankfurter_file_suffix}"
try:
with httpx.Client() as client:
response = client.get(url, params=params, timeout=10)
except (httpx.ConnectError, httpx.ReadTimeout):
return read_cached_rates(latest_valid_path, currencies)
try:
data = json.loads(response.text, parse_float=Decimal)
except json.decoder.JSONDecodeError:
return read_cached_rates(latest_valid_path, currencies)
frankfurter_rates = _frankfurter_rates(data, currencies)
if not frankfurter_rates:
with open(os.path.join(fx_dir, filename), "w") as file:
file.write(response.text)
return read_cached_rates(latest_valid_path, currencies)
with open(os.path.join(fx_dir, filename), "w") as file:
file.write(response.text)
return frankfurter_rates

27
agenda/gandi.py
View file

@ -1,27 +0,0 @@
"""Gandi domain renewal dates."""
import os
from .event import Event
from datetime import datetime
import json
def get_events(data_dir: str) -> list[Event]:
"""Get subscription renewal dates."""
filename = os.path.join(data_dir, "gandi_domains.json")
with open(filename) as f:
items = json.load(f)
assert isinstance(items, list)
assert all(item["fqdn"] and item["dates"]["registry_ends_at"] for item in items)
return [
Event(
date=datetime.fromisoformat(item["dates"]["registry_ends_at"]).date(),
name="domain",
title=item["fqdn"] + " renewal",
)
for item in items
]

483
agenda/generate_booking_yaml.py
View file

@ -1,483 +0,0 @@
"""Generate travel booking YAML from booking text or a booking URL."""
import argparse
import configparser
import importlib
import json
import os
import sys
import typing
from dataclasses import dataclass
from datetime import date, datetime
from pathlib import Path
import html2text
import lxml.html
import openai
import requests
import yaml
USER_AGENT = "generate-booking-yaml/0.1"
REPO_ROOT = Path(__file__).resolve().parent.parent
SPEC_PATH = REPO_ROOT / "docs" / "personal-data-yaml.md"
PERSONAL_DATA_DIR = Path("~/src/personal-data").expanduser()
class TripLike(typing.Protocol):
"""Trip attributes needed for booking import matching."""
start: date
end: date | None
@dataclass(frozen=True)
class BookingConfig:
"""Configuration for one travel booking YAML generator."""
booking_type: str
yaml_filename: str
spec_heading: str
start_field: str
excluded_top_level_keys: tuple[str, ...]
json_key: str = "booking"
BOOKING_CONFIGS: dict[str, BookingConfig] = {
"flight": BookingConfig(
booking_type="flight",
yaml_filename="flights.yaml",
spec_heading="flights.yaml",
start_field="depart",
excluded_top_level_keys=("trip",),
),
"train": BookingConfig(
booking_type="train",
yaml_filename="trains.yaml",
spec_heading="trains.yaml",
start_field="depart",
excluded_top_level_keys=("trip",),
),
"accommodation": BookingConfig(
booking_type="accommodation",
yaml_filename="accommodation.yaml",
spec_heading="accommodation.yaml",
start_field="from",
excluded_top_level_keys=("trip", "latitude", "longitude"),
),
}
def read_api_key() -> str:
"""Read API key from ~/.config/openai/config."""
config_path = os.path.expanduser("~/.config/openai/config")
parser = configparser.ConfigParser()
parser.read(config_path)
return parser["openai"]["api_key"]
def read_markdown_section(markdown_text: str, heading: str) -> str:
"""Return one second-level markdown section by heading text."""
section_headings = (f"## `{heading}`", f"## {heading}")
start = -1
matched_heading = ""
for section_heading in section_headings:
start = markdown_text.find(section_heading)
if start != -1:
matched_heading = section_heading
break
if start == -1:
raise ValueError(f"Could not find section for heading {heading!r}")
next_heading = markdown_text.find("\n## ", start + len(matched_heading))
if next_heading == -1:
return markdown_text[start:].strip()
return markdown_text[start:next_heading].strip()
def yaml_format_description(config: BookingConfig) -> str:
"""Return relevant personal-data YAML documentation for the prompt."""
spec_text = SPEC_PATH.read_text()
sections = [
read_markdown_section(spec_text, "General Rules"),
read_markdown_section(spec_text, "Cross-File References"),
read_markdown_section(spec_text, config.spec_heading),
]
return "\n\n".join(sections)
def read_existing_bookings(
config: BookingConfig, data_dir: Path = PERSONAL_DATA_DIR
) -> str:
"""Read the existing YAML file for examples and local style."""
path = data_dir / config.yaml_filename
return path.read_text()
def build_prompt(
booking_text: str,
config: BookingConfig,
current_bookings: str | None = None,
) -> str:
"""Build prompt to pass to the LLM."""
bookings = current_bookings
if bookings is None:
bookings = read_existing_bookings(config)
excluded_keys = ", ".join(f'"{key}"' for key in config.excluded_top_level_keys)
return f"""
I keep a record of all my {config.booking_type} bookings in a YAML file.
Use this YAML format specification:
{yaml_format_description(config)}
Here's my current list of bookings for examples of local style and known
references.
===
{bookings}
===
Here's a new booking I just made.
Return the YAML representation for this booking using the documented format and
the same local style as my existing bookings.
Rules:
- Wrap the response in a JSON object with a single key "{config.json_key}" that
contains the booking in YAML.
- The value of "{config.json_key}" must be YAML text, not JSON.
- Exclude these top-level keys from the YAML: {excluded_keys}.
- Do not invent details that are not present in the booking text.
- Quote prices and identifiers that might otherwise be parsed as numbers.
===
{booking_text}
"""
def get_from_open_ai(prompt: str, model: str = "gpt-5.4") -> dict[str, str]:
"""Pass prompt to OpenAI and get reply."""
client = openai.OpenAI(api_key=read_api_key())
response = client.chat.completions.create(
messages=[{"role": "user", "content": prompt}],
model=model,
response_format={"type": "json_object"},
)
reply = response.choices[0].message.content
assert isinstance(reply, str)
return typing.cast(dict[str, str], json.loads(reply))
def fetch_webpage(url: str) -> lxml.html.HtmlElement:
"""Fetch webpage HTML and parse it."""
response = requests.get(url, headers={"User-Agent": USER_AGENT})
response.raise_for_status()
return lxml.html.fromstring(response.content)
def webpage_to_text(root: lxml.html.HtmlElement) -> str:
"""Convert parsed HTML into readable text content."""
root_copy = lxml.html.fromstring(lxml.html.tostring(root))
for script_or_style in root_copy.xpath("//script|//style"):
script_or_style.drop_tree()
text_maker = html2text.HTML2Text()
text_maker.ignore_links = False
text_maker.ignore_images = True
return text_maker.handle(lxml.html.tostring(root_copy, encoding="unicode"))
def url_to_booking_text(url: str) -> str:
"""Fetch a URL and convert it to source text for the model."""
return webpage_to_text(fetch_webpage(url))
def booking_text_from_args(args: list[str]) -> str:
"""Return booking text from a URL argument or stdin."""
if args:
if len(args) != 1:
raise SystemExit("Usage: generate-BOOKING-booking-yaml [URL]")
return url_to_booking_text(args[0])
return sys.stdin.read()
def generate_booking_yaml(
booking_text: str, config: BookingConfig, model: str = "gpt-5.4"
) -> str:
"""Generate booking YAML from source text."""
prompt = build_prompt(booking_text, config)
return get_from_open_ai(prompt, model=model)[config.json_key]
def datetime_from_yaml_value(value: typing.Any) -> datetime:
"""Convert a YAML date/datetime/string value into a datetime."""
if isinstance(value, datetime):
return value
if isinstance(value, date):
return datetime.combine(value, datetime.min.time())
if isinstance(value, str):
parsed = datetime.fromisoformat(value)
return parsed
raise TypeError(f"Unsupported departure value: {value!r}")
def first_departure(booking: dict[str, typing.Any], config: BookingConfig) -> datetime:
"""Return the first departure datetime for a generated booking."""
if config.booking_type == "flight":
flights = booking["flights"]
assert isinstance(flights, list)
first_flight = flights[0]
assert isinstance(first_flight, dict)
return datetime_from_yaml_value(first_flight["depart"])
return datetime_from_yaml_value(booking[config.start_field])
def comparable_departure(
booking: dict[str, typing.Any], config: BookingConfig
) -> datetime:
"""Return a timezone-naive departure datetime for sorting."""
return first_departure(booking, config).replace(tzinfo=None)
def generated_bookings_from_yaml(yaml_text: str) -> list[dict[str, typing.Any]]:
"""Parse generated booking YAML into a list of booking mappings."""
loaded = yaml.safe_load(yaml_text)
if isinstance(loaded, dict):
return [typing.cast(dict[str, typing.Any], loaded)]
if isinstance(loaded, list) and all(isinstance(item, dict) for item in loaded):
return typing.cast(list[dict[str, typing.Any]], loaded)
raise ValueError("Generated booking YAML must be a mapping or list of mappings.")
def trip_key_position(booking: dict[str, typing.Any], config: BookingConfig) -> int:
"""Return the preferred insertion position for the top-level trip key."""
keys = list(booking)
if config.booking_type == "flight":
if "booking_reference" in booking:
return keys.index("booking_reference") + 1
return 0
if config.booking_type == "accommodation":
if "country" in booking:
return keys.index("country") + 1
if "location" in booking:
return keys.index("location") + 1
return 0
if "to" in booking:
return keys.index("to") + 1
if "from" in booking:
return keys.index("from") + 1
return 0
def set_trip_key(
booking: dict[str, typing.Any], config: BookingConfig, trip_date: date
) -> dict[str, typing.Any]:
"""Set trip in the usual top-level position while preserving other key order."""
without_trip = {key: value for key, value in booking.items() if key != "trip"}
keys = list(without_trip)
position = trip_key_position(without_trip, config)
reordered: dict[str, typing.Any] = {}
for index, key in enumerate(keys):
if index == position:
reordered["trip"] = trip_date
reordered[key] = without_trip[key]
if "trip" not in reordered:
reordered["trip"] = trip_date
booking.clear()
booking.update(reordered)
return booking
def build_trips(data_dir: Path) -> list[TripLike]:
"""Build trips from personal data without importing agenda.trip at module load."""
trip_module = importlib.import_module("agenda.trip")
build_trip_list = typing.cast(
typing.Callable[..., list[TripLike]], trip_module.build_trip_list
)
return build_trip_list(data_dir=str(data_dir))
def matching_trip_date(depart: datetime, data_dir: Path = PERSONAL_DATA_DIR) -> date:
"""Find the trip grouping date for a departure, falling back to departure date."""
depart_date = depart.date()
matching_starts: list[date] = []
for trip in build_trips(data_dir):
trip_end = trip.end or trip.start
if trip.start <= depart_date <= trip_end:
matching_starts.append(trip.start)
if matching_starts:
return max(matching_starts)
return depart_date
def add_trip_dates(
bookings: list[dict[str, typing.Any]],
config: BookingConfig,
data_dir: Path = PERSONAL_DATA_DIR,
) -> None:
"""Add the top-level trip key to generated booking mappings."""
for booking in bookings:
trip_date = matching_trip_date(first_departure(booking, config), data_dir)
set_trip_key(booking, config, trip_date)
def dump_generated_bookings(bookings: list[dict[str, typing.Any]]) -> str:
"""Dump only generated bookings for insertion into an existing YAML list."""
text = yaml.dump(bookings, sort_keys=False, allow_unicode=True)
return text.lstrip()
def join_yaml_list_blocks(preamble: str, blocks: list[str], trailing: str = "") -> str:
"""Join top-level YAML list blocks with a blank line between items."""
body = "\n\n".join(block.rstrip("\n") for block in blocks)
return preamble + body + "\n" + trailing
def split_yaml_list_blocks(text: str) -> tuple[str, list[str], str]:
"""Split a top-level YAML list into preamble, item blocks, and trailing text."""
lines = text.splitlines(keepends=True)
first_item = next(
(index for index, line in enumerate(lines) if line.startswith("- ")), None
)
if first_item is None:
return text, [], ""
item_starts = [
index
for index, line in enumerate(lines[first_item:], start=first_item)
if line.startswith("- ")
]
blocks = [
"".join(lines[start:end])
for start, end in zip(item_starts, item_starts[1:] + [len(lines)])
]
return "".join(lines[:first_item]), blocks, ""
def existing_bookings_from_blocks(blocks: list[str]) -> list[dict[str, typing.Any]]:
"""Parse split YAML item blocks into booking mappings."""
bookings = []
for block in blocks:
loaded = yaml.safe_load(block)
if not isinstance(loaded, list) or len(loaded) != 1:
raise ValueError("Could not parse existing booking block.")
item = loaded[0]
if not isinstance(item, dict):
raise ValueError("Existing booking block is not a mapping.")
bookings.append(typing.cast(dict[str, typing.Any], item))
return bookings
def insertion_index(
existing_bookings: list[dict[str, typing.Any]],
new_bookings: list[dict[str, typing.Any]],
config: BookingConfig,
) -> int:
"""Return the chronological insertion index for generated bookings."""
new_depart = min(comparable_departure(booking, config) for booking in new_bookings)
for index, booking in enumerate(existing_bookings):
if comparable_departure(booking, config) > new_depart:
return index
return len(existing_bookings)
def insert_booking_text(
existing_text: str,
new_yaml_text: str,
config: BookingConfig,
) -> str:
"""Insert generated booking YAML into an existing top-level YAML list."""
preamble, blocks, trailing = split_yaml_list_blocks(existing_text)
new_bookings = generated_bookings_from_yaml(new_yaml_text)
_, new_blocks, _ = split_yaml_list_blocks(dump_generated_bookings(new_bookings))
if len(new_blocks) != len(new_bookings):
raise ValueError("Could not split generated booking YAML into item blocks.")
existing_bookings = existing_bookings_from_blocks(blocks)
new_items = sorted(
zip(new_bookings, new_blocks),
key=lambda item: comparable_departure(item[0], config),
)
for booking, block in new_items:
insert_at = insertion_index(existing_bookings, [booking], config)
existing_bookings.insert(insert_at, booking)
blocks.insert(insert_at, block)
return join_yaml_list_blocks(preamble, blocks, trailing)
def import_booking_yaml(
generated_yaml: str,
config: BookingConfig,
data_dir: Path = PERSONAL_DATA_DIR,
) -> int:
"""Add generated booking YAML to the configured personal-data file."""
bookings = generated_bookings_from_yaml(generated_yaml)
add_trip_dates(bookings, config, data_dir)
new_yaml = dump_generated_bookings(bookings)
yaml_path = data_dir / config.yaml_filename
existing_text = yaml_path.read_text()
updated_text = insert_booking_text(existing_text, new_yaml, config)
yaml_path.write_text(updated_text)
return len(bookings)
def main_for_type(booking_type: str, argv: list[str] | None = None) -> int:
"""CLI entrypoint for a specific booking type."""
parser = argparse.ArgumentParser(
description=(
f"Generate {booking_type} booking YAML from stdin or a URL and import it."
)
)
parser.add_argument("url", nargs="?", help="Booking URL to fetch")
parser.add_argument("--model", default=os.environ.get("OPENAI_MODEL", "gpt-5.4"))
parser.add_argument(
"--data-dir",
default=str(PERSONAL_DATA_DIR),
help="Directory containing personal-data YAML files.",
)
parser.add_argument(
"--print-only",
action="store_true",
help="Print generated YAML instead of editing the personal-data file.",
)
parsed = parser.parse_args(argv)
config = BOOKING_CONFIGS[booking_type]
args = [parsed.url] if parsed.url else []
booking_text = booking_text_from_args(args)
new_yaml = generate_booking_yaml(booking_text, config, model=parsed.model)
if parsed.print_only:
print(new_yaml)
return 0
count = import_booking_yaml(new_yaml, config, data_dir=Path(parsed.data_dir))
print(f"Imported {count} {booking_type} booking(s).")
return 0
def train_main(argv: list[str] | None = None) -> int:
"""CLI entrypoint for train booking YAML generation."""
return main_for_type("train", argv)
def flight_main(argv: list[str] | None = None) -> int:
"""CLI entrypoint for flight booking YAML generation."""
return main_for_type("flight", argv)
def accommodation_main(argv: list[str] | None = None) -> int:
"""CLI entrypoint for accommodation booking YAML generation."""
return main_for_type("accommodation", argv)

225
agenda/geomob.py
View file

@ -1,225 +0,0 @@
"""Geomob events."""
import logging
import os
import subprocess
from pathlib import Path
from dataclasses import dataclass
from datetime import date, datetime
from typing import List
import dateutil.parser
import flask
import lxml.html # type: ignore[import-untyped]
import requests
import agenda.add_new_conference
import agenda.conference_list
import agenda.mail
import agenda.utils
logger = logging.getLogger(__name__)
AGENDA_BASE_URL = "https://edwardbetts.com/agenda"
@dataclass(frozen=True)
class GeomobEvent:
"""Geomob event."""
date: date
href: str
hashtag: str
def extract_events(
tree: lxml.html.HtmlElement,
) -> List[GeomobEvent]:
"""Extract upcoming events from the HTML content."""
events = []
for event in tree.xpath('//ol[@class="event-list"]/li/a'):
date_str, _, hashtag = event.text_content().strip().rpartition(" ")
events.append(
GeomobEvent(
date=dateutil.parser.parse(date_str).date(),
href=event.get("href"),
hashtag=hashtag,
)
)
return events
def find_new_events(
prev: list[GeomobEvent], cur: list[GeomobEvent]
) -> list[GeomobEvent]:
"""Find new events that appear in cur but not in prev."""
return list(set(cur) - set(prev))
def geomob_email(
new_events: list[GeomobEvent],
base_url: str,
recording_results: dict[GeomobEvent, str] | None = None,
conference_links: dict[GeomobEvent, str] | None = None,
) -> tuple[str, str]:
"""Generate email subject and body for new events.
Args:
new_events (List[Event]): List of new events.
base_url (str): The base URL of the website.
Returns:
tuple[str, str]: Email subject and body.
"""
assert new_events
subject = f"{len(new_events)} New Geomob Event(s) Announced"
body_lines = ["Hello,\n", "Here are the new Geomob events:\n"]
for event in new_events:
url = base_url + event.href
# Check for double slashes in the path part only (after protocol)
if "://" in url:
protocol, rest = url.split("://", 1)
assert "//" not in rest, f"Double slash found in URL path: {url}"
else:
assert "//" not in url, f"Double slash found in URL: {url}"
event_details = f"Date: {event.date}\nURL: {url}\nHashtag: {event.hashtag}\n"
if conference_links and event in conference_links:
event_details += f"Conference page: {conference_links[event]}\n"
if recording_results is not None:
event_details += f"Recording: {recording_results[event]}\n"
body_lines.append(event_details)
body_lines.append("-" * 40)
body = "\n".join(body_lines)
return (subject, body)
def conference_page_links(
events: list[GeomobEvent], base_url: str, personal_data: str
) -> dict[GeomobEvent, str]:
"""Find saved events using the same dated identities as the conference pages."""
try:
conferences = agenda.conference_list.build_conference_list(personal_data, [])
except Exception:
logger.warning(
"Could not find conference links for Geomob email", exc_info=True
)
return {}
by_identity = {
(conference.get("url"), conference["start_date"]): conference
for conference in conferences
}
links = {}
for event in events:
conference = by_identity.get((base_url + event.href, event.date))
if conference is not None:
links[event] = (
f"{AGENDA_BASE_URL}/conference/{conference['sort_date'].isoformat()}/{conference['slug']}"
)
return links
def record_event(event: GeomobEvent, base_url: str) -> str:
"""Record an event, reporting failures without suppressing its notification.
Leave failed changes in place for inspection. Only conferences.yaml is staged
and committed, and an initially dirty repository is never modified.
"""
repository = Path("~/src/personal-data").expanduser()
yaml_path = repository / "conferences.yaml"
validator = Path(__file__).resolve().parent.parent / "validate_yaml.py"
stage = "check repository"
recorded = False
def run(command: list[str]) -> str:
result = subprocess.run(
command,
cwd=repository,
capture_output=True,
text=True,
timeout=300,
check=True,
)
return result.stdout.strip()
try:
if run(["git", "status", "--porcelain"]):
return "FAILED at check repository: uncommitted changes; no steps run."
stage = "git pull --rebase"
run(["git", "pull", "--rebase"])
stage = "add conference"
added = agenda.add_new_conference.add_new_conference(
base_url + event.href, str(yaml_path)
)
recorded = True
stage = "validate_yaml.py"
run(["python3", str(validator)])
if added:
stage = "git add conferences.yaml"
run(["git", "add", "--", "conferences.yaml"])
stage = "git commit"
run(
[
"git",
"commit",
"--only",
"-m",
f"Add {event.hashtag} event on {event.date}",
"--",
"conferences.yaml",
]
)
stage = "git push"
run(["git", "push"])
return (
"Recorded in conferences.yaml, validated, committed and pushed."
if added
else "Already recorded in conferences.yaml; validated and pushed."
)
except Exception as exc:
details = str(exc)
if isinstance(exc, subprocess.CalledProcessError):
details += f"\n{exc.stdout or ''}\n{exc.stderr or ''}"
elif isinstance(exc, subprocess.TimeoutExpired):
details += f"\n{exc.stdout!r}\n{exc.stderr!r}"
state = "Recorded locally; " if recorded else ""
return f"{state}FAILED at {stage}: {details.strip()}"
def get_cached_upcoming_events_list(geomob_dir: str) -> list[GeomobEvent]:
"""Get known geomob events."""
filename = agenda.utils.get_most_recent_file(geomob_dir, "html")
return extract_events(lxml.html.parse(filename).getroot()) if filename else []
def update(config: flask.config.Config) -> None:
"""Get upcoming Geomob events and report new ones."""
geomob_dir = os.path.join(config["DATA_DIR"], "geomob")
prev_events = get_cached_upcoming_events_list(geomob_dir)
r = requests.get("https://thegeomob.com/")
cur_events = extract_events(lxml.html.fromstring(r.content))
if cur_events == prev_events:
return # no change
now = datetime.now()
new_filename = os.path.join(geomob_dir, now.strftime("%Y-%m-%d_%H:%M:%S.html"))
open(new_filename, "w").write(r.text)
new_events = list(set(cur_events) - set(prev_events))
if not new_events:
return
base_url = "https://thegeomob.com"
recording_results = {event: record_event(event, base_url) for event in new_events}
links = (
conference_page_links(new_events, base_url, config["PERSONAL_DATA"])
if config.get("PERSONAL_DATA")
else {}
)
subject, body = geomob_email(new_events, base_url, recording_results, links)
agenda.mail.send_mail(config, subject, body)

417
agenda/google_flights.py
View file

@ -1,417 +0,0 @@
"""Read Google Flights in Chromium via Playwright, without a flight API client.
Query schema and embedded result format documented by AWeirdDev/fast-flights.
See licenses/fast-flights-MIT.txt. Google's undocumented format may change.
"""
import base64
import contextvars
import json
import logging
import os
import re
import shutil
import typing
from contextlib import contextmanager
from datetime import date, datetime
from pathlib import Path
from urllib.parse import urlencode, urlsplit
from playwright.sync_api import (
BrowserContext,
Page,
Playwright,
Response,
sync_playwright,
)
from agenda import flight_diagnostics, flight_search_cache
from agenda.types import StrDict
logger = logging.getLogger(__name__)
PARSER_VERSION = 2
LONDON_AIRPORTS = ("LHR", "LGW", "STN", "LTN", "LCY", "SEN")
class FlightDateUnavailableError(ValueError):
"""Google explicitly rejected a search beyond its available date range."""
def date_limit_message(text: str) -> str | None:
"""Recognize a provider date-range error without assuming a fixed horizon."""
normalized_text = " ".join(text.split())
if re.search(
r"(?:date|flight|departure|return)[^.]{0,160}too far[^.]{0,80}future",
normalized_text,
re.IGNORECASE,
):
return "Requested flight date is too far in the future. Please try again closer to departure."
return None
def chromium_executable() -> str:
"""Avoid Debian's shell wrapper: Apache hides its /proc/cpuinfo probe."""
binary = Path("/usr/lib/chromium/chromium")
if binary.is_file() and os.access(binary, os.X_OK):
return str(binary)
executable = shutil.which("chromium")
if executable is None:
raise RuntimeError("Chromium is not installed")
return executable
def varint(value: int) -> bytes:
"""Encode an unsigned protobuf integer."""
result = bytearray()
while value > 127:
result.append((value & 127) | 128)
value >>= 7
result.append(value)
return bytes(result)
def field(number: int, value: str | bytes | int) -> bytes:
"""Encode the small subset of protobuf needed by a search URL."""
if isinstance(value, int):
return varint(number << 3) + varint(value)
data = value.encode() if isinstance(value, str) else value
return varint((number << 3) | 2) + varint(len(data)) + data
def search_segment(origin: str, destination: str, day: date, max_stops: int) -> bytes:
"""Encode airports, departure date and stop limit for one direction."""
segment = field(2, day.isoformat()) + field(5, max_stops)
for number, code in ((13, origin), (14, destination)):
for member in code.split(","):
for airport in LONDON_AIRPORTS if member == "LON" else (member,):
segment += field(number, field(1, 1) + field(2, airport))
return segment
def search_url(
origin: str,
destination: str,
day: date,
max_stops: int,
return_day: date | None = None,
) -> str:
"""One adult, economy, British locale; one-way or a prefilled return trip."""
info = field(3, search_segment(origin, destination, day, max_stops))
if return_day is not None:
info += field(3, search_segment(destination, origin, return_day, max_stops))
info += field(8, 1) + field(9, 1) + field(19, 1 if return_day is not None else 2)
return "https://www.google.com/travel/flights/search?" + urlencode(
{
"tfs": base64.urlsafe_b64encode(info).decode().rstrip("="),
"tfu": "EgQIABABIgA",
"hl": "en-GB",
"gl": "GB",
"curr": "GBP",
}
)
def timestamp(values: list[int], clock: list[int | None] | None) -> str:
"""Google omits zero hours/minutes; preserve midnight and overnight dates."""
padded = [*(clock or []), None, None]
return datetime(
values[0], values[1], values[2], padded[0] or 0, padded[1] or 0
).isoformat()
def parse_results(script: str, url: str) -> list[StrDict]:
"""Read Google's best flights followed by its remaining itineraries."""
marker = re.search(r"\bdata\s*:\s*", script)
if marker is None:
raise ValueError("Google Flights page did not contain flight data")
payload, _ = json.JSONDecoder().raw_decode(script[marker.end() :])
if not isinstance(payload, list) or len(payload) < 4:
message = date_limit_message(json.dumps(payload))
if message:
raise FlightDateUnavailableError(message)
raise ValueError("Unrecognized Google Flights result format")
airlines: dict[str, str] = {}
try:
airlines = dict(payload[7][1][1])
except (IndexError, TypeError, ValueError):
pass
rows: list[StrDict] = []
malformed = 0
for section in payload[2:4]:
if not section:
continue
for item in section[0] or []:
try:
itinerary = item[0]
raw_legs = itinerary[2]
if not raw_legs:
continue
legs = []
for leg in raw_legs:
airline = leg[22]
code = str(airline[0])
operating = airline[2] if len(airline) > 2 else None
legs.append(
{
"airline": airlines.get(code, code),
"airline_code": code,
"operating_airline_code": operating or code,
"number": f"{code} {airline[1]}",
"origin": leg[3],
"destination": leg[6],
}
)
fare = item[1][0] if item[1] else []
price = fare[1] if len(fare) > 1 else None
rows.append(
{
"price": price,
"currency": "GBP",
"stops": len(legs) - 1,
"duration": itinerary[9],
"departure": timestamp(raw_legs[0][20], raw_legs[0][8]),
"arrival": timestamp(raw_legs[-1][21], raw_legs[-1][10]),
"legs": legs,
"url": url,
}
)
except (IndexError, TypeError, ValueError) as exc:
malformed += 1
if not rows:
last_error = exc
if malformed and not rows:
raise ValueError("Could not decode Google Flights itineraries") from last_error
return rows
class BrowserSearch:
"""One lazy browser session shared by the dates in a conference lookup."""
def __init__(self, directory: Path) -> None:
self.directory = directory
self.playwright: Playwright | None = None
self.context: BrowserContext | None = None
self.page: Page | None = None
self.rate_limited = False
self.response: Response | None = None
self.error_responses: list[Response] = []
self.flight_script: str | None = None
def start(self) -> Page:
"""Use installed Chromium, including when running as Apache's www-data."""
if self.page is None:
executable = chromium_executable()
self.playwright = sync_playwright().start()
self.context = self.playwright.chromium.launch_persistent_context(
str(self.directory / "browser-profile"),
executable_path=executable,
headless=True,
locale="en-GB",
timezone_id="Europe/London",
args=["--disable-dev-shm-usage"],
env={
**os.environ,
"XDG_CONFIG_HOME": str(self.directory / "browser-config"),
"XDG_CACHE_HOME": str(self.directory / "browser-cache"),
},
)
self.page = self.context.new_page()
self.page.on("response", self.check_response)
self.page.set_default_timeout(20000)
return self.page
def check_response(self, response: Response) -> None:
"""Observe rate limits on flight data requests as well as navigation."""
host = urlsplit(response.url).hostname or ""
is_google = host == "google.com" or host.endswith(".google.com")
if response.status >= 400 and is_google and len(self.error_responses) < 5:
self.error_responses.append(response)
if response.status == 429 and is_google:
if not self.rate_limited:
flight_search_cache.block_requests(self.directory)
self.rate_limited = True
def check_cooldown(self) -> None:
"""Surface a subrequest's 429 instead of reporting missing flight data."""
if self.rate_limited:
raise flight_search_cache.RateLimitCooldownError(
flight_search_cache.cooldown_message(self.directory)
)
def search(
self, origin: str, destination: str, day: date, max_stops: int
) -> list[StrDict]:
"""Pace searches and capture failed responses without extra Google requests."""
flight_search_cache.wait_for_request(self.directory)
self.response = None
self.error_responses = []
self.flight_script = None
try:
return self.perform_search(origin, destination, day, max_stops)
except Exception as exc:
try:
self.record_failure(origin, destination, day, max_stops, exc)
except Exception:
logger.warning(
"Unable to capture flight failure diagnostics", exc_info=True
)
raise
def perform_search(
self, origin: str, destination: str, day: date, max_stops: int
) -> list[StrDict]:
"""Navigate once, honour consent, and stop immediately on rate limiting."""
page = self.start()
url = search_url(origin, destination, day, max_stops)
response = page.goto(url, wait_until="domcontentloaded", timeout=45000)
self.response = response
if response is not None and response.status == 429:
flight_search_cache.block_requests(self.directory)
raise flight_search_cache.RateLimitCooldownError(
flight_search_cache.cooldown_message(self.directory)
)
self.check_cooldown()
if response is not None and response.status >= 400:
raise RuntimeError(f"Google Flights returned HTTP {response.status}")
if "consent.google." in page.url:
page.get_by_role("button", name="Reject all", exact=True).click()
page.wait_for_url("**/travel/flights/**", timeout=20000)
script = page.locator("script.ds\\:1")
try:
script.wait_for(state="attached", timeout=20000)
except Exception as exc:
self.check_cooldown()
text = " ".join(page.locator("body").inner_text().split())[:400]
raise RuntimeError(
f"Google Flights did not load results ({page.url}): {text}"
) from exc
self.check_cooldown()
self.flight_script = script.inner_text()
return self.read_results(page, self.flight_script, url, day)
def read_results(
self, page: Page, script: str, url: str, day: date
) -> list[StrDict]:
"""Interpret provider messages when the embedded data isn't a result list."""
try:
return parse_results(script, url)
except FlightDateUnavailableError:
raise
except ValueError as exc:
text = " ".join(page.locator("body").inner_text().split())
message = date_limit_message(text)
if message:
raise FlightDateUnavailableError(message) from None
if re.search(
r"\b(?:no (?:(?:non[-\u2010-\u2015 ]?stop|direct) )?flights found|no results found|no matching flights)\b",
text,
re.IGNORECASE,
):
return []
ahead = (day - date.today()).days
context = f"Google Flights returned unusable search data for {day.strftime('%a %-d %b %Y')} ({ahead} days ahead)."
if ahead > 300:
context += " This date may be beyond the available airline schedules; try again closer to departure."
raise ValueError(context) from exc
def record_failure(
self, origin: str, destination: str, day: date, max_stops: int, exc: Exception
) -> None:
"""Capture already-loaded data; unavailable page fields are recorded explicitly."""
from agenda.conference_flights import error_detail
details: StrDict = {
"origin": origin,
"destination": destination,
"departure_date": day.isoformat(),
"max_stops": max_stops,
"currency": "GBP",
"language": "en-GB",
"country": "GB",
"requested_url": search_url(origin, destination, day, max_stops),
"error": error_detail(exc),
"flight_script": self.flight_script,
"transport": flight_search_cache.TRANSPORT,
"parser_version": PARSER_VERSION,
}
capture_errors: list[str] = []
if self.page is not None:
captures: tuple[tuple[str, typing.Callable[[], str]], ...] = (
("page_url", lambda: self.page.url if self.page else ""),
("page_html", lambda: self.page.content() if self.page else ""),
(
"page_text",
lambda: (
self.page.locator("body").inner_text(timeout=2000)
if self.page
else ""
),
),
)
for field, fetch in captures:
try:
details[field] = fetch()
except Exception as capture_error:
capture_errors.append(
f"{field}: {type(capture_error).__name__}: {capture_error}"
)
responses = list(self.error_responses[:5])
if self.response is not None:
details["http_status"] = self.response.status
if self.response.status >= 400 and self.response not in responses:
responses = [self.response, *responses][:5]
errors = []
for response in responses:
try:
errors.append(
{
"url": response.url,
"status": response.status,
"body": response.text(),
}
)
except Exception as capture_error:
capture_errors.append(
f"response body: {type(capture_error).__name__}: {capture_error}"
)
details["http_errors"] = errors
details["capture_errors"] = capture_errors
flight_diagnostics.write_failure(self.directory, details)
def close(self) -> None:
"""Always release Chromium, including on failed searches."""
try:
if self.context is not None:
self.context.close()
finally:
if self.playwright is not None:
self.playwright.stop()
session: contextvars.ContextVar[BrowserSearch | None] = contextvars.ContextVar(
"conference_browser_session", default=None
)
@contextmanager
def use_browser(directory: Path) -> typing.Iterator[None]:
"""Lazy initialization keeps cache hits free of browser startup costs."""
browser = BrowserSearch(directory)
token = session.set(browser)
try:
yield
finally:
session.reset(token)
browser.close()
def search(origin: str, destination: str, day: date, max_stops: int) -> list[StrDict]:
"""Attach the route/date to browser failures for display on the page."""
browser = session.get()
if browser is None:
raise RuntimeError("Flight searches require a conference browser session")
try:
return browser.search(origin, destination, day, max_stops)
except Exception as exc:
raise RuntimeError(f"{origin} → {destination} on {day.isoformat()}") from exc

47
agenda/gwr.py
View file

@ -3,7 +3,6 @@
import os
import re
from datetime import date, datetime
from html import unescape
from time import time
import httpx
@ -11,31 +10,6 @@ import httpx
url = "https://www.gwr.com/your-tickets/choosing-your-ticket/advance-tickets"
def parse_date_string(date_str: str) -> date:
"""Parse date string from HTML."""
date_str = " ".join(unescape(date_str).split())
if not date_str[-1].isdigit(): # If the year is missing, use the current year
date_str += f" {date.today().year}"
return datetime.strptime(date_str, "%A %d %B %Y").date()
def extract_dates(html: str) -> None | dict[str, date]:
"""Extract dates from HTML."""
pattern = re.compile(
r"<tr>\s*<td>(Weekdays|Saturdays|Sundays)</td>*"
+ r"\s*<td>(.*?)(?:\*\*)?</td>\s*</tr>",
)
if not pattern.search(html):
return None
return {
match.group(1): parse_date_string(match.group(2))
for match in pattern.finditer(html)
}
def extract_weekday_date(html: str) -> date | None:
"""Furthest date of GWR advance ticket booking."""
# Compile a regular expression pattern to match the relevant table row
@ -44,19 +18,22 @@ def extract_weekday_date(html: str) -> date | None:
)
# Search the HTML for the pattern
if match := pattern.search(html):
return parse_date_string(match.group(1))
else:
if not (match := pattern.search(html)):
return None
date_str = match.group(1)
# If the year is missing, use the current year
if not date_str[-1].isdigit():
date_str += f" {date.today().year}"
return datetime.strptime(date_str, "%A %d %B %Y").date()
async def advance_tickets_page_html(
data_dir: str, ttl: int = 60 * 60 * 6, force_cache: bool = False
) -> str:
async def advance_tickets_page_html(data_dir: str, ttl: int = 60 * 60 * 6) -> str:
"""Get advance-tickets web page HTML with cache."""
filename = os.path.join(data_dir, "advance-tickets.html")
mtime = os.path.getmtime(filename) if os.path.exists(filename) else 0
if force_cache or (time() - mtime) < ttl: # use cache
if (time() - mtime) < ttl: # use cache
return open(filename).read()
async with httpx.AsyncClient() as client:
r = await client.get(url)
@ -65,7 +42,7 @@ async def advance_tickets_page_html(
return html
async def advance_ticket_date(data_dir: str, force_cache: bool = False) -> date | None:
async def advance_ticket_date(data_dir: str) -> date | None:
"""Get GWR advance tickets date with cache."""
html = await advance_tickets_page_html(data_dir, force_cache=force_cache)
html = await advance_tickets_page_html(data_dir)
return extract_weekday_date(html)

2
agenda/hn.py
View file

@ -5,7 +5,7 @@ from datetime import date, datetime, time, timedelta
import pytz
from dateutil.relativedelta import relativedelta
from .event import Event
from .types import Event
eastern_time = pytz.timezone("America/New_York")

211
agenda/holidays.py
View file

@ -1,211 +0,0 @@
"""Holidays."""
import collections
from datetime import date, timedelta
import flask
import agenda.uk_holiday
import holidays
from agenda.uk_school_holiday import school_holiday_list
from .event import Event
from .types import Holiday, Trip
def get_trip_holidays(trip: Trip) -> list[Holiday]:
"""Get holidays happening during trip."""
if not trip.end:
return []
countries = {"GB", *(c.alpha_2 for c in trip.countries)}
return sorted(
(
hol
for hol in get_all(
trip.start, trip.end, flask.current_app.config["DATA_DIR"]
)
if hol.country.upper() in countries
),
key=lambda item: (item.date, item.country),
)
def get_school_holidays(start_date: date, end_date: date, data_dir: str) -> list[Event]:
"""Get UK school holidays from cache."""
return school_holiday_list(start_date, end_date, data_dir)
def get_trip_school_holidays(trip: Trip) -> list[Event]:
"""Get UK school holidays happening during trip."""
if not trip.end:
return []
return get_school_holidays(
trip.start,
trip.end,
flask.current_app.config["DATA_DIR"],
)
def us_holidays(start_date: date, end_date: date) -> list[Holiday]:
"""Get US holidays."""
found: list[Holiday] = []
for year in range(start_date.year, end_date.year + 1):
hols = holidays.country_holidays("US", years=year, language="en")
found += [
Holiday(date=hol_date, name=title, country="us")
for hol_date, title in hols.items()
if start_date < hol_date < end_date
]
extra = []
for h in found:
if h.name != "Thanksgiving":
continue
extra += [
Holiday(date=h.date + timedelta(days=1), name="Black Friday", country="us"),
Holiday(date=h.date + timedelta(days=4), name="Cyber Monday", country="us"),
]
return found + extra
def get_nyse_holidays(
start_date: date, end_date: date, us_hols: list[Holiday]
) -> list[Event]:
"""NYSE holidays."""
known_us_hols = {(h.date, h.name) for h in us_hols}
found: list[Event] = []
rename = {"Thanksgiving Day": "Thanksgiving"}
for year in range(start_date.year, end_date.year + 1):
hols = holidays.financial_holidays("NYSE", years=year)
found += [
Event(
name="holiday",
date=hol_date,
title=rename.get(title, title),
)
for hol_date, title in hols.items()
if start_date <= hol_date <= end_date
]
found = [hol for hol in found if (hol.date, hol.title) not in known_us_hols]
for hol in found:
assert hol.title
hol.title += " (NYSE)"
return found
def get_holidays(country: str, start_date: date, end_date: date) -> list[Holiday]:
"""Get holidays."""
found: list[Holiday] = []
uc_country = country.upper()
holiday_country = getattr(holidays, uc_country)
default_language = holiday_country.default_language
def skip_holiday(holiday_name: str) -> bool:
"""Skip holiday."""
return country == "se" and holiday_name == "Sunday"
for year in range(start_date.year, end_date.year + 1):
en_hols = holidays.country_holidays(uc_country, years=year, language="en_US")
local_lang = holidays.country_holidays(
uc_country, years=year, language=default_language
)
found += [
Holiday(
date=hol_date,
name=title,
local_name=local_lang[hol_date],
country=country.lower(),
)
for hol_date, title in en_hols.items()
if start_date <= hol_date <= end_date and not skip_holiday(title)
]
return found
def combine_holidays(holidays: list[Holiday]) -> list[Event]:
"""Combine UK and US holidays with the same date and title."""
all_countries = {h.country for h in holidays}
standard_name = {
(1, 1): "New Year's Day",
(1, 6): "Epiphany",
(5, 1): "Labour Day",
(8, 15): "Assumption Day",
(12, 8): "Immaculate conception",
(12, 25): "Christmas Day",
(12, 26): "Boxing Day",
}
combined: collections.defaultdict[tuple[date, str], set[str]] = (
collections.defaultdict(set)
)
for h in holidays:
assert isinstance(h.name, str) and isinstance(h.date, date)
event_key = (h.date, standard_name.get((h.date.month, h.date.day), h.name))
combined[event_key].add(h.country)
events: list[Event] = []
for (d, name), countries in combined.items():
if len(countries) == len(all_countries):
country_list = ""
elif len(countries) < len(all_countries) / 2:
country_list = ", ".join(sorted(country.upper() for country in countries))
else:
country_list = "not " + ", ".join(
sorted(country.upper() for country in all_countries - set(countries))
)
e = Event(
name="holiday",
date=d,
title=f"{name} ({country_list})" if country_list else name,
)
events.append(e)
return events
def get_all(last_year: date, next_year: date, data_dir: str) -> list[Holiday]:
"""Get holidays for various countries and return as a list."""
us_hols = us_holidays(last_year, next_year)
bank_holidays = agenda.uk_holiday.bank_holiday_list(last_year, next_year, data_dir)
holiday_list: list[Holiday] = bank_holidays + us_hols
for country in flask.current_app.config["HOLIDAY_COUNTRIES"]:
holiday_list += get_holidays(country, last_year, next_year)
return holiday_list
def trip_school_holiday_map(trips: list[Trip], data_dir: str) -> dict[str, list[Event]]:
"""Map trip-start ISO date to overlapping UK school holidays."""
if not trips:
return {}
starts = [trip.start for trip in trips]
ends = [trip.end or trip.start for trip in trips]
school_holidays = get_school_holidays(
min(starts),
max(ends),
data_dir,
)
result: dict[str, list[Event]] = {}
for trip in trips:
trip_end = trip.end or trip.start
overlaps = [
school_holiday
for school_holiday in school_holidays
if school_holiday.as_date <= trip_end
and school_holiday.end_as_date >= trip.start
]
result[trip.start.isoformat()] = overlaps
return result

53
agenda/ical.py
View file

@ -1,53 +0,0 @@
"""Shared helpers for generating iCalendar feeds."""
from __future__ import annotations
from datetime import date, datetime, timezone
from typing import Iterable
def escape_text(value: str) -> str:
"""Escape text for safer ICS output."""
return (
value.replace("\\", "\\\\")
.replace(";", "\\;")
.replace(",", "\\,")
.replace("\n", "\\n")
)
def _fold_line(value: str) -> Iterable[str]:
"""Yield RFC5545 folded lines."""
if len(value) <= 75:
yield value
return
remaining = value
first = True
while remaining:
segment = remaining[:75]
remaining = remaining[75:]
if not first:
segment = " " + segment
yield segment
first = False
def append_property(lines: list[str], name: str, value: str) -> None:
"""Append a folded property line to the ICS output."""
for line in _fold_line(f"{name}:{value}"):
lines.append(line)
def format_datetime_utc(dt_value: datetime) -> str:
"""Return datetime formatted in UTC for ICS."""
if dt_value.tzinfo is None:
dt_value = dt_value.replace(tzinfo=timezone.utc)
else:
dt_value = dt_value.astimezone(timezone.utc)
return dt_value.strftime("%Y%m%dT%H%M%SZ")
def format_date(date_value: date) -> str:
"""Return date formatted for all-day ICS events."""
return date_value.strftime("%Y%m%d")

28
agenda/mail.py
View file

@ -1,28 +0,0 @@
"""Send e-mail."""
import smtplib
from email.message import EmailMessage
from email.utils import formatdate, make_msgid
import flask
def send_mail(config: flask.config.Config, subject: str, body: str) -> None:
"""Send an e-mail."""
msg = EmailMessage()
msg["Subject"] = subject
msg["To"] = f"{config['NAME']} <{config['MAIL_TO']}>"
msg["From"] = f"{config['NAME']} <{config['MAIL_FROM']}>"
msg["Date"] = formatdate()
msg["Message-ID"] = make_msgid()
# Add extra mail headers
for header, value in config["MAIL_HEADERS"]:
msg[header] = value
msg.set_content(body)
s = smtplib.SMTP(config["SMTP_HOST"])
s.sendmail(config["MAIL_TO"], [config["MAIL_TO"]], msg.as_string())
s.quit()

4
agenda/meetup.py
View file

@ -4,7 +4,7 @@ import json
import os.path
from datetime import datetime
from .event import Event
from .types import Event
def get_events(data_dir: str) -> list[Event]:
@ -21,7 +21,7 @@ def get_events(data_dir: str) -> list[Event]:
date=start,
end_date=end,
name="meetup",
title=item_event["title"],
title="👥" + item_event["title"],
url=item_event["eventUrl"],
)
events.append(e)

270
agenda/meteors.py
View file

@ -1,270 +0,0 @@
"""Meteor shower data calculations."""
import typing
from datetime import datetime, timedelta
import ephem # type: ignore
MeteorShower = dict[str, typing.Any]
# Meteor shower definitions with parent comet orbital elements
METEOR_SHOWERS = {
"Quadrantids": {
"name": "Quadrantids",
"radiant_ra": "15h20m", # Right ascension at peak
"radiant_dec": "+49.5°", # Declination at peak
"peak_solar_longitude": 283.16, # Solar longitude at peak
"activity_start": 283.16 - 10, # Activity period
"activity_end": 283.16 + 10,
"rate_max": 120,
"rate_min": 50,
"parent_body": "2003 EH1",
"visibility": "Northern Hemisphere",
"description": "The year kicks off with the Quadrantids, known for their brief but intense peak lasting only about 4 hours.",
"velocity_kms": 41,
},
"Lyrids": {
"name": "Lyrids",
"radiant_ra": "18h04m",
"radiant_dec": "+32.32°",
"peak_solar_longitude": 32.32,
"activity_start": 32.32 - 10,
"activity_end": 32.32 + 10,
"rate_max": 18,
"rate_min": 10,
"parent_body": "C/1861 G1 (Thatcher)",
"visibility": "Both hemispheres",
"description": "The Lyrids are one of the oldest recorded meteor showers, with observations dating back 2,700 years.",
"velocity_kms": 49,
},
"Eta Aquariids": {
"name": "Eta Aquariids",
"radiant_ra": "22h32m",
"radiant_dec": "-1.0°",
"peak_solar_longitude": 45.5,
"activity_start": 45.5 - 15,
"activity_end": 45.5 + 15,
"rate_max": 60,
"rate_min": 30,
"parent_body": "1P/Halley",
"visibility": "Southern Hemisphere (best)",
"description": "Created by debris from Halley's Comet, these meteors are fast and often leave glowing trails.",
"velocity_kms": 66,
},
"Perseids": {
"name": "Perseids",
"radiant_ra": "03h04m",
"radiant_dec": "+58.0°",
"peak_solar_longitude": 140.0,
"activity_start": 140.0 - 15,
"activity_end": 140.0 + 15,
"rate_max": 100,
"rate_min": 50,
"parent_body": "109P/Swift-Tuttle",
"visibility": "Northern Hemisphere",
"description": "One of the most popular meteor showers, viewing conditions vary by year based on moon phase.",
"velocity_kms": 59,
},
"Orionids": {
"name": "Orionids",
"radiant_ra": "06h20m",
"radiant_dec": "+16.0°",
"peak_solar_longitude": 208.0,
"activity_start": 208.0 - 15,
"activity_end": 208.0 + 15,
"rate_max": 25,
"rate_min": 15,
"parent_body": "1P/Halley",
"visibility": "Both hemispheres",
"description": "Another shower created by Halley's Comet debris, known for their speed and brightness.",
"velocity_kms": 66,
},
"Geminids": {
"name": "Geminids",
"radiant_ra": "07h28m",
"radiant_dec": "+32.0°",
"peak_solar_longitude": 262.2,
"activity_start": 262.2 - 10,
"activity_end": 262.2 + 10,
"rate_max": 120,
"rate_min": 60,
"parent_body": "3200 Phaethon",
"visibility": "Both hemispheres",
"description": "The best shower of most years with the highest rates. Unusual for being caused by an asteroid rather than a comet.",
"velocity_kms": 35,
},
"Ursids": {
"name": "Ursids",
"radiant_ra": "14h28m",
"radiant_dec": "+75.0°",
"peak_solar_longitude": 270.7,
"activity_start": 270.7 - 5,
"activity_end": 270.7 + 5,
"rate_max": 10,
"rate_min": 5,
"parent_body": "8P/Tuttle",
"visibility": "Northern Hemisphere",
"description": "A minor shower that closes out the year, best viewed from dark locations away from city lights.",
"velocity_kms": 33,
},
}
def calculate_solar_longitude_date(year: int, target_longitude: float) -> datetime:
"""Calculate the date when the Sun reaches a specific longitude for a given year."""
# Start from beginning of year
start_date = datetime(year, 1, 1)
# Use PyEphem to calculate solar longitude
observer = ephem.Observer()
observer.lat = "0" # Equator
observer.lon = "0" # Greenwich
observer.date = start_date
sun = ephem.Sun(observer)
# Search for the date when solar longitude matches target
# Solar longitude 0° = Spring Equinox (around March 20)
# We need to find when sun reaches the target longitude
# Approximate: start search from reasonable date based on longitude
if target_longitude < 90: # Spring (Mar-Jun)
search_start = datetime(year, 3, 1)
elif target_longitude < 180: # Summer (Jun-Sep)
search_start = datetime(year, 6, 1)
elif target_longitude < 270: # Fall (Sep-Dec)
search_start = datetime(year, 9, 1)
else: # Winter (Dec-Mar)
search_start = datetime(year, 12, 1)
observer.date = search_start
# Search within a reasonable range (±60 days)
for day_offset in range(-60, 61):
test_date = search_start + timedelta(days=day_offset)
observer.date = test_date
sun.compute(observer)
# Convert ecliptic longitude to degrees
sun_longitude = float(sun.hlon) * 180 / ephem.pi
# Check if we're close to target longitude (within 0.5 degrees)
if abs(sun_longitude - target_longitude) < 0.5:
return test_date
# Fallback: return approximation based on solar longitude
# Rough approximation: solar longitude increases ~1° per day
days_from_equinox = target_longitude
equinox_date = datetime(year, 3, 20) # Approximate spring equinox
return equinox_date + timedelta(days=days_from_equinox)
def calculate_moon_phase(date_obj: datetime) -> tuple[float, str]:
"""Calculate moon phase for a given date."""
observer = ephem.Observer()
observer.date = date_obj
moon = ephem.Moon(observer)
moon.compute(observer)
# Moon phase (0 = new moon, 0.5 = full moon, 1 = new moon again)
phase = moon.phase / 100.0
# Determine moon phase name and viewing quality
if phase < 0.1:
phase_name = "New Moon"
viewing_quality = "excellent"
elif phase < 0.3:
phase_name = "Waxing Crescent"
viewing_quality = "good"
elif phase < 0.7:
phase_name = "First Quarter" if phase < 0.5 else "Waxing Gibbous"
viewing_quality = "moderate"
elif phase < 0.9:
phase_name = "Full Moon"
viewing_quality = "poor"
else:
phase_name = "Waning Crescent"
viewing_quality = "good"
return phase, f"{phase_name} ({viewing_quality} viewing)"
def calculate_meteor_shower_data(year: int) -> list[MeteorShower]:
"""Calculate meteor shower data for a given year using astronomical calculations."""
meteor_data = []
for shower_id, shower_info in METEOR_SHOWERS.items():
# Calculate peak date based on solar longitude
peak_longitude = shower_info["peak_solar_longitude"]
assert isinstance(peak_longitude, (int, float))
peak_date = calculate_solar_longitude_date(year, peak_longitude)
# Calculate activity period
start_longitude = shower_info["activity_start"]
assert isinstance(start_longitude, (int, float))
activity_start = calculate_solar_longitude_date(year, start_longitude)
end_longitude = shower_info["activity_end"]
assert isinstance(end_longitude, (int, float))
activity_end = calculate_solar_longitude_date(year, end_longitude)
# Calculate moon phase at peak
moon_illumination, moon_phase_desc = calculate_moon_phase(peak_date)
# Format dates
peak_formatted = peak_date.strftime("%B %d")
if peak_date.day != (peak_date + timedelta(days=1)).day:
peak_formatted += f"-{(peak_date + timedelta(days=1)).strftime('%d')}"
active_formatted = (
f"{activity_start.strftime('%B %d')} - {activity_end.strftime('%B %d')}"
)
# Determine viewing quality based on moon phase
viewing_quality = (
"excellent"
if moon_illumination < 0.3
else (
"good"
if moon_illumination < 0.7
else "moderate" if moon_illumination < 0.9 else "poor"
)
)
meteor_shower = {
"name": shower_info["name"],
"peak": peak_formatted,
"active": active_formatted,
"rate": f"{shower_info['rate_min']}-{shower_info['rate_max']} meteors per hour",
"radiant": str(shower_info["radiant_ra"]).split("h")[0]
+ "h "
+ str(shower_info["radiant_dec"]),
"moon_phase": moon_phase_desc,
"visibility": shower_info["visibility"],
"description": shower_info["description"],
"peak_date": peak_date.strftime("%Y-%m-%d"),
"start_date": activity_start.strftime("%Y-%m-%d"),
"end_date": activity_end.strftime("%Y-%m-%d"),
"rate_min": shower_info["rate_min"],
"rate_max": shower_info["rate_max"],
"moon_illumination": moon_illumination,
"viewing_quality": viewing_quality,
"parent_body": shower_info["parent_body"],
"velocity_kms": shower_info["velocity_kms"],
}
meteor_data.append(meteor_shower)
# Sort by peak date
meteor_data.sort(key=lambda x: datetime.strptime(str(x["peak_date"]), "%Y-%m-%d"))
return meteor_data
def get_meteor_data(year: int | None = None) -> list[MeteorShower]:
"""Get meteor shower data for a specific year using astronomical calculations."""
if year is None:
year = datetime.now().year
return calculate_meteor_shower_data(year)

93
agenda/n_somerset_waste.py
View file

@ -1,93 +0,0 @@
"""Waste collection schedules."""
import os
import re
from collections import defaultdict
from datetime import date, datetime, time, timedelta
import httpx
import lxml.html
from . import uk_time
from .event import Event
from .utils import make_waste_dir
ttl_hours = 12
async def get_html(
data_dir: str, postcode: str, uprn: str, force_cache: bool = False
) -> str:
"""Get waste schedule."""
now = datetime.now()
waste_dir = os.path.join(data_dir, "waste")
make_waste_dir(data_dir)
existing_data = os.listdir(waste_dir)
existing = [f for f in existing_data if f.endswith(".html")]
if existing:
recent_filename = max(existing)
recent = datetime.strptime(recent_filename, "%Y-%m-%d_%H:%M.html")
delta = now - recent
if existing and (force_cache or delta < timedelta(hours=ttl_hours)):
return open(os.path.join(waste_dir, recent_filename)).read()
now_str = now.strftime("%Y-%m-%d_%H:%M")
filename = f"{waste_dir}/{now_str}.html"
forms_base_url = "https://forms.n-somerset.gov.uk"
url = "https://forms.n-somerset.gov.uk/Waste/CollectionSchedule"
async with httpx.AsyncClient() as client:
r = await client.post(
url,
data={
"PreviousHouse": "",
"PreviousPostcode": "-",
"Postcode": postcode,
"SelectedUprn": uprn,
},
)
form_post_html = r.text
pattern = r'<h2>Object moved to <a href="([^"]*)">here<\/a>\.<\/h2>'
m = re.search(pattern, form_post_html)
if m:
r = await client.get(forms_base_url + m.group(1))
html = r.text
open(filename, "w").write(html)
return html
def parse_waste_schedule_date(day_and_month: str) -> date:
"""Parse waste schedule date."""
today = date.today()
fmt = "%A %d %B %Y"
d = datetime.strptime(f"{day_and_month} {today.year}", fmt).date()
if d < today:
d = datetime.strptime(f"{day_and_month} {today.year + 1}", fmt).date()
return d
def parse(root: lxml.html.HtmlElement) -> list[Event]:
"""Parse waste schedule."""
tbody = root.find(".//table/tbody")
assert tbody is not None
by_date = defaultdict(list)
for e_service, e_next_date, e_following in tbody:
assert e_service.text and e_next_date.text and e_following.text
service = e_service.text
next_date = parse_waste_schedule_date(e_next_date.text)
following_date = parse_waste_schedule_date(e_following.text)
by_date[next_date].append(service)
by_date[following_date].append(service)
return [
Event(
name="waste_schedule",
date=uk_time(d, time(6, 30)),
title="Backwell: " + ", ".join(services),
)
for d, services in by_date.items()
]

299
agenda/schengen.py
View file

@ -1,299 +0,0 @@
"""Schengen area rolling time calculator for travel tracking."""
from datetime import date, datetime, timedelta
from .types import SchengenCalculation, SchengenStay, StrDict
from .utils import depart_datetime
# Schengen Area countries as of 2025
SCHENGEN_COUNTRIES = {
# EU countries in Schengen
"at", # Austria
"be", # Belgium
"bg", # Bulgaria (joined January 2025)
"hr", # Croatia
"cz", # Czech Republic
"dk", # Denmark
"ee", # Estonia
"fi", # Finland
"fr", # France
"de", # Germany
"gr", # Greece
"hu", # Hungary
"it", # Italy
"lv", # Latvia
"lt", # Lithuania
"lu", # Luxembourg
"mt", # Malta
"nl", # Netherlands
"pl", # Poland
"pt", # Portugal
"ro", # Romania (joined January 2025)
"sk", # Slovakia
"si", # Slovenia
"es", # Spain
"se", # Sweden
# Non-EU countries in Schengen
"is", # Iceland
"li", # Liechtenstein
"no", # Norway
"ch", # Switzerland
}
def is_schengen_country(country_code: str) -> bool:
"""Check if a country is in the Schengen area."""
if not country_code or not isinstance(country_code, str):
return False
return country_code.lower() in SCHENGEN_COUNTRIES
def extract_schengen_stays_from_travel(
travel_items: list[StrDict],
) -> list[SchengenStay]:
"""Extract Schengen stays from travel items."""
stays: list[SchengenStay] = []
current_location = None
entry_date = None
# Handle empty travel items
if not travel_items:
return stays
# Sort travel items by departure date, filtering out items without depart date
sorted_items = sorted(
[item for item in travel_items if item.get("depart")],
key=lambda x: depart_datetime(x),
)
for item in sorted_items:
from_country = None
to_country = None
travel_date = item.get("depart")
if not travel_date:
continue
# Extract travel date
if isinstance(travel_date, datetime):
travel_date = travel_date.date()
elif not isinstance(travel_date, date):
# Skip items with invalid travel dates
continue
# Determine origin and destination countries
if item.get("type") == "flight":
from_airport = item.get("from_airport", {})
to_airport = item.get("to_airport", {})
from_country = from_airport.get("country")
to_country = to_airport.get("country")
elif item.get("type") == "train":
from_station = item.get("from_station", {})
to_station = item.get("to_station", {})
from_country = from_station.get("country")
to_country = to_station.get("country")
elif item.get("type") == "ferry":
from_terminal = item.get("from_terminal", {})
to_terminal = item.get("to_terminal", {})
from_country = from_terminal.get("country")
to_country = to_terminal.get("country")
# Handle entering/exiting Schengen
if current_location and is_schengen_country(current_location):
# Currently in Schengen
if to_country and not is_schengen_country(to_country):
# Exiting Schengen - use departure date
if entry_date:
stays.append(
SchengenStay(
entry_date=entry_date,
exit_date=travel_date,
country=current_location,
days=0, # Will be calculated in __post_init__
)
)
entry_date = None
else:
# Currently outside Schengen
if to_country and is_schengen_country(to_country):
# Entering Schengen - use arrival date for long-haul flights
arrive_date = item.get("arrive")
assert arrive_date
if isinstance(arrive_date, datetime):
arrive_date = arrive_date.date()
assert isinstance(arrive_date, date)
entry_date = arrive_date
current_location = to_country
# Handle case where still in Schengen
if current_location and is_schengen_country(current_location) and entry_date:
stays.append(
SchengenStay(
entry_date=entry_date,
exit_date=None, # Still in Schengen
country=current_location,
days=0, # Will be calculated in __post_init__
)
)
return stays
def calculate_schengen_time(
travel_items: list[StrDict], calculation_date: date | None = None
) -> SchengenCalculation:
"""
Calculate Schengen rolling time compliance.
Args:
travel_items: List of travel items from the trip system
calculation_date: Date to calculate from (defaults to today)
Returns:
SchengenCalculation with compliance status and details
"""
if calculation_date is None:
calculation_date = date.today()
# Extract Schengen stays from travel data
stays = extract_schengen_stays_from_travel(travel_items)
# Calculate 180-day window (ending on calculation_date)
window_start = calculation_date - timedelta(days=179)
window_end = calculation_date
# Find stays that overlap with the 180-day window
relevant_stays = []
total_days = 0
for stay in stays:
# Check if stay overlaps with our 180-day window
stay_start = stay.entry_date
stay_end = stay.exit_date or calculation_date
if stay_end >= window_start and stay_start <= window_end:
# Calculate overlap with window
overlap_start = max(stay_start, window_start)
overlap_end = min(stay_end, window_end)
overlap_days = (overlap_end - overlap_start).days + 1
if overlap_days > 0:
# Create a new stay object for the overlapping period
overlapping_stay = SchengenStay(
entry_date=overlap_start,
exit_date=overlap_end if overlap_end != calculation_date else None,
country=stay.country,
days=overlap_days,
)
relevant_stays.append(overlapping_stay)
total_days += overlap_days
# Calculate compliance
is_compliant = total_days <= 90
days_remaining = max(0, 90 - total_days)
# Calculate next reset date (when earliest stay in window expires)
next_reset_date = None
if relevant_stays:
earliest_stay = min(relevant_stays, key=lambda s: s.entry_date)
next_reset_date = earliest_stay.entry_date + timedelta(days=180)
return SchengenCalculation(
total_days_used=total_days,
days_remaining=days_remaining,
is_compliant=is_compliant,
current_180_day_period=(window_start, window_end),
stays_in_period=relevant_stays,
next_reset_date=next_reset_date,
)
def format_schengen_report(calculation: SchengenCalculation) -> str:
"""Format a human-readable Schengen compliance report."""
report = []
# Header
report.append("=== SCHENGEN AREA COMPLIANCE REPORT ===")
report.append(
f"Calculation period: {calculation.current_180_day_period[0]} to {calculation.current_180_day_period[1]}"
)
report.append("")
# Summary
status = "✅ COMPLIANT" if calculation.is_compliant else "❌ NON-COMPLIANT"
report.append(f"Status: {status}")
report.append(f"Days used: {calculation.total_days_used}/90")
if calculation.is_compliant:
report.append(f"Days remaining: {calculation.days_remaining}")
else:
report.append(f"Days over limit: {calculation.days_over_limit}")
if calculation.next_reset_date:
report.append(f"Next reset date: {calculation.next_reset_date}")
report.append("")
# Detailed stays
if calculation.stays_in_period:
report.append("Stays in current 180-day period:")
for stay in calculation.stays_in_period:
exit_str = (
stay.exit_date.strftime("%Y-%m-%d") if stay.exit_date else "ongoing"
)
report.append(
f" • {stay.entry_date.strftime('%Y-%m-%d')} to {exit_str} "
f"({stay.country.upper()}): {stay.days} days"
)
else:
report.append("No Schengen stays in current 180-day period.")
return "\n".join(report)
def get_schengen_countries_list() -> list[str]:
"""Get a list of all Schengen area country codes."""
return sorted(list(SCHENGEN_COUNTRIES))
def predict_future_compliance(
travel_items: list[StrDict],
future_travel: list[tuple[date, date, str]], # (entry_date, exit_date, country)
) -> list[SchengenCalculation]:
"""
Predict future Schengen compliance with planned travel.
Args:
travel_items: Existing travel history
future_travel: List of planned trips as (entry_date, exit_date, country)
Returns:
List of SchengenCalculation objects for each planned trip
"""
predictions = []
# Create mock travel items for future trips
extended_travel = travel_items.copy()
for entry_date, exit_date, country in future_travel:
# Add entry
extended_travel.append(
{"type": "flight", "depart": entry_date, "to_airport": {"country": country}}
)
# Add exit
extended_travel.append(
{
"type": "flight",
"depart": exit_date,
"from_airport": {"country": country},
"to_airport": {"country": "gb"}, # Assuming return to UK
}
)
# Calculate compliance at the exit date
calculation = calculate_schengen_time(extended_travel, exit_date)
predictions.append(calculation)
return predictions

188
agenda/stats.py
View file

@ -1,188 +0,0 @@
"""Trip statistic functions."""
from collections import defaultdict
import typing
from typing import TYPE_CHECKING, Counter, Mapping
import agenda
if TYPE_CHECKING:
from agenda.types import Trip
StrDict = dict[str, typing.Any]
def travel_legs(trip: Trip, stats: StrDict) -> None:
"""Calculate stats for travel legs."""
for leg in trip.travel:
stats.setdefault("co2_kg", 0)
stats.setdefault("co2_by_transport_type", {})
if "co2_kg" in leg:
stats["co2_kg"] += leg["co2_kg"]
transport_type = leg["type"]
stats["co2_by_transport_type"].setdefault(transport_type, 0)
stats["co2_by_transport_type"][transport_type] += leg["co2_kg"]
if leg["type"] == "flight":
from agenda.types import airport_label
stats.setdefault("flight_count", 0)
stats.setdefault("airlines", Counter())
stats.setdefault("airports", Counter())
stats["flight_count"] += 1
stats["airlines"][leg["airline_detail"]["name"]] += 1
for field in ("from_airport", "to_airport"):
airport = leg.get(field)
if airport:
country = agenda.get_country(airport.get("country"))
label = airport_label(airport)
display = f"{country.flag} {label}" if country else label
stats["airports"][display] += 1
if leg["type"] == "train":
stats.setdefault("train_count", 0)
stats["train_count"] += 1
stats.setdefault("stations", Counter())
train_legs = leg.get("legs", [])
if train_legs:
for train_leg in train_legs:
for field in ("from_station", "to_station"):
station = train_leg.get(field)
if station:
country = agenda.get_country(station.get("country"))
label = station["name"]
display = f"{country.flag} {label}" if country else label
stats["stations"][display] += 1
else:
for field in ("from_station", "to_station"):
station = leg.get(field)
if station:
country = agenda.get_country(station.get("country"))
label = station["name"]
display = f"{country.flag} {label}" if country else label
stats["stations"][display] += 1
def conferences(trip: Trip, yearly_stats: Mapping[int, StrDict]) -> None:
"""Calculate conference stats."""
for c in trip.conferences:
yearly_stats[c["start"].year].setdefault("conferences", 0)
yearly_stats[c["start"].year]["conferences"] += 1
def calculate_overall_stats(yearly_stats: dict[int, StrDict]) -> StrDict:
"""Aggregate yearly stats into overall stats for airlines, airports, stations."""
overall: StrDict = {
"airlines": Counter(),
"airports": Counter(),
"stations": Counter(),
"flight_count": 0,
"train_count": 0,
"co2_kg": 0.0,
"co2_by_transport_type": {},
}
for year_stats in yearly_stats.values():
if "airlines" in year_stats:
overall["airlines"] += year_stats["airlines"]
if "airports" in year_stats:
overall["airports"] += year_stats["airports"]
if "stations" in year_stats:
overall["stations"] += year_stats["stations"]
overall["flight_count"] += year_stats.get("flight_count", 0)
overall["train_count"] += year_stats.get("train_count", 0)
overall["co2_kg"] += year_stats.get("co2_kg", 0)
for transport_type, co2_kg in year_stats.get(
"co2_by_transport_type", {}
).items():
overall["co2_by_transport_type"].setdefault(transport_type, 0.0)
overall["co2_by_transport_type"][transport_type] += co2_kg
return overall
def calculate_yearly_stats(
trips: list[Trip], previously_visited: set[str] | None = None
) -> dict[int, StrDict]:
"""Calculate total distance and distance by transport type grouped by year."""
yearly_stats: defaultdict[int, StrDict] = defaultdict(dict)
first_visit_year: dict[str, int] = {}
excluded_new: set[str] = previously_visited or set()
for trip in trips:
year = trip.start.year
for country in trip.countries:
if country.alpha_2 == "GB":
continue
alpha_2 = country.alpha_2
if alpha_2 not in first_visit_year or year < first_visit_year[alpha_2]:
first_visit_year[alpha_2] = year
for trip in trips:
year = trip.start.year
dist = trip.total_distance()
yearly_stats[year].setdefault("count", 0)
yearly_stats[year]["count"] += 1
conferences(trip, yearly_stats)
if dist:
yearly_stats[year]["total_distance"] = (
yearly_stats[year].get("total_distance", 0) + trip.total_distance()
)
for transport_type, distance in trip.distances_by_transport_type():
yearly_stats[year].setdefault("distances_by_transport_type", {})
yearly_stats[year]["distances_by_transport_type"][transport_type] = (
yearly_stats[year]["distances_by_transport_type"].get(transport_type, 0)
+ distance
)
for country in trip.countries:
if country.alpha_2 == "GB":
continue
yearly_stats[year].setdefault("countries", set())
yearly_stats[year]["countries"].add(country)
if (
first_visit_year.get(country.alpha_2) == year
and country.alpha_2 not in excluded_new
):
yearly_stats[year].setdefault("new_countries", set())
yearly_stats[year]["new_countries"].add(country)
travel_legs(trip, yearly_stats[year])
return dict(yearly_stats)
def calc_total_distance(trips: list[Trip]) -> float:
"""Total distance for trips."""
total = 0.0
for item in trips:
if dist := item.total_distance():
total += dist
return total
def calc_total_co2_kg(trips: list[Trip]) -> float:
"""Total CO₂ for trips."""
return sum(item.total_co2_kg() or 0.0 for item in trips)
def sum_distances_by_transport_type(trips: list[Trip]) -> list[tuple[str, float]]:
"""Sum distances by transport type."""
distances_by_transport_type: defaultdict[str, float] = defaultdict(float)
for trip in trips:
for transport_type, dist in trip.distances_by_transport_type():
distances_by_transport_type[transport_type] += dist
return list(distances_by_transport_type.items())
def sum_co2_by_transport_type(trips: list[Trip]) -> list[tuple[str, float]]:
"""Sum CO₂ by transport type."""
co2_by_transport_type: defaultdict[str, float] = defaultdict(float)
for trip in trips:
for transport_type, co2_kg in trip.co2_by_transport_type():
co2_by_transport_type[transport_type] += co2_kg
return list(co2_by_transport_type.items())

24
agenda/stock_market.py
View file

@ -3,14 +3,26 @@
from datetime import timedelta, timezone
import dateutil.tz
import exchange_calendars # type: ignore
import exchange_calendars
import pandas
from . import utils
here = dateutil.tz.tzlocal()
def timedelta_display(delta: timedelta) -> str:
"""Format timedelta as a human readable string."""
total_seconds = int(delta.total_seconds())
days, remainder = divmod(total_seconds, 24 * 60 * 60)
hours, remainder = divmod(remainder, 60 * 60)
mins, secs = divmod(remainder, 60)
return " ".join(
f"{v:>3} {label}"
for v, label in ((days, "days"), (hours, "hrs"), (mins, "mins"))
if v
)
def open_and_close() -> list[str]:
"""Stock markets open and close times."""
# The trading calendars code is slow, maybe there is a faster way to do this
@ -28,11 +40,11 @@ def open_and_close() -> list[str]:
if cal.is_open_on_minute(now_local):
next_close = cal.next_close(now).tz_convert(here)
next_close = next_close.replace(minute=round(next_close.minute, -1))
delta_close = utils.timedelta_display(next_close - now_local)
delta_close = timedelta_display(next_close - now_local)
prev_open = cal.previous_open(now).tz_convert(here)
prev_open = prev_open.replace(minute=round(prev_open.minute, -1))
delta_open = utils.timedelta_display(now_local - prev_open)
delta_open = timedelta_display(now_local - prev_open)
msg = (
f"{label:>6} market opened {delta_open} ago, "
@ -42,7 +54,7 @@ def open_and_close() -> list[str]:
ts = cal.next_open(now)
ts = ts.replace(minute=round(ts.minute, -1))
ts = ts.tz_convert(here)
delta = utils.timedelta_display(ts - now_local)
delta = timedelta_display(ts - now_local)
msg = f"{label:>6} market opens in {delta}" + (
f" ({ts:%H:%M})" if (ts - now_local) < timedelta(days=1) else ""
)

2
agenda/subscription.py
View file

@ -2,7 +2,7 @@
import yaml
from .event import Event
from .types import Event
def get_events(filepath: str) -> list[Event]:

2
agenda/sun.py
View file

@ -3,7 +3,7 @@
import typing
from datetime import datetime
import ephem # type: ignore
import ephem
def bristol() -> ephem.Observer:

509
agenda/thespacedevs.py
View file

@ -1,302 +1,37 @@
"""Get details of upcoming space launches."""
import fcntl
import json
import re
import time
import os
import typing
import tempfile
from datetime import datetime
import requests
from .types import StrDict
from .utils import filename_timestamp
import httpx
Launch = dict[str, typing.Any]
Summary = dict[str, typing.Any]
ttl = 60 * 60 * 2 # two hours
LIMIT = 500
ACTIVE_CREWED_FLIGHTS_CACHE_FILE = "active_crewed_flights.json"
def write_json_cache(filename: str, payload: StrDict) -> None:
"""Publish a complete cache atomically, preserving old data on write failure."""
contents = json.dumps(payload)
temporary: str | None = None
try:
with tempfile.NamedTemporaryFile(
mode="w", dir=os.path.dirname(filename), prefix=".cache-", delete=False
) as cache:
temporary = cache.name
cache.write(contents)
cache.flush()
# The updater and web server run as different users.
os.fchmod(cache.fileno(), 0o644)
os.fsync(cache.fileno())
os.replace(temporary, filename)
finally:
if temporary is not None and os.path.exists(temporary):
os.unlink(temporary)
def read_json_cache(filename: str) -> StrDict | None:
"""Skip empty, unreadable, or damaged cache files."""
try:
with open(filename) as cache:
if os.fstat(cache.fileno()).st_size == 0:
return None
data = json.load(cache)
except (OSError, ValueError):
return None
return data if isinstance(data, dict) else None
def latest_launches_cache(rocket_dir: str) -> tuple[datetime, StrDict] | None:
"""Find the newest usable timestamped launch cache."""
existing = sorted(
(
entry
for f in os.listdir(rocket_dir)
if (entry := filename_timestamp(f, "json")) is not None
),
reverse=True,
)
for updated, filename in existing:
data = read_json_cache(os.path.join(rocket_dir, filename))
if data is not None and isinstance(data.get("results"), list):
return updated, data
return None
class RateLimitDeferred(Exception):
"""The shared API budget or server cooldown requires waiting."""
def api_get(
rocket_dir: str, url: str, params: dict[str, str | int] | None = None
) -> requests.Response:
"""Share a conservative rolling request budget across updater processes."""
# Keep three calls spare for other clients sharing the public IP.
filename = os.path.join(rocket_dir, "api_budget.json")
with open(filename + ".lock", "a") as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
now = time.time()
state = read_json_cache(filename) or {}
calls = [float(t) for t in state.get("calls", []) if float(t) > now - 3600]
retry_at = float(state.get("retry_at", 0))
if now < retry_at or len(calls) >= 12:
raise RateLimitDeferred()
calls.append(now)
# Count attempts even if the connection fails.
write_json_cache(filename, {"calls": calls, "retry_at": retry_at})
response = requests.get(url, params=params, timeout=30)
if response.status_code == 429:
delay = 3600.0
try:
delay = float(response.headers["Retry-After"])
except (KeyError, ValueError):
match = re.search(r"available in (\d+) seconds", response.text)
if match:
delay = float(match[1])
write_json_cache(filename, {"calls": calls, "retry_at": now + delay + 1})
raise RateLimitDeferred()
return response
def next_launch_api_data(rocket_dir: str, limit: int = LIMIT) -> StrDict | None:
async def next_launch_api(rocket_dir: str, limit: int = 200) -> list[Launch]:
"""Get the next upcoming launches from the API."""
now = datetime.now()
filename = os.path.join(rocket_dir, now.strftime("%Y-%m-%d_%H:%M:%S.json"))
url = "https://ll.thespacedevs.com/2.2.0/launch/upcoming/"
params: dict[str, str | int] = {"limit": limit}
r = api_get(rocket_dir, url, params)
try:
data: StrDict = r.json()
except requests.exceptions.JSONDecodeError:
return None
# Only persist valid launch payloads; rate-limit / error responses must not
# overwrite the cache or they become the "most recent" file.
if isinstance(data.get("results"), list):
write_json_cache(filename, data)
return data
def next_launch_api(rocket_dir: str, limit: int = LIMIT) -> list[Summary] | None:
"""Get the next upcoming launches from the API."""
data = next_launch_api_data(rocket_dir, limit)
if not data:
return None
async with httpx.AsyncClient() as client:
r = await client.get(url, params=params)
open(filename, "w").write(r.text)
data = r.json()
return [summarize_launch(launch) for launch in data["results"]]
def parse_api_datetime(value: typing.Any) -> datetime | None:
"""Parse API datetime strings into datetime objects."""
if not isinstance(value, str):
return None
def filename_timestamp(filename: str) -> tuple[datetime, str] | None:
"""Get datetime from filename."""
try:
return datetime.fromisoformat(value.replace("Z", "+00:00"))
ts = datetime.strptime(filename, "%Y-%m-%d_%H:%M:%S.json")
except ValueError:
return None
def is_crewed_spaceflight(flight: Launch) -> bool:
"""Return True when a spaceflight is crewed/human-rated."""
spacecraft_config = get_nested(flight, ["spacecraft", "spacecraft_config"])
if isinstance(spacecraft_config, dict) and spacecraft_config.get("human_rated"):
return True
mission_type = get_nested(flight, ["launch", "mission", "type"])
if isinstance(mission_type, str) and "human" in mission_type.lower():
return True
return False
def is_active_crewed_spaceflight(flight: Launch, now: datetime) -> bool:
"""Return True when a crewed spaceflight is active (mission not yet ended)."""
launch = flight.get("launch")
if not isinstance(launch, dict):
return False
if not is_crewed_spaceflight(flight):
return False
launch_net = parse_api_datetime(launch.get("net"))
if launch_net and launch_net > now:
return False
mission_end = parse_api_datetime(flight.get("mission_end"))
if mission_end and mission_end <= now:
return False
spacecraft_in_space = get_nested(flight, ["spacecraft", "in_space"])
if spacecraft_in_space is True:
return True
if mission_end is None:
landing_success = get_nested(flight, ["landing", "success"])
if landing_success is True:
return False
return True
return True
def active_crewed_flights_api(
rocket_dir: str, limit: int = LIMIT
) -> list[Summary] | None:
"""
Get active crewed spaceflights from the SpaceDevs API.
The API does not reliably expose a direct filter for active flights, so this
paginates through results and applies local filtering.
"""
now = datetime.now().astimezone()
url = "https://ll.thespacedevs.com/2.2.0/spacecraft/flight/"
params: dict[str, str | int] = {"limit": limit}
launches: list[Summary] = []
seen_slugs: set[str] = set()
page = 0
max_pages = 20
while url and page < max_pages:
r = api_get(rocket_dir, url, params if page == 0 else None)
if not r.ok:
return None
try:
data: StrDict = r.json()
except requests.exceptions.JSONDecodeError:
return None
results = data.get("results")
if not isinstance(results, list):
return None
for flight in results:
if not isinstance(flight, dict):
continue
if not is_active_crewed_spaceflight(flight, now):
continue
launch = flight.get("launch")
if not isinstance(launch, dict):
continue
launch_summary = summarize_launch(typing.cast(Launch, launch))
slug = launch_summary.get("slug")
if not isinstance(slug, str):
continue
if slug in seen_slugs:
continue
seen_slugs.add(slug)
launches.append(launch_summary)
next_url = data.get("next")
url = next_url if isinstance(next_url, str) else ""
params = {}
page += 1
# Never replace the cache with an incomplete paginated result.
return None if url else launches
def get_active_crewed_flights_cache_filename(rocket_dir: str) -> str:
"""Path of the active crewed flights cache file."""
return os.path.join(rocket_dir, ACTIVE_CREWED_FLIGHTS_CACHE_FILE)
def load_active_crewed_flights_cache(
rocket_dir: str,
) -> tuple[datetime, list[Summary]] | None:
"""Load active crewed flights cache file."""
filename = get_active_crewed_flights_cache_filename(rocket_dir)
cache_data = read_json_cache(filename)
if cache_data is None:
return None
updated_str = cache_data.get("updated")
updated = parse_api_datetime(updated_str)
results = cache_data.get("results")
if not updated or not isinstance(results, list):
return None
summary_results: list[Summary] = [r for r in results if isinstance(r, dict)]
return (updated.replace(tzinfo=None), summary_results)
def write_active_crewed_flights_cache(rocket_dir: str, launches: list[Summary]) -> None:
"""Write active crewed flights cache file."""
filename = get_active_crewed_flights_cache_filename(rocket_dir)
payload: StrDict = {
"updated": datetime.now().isoformat(),
"results": launches,
}
write_json_cache(filename, payload)
def get_active_crewed_flights(
rocket_dir: str, refresh: bool = False
) -> list[Summary] | None:
"""Get active crewed flights with cache and API fallback."""
now = datetime.now()
cached = load_active_crewed_flights_cache(rocket_dir)
if cached and not refresh and (now - cached[0]).total_seconds() <= ttl:
return cached[1]
try:
active_flights = active_crewed_flights_api(rocket_dir)
if active_flights is not None:
write_active_crewed_flights_cache(rocket_dir, active_flights)
return active_flights
except Exception:
pass
return cached[1] if cached else []
return (ts, filename)
def format_time(time_str: str, net_precision: str) -> tuple[str, str | None]:
@ -324,8 +59,6 @@ def format_time(time_str: str, net_precision: str) -> tuple[str, str | None]:
include_time = True
case _ if net_precision and net_precision.startswith("Quarter "):
time_format = f"Q{net_precision[-1]} %Y"
case _ if net_precision and net_precision.startswith("Year Half "):
time_format = f"H{net_precision[-1]} %Y"
case _:
time_format = None
@ -383,7 +116,6 @@ def summarize_launch(launch: Launch) -> Summary:
return {
"name": launch.get("name"),
"slug": launch["slug"],
"status": launch.get("status"),
"net": launch.get("net"),
"net_precision": net_precision,
@ -394,7 +126,7 @@ def summarize_launch(launch: Launch) -> Summary:
"launch_provider": launch_provider,
"launch_provider_abbrev": launch_provider_abbrev,
"launch_provider_type": get_nested(launch, ["launch_service_provider", "type"]),
"rocket": launch["rocket"]["configuration"],
"rocket": launch["rocket"]["configuration"]["full_name"],
"mission": launch.get("mission"),
"mission_name": get_nested(launch, ["mission", "name"]),
"pad_name": launch["pad"]["name"],
@ -402,215 +134,24 @@ def summarize_launch(launch: Launch) -> Summary:
"location": launch["pad"]["location"]["name"],
"country_code": launch["pad"]["country_code"],
"orbit": get_nested(launch, ["mission", "orbit"]),
"probability": launch["probability"],
"weather_concerns": launch["weather_concerns"],
"image": launch.get("image"),
}
def is_launches_cache_fresh(rocket_dir: str) -> bool:
"""Return True if the launches cache is younger than the TTL."""
cached = latest_launches_cache(rocket_dir)
return cached is not None and (datetime.now() - cached[0]).total_seconds() <= ttl
def load_cached_launches(rocket_dir: str) -> StrDict | None:
"""Read the most recent usable cache of launches."""
cached = latest_launches_cache(rocket_dir)
return cached[1] if cached is not None else None
def read_cached_launches(rocket_dir: str) -> list[Summary]:
"""Read cached launches."""
data = load_cached_launches(rocket_dir)
if not data or not isinstance(data.get("results"), list):
return []
return [summarize_launch(launch) for launch in data["results"]]
def get_launches(
rocket_dir: str, limit: int = LIMIT, refresh: bool = False
) -> list[Summary] | None:
async def get_launches(rocket_dir: str, limit: int = 200) -> list[Summary]:
"""Get rocket launches with caching."""
if refresh or not is_launches_cache_fresh(rocket_dir):
now = datetime.now()
existing = [x for x in (filename_timestamp(f) for f in os.listdir(rocket_dir)) if x]
existing.sort(reverse=True)
if not existing or (now - existing[0][0]).seconds > 3600: # one hour
try:
upcoming = next_launch_api(rocket_dir, limit=limit)
if upcoming is None:
raise RuntimeError("unable to fetch upcoming launches")
active_crewed = get_active_crewed_flights(rocket_dir, refresh=refresh) or []
by_slug = {
typing.cast(str, launch["slug"]): launch
for launch in upcoming
if isinstance(launch.get("slug"), str)
}
for launch in active_crewed:
slug = launch.get("slug")
if isinstance(slug, str) and slug not in by_slug:
by_slug[slug] = launch
return sorted(by_slug.values(), key=lambda launch: str(launch.get("net")))
except Exception:
pass # fallback to cached version
return await next_launch_api(rocket_dir, limit=limit)
except httpx.ReadTimeout:
pass
data = load_cached_launches(rocket_dir)
if not data:
return []
upcoming = [summarize_launch(launch) for launch in data["results"]]
active_crewed = get_active_crewed_flights(rocket_dir, refresh=refresh) or []
by_slug = {
typing.cast(str, launch["slug"]): launch
for launch in upcoming
if isinstance(launch.get("slug"), str)
}
for launch in active_crewed:
slug = launch.get("slug")
if isinstance(slug, str) and slug not in by_slug:
by_slug[slug] = launch
return sorted(by_slug.values(), key=lambda launch: str(launch.get("net")))
f = existing[0][1]
def format_date(dt: datetime) -> str:
"""Human readable date."""
return dt.strftime("%d %b %Y at %H:%M UTC")
def format_datetime_change(field_name: str, old_val: str, new_val: str) -> str:
"""Format a datetime field change with proper error handling."""
try:
old_dt = datetime.fromisoformat(old_val.replace("Z", "+00:00"))
new_dt = datetime.fromisoformat(new_val.replace("Z", "+00:00"))
return (
f"{field_name} changed from {format_date(old_dt)} to {format_date(new_dt)}"
)
except (ValueError, AttributeError):
return f"{field_name} changed from {old_val} to {new_val}"
def format_datetime_update(field_name: str, new_val: str) -> str:
"""Format a datetime field update (showing only the new value)."""
try:
new_dt = datetime.fromisoformat(new_val.replace("Z", "+00:00"))
return f"{field_name}: {format_date(new_dt)}"
except (ValueError, AttributeError):
return f"{field_name}: {new_val}"
def format_probability_change(old_val: int, new_val: int) -> str:
"""Format probability field changes."""
if old_val is None:
return f"Launch probability set to {new_val}%"
elif new_val is None:
return "Launch probability removed"
else:
return f"Launch probability changed from {old_val}% to {new_val}%"
def format_launch_changes(differences: StrDict) -> str:
"""Convert deepdiff output to human-readable format."""
changes: list[str] = []
processed_paths: set[str] = set()
SKIP_FIELDS = {
"agency_launch_attempt_count",
"agency_launch_attempt_count_year",
"location_launch_attempt_count",
"location_launch_attempt_count_year",
"pad_launch_attempt_count",
"pad_launch_attempt_count_year",
"orbital_launch_attempt_count",
"orbital_launch_attempt_count_year",
}
# --- 1. Handle Special Group Value Changes ---
# Process high-level, user-friendly summaries first.
values = differences.get("values_changed", {})
if "root['status']['name']" in values:
old_val = values["root['status']['name']"]["old_value"]
new_val = values["root['status']['name']"]["new_value"]
changes.append(f"Status changed from '{old_val}' to '{new_val}'")
processed_paths.add("root['status']")
if "root['net_precision']['name']" in values:
old_val = values["root['net_precision']['name']"]["old_value"]
new_val = values["root['net_precision']['name']"]["new_value"]
changes.append(f"Launch precision changed from '{old_val}' to '{new_val}'")
processed_paths.add("root['net_precision']")
# --- 2. Handle Type Changes ---
# This is often more significant than a value change (e.g., probability becoming None).
if "type_changes" in differences:
for path, change in differences["type_changes"].items():
if any(path.startswith(p) for p in processed_paths):
continue
field = path.replace("root['", "").replace("']", "").replace("root.", "")
if field == "probability":
# Use custom formatter only for meaningful None transitions.
if change["old_type"] is type(None) or change["new_type"] is type(None):
changes.append(
format_probability_change(
change["old_value"], change["new_value"]
)
)
else: # For other type changes (e.g., int to str), use the generic message.
changes.append(
f"{field.replace('_', ' ').title()} type changed "
+ f"from {change['old_type'].__name__} to {change['new_type'].__name__}"
)
else:
changes.append(
f"{field.replace('_', ' ').title()} type changed "
+ f"from {change['old_type'].__name__} to {change['new_type'].__name__}"
)
processed_paths.add(path)
# --- 3. Handle Remaining Value Changes ---
for path, change in values.items():
if any(path.startswith(p) for p in processed_paths):
continue
field = path.replace("root['", "").replace("']", "").replace("root.", "")
if field in SKIP_FIELDS:
continue
old_val = change["old_value"]
new_val = change["new_value"]
match field:
case "net":
changes.append(format_datetime_change("Launch time", old_val, new_val))
case "window_start":
changes.append(
format_datetime_change("Launch window start", old_val, new_val)
)
case "window_end":
changes.append(
format_datetime_change("Launch window end", old_val, new_val)
)
case "last_updated":
changes.append(format_datetime_update("Last updated", new_val))
case "name":
changes.append(f"Mission name changed from '{old_val}' to '{new_val}'")
case "probability":
changes.append(format_probability_change(old_val, new_val))
case _:
changes.append(f"{field} changed from '{old_val}' to '{new_val}'")
processed_paths.add(path)
# --- 4. Handle Added/Removed Fields ---
if "dictionary_item_added" in differences:
for path in differences["dictionary_item_added"]:
field = path.replace("root['", "").replace("']", "").replace("root.", "")
changes.append(f"New field added: {field.replace('_', ' ').title()}")
if "dictionary_item_removed" in differences:
for path in differences["dictionary_item_removed"]:
field = path.replace("root['", "").replace("']", "").replace("root.", "")
changes.append(f"Field removed: {field.replace('_', ' ').title()}")
# Sort changes for deterministic output in tests
return (
"\n".join(f"• {change}" for change in sorted(changes))
if changes
else "No specific changes detected"
)
filename = os.path.join(rocket_dir, f)
data = json.load(open(filename))
return [summarize_launch(launch) for launch in data["results"]]

120
agenda/travel.py
View file

@ -1,86 +1,36 @@
"""Travel."""
import decimal
import json
import os
import typing
import flask
import yaml
from geopy.distance import geodesic # type: ignore
from .event import Event
from .types import StrDict
from .types import Event
Leg = dict[str, str]
TravelList = list[dict[str, typing.Any]]
RouteDistances = dict[tuple[str, str], float]
def coords(airport: StrDict) -> tuple[float, float]:
"""Longitude / Latitude as coordinate tuples."""
# return (airport["longitude"], airport["latitude"])
return (airport["latitude"], airport["longitude"])
def flight_distance(f: StrDict) -> float:
"""Distance of flight."""
return float(geodesic(coords(f["from_airport"]), coords(f["to_airport"])).km)
def route_distances_as_json(route_distances: RouteDistances) -> str:
"""Format route distances as JSON string."""
return (
"[\n"
+ ",\n".join(
" " + json.dumps([s1, s2, dist])
for (s1, s2), dist in route_distances.items()
)
+ "\n]"
)
def parse_yaml(travel_type: str, data_dir: str) -> TravelList:
"""Parse flights YAML and return list of travel."""
filepath = os.path.join(data_dir, travel_type + ".yaml")
items: TravelList = yaml.safe_load(open(filepath))
if not all(isinstance(item, dict) for item in items):
return items
for item in items:
price = item.get("price")
if price:
item["price"] = decimal.Decimal(price)
return items
return typing.cast(TravelList, yaml.safe_load(open(filepath)))
def get_flights(data_dir: str) -> list[Event]:
"""Get travel events."""
bookings = parse_yaml("flights", data_dir)
airlines = parse_yaml("airlines", data_dir)
by_iata = {a["iata"]: a for a in airlines}
events = []
for booking in bookings:
for item in booking["flights"]:
if not item["depart"].date():
continue
airline = by_iata[item["airline"]]
item["airline_code"] = airline[
"iata" if not airline.get("flight_number_prefer_icao") else "icao"
]
e = Event(
date=item["depart"],
end_date=item.get("arrive"),
name="transport",
title=f'✈️ {item["from"]} to {item["to"]} ({flight_number(item)})',
url=(item.get("url") if flask.g.user.is_authenticated else None),
)
events.append(e)
return events
return [
Event(
date=item["depart"],
end_date=item.get("arrive"),
name="transport",
title=f'✈️ {item["from"]} to {item["to"]} ({flight_number(item)})',
url=item.get("url"),
)
for item in parse_yaml("flights", data_dir)
if item["depart"].date()
]
def get_trains(data_dir: str) -> list[Event]:
@ -93,7 +43,7 @@ def get_trains(data_dir: str) -> list[Event]:
end_date=leg["arrive"],
name="transport",
title=f'🚆 {leg["from"]} to {leg["to"]}',
url=(item.get("url") if flask.g.user.is_authenticated else None),
url=item.get("url"),
)
for leg in item["legs"]
]
@ -102,7 +52,7 @@ def get_trains(data_dir: str) -> list[Event]:
def flight_number(flight: Leg) -> str:
"""Flight number."""
airline_code = flight["airline_code"]
airline_code = flight["airline"]
# make sure this is the airline code, not the airline name
assert " " not in airline_code and not any(c.islower() for c in airline_code)
@ -112,43 +62,3 @@ def flight_number(flight: Leg) -> str:
def all_events(data_dir: str) -> list[Event]:
"""Get all flights and rail journeys."""
return get_trains(data_dir) + get_flights(data_dir)
def train_leg_distance(geojson_data: StrDict) -> float:
"""Calculate the total length of a LineString in kilometers from GeoJSON data."""
# Extract coordinates
first_object = geojson_data["features"][0]["geometry"]
assert first_object["type"] in ("LineString", "MultiLineString")
if first_object["type"] == "LineString":
coord_list = [first_object["coordinates"]]
else:
first_object["type"] == "MultiLineString"
coord_list = first_object["coordinates"]
total_length_km = 0.0
for coordinates in coord_list:
total_length_km += sum(
float(geodesic(coordinates[i], coordinates[i + 1]).km)
for i in range(len(coordinates) - 1)
)
return total_length_km
def load_route_distances(data_dir: str) -> RouteDistances:
"""Load cache of route distances."""
route_distances: RouteDistances = {}
with open(os.path.join(data_dir, "route_distances.json")) as f:
for s1, s2, dist in json.load(f):
route_distances[(s1, s2)] = dist
return route_distances
def add_leg_route_distance(leg: StrDict, route_distances: RouteDistances) -> None:
s1, s2 = sorted([leg["from"], leg["to"]])
dist = route_distances.get((s1, s2))
if dist:
leg["distance"] = dist

1355
agenda/trip.py
View file

File diff suppressed because it is too large Load diff

61
agenda/trip_debug.py
View file

@ -1,61 +0,0 @@
"""Serialize trip details for the debug page."""
import json
import yaml
from agenda.types import Trip
def serialize_trip(trip: Trip) -> tuple[str, str]:
"""Return JSON and YAML including computed trip properties."""
# Convert trip object to dictionary for display
trip_dict = {
"start": trip.start.isoformat(),
"name": trip.name,
"private": trip.private,
"travel": trip.travel,
"accommodation": trip.accommodation,
"conferences": trip.conferences,
"events": trip.events,
"flight_bookings": trip.flight_bookings,
"computed_properties": {
"title": trip.title,
"end": trip.end.isoformat() if trip.end else None,
"countries": [
{"name": c.name, "alpha_2": c.alpha_2, "flag": c.flag}
for c in trip.countries
],
"locations": [
{
"location": loc,
"country": {"name": country.name, "alpha_2": country.alpha_2},
}
for loc, country in trip.locations()
],
"total_distance": trip.total_distance(),
"total_co2_kg": trip.total_co2_kg(),
"distances_by_transport_type": trip.distances_by_transport_type(),
"co2_by_transport_type": trip.co2_by_transport_type(),
},
"schengen_compliance": (
{
"total_days_used": trip.schengen_compliance.total_days_used,
"days_remaining": trip.schengen_compliance.days_remaining,
"is_compliant": trip.schengen_compliance.is_compliant,
"current_180_day_period": [
trip.schengen_compliance.current_180_day_period[0].isoformat(),
trip.schengen_compliance.current_180_day_period[1].isoformat(),
],
"days_over_limit": trip.schengen_compliance.days_over_limit,
}
if trip.schengen_compliance
else None
),
}
# Convert to JSON for pretty printing
trip_json = json.dumps(trip_dict, indent=2, default=str)
trip_yaml = yaml.safe_dump(json.loads(trip_json), sort_keys=False)
return trip_json, trip_yaml

279
agenda/trip_schengen.py
View file

@ -1,279 +0,0 @@
"""Integration of Schengen calculator with the existing trip system."""
import logging
import typing
from datetime import date, timedelta
import flask
from . import get_country, trip
from .schengen import (
SCHENGEN_COUNTRIES,
calculate_schengen_time,
extract_schengen_stays_from_travel,
)
from .types import SchengenCalculation, SchengenStay, StrDict, Trip
def trip_includes_schengen(trip: Trip) -> bool:
return bool({c.alpha_2.lower() for c in trip.countries} & SCHENGEN_COUNTRIES)
def add_schengen_compliance_to_trip(trip: Trip) -> Trip:
"""Add Schengen compliance information to a trip object."""
if not trip_includes_schengen(trip):
return trip
try:
# Calculate Schengen compliance for the trip
calculation = calculate_schengen_time(trip.travel)
# Add the calculation to the trip object
trip.schengen_compliance = calculation
except Exception as e:
# Log the error but don't fail the trip loading
logging.warning(
f"Failed to calculate Schengen compliance for trip {trip.start}: {e}"
)
trip.schengen_compliance = None
return trip
def get_schengen_compliance_for_all_trips(
trip_list: list[Trip],
) -> dict[date, SchengenCalculation]:
"""Calculate Schengen compliance for all trips."""
compliance_by_date = {}
# Collect all travel items across all trips
all_travel_items = []
for trip_obj in trip_list:
all_travel_items.extend(trip_obj.travel)
# Calculate compliance for each trip's start date
for trip_obj in trip_list:
calculation = calculate_schengen_time(all_travel_items, trip_obj.start)
compliance_by_date[trip_obj.start] = calculation
return compliance_by_date
def generate_schengen_warnings(trip_list: list[Trip]) -> list[dict[str, typing.Any]]:
"""Generate warnings for potential Schengen violations."""
warnings = []
# Get compliance for all trips
compliance_by_date = get_schengen_compliance_for_all_trips(trip_list)
for trip_date, calculation in compliance_by_date.items():
if not calculation.is_compliant:
warnings.append(
{
"type": "schengen_violation",
"date": trip_date,
"message": f"Schengen violation on {trip_date}: {calculation.days_over_limit} days over limit",
"severity": "error",
"calculation": calculation,
}
)
elif calculation.days_remaining < 10:
warnings.append(
{
"type": "schengen_warning",
"date": trip_date,
"message": f"Low Schengen allowance on {trip_date}: only {calculation.days_remaining} days remaining",
"severity": "warning",
"calculation": calculation,
}
)
return warnings
def extract_schengen_stays_with_trip_info(
trip_list: list[Trip],
) -> list[SchengenStay]:
"""Extract Schengen stays from travel items with trip information."""
# Create a mapping of travel items to their trips
travel_to_trip_map = {}
for trip_obj in trip_list:
for travel_item in trip_obj.travel:
travel_to_trip_map[id(travel_item)] = trip_obj
# Collect all travel items
all_travel_items = []
for trip_obj in trip_list:
all_travel_items.extend(trip_obj.travel)
# Get stays with trip information
stays = extract_schengen_stays_from_travel(all_travel_items)
# Try to associate stays with trips based on entry dates
for stay in stays:
# Find the trip that contains this stay's entry date
for trip_obj in trip_list:
if trip_obj.start <= stay.entry_date <= (trip_obj.end or stay.entry_date):
stay.trip_date = trip_obj.start
stay.trip_name = trip_obj.title
break
# If no exact match, find the closest trip by start date
if stay.trip_date is None:
closest_trip = min(
trip_list,
key=lambda t: abs((t.start - stay.entry_date).days),
default=None,
)
if closest_trip:
stay.trip_date = closest_trip.start
stay.trip_name = closest_trip.title
return stays
def schengen_dashboard_data(data_dir: str | None = None) -> dict[str, typing.Any]:
"""Generate dashboard data for Schengen compliance."""
if data_dir is None:
data_dir = flask.current_app.config["PERSONAL_DATA"]
# Load all trips
trip_list = [
trip for trip in trip.build_trip_list(data_dir) if trip_includes_schengen(trip)
]
# Calculate current compliance with trip information
all_travel_items = []
for trip_obj in trip_list:
all_travel_items.extend(trip_obj.travel)
current_calculation = calculate_schengen_time(all_travel_items)
# Get stays with trip information
stays_with_trip_info = extract_schengen_stays_with_trip_info(trip_list)
# Update current calculation with trip information
current_calculation.stays_in_period = [
stay
for stay in stays_with_trip_info
if stay.entry_date >= current_calculation.current_180_day_period[0]
and stay.entry_date <= current_calculation.current_180_day_period[1]
]
# Generate warnings
warnings = generate_schengen_warnings(trip_list)
# Get compliance history for the last year
compliance_history = []
current_date = date.today()
for i in range(365, 0, -7): # Weekly snapshots for the last year
snapshot_date = current_date - timedelta(days=i)
calculation = calculate_schengen_time(all_travel_items, snapshot_date)
compliance_history.append(
{
"date": snapshot_date,
"days_used": calculation.total_days_used,
"is_compliant": calculation.is_compliant,
}
)
return {
"current_compliance": current_calculation,
"warnings": warnings,
"compliance_history": compliance_history,
"trips_with_compliance": get_schengen_compliance_for_all_trips(trip_list),
}
def flask_route_schengen_report() -> str:
"""Flask route for Schengen compliance report."""
data_dir = flask.current_app.config["PERSONAL_DATA"]
dashboard_data = schengen_dashboard_data(data_dir)
# Load trips for the template
trip_list = trip.build_trip_list(data_dir)
return flask.render_template(
"schengen_report.html",
trip_list=trip_list,
get_country=get_country,
**dashboard_data,
)
def export_schengen_data_for_external_calculator(
travel_items: list[StrDict],
) -> list[dict[str, typing.Any]]:
"""Export travel data in format suitable for external Schengen calculators."""
stays = extract_schengen_stays_from_travel(travel_items)
export_data = []
for stay in stays:
export_data.append(
{
"entry_date": stay.entry_date.strftime("%Y-%m-%d"),
"exit_date": (
stay.exit_date.strftime("%Y-%m-%d") if stay.exit_date else None
),
"country": stay.country.upper(),
"days": stay.days,
}
)
return export_data
# Integration with existing trip.py functions
def enhanced_build_trip_list(
data_dir: str | None = None,
route_distances: typing.Any = None,
include_schengen: bool = True,
) -> list[Trip]:
"""Enhanced version of build_trip_list that includes Schengen compliance."""
# Use the original function
trip_list = trip.build_trip_list(data_dir, route_distances)
if include_schengen:
# Add Schengen compliance to each trip
for trip_obj in trip_list:
add_schengen_compliance_to_trip(trip_obj)
return trip_list
def check_schengen_compliance_for_new_trip(
existing_trips: list[Trip],
new_trip_dates: tuple[date, date],
destination_country: str,
) -> SchengenCalculation:
"""Check if a new trip would violate Schengen rules."""
# Collect all existing travel
all_travel_items = []
for trip_obj in existing_trips:
all_travel_items.extend(trip_obj.travel)
# Add the new trip as mock travel items
entry_date, exit_date = new_trip_dates
# Mock entry to Schengen
all_travel_items.append(
{
"type": "flight",
"depart": entry_date,
"to_airport": {"country": destination_country},
}
)
# Mock exit from Schengen
all_travel_items.append(
{
"type": "flight",
"depart": exit_date,
"from_airport": {"country": destination_country},
"to_airport": {"country": "gb"}, # Assuming return to UK
}
)
# Calculate compliance at the exit date
return calculate_schengen_time(all_travel_items, exit_date)

341
agenda/trip_timezones.py
View file

@ -1,341 +0,0 @@
"""Resolve trip destination timezones and local journey display times."""
import functools
import importlib
import json
import os.path
import typing
from collections import defaultdict
from datetime import date, datetime, timedelta, timezone
from zoneinfo import ZoneInfo
import pytz
from pycountry.db import Country
import agenda.trip
from agenda.types import StrDict, Trip
def _timezone_name_from_datetime(value: typing.Any) -> str | None:
"""Get IANA timezone name from a datetime value if available."""
if not isinstance(value, datetime) or value.tzinfo is None:
return None
key = getattr(value.tzinfo, "key", None)
if isinstance(key, str):
return key
zone = getattr(value.tzinfo, "zone", None)
if isinstance(zone, str):
return zone
return None
def _format_offset_from_uk(offset_minutes: int) -> str:
"""Format offset from UK in +/-HH:MM."""
if offset_minutes == 0:
return "No difference"
sign = "+" if offset_minutes > 0 else "-"
hours, mins = divmod(abs(offset_minutes), 60)
return f"{sign}{hours:02d}:{mins:02d} vs UK"
def _trip_offset_minutes(
trip_start: date, trip_end: date, destination_timezone: str
) -> list[int]:
"""Unique UTC offset differences vs UK across the trip date range."""
destination_tz = ZoneInfo(destination_timezone)
uk_timezone = ZoneInfo("Europe/London")
current = trip_start
offsets: set[int] = set()
while current <= trip_end:
instant = datetime(
current.year, current.month, current.day, 12, tzinfo=timezone.utc
)
destination_offset = instant.astimezone(destination_tz).utcoffset()
uk_offset = instant.astimezone(uk_timezone).utcoffset()
if destination_offset is not None and uk_offset is not None:
offsets.add(int((destination_offset - uk_offset).total_seconds() // 60))
current += timedelta(days=1)
return sorted(offsets)
def _format_trip_offset_display(offsets: list[int]) -> str:
"""Format trip-range offsets; include variation if DST changes during trip."""
if not offsets:
return "Timezone unknown"
if len(offsets) == 1:
return _format_offset_from_uk(offsets[0])
return (
"Varies during trip: "
f"{_format_offset_from_uk(offsets[0])} to {_format_offset_from_uk(offsets[-1])}"
)
def _timezone_from_coordinates(latitude: float, longitude: float) -> str | None:
"""Resolve IANA timezone name from coordinates."""
timezone_finder = _get_timezone_finder()
if timezone_finder is None:
return None
for method_name in ("timezone_at", "certain_timezone_at", "closest_timezone_at"):
finder_method = getattr(timezone_finder, method_name, None)
if not callable(finder_method):
continue
tz_name = finder_method(lng=longitude, lat=latitude)
if isinstance(tz_name, str):
return tz_name
return None
@functools.lru_cache(maxsize=1)
def _get_timezone_finder() -> typing.Any:
"""Get timezone finder instance if dependency is available."""
try:
timezonefinder_module = importlib.import_module("timezonefinder")
except ModuleNotFoundError:
return None
timezone_finder_cls = getattr(timezonefinder_module, "TimezoneFinder", None)
if timezone_finder_cls is None:
return None
return timezone_finder_cls()
def _coordinates_from_location(location: typing.Any) -> tuple[float, float] | None:
"""Extract latitude/longitude from a location mapping."""
if not isinstance(location, dict):
return None
latitude = location.get("latitude")
longitude = location.get("longitude")
if not isinstance(latitude, (int, float)) or not isinstance(
longitude, (int, float)
):
return None
return (float(latitude), float(longitude))
def _route_endpoints_for_car_item(
item: StrDict,
data_dir: str,
route_cache: dict[str, tuple[tuple[float, float], tuple[float, float]] | None],
) -> tuple[tuple[float, float], tuple[float, float]] | None:
"""Return route endpoints for a car journey when available."""
route_filename = item.get("geojson_filename")
if not isinstance(route_filename, str):
return None
if route_filename not in route_cache:
geojson_text = agenda.trip.read_geojson(
data_dir, os.path.join("car_routes", route_filename)
)
endpoints = agenda.trip.geojson_route_endpoints(json.loads(geojson_text))
route_cache[route_filename] = endpoints
return route_cache[route_filename]
def _timezone_name_for_car_item(
item: StrDict,
data_dir: str,
route_cache: dict[str, tuple[tuple[float, float], tuple[float, float]] | None],
timezone_cache: dict[tuple[float, float], str | None],
) -> str | None:
"""Resolve the local timezone for a car journey from endpoints."""
candidate_coords: list[tuple[float, float]] = []
for field in ("from_location", "to_location"):
coord = _coordinates_from_location(item.get(field))
if coord is not None:
candidate_coords.append(coord)
if not candidate_coords:
endpoints = _route_endpoints_for_car_item(item, data_dir, route_cache)
if endpoints is not None:
candidate_coords.extend(endpoints)
for coord in candidate_coords:
if coord not in timezone_cache:
timezone_cache[coord] = _timezone_from_coordinates(coord[0], coord[1])
timezone_name = timezone_cache[coord]
if timezone_name:
return timezone_name
return None
def localize_trip_car_journey_display_times(trip: Trip, data_dir: str) -> None:
"""Add route-local display timestamps for car journeys on the trip page."""
route_cache: dict[str, tuple[tuple[float, float], tuple[float, float]] | None] = {}
timezone_cache: dict[tuple[float, float], str | None] = {}
for item in trip.travel:
if item.get("type") != "car":
continue
depart = item.get("depart")
arrive = item.get("arrive")
if not isinstance(depart, datetime) or not isinstance(arrive, datetime):
continue
timezone_name = _timezone_name_for_car_item(
item, data_dir, route_cache, timezone_cache
)
if not timezone_name:
continue
item["display_depart"] = depart.astimezone(ZoneInfo(timezone_name))
item["display_arrive"] = arrive.astimezone(ZoneInfo(timezone_name))
def get_destination_timezones(trip: Trip) -> list[StrDict]:
"""Build destination timezone metadata for the trip page."""
per_location: dict[tuple[str, str], list[str]] = defaultdict(list)
location_coords: dict[tuple[str, str], tuple[float, float]] = {}
for item in trip.accommodation + trip.conferences + trip.events:
location = item.get("location")
country = item.get("country")
if not isinstance(location, str) or not isinstance(country, str):
continue
key = (location, country.lower())
timezone_name = item.get("timezone")
if isinstance(timezone_name, str):
per_location[key].append(timezone_name)
latitude = item.get("latitude")
longitude = item.get("longitude")
if isinstance(latitude, (int, float)) and isinstance(longitude, (int, float)):
location_coords[key] = (float(latitude), float(longitude))
for field in (
"from",
"to",
"date",
"start",
"end",
"attend_start",
"attend_end",
):
candidate = _timezone_name_from_datetime(item.get(field))
if candidate:
per_location[key].append(candidate)
# Also collect airport locations from flights, for transit countries
flight_locations: list[tuple[str, Country]] = []
seen_flight_keys: set[tuple[str, str]] = set()
for item in trip.travel:
if item.get("type") != "flight":
continue
for airport_key in ("from_airport", "to_airport"):
airport = item.get(airport_key)
if not isinstance(airport, dict):
continue
city = airport.get("city")
country_code = airport.get("country")
if not isinstance(city, str) or not isinstance(country_code, str):
continue
if country_code == "gb":
continue
key = (city, country_code.lower())
lat = airport.get("latitude")
lon = airport.get("longitude")
if isinstance(lat, (int, float)) and isinstance(lon, (int, float)):
location_coords.setdefault(key, (float(lat), float(lon)))
if key not in seen_flight_keys:
seen_flight_keys.add(key)
flight_country = agenda.get_country(country_code)
if flight_country:
flight_locations.append((city, flight_country))
existing_location_keys = {(loc, c.alpha_2.lower()) for loc, c in trip.locations()}
all_locations = list(trip.locations()) + [
(city, country)
for city, country in flight_locations
if (city, country.alpha_2.lower()) not in existing_location_keys
]
destination_times: list[StrDict] = []
trip_end = trip.end or trip.start
for location, country in all_locations:
country_code = country.alpha_2.lower()
key = (location, country_code)
timezone_name = None
for candidate in per_location.get(key, []):
try:
ZoneInfo(candidate)
timezone_name = candidate
break
except Exception:
continue
if not timezone_name and key in location_coords:
latitude, longitude = location_coords[key]
coordinate_timezone = _timezone_from_coordinates(latitude, longitude)
if coordinate_timezone:
timezone_name = coordinate_timezone
if not timezone_name:
country_timezones = pytz.country_timezones.get(country_code, [])
if len(country_timezones) == 1:
timezone_name = country_timezones[0]
offset_display = "Timezone unknown"
if timezone_name:
offset_display = _format_trip_offset_display(
_trip_offset_minutes(trip.start, trip_end, timezone_name)
)
destination_times.append(
{
"location": location,
"country_name": country.name,
"country_flag": country.flag,
"timezone": timezone_name,
"offset_display": offset_display,
}
)
grouped: list[StrDict] = []
grouped_index: dict[tuple[str, str, str | None], int] = {}
for item in destination_times:
group_key = (item["country_name"], item["country_flag"], item["timezone"])
if group_key in grouped_index:
existing = grouped[grouped_index[group_key]]
existing_locations = typing.cast(list[str], existing["locations"])
existing_locations.append(typing.cast(str, item["location"]))
existing["location_count"] = (
typing.cast(int, existing["location_count"]) + 1
)
continue
grouped_index[group_key] = len(grouped)
grouped.append(
{
**item,
"locations": [item["location"]],
"location_count": 1,
}
)
for item in grouped:
location_count = typing.cast(int, item["location_count"])
country_name = typing.cast(str, item["country_name"])
country_flag = typing.cast(str, item["country_flag"])
if location_count > 1:
label = f"{country_name} ({location_count} locations)"
else:
label = f"{item['location']} ({country_name})"
if trip.show_flags:
label = f"{label} {country_flag}"
item["destination_label"] = label
return grouped

572
agenda/types.py
View file

@ -1,498 +1,104 @@
"""Types."""
import collections
import dataclasses
import datetime
import functools
import typing
from collections import defaultdict
from dataclasses import dataclass, field
from datetime import date
import emoji
from pycountry.db import Country
import agenda
from agenda import format_list_with_ampersand
from . import utils
StrDict = dict[str, typing.Any]
DateOrDateTime = datetime.datetime | datetime.date
@dataclass
class TripElement:
"""Trip element."""
start_time: DateOrDateTime
title: str
element_type: str
detail: StrDict
end_time: DateOrDateTime | None = None
start_loc: str | None = None
end_loc: str | None = None
start_country: Country | None = None
end_country: Country | None = None
all_day: bool = False
def get_emoji(self) -> str | None:
"""Emoji for trip element."""
emoji_map = {
"check-in": ":hotel:",
"check-out": ":hotel:",
"train": ":train:",
"flight": ":airplane:",
"ferry": ":ferry:",
"coach": ":bus:",
"bus": ":bus:",
"car": ":automobile:",
}
alias = emoji_map.get(self.element_type)
return emoji.emojize(alias, language="alias") if alias else None
def airport_label(airport: StrDict) -> str:
"""Airport label: name and iata."""
name = airport.get("alt_name") or airport["city"]
return f"{name} ({airport['iata']})"
def is_generated_drive_stop(label: str) -> bool:
"""Return whether a car stop label is synthetic timeline filler."""
prefix = "Drive stop "
return label.startswith(prefix) and label[len(prefix) :].isdigit()
@dataclass
class SchengenStay:
"""Represents a stay in the Schengen area."""
entry_date: date
exit_date: date | None # None if currently in Schengen
country: str
days: int
trip_date: date | None = None # Trip start date for linking
trip_name: str | None = None # Trip name for display
def __post_init__(self) -> None:
"""Post init."""
if self.exit_date is None:
# Currently in Schengen, calculate days up to today
self.days = (date.today() - self.entry_date).days + 1
else:
self.days = (self.exit_date - self.entry_date).days + 1
@dataclass
class SchengenCalculation:
"""Result of Schengen time calculation."""
total_days_used: int
days_remaining: int
is_compliant: bool
current_180_day_period: tuple[date, date] # (start, end)
stays_in_period: list[SchengenStay]
next_reset_date: typing.Optional[date] # When the 180-day window resets
@property
def days_over_limit(self) -> int:
"""Days over the 90-day limit."""
return max(0, self.total_days_used - 90)
@dataclass
class Trip:
"""Trip."""
start: datetime.date
travel: list[StrDict] = field(default_factory=list)
accommodation: list[StrDict] = field(default_factory=list)
conferences: list[StrDict] = field(default_factory=list)
events: list[StrDict] = field(default_factory=list)
flight_bookings: list[StrDict] = field(default_factory=list)
name: str | None = None
private: bool = False
schengen_compliance: SchengenCalculation | None = None
@property
def title(self) -> str:
"""Trip title."""
if self.name:
return self.name
titles: list[str] = [conf["name"] for conf in self.conferences] + [
event["title"] for event in self.events
] or self.titles_from_travel()
if not titles:
titles = [acc["location"] for acc in self.accommodation]
return format_list_with_ampersand(titles) or "[unnamed trip]"
def titles_from_travel(self) -> list[str]:
"""Titles from travel."""
titles = []
for travel in self.travel:
if not (depart := (travel["depart"] and utils.as_date(travel["depart"]))):
continue
for when, from_or_to in ((self.start, "from"), (self.end, "to")):
if depart == when:
continue
label = travel.get(from_or_to)
if not isinstance(label, str) or is_generated_drive_stop(label):
continue
if label not in titles:
titles.append(label)
return titles
@property
def end(self) -> datetime.date | None:
"""End date for trip."""
max_conference_end = (
max(utils.as_date(item["end"]) for item in self.conferences)
if self.conferences
else datetime.date.min
)
assert isinstance(max_conference_end, datetime.date)
arrive = [
utils.as_date(item["arrive"]) for item in self.travel if "arrive" in item
]
travel_end = max(arrive) if arrive else datetime.date.min
assert isinstance(travel_end, datetime.date)
accommodation_end = (
max(utils.as_date(item["to"]) for item in self.accommodation)
if self.accommodation
else datetime.date.min
)
assert isinstance(accommodation_end, datetime.date)
max_date = max(max_conference_end, travel_end, accommodation_end)
return max_date if max_date != datetime.date.min else None
def locations(self) -> list[tuple[str, Country]]:
"""Locations for trip."""
seen: set[tuple[str, str]] = set()
items = []
for item in self.conferences + self.accommodation + self.events:
if "country" not in item or "location" not in item:
continue
key = (item["location"], item["country"])
if key in seen:
continue
seen.add(key)
country = agenda.get_country(item["country"])
assert country
items.append((item["location"], country))
return items
@property
def countries(self) -> list[Country]:
"""Countries visited as part of trip, in order."""
seen: set[str] = set()
items: list[Country] = []
for item in self.conferences + self.accommodation + self.events:
if "country" not in item:
continue
if item["country"] in seen:
continue
seen.add(item["country"])
country = agenda.get_country(item["country"])
assert country
items.append(country)
for item in self.travel:
travel_countries = set()
if item["type"] == "flight":
for key in "from_airport", "to_airport":
c = item[key]["country"]
travel_countries.add(c)
if item["type"] == "train":
for leg in item["legs"]:
for key in "from_station", "to_station":
c = leg[key]["country"]
travel_countries.add(c)
for c in travel_countries - seen:
seen.add(c)
country = agenda.get_country(c)
assert country
items.append(country)
# Don't include GB in countries visited unless entire trip was GB based
return [c for c in items if c.alpha_2 != "GB"] or items
@functools.cached_property
def show_flags(self) -> bool:
"""Show flags for international trips."""
return len({c for c in self.countries if c.name != "United Kingdom"}) > 1
@property
def countries_str(self) -> str:
"""List of countries visited on this trip."""
return format_list_with_ampersand(
[f"{c.name} {c.flag}" for c in self.countries]
)
@property
def locations_str(self) -> str:
"""List of countries visited on this trip."""
return format_list_with_ampersand(
[
f"{location} ({c.name})" + (f" {c.flag}" if self.show_flags else "")
for location, c in self.locations()
]
)
@property
def country_flags(self) -> str:
"""Countries flags for trip."""
return "".join(c.flag for c in self.countries)
def total_distance(self) -> float:
"""Total distance for trip.
Sums distances for travel items where a distance value is present.
Ignores legs with missing or falsy distances rather than returning None.
"""
total = 0.0
for t in self.travel:
distance = t.get("distance")
if distance:
total += distance
return total
def total_co2_kg(self) -> float | None:
"""Total CO₂ for trip."""
return sum(float(t.get("co2_kg", 0)) for t in self.travel)
@property
def flights(self) -> list[StrDict]:
"""Flights."""
return [item for item in self.travel if item["type"] == "flight"]
def distances_by_transport_type(self) -> list[tuple[str, float]]:
"""Calculate the total distance travelled for each type of transport.
Any travel item with a missing or None 'distance' field is ignored.
"""
transport_distances: defaultdict[str, float] = defaultdict(float)
for item in self.travel:
distance = item.get("distance")
if distance:
transport_type: str = item.get("type", "unknown")
transport_distances[transport_type] += distance
return list(transport_distances.items())
def co2_by_transport_type(self) -> list[tuple[str, float]]:
"""Calculate the total CO₂ emissions for each type of transport.
Any travel item with a missing or None 'co2_kg' field is ignored.
"""
transport_co2: defaultdict[str, float] = defaultdict(float)
for item in self.travel:
co2_kg = item.get("co2_kg")
if co2_kg:
transport_type: str = item.get("type", "unknown")
transport_co2[transport_type] += float(co2_kg)
return list(transport_co2.items())
def elements(self) -> list[TripElement]:
"""Trip elements ordered by time."""
elements: list[TripElement] = []
for item in self.accommodation:
title = "Airbnb" if item.get("operator") == "airbnb" else item["name"]
start = TripElement(
start_time=item["from"],
title=title,
detail=item,
element_type="check-in",
)
elements.append(start)
end = TripElement(
start_time=item["to"],
title=title,
detail=item,
element_type="check-out",
)
elements.append(end)
for item in self.conferences:
location_label = item.get("venue") or item.get("location") or item["name"]
country = agenda.get_country(item.get("country"))
elements.append(
TripElement(
start_time=item["start"],
end_time=item["end"],
title=item["name"],
detail=item,
element_type="conference",
start_loc=location_label,
start_country=country,
end_country=country,
all_day=True,
)
)
for item in self.travel:
if item["type"] == "flight":
name = (
f"{airport_label(item['from_airport'])} → "
+ f"{airport_label(item['to_airport'])}"
)
from_country = agenda.get_country(item["from_airport"]["country"])
to_country = agenda.get_country(item["to_airport"]["country"])
elements.append(
TripElement(
start_time=item["depart"],
end_time=item.get("arrive"),
title=name,
detail=item,
element_type="flight",
start_loc=airport_label(item["from_airport"]),
end_loc=airport_label(item["to_airport"]),
start_country=from_country,
end_country=to_country,
)
)
if item["type"] == "train":
for leg in item["legs"]:
from_country = agenda.get_country(leg["from_station"]["country"])
to_country = agenda.get_country(leg["to_station"]["country"])
assert from_country and to_country
name = f"{leg['from']} → {leg['to']}"
elements.append(
TripElement(
start_time=leg["depart"],
end_time=leg["arrive"],
title=name,
detail=leg,
element_type="train",
start_loc=leg["from"],
end_loc=leg["to"],
start_country=from_country,
end_country=to_country,
)
)
if item["type"] == "ferry":
from_country = agenda.get_country(item["from_terminal"]["country"])
to_country = agenda.get_country(item["to_terminal"]["country"])
name = f"{item['from']} → {item['to']}"
elements.append(
TripElement(
start_time=item["depart"],
end_time=item["arrive"],
title=name,
detail=item,
element_type="ferry",
start_loc=item["from"],
end_loc=item["to"],
start_country=from_country,
end_country=to_country,
)
)
if item["type"] in ("coach", "bus"):
from_country = agenda.get_country(item["from_station"]["country"])
to_country = agenda.get_country(item["to_station"]["country"])
name = f"{item['from']} → {item['to']}"
elements.append(
TripElement(
start_time=item["depart"],
end_time=item.get("arrive"),
title=name,
detail=item,
element_type=item["type"],
start_loc=item["from"],
end_loc=item["to"],
start_country=from_country,
end_country=to_country,
)
)
if item["type"] == "car":
start_location = item.get("from_location", {})
end_location = item.get("to_location", {})
from_country = agenda.get_country(start_location.get("country"))
to_country = agenda.get_country(end_location.get("country"))
name = f"{item.get('from', 'Car journey')} → {item.get('to', 'destination')}"
elements.append(
TripElement(
start_time=item["depart"],
end_time=item.get("arrive"),
title=name,
detail=item,
element_type="car",
start_loc=item.get("from"),
end_loc=item.get("to"),
start_country=from_country,
end_country=to_country,
)
)
return sorted(elements, key=lambda e: utils.as_datetime(e.start_time))
def elements_grouped_by_day(self) -> list[tuple[datetime.date, list[TripElement]]]:
"""Group trip elements by day."""
# Create a dictionary to hold lists of TripElements grouped by their date
grouped_elements: collections.defaultdict[datetime.date, list[TripElement]] = (
collections.defaultdict(list)
)
for element in self.elements():
# Extract the date part of the 'when' attribute
day = utils.as_date(element.start_time)
grouped_elements[day].append(element)
# Sort elements within each day
for day in grouped_elements:
grouped_elements[day].sort(
key=lambda e: (
e.element_type == "check-in", # check-out elements last
e.element_type != "check-out", # check-in elements first
utils.as_datetime(e.start_time), # then sort by time
)
)
# Convert the dictionary to a sorted list of tuples
grouped_elements_list = sorted(grouped_elements.items())
return grouped_elements_list
# Example usage:
# You would call the function with your travel list here to get the results.
@dataclass
@dataclasses.dataclass
class Holiday:
"""Holiay."""
name: str
country: str
date: datetime.date
local_name: str | None = None
@dataclasses.dataclass
class Event:
"""Event."""
name: str
date: datetime.date | datetime.datetime
end_date: datetime.date | datetime.datetime | None = None
title: str | None = None
url: str | None = None
going: bool | None = None
@property
def display_name(self) -> str:
"""Format name for display."""
def as_datetime(self) -> datetime.datetime:
"""Date/time of event."""
d = self.date
t0 = datetime.datetime.min.time()
return (
f"{self.name} ({self.local_name})"
if self.local_name and self.local_name != self.name
else self.name
d
if isinstance(d, datetime.datetime)
else datetime.datetime.combine(d, t0).replace(tzinfo=datetime.timezone.utc)
)
@property
def has_time(self) -> bool:
"""Event has a time associated with it."""
return isinstance(self.date, datetime.datetime)
@property
def as_date(self) -> datetime.date:
"""Date of event."""
return (
self.date.date() if isinstance(self.date, datetime.datetime) else self.date
)
@property
def end_as_date(self) -> datetime.date:
"""Date of event."""
return (
(
self.end_date.date()
if isinstance(self.end_date, datetime.datetime)
else self.end_date
)
if self.end_date
else self.as_date
)
@property
def display_time(self) -> str | None:
"""Time for display on web page."""
return (
self.date.strftime("%H:%M")
if isinstance(self.date, datetime.datetime)
else None
)
@property
def display_timezone(self) -> str | None:
"""Timezone for display on web page."""
return (
self.date.strftime("%z")
if isinstance(self.date, datetime.datetime)
else None
)
def delta_days(self, today: datetime.date) -> str:
"""Return number of days from today as a string."""
delta = (self.as_date - today).days
match delta:
case 0:
return "today"
case 1:
return "1 day"
case _:
return f"{delta:,d} days"
@property
def display_date(self) -> str:
"""Date for display on web page."""
if isinstance(self.date, datetime.datetime):
return self.date.strftime("%a, %d, %b %Y %H:%M %z")
else:
return self.date.strftime("%a, %d, %b %Y")
@property
def display_title(self) -> str:
"""Name for display."""
return self.title or self.name

37
agenda/uk_holiday.py
View file

@ -3,38 +3,29 @@
import json
import os
from datetime import date, datetime, timedelta
from time import time
import httpx
from dateutil.easter import easter
from .types import Holiday, StrDict
url = "https://www.gov.uk/bank-holidays.json"
from .types import Holiday
def json_filename(data_dir: str) -> str:
"""Filename for cached bank holidays."""
assert os.path.exists(data_dir)
return os.path.join(data_dir, "bank-holidays.json")
async def get_holiday_list(data_dir: str) -> list[StrDict]:
"""Download holiday list and save cache."""
filename = json_filename(data_dir)
async with httpx.AsyncClient() as client:
r = await client.get(url)
events: list[StrDict] = r.json()["england-and-wales"]["events"] # check valid
open(filename, "w").write(r.text)
return events
def bank_holiday_list(start_date: date, end_date: date, data_dir: str) -> list[Holiday]:
async def bank_holiday_list(
start_date: date, end_date: date, data_dir: str
) -> list[Holiday]:
"""Date and name of the next UK bank holiday."""
filename = json_filename(data_dir)
url = "https://www.gov.uk/bank-holidays.json"
filename = os.path.join(data_dir, "bank-holidays.json")
mtime = os.path.getmtime(filename)
if (time() - mtime) > 60 * 60 * 6: # six hours
async with httpx.AsyncClient() as client:
r = await client.get(url)
open(filename, "w").write(r.text)
events = json.load(open(filename))["england-and-wales"]["events"]
hols: list[Holiday] = []
for event in json.load(open(filename))["england-and-wales"]["events"]:
for event in events:
event_date = datetime.strptime(event["date"], "%Y-%m-%d").date()
if event_date < start_date:
continue

249
agenda/uk_school_holiday.py
View file

@ -1,249 +0,0 @@
"""UK school holidays (Bristol) via iCalendar."""
from __future__ import annotations
import datetime
import json
import os
import httpx
from .event import Event
school_holiday_page_url = (
"https://www.bristol.gov.uk/residents/schools-learning-and-early-years/"
"school-term-and-holiday-dates"
)
school_holiday_ics_url = (
"https://www.bristol.gov.uk/files/documents/"
"4641-bristol-school-term-and-holiday-dates-2021-2022-and-2022-2023-and-2023-"
"2024-calendar"
)
def ics_filename(data_dir: str) -> str:
"""Filename for cached school-holiday ICS."""
assert os.path.exists(data_dir)
return os.path.join(data_dir, "bristol-school-holidays.ics")
def json_filename(data_dir: str) -> str:
"""Filename for cached parsed school-holiday data."""
assert os.path.exists(data_dir)
return os.path.join(data_dir, "bristol-school-holidays.json")
def _unescape_ics_text(value: str) -> str:
"""Decode escaped ICS text values."""
return (
value.replace("\\n", " ")
.replace("\\N", " ")
.replace("\\,", ",")
.replace("\\;", ";")
.replace("\\\\", "\\")
).strip()
def unfold_ics_lines(ics_text: str) -> list[str]:
"""Unfold folded ICS lines (RFC5545)."""
unfolded: list[str] = []
for raw_line in ics_text.splitlines():
line = raw_line.rstrip("\r\n")
if not line:
continue
if unfolded and line[:1] in {" ", "\t"}:
unfolded[-1] += line[1:]
else:
unfolded.append(line)
return unfolded
def _parse_ics_date(value: str) -> datetime.date:
"""Parse date/date-time values in ICS."""
value = value.strip()
if "T" in value:
date_part = value.split("T", 1)[0]
return datetime.datetime.strptime(date_part, "%Y%m%d").date()
return datetime.datetime.strptime(value, "%Y%m%d").date()
def _is_school_holiday_summary(summary: str) -> bool:
"""Return True if summary looks like a school holiday event."""
lower = summary.lower()
if "holiday" not in lower:
return False
if "bank holiday" in lower:
return False
return True
def _clean_summary(summary: str) -> str:
"""Normalise holiday summary text for display."""
summary = _unescape_ics_text(summary)
# The feed embeds long policy notes in parentheses after the name.
if " (" in summary:
summary = summary.split(" (", 1)[0]
return summary.strip()
def parse_school_holidays_from_ics(ics_text: str) -> list[Event]:
"""Parse school holiday ranges from an ICS file as Events."""
events: list[Event] = []
current: dict[str, str] = {}
def flush_current() -> None:
summary = current.get("SUMMARY")
dtstart = current.get("DTSTART")
dtend = current.get("DTEND")
if not summary or not dtstart or not dtend:
return
clean_summary = _clean_summary(summary)
if not _is_school_holiday_summary(clean_summary):
return
start_date = _parse_ics_date(dtstart)
end_exclusive = _parse_ics_date(dtend)
end_date = end_exclusive - datetime.timedelta(days=1)
if end_date < start_date:
return
events.append(
Event(
name="uk_school_holiday",
date=start_date,
end_date=end_date,
title=clean_summary,
url=school_holiday_page_url,
)
)
for line in unfold_ics_lines(ics_text):
if line == "BEGIN:VEVENT":
current = {}
continue
if line == "END:VEVENT":
flush_current()
current = {}
continue
if ":" not in line:
continue
key_part, value = line.split(":", 1)
key = key_part.split(";", 1)[0].upper()
if key in {"SUMMARY", "DTSTART", "DTEND"}:
current[key] = value.strip()
# De-duplicate by title/date-range.
unique: dict[tuple[str, datetime.date, datetime.date], Event] = {}
for event in events:
end_date = event.end_as_date
unique[(event.title or event.name, event.as_date, end_date)] = event
return sorted(unique.values(), key=lambda item: (item.as_date, item.end_as_date))
def write_school_holidays_json(events: list[Event], data_dir: str) -> None:
"""Write parsed school-holiday events to JSON cache."""
filename = json_filename(data_dir)
payload: list[dict[str, str]] = [
{
"name": event.name,
"title": event.title or event.name,
"start": event.as_date.isoformat(),
"end": event.end_as_date.isoformat(),
"url": event.url or "",
}
for event in events
]
with open(filename, "w", encoding="utf-8") as out:
json.dump(payload, out, indent=2)
def read_school_holidays_json(data_dir: str) -> list[Event]:
"""Read parsed school-holiday events from JSON cache."""
filename = json_filename(data_dir)
if not os.path.exists(filename):
return []
with open(filename, encoding="utf-8") as in_file:
loaded = json.load(in_file)
if not isinstance(loaded, list):
return []
parsed_events: list[Event] = []
for raw_item in loaded:
if not isinstance(raw_item, dict):
continue
title = raw_item.get("title")
start_value = raw_item.get("start")
end_value = raw_item.get("end")
if not (
isinstance(title, str)
and isinstance(start_value, str)
and isinstance(end_value, str)
):
continue
try:
start_date = datetime.date.fromisoformat(start_value)
end_date = datetime.date.fromisoformat(end_value)
except ValueError:
continue
event_url = raw_item.get("url")
parsed_events.append(
Event(
name="uk_school_holiday",
date=start_date,
end_date=end_date,
title=title,
url=event_url if isinstance(event_url, str) and event_url else None,
)
)
return sorted(parsed_events, key=lambda item: (item.as_date, item.end_as_date))
def school_holiday_list(
start_date: datetime.date,
end_date: datetime.date,
data_dir: str,
) -> list[Event]:
"""Get cached school-holiday events overlapping the supplied range."""
items = read_school_holidays_json(data_dir)
return [
item
for item in items
if item.as_date <= end_date and item.end_as_date >= start_date
]
async def get_holiday_list(data_dir: str) -> list[Event]:
"""Download, parse and cache school-holiday data."""
headers = {
"User-Agent": (
"Mozilla/5.0 (X11; Linux x86_64) "
"AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0 Safari/537.36"
),
"Accept": "text/calendar,*/*;q=0.9",
"Referer": school_holiday_page_url,
}
async with httpx.AsyncClient(timeout=60.0, follow_redirects=True) as client:
response = await client.get(school_holiday_ics_url, headers=headers)
response.raise_for_status()
content_type = response.headers.get("content-type", "")
ics_text = response.text
if "text/calendar" not in content_type and "BEGIN:VCALENDAR" not in ics_text:
raise ValueError("School holiday ICS download did not return calendar content")
with open(ics_filename(data_dir), "w", encoding="utf-8") as out:
out.write(ics_text)
events = parse_school_holidays_from_ics(ics_text)
write_school_holidays_json(events, data_dir)
return events

135
agenda/utils.py
View file

@ -1,135 +0,0 @@
"""Utility functions."""
import os
import typing
from datetime import date, datetime, time, timedelta, timezone
from time import time as unixtime
from zoneinfo import ZoneInfo
StrDict = dict[str, typing.Any]
def as_date(d: datetime | date) -> date:
"""Convert datetime to date."""
match d:
case datetime():
return d.date()
case date():
return d
case _:
raise TypeError(f"Unsupported type: {type(d)}")
def as_datetime(d: datetime | date) -> datetime:
"""Date/time of event."""
match d:
case datetime():
return d
case date():
return datetime.combine(d, datetime.min.time()).replace(tzinfo=timezone.utc)
case _:
raise TypeError(f"Unsupported type: {type(d)}")
def timedelta_display(delta: timedelta) -> str:
"""Format timedelta as a human readable string."""
total_seconds = int(delta.total_seconds())
days, remainder = divmod(total_seconds, 24 * 60 * 60)
hours, remainder = divmod(remainder, 60 * 60)
mins, secs = divmod(remainder, 60)
return " ".join(
f"{v} {label}"
for v, label in ((days, "days"), (hours, "hrs"), (mins, "mins"))
if v
)
def plural(value: int, unit: str) -> str:
"""Value + unit with unit written as singular or plural as appropriate."""
return f"{value} {unit}{'s' if value > 1 else ''}"
def human_readable_delta(future_date: date) -> str | None:
"""
Calculate the human-readable time delta for a given future date.
Args:
future_date (date): The future date as a datetime.date object.
Returns:
str: Human-readable time delta.
"""
# Ensure the input is a future date
if future_date <= date.today():
return None
# Calculate the delta
delta = future_date - date.today()
# Convert delta to a more human-readable format
months, days = divmod(delta.days, 30)
weeks, days = divmod(days, 7)
# Formatting the output
parts = [
plural(value, unit)
for value, unit in ((months, "month"), (weeks, "week"), (days, "day"))
if value > 0
]
return " ".join(parts) if parts else None
def filename_timestamp(filename: str, ext: str) -> tuple[datetime, str] | None:
"""Get datetime from filename."""
try:
ts = datetime.strptime(filename, f"%Y-%m-%d_%H:%M:%S.{ext}")
except ValueError:
return None
return (ts, filename)
def get_most_recent_file(directory: str, ext: str) -> str | None:
"""Get most recent file from directory."""
existing = [
x for x in (filename_timestamp(f, ext) for f in os.listdir(directory)) if x
]
if not existing:
return None
existing.sort(reverse=True)
return os.path.join(directory, existing[0][1])
def make_waste_dir(data_dir: str) -> None:
"""Make waste dir if missing."""
waste_dir = os.path.join(data_dir, "waste")
if not os.path.exists(waste_dir):
os.mkdir(waste_dir)
async def time_function(
name: str,
func: typing.Callable[..., typing.Coroutine[typing.Any, typing.Any, typing.Any]],
*args: typing.Any,
**kwargs: typing.Any,
) -> tuple[str, typing.Any, float, Exception | None]:
"""Time the execution of an asynchronous function."""
start_time, result, exception = unixtime(), None, None
try:
result = await func(*args, **kwargs)
except Exception as e:
exception = e
end_time = unixtime()
return name, result, end_time - start_time, exception
def depart_datetime(item: StrDict) -> datetime:
"""Return a datetime for this travel item.
If the travel item already has a datetime return that, otherwise if the
departure time is just a date return midnight UTC for that date.
"""
depart = item["depart"]
if isinstance(depart, datetime):
return depart
return datetime.combine(depart, time.min).replace(tzinfo=ZoneInfo("UTC"))

209
agenda/waste_schedule.py Normal file
View file

@ -0,0 +1,209 @@
"""Waste collection schedules."""
import json
import os
import re
import typing
from collections import defaultdict
from datetime import date, datetime, time, timedelta
import httpx
import lxml.html
from . import uk_time
from .types import Event
ttl_hours = 12
def make_waste_dir(data_dir: str) -> None:
"""Make waste dir if missing."""
waste_dir = os.path.join(data_dir, "waste")
if not os.path.exists(waste_dir):
os.mkdir(waste_dir)
async def get_html(data_dir: str, postcode: str, uprn: str) -> str:
"""Get waste schedule."""
now = datetime.now()
waste_dir = os.path.join(data_dir, "waste")
make_waste_dir(data_dir)
existing_data = os.listdir(waste_dir)
existing = [f for f in existing_data if f.endswith(".html")]
if existing:
recent_filename = max(existing)
recent = datetime.strptime(recent_filename, "%Y-%m-%d_%H:%M.html")
delta = now - recent
if existing and delta < timedelta(hours=ttl_hours):
return open(os.path.join(waste_dir, recent_filename)).read()
now_str = now.strftime("%Y-%m-%d_%H:%M")
filename = f"{waste_dir}/{now_str}.html"
forms_base_url = "https://forms.n-somerset.gov.uk"
# url2 = "https://forms.n-somerset.gov.uk/Waste/CollectionSchedule/ViewSchedule"
url = "https://forms.n-somerset.gov.uk/Waste/CollectionSchedule"
async with httpx.AsyncClient() as client:
r = await client.post(
url,
data={
"PreviousHouse": "",
"PreviousPostcode": "-",
"Postcode": postcode,
"SelectedUprn": uprn,
},
)
form_post_html = r.text
pattern = r'<h2>Object moved to <a href="([^"]*)">here<\/a>\.<\/h2>'
m = re.search(pattern, form_post_html)
if m:
r = await client.get(forms_base_url + m.group(1))
html = r.text
open(filename, "w").write(html)
return html
def parse_waste_schedule_date(day_and_month: str) -> date:
"""Parse waste schedule date."""
today = date.today()
this_year = today.year
date_format = "%A %d %B %Y"
d = datetime.strptime(f"{day_and_month} {this_year}", date_format).date()
if d < today:
d = datetime.strptime(f"{day_and_month} {this_year + 1}", date_format).date()
return d
def parse(root: lxml.html.HtmlElement) -> list[Event]:
"""Parse waste schedule."""
tbody = root.find(".//table/tbody")
assert tbody is not None
by_date = defaultdict(list)
for e_service, e_next_date, e_following in tbody:
assert e_service.text and e_next_date.text and e_following.text
service = e_service.text
next_date = parse_waste_schedule_date(e_next_date.text)
following_date = parse_waste_schedule_date(e_following.text)
by_date[next_date].append(service)
by_date[following_date].append(service)
return [
Event(
name="waste_schedule",
date=uk_time(d, time(6, 30)),
title="🗑️ Backwell: " + ", ".join(services),
)
for d, services in by_date.items()
]
BristolSchedule = list[dict[str, typing.Any]]
async def get_bristol_data(data_dir: str, uprn: str) -> BristolSchedule:
"""Get Bristol Waste schedule, with cache."""
now = datetime.now()
waste_dir = os.path.join(data_dir, "waste")
make_waste_dir(data_dir)
existing_data = os.listdir(waste_dir)
existing = [f for f in existing_data if f.endswith(f"_{uprn}.json")]
if existing:
recent_filename = max(existing)
recent = datetime.strptime(recent_filename, f"%Y-%m-%d_%H:%M_{uprn}.json")
delta = now - recent
def get_from_recent() -> BristolSchedule:
json_data = json.load(open(os.path.join(waste_dir, recent_filename)))
return typing.cast(BristolSchedule, json_data["data"])
if existing and delta < timedelta(hours=ttl_hours):
return get_from_recent()
try:
r = await get_bristol_gov_uk_data(uprn)
except httpx.ReadTimeout:
return get_from_recent()
with open(f'{waste_dir}/{now.strftime("%Y-%m-%d_%H:%M")}_{uprn}.json', "wb") as out:
out.write(r.content)
return typing.cast(BristolSchedule, r.json()["data"])
async def get_bristol_gov_uk_data(uprn: str) -> httpx.Response:
"""Get JSON from Bristol City Council."""
UA = "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"
HEADERS = {
"Accept": "*/*",
"Accept-Language": "en-GB,en;q=0.9",
"Connection": "keep-alive",
"Ocp-Apim-Subscription-Key": "47ffd667d69c4a858f92fc38dc24b150",
"Ocp-Apim-Trace": "true",
"Origin": "https://bristolcouncil.powerappsportals.com",
"Referer": "https://bristolcouncil.powerappsportals.com/",
"Sec-Fetch-Dest": "empty",
"Sec-Fetch-Mode": "cors",
"Sec-Fetch-Site": "cross-site",
"Sec-GPC": "1",
"User-Agent": UA,
}
_uprn = str(uprn).zfill(12)
async with httpx.AsyncClient(timeout=20) as client:
# Initialise form
payload = {"servicetypeid": "7dce896c-b3ba-ea11-a812-000d3a7f1cdc"}
response = await client.get(
"https://bristolcouncil.powerappsportals.com/completedynamicformunauth/",
headers=HEADERS,
params=payload,
)
host = "bcprdapidyna002.azure-api.net"
# Set the search criteria
payload = {"Uprn": "UPRN" + _uprn}
response = await client.post(
f"https://{host}/bcprdfundyna001-llpg/DetailedLLPG",
headers=HEADERS,
json=payload,
)
# Retrieve the schedule
payload = {"uprn": _uprn}
response = await client.post(
f"https://{host}/bcprdfundyna001-alloy/NextCollectionDates",
headers=HEADERS,
json=payload,
)
return response
async def get_bristol_gov_uk(start_date: date, data_dir: str, uprn: str) -> list[Event]:
"""Get waste collection schedule from Bristol City Council."""
data = await get_bristol_data(data_dir, uprn)
by_date: defaultdict[date, list[str]] = defaultdict(list)
for item in data:
service = item["containerName"]
service = "Recycling" if "Recycling" in service else service.partition(" ")[2]
for collection in item["collection"]:
for collection_date_key in ["nextCollectionDate", "lastCollectionDate"]:
d = date.fromisoformat(collection[collection_date_key][:10])
if d < start_date:
continue
if service not in by_date[d]:
by_date[d].append(service)
return [
Event(name="waste_schedule", date=d, title="🗑️ Bristol: " + ", ".join(services))
for d, services in by_date.items()
]

105
agenda/weather.py
View file

@ -1,105 +0,0 @@
"""Weather forecast using OpenWeatherMap One Call API."""
import json
import os
from datetime import datetime
import pyowm
def _cache_path(data_dir: str, lat: float, lon: float) -> str:
"""Path for weather cache file."""
weather_dir = os.path.join(data_dir, "weather")
os.makedirs(weather_dir, exist_ok=True)
return os.path.join(weather_dir, f"{lat:.2f}_{lon:.2f}.json")
def _is_fresh(path: str, max_age_hours: int = 24) -> bool:
"""Return True if the cache file exists and is recent enough."""
if not os.path.exists(path):
return False
age = datetime.now().timestamp() - os.path.getmtime(path)
return age < max_age_hours * 3600
def get_forecast(
data_dir: str,
api_key: str,
lat: float,
lon: float,
cache_only: bool = False,
) -> list[dict]:
"""Return 8-day daily forecast for lat/lon, caching results for 24 hours.
If cache_only=True, return cached data if available (even if stale) and
never call the API. Returns [] if no cache exists.
"""
cache_file = _cache_path(data_dir, lat, lon)
if _is_fresh(cache_file):
with open(cache_file) as f:
return json.load(f) # type: ignore[no-any-return]
if cache_only:
if os.path.exists(cache_file):
with open(cache_file) as f:
return json.load(f) # type: ignore[no-any-return]
return []
owm = pyowm.OWM(api_key)
mgr = owm.weather_manager()
result = mgr.one_call(lat=lat, lon=lon)
forecasts = []
for day in result.forecast_daily:
dt = datetime.fromtimestamp(day.ref_time)
temp = day.temperature("celsius")
forecasts.append(
{
"date": dt.date().isoformat(),
"status": day.status,
"detailed_status": day.detailed_status,
"temp_min": round(temp["min"]),
"temp_max": round(temp["max"]),
"precipitation_probability": day.precipitation_probability,
"icon": day.weather_icon_name,
}
)
with open(cache_file, "w") as f:
json.dump(forecasts, f)
return forecasts
def trip_latlon(trip: object) -> tuple[float, float] | None:
"""Return (lat, lon) for the primary destination of a trip, or None."""
from agenda.types import Trip
assert isinstance(trip, Trip)
for item in list(trip.accommodation) + list(trip.conferences):
if "latitude" in item and "longitude" in item:
return (float(item["latitude"]), float(item["longitude"]))
return None
def get_trip_weather(
data_dir: str,
api_key: str,
trip: object,
cache_only: bool = False,
) -> dict[str, dict]:
"""Return forecast for a trip keyed by date ISO string.
Returns an empty dict if no location is known or the API call fails.
If cache_only=True, never call the API (returns stale or empty data).
"""
latlon = trip_latlon(trip)
if not latlon:
return {}
lat, lon = latlon
try:
forecasts = get_forecast(data_dir, api_key, lat, lon, cache_only=cache_only)
except Exception:
return {}
return {f["date"]: f for f in forecasts}

1002
docs/personal-data-yaml.md
View file

File diff suppressed because it is too large Load diff

0
frontend/index.js
View file

224
get_airport.py
View file

@ -1,224 +0,0 @@
#!/usr/bin/python3
import sys
from typing import Any, Dict, List
import requests
import yaml
# Define the base URL for the Wikidata API
WIKIDATA_API_URL = "https://www.wikidata.org/w/api.php"
def get_entity_label(qid: str) -> str | None:
"""
Fetches the English label for a given Wikidata entity QID.
Args:
qid (str): The Wikidata entity ID (e.g., "Q6106").
Returns:
Optional[str]: The English label of the entity, or None if not found.
"""
params: Dict[str, str] = {
"action": "wbgetentities",
"ids": qid,
"format": "json",
"props": "labels",
"languages": "en",
}
try:
response = requests.get(WIKIDATA_API_URL, params=params)
response.raise_for_status()
entity = response.json().get("entities", {}).get(qid, {})
return entity.get("labels", {}).get("en", {}).get("value")
except requests.exceptions.RequestException as e:
print(f"Error fetching label for QID {qid}: {e}", file=sys.stderr)
return None
def get_entity_details(qid: str) -> dict[str, Any] | None:
"""
Fetches and processes detailed information for a given airport QID.
Args:
qid (str): The QID of the airport Wikidata entity.
Returns:
Optional[Dict[str, Any]]: A dictionary containing the detailed airport data.
"""
params: Dict[str, str] = {
"action": "wbgetentities",
"ids": qid,
"format": "json",
"props": "claims|labels",
}
try:
response = requests.get(WIKIDATA_API_URL, params=params)
response.raise_for_status()
entity = response.json().get("entities", {}).get(qid, {})
if not entity:
return None
claims = entity.get("claims", {})
# Helper to safely extract claim values
def get_simple_claim_value(prop_id: str) -> str | None:
claim = claims.get(prop_id)
if not claim:
return None
v = claim[0].get("mainsnak", {}).get("datavalue", {}).get("value")
assert isinstance(v, str) or v is None
return v
# Get IATA code, name, and website
iata = get_simple_claim_value("P238")
name = entity.get("labels", {}).get("en", {}).get("value")
website = get_simple_claim_value("P856")
# Get City Name by resolving its QID
city_qid_claim = claims.get("P131")
city_name = None
if city_qid_claim:
city_qid = (
city_qid_claim[0]
.get("mainsnak", {})
.get("datavalue", {})
.get("value", {})
.get("id")
)
if city_qid:
city_name = get_entity_label(city_qid)
# Get coordinates
coords_claim = claims.get("P625")
latitude, longitude = None, None
if coords_claim:
coords = (
coords_claim[0]
.get("mainsnak", {})
.get("datavalue", {})
.get("value", {})
)
latitude = coords.get("latitude")
longitude = coords.get("longitude")
# Get elevation
elevation_claim = claims.get("P2044")
elevation = None
if elevation_claim:
amount_str = (
elevation_claim[0]
.get("mainsnak", {})
.get("datavalue", {})
.get("value", {})
.get("amount")
)
if amount_str:
elevation = float(amount_str) if "." in amount_str else int(amount_str)
# Get Country Code
country_claim = claims.get("P17")
country_code = None
if country_claim:
country_qid = (
country_claim[0]
.get("mainsnak", {})
.get("datavalue", {})
.get("value", {})
.get("id")
)
if country_qid:
# Fetch the ISO 3166-1 alpha-2 code (P297) for the country entity
country_code_params = {
"action": "wbgetclaims",
"entity": country_qid,
"property": "P297",
"format": "json",
}
country_res = requests.get(WIKIDATA_API_URL, params=country_code_params)
country_res.raise_for_status()
country_claims = country_res.json().get("claims", {}).get("P297")
if country_claims:
code = (
country_claims[0]
.get("mainsnak", {})
.get("datavalue", {})
.get("value")
)
if code:
country_code = code.lower()
data = {
"iata": iata,
"name": name,
"city": city_name,
"qid": qid,
"latitude": latitude,
"longitude": longitude,
"elevation": elevation,
"website": website,
"country": country_code,
}
# Return the final structure, filtering out null values for cleaner output
return {iata: {k: v for k, v in data.items() if v is not None}}
except requests.exceptions.RequestException as e:
print(f"Error fetching entity details for QID {qid}: {e}", file=sys.stderr)
return None
def find_airport_by_iata(iata_code: str) -> dict[str, Any] | None:
"""
Finds an airport by its IATA code using Wikidata's search API.
Args:
iata_code (str): The IATA code of the airport (e.g., "PDX").
Returns:
Optional[Dict[str, Any]]: A dictionary with the airport data or None.
"""
params: Dict[str, str] = {
"action": "query",
"list": "search",
"srsearch": f"haswbstatement:P238={iata_code.upper()}",
"format": "json",
}
try:
response = requests.get(WIKIDATA_API_URL, params=params)
response.raise_for_status()
search_results: List[Dict[str, Any]] = (
response.json().get("query", {}).get("search", [])
)
if not search_results:
print(f"No airport found with IATA code: {iata_code}", file=sys.stderr)
return None
qid = search_results[0]["title"]
return get_entity_details(qid)
except requests.exceptions.RequestException as e:
print(f"Error searching on Wikidata API: {e}", file=sys.stderr)
return None
if __name__ == "__main__":
if len(sys.argv) != 2:
print("Usage: python airport_lookup.py <IATA_CODE>", file=sys.stderr)
sys.exit(1)
iata_code_arg = sys.argv[1]
airport_data = find_airport_by_iata(iata_code_arg)
if airport_data:
print(
yaml.safe_dump(
airport_data,
default_flow_style=False,
allow_unicode=True,
sort_keys=False,
),
end="",
)

21
licenses/fast-flights-MIT.txt
View file

@ -1,21 +0,0 @@
MIT License
Copyright (c) 2024-PRESENT fast-flights Contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

28
package.json
View file

@ -1,28 +0,0 @@
{
"name": "agenda",
"version": "1.0.0",
"directories": {
"test": "tests"
},
"repository": {
"type": "git",
"url": "https://git.4angle.com/edward/agenda.git"
},
"license": "ISC",
"devDependencies": {
"copy-webpack-plugin": "^12.0.2",
"eslint": "^9.2.0",
"webpack": "^5.91.0",
"webpack-cli": "^5.1.4"
},
"dependencies": {
"@fullcalendar/core": "^6.1.11",
"@fullcalendar/daygrid": "^6.1.11",
"@fullcalendar/list": "^6.1.11",
"@fullcalendar/timegrid": "^6.1.11",
"bootstrap": "^5.3.3",
"es-module-shims": "^1.8.3",
"leaflet": "^1.9.4",
"leaflet.geodesic": "^2.7.1"
}
}

19
parse_airbnb.py
View file

@ -1,19 +0,0 @@
#!/usr/bin/python3
import sys
import yaml
from agenda.airbnb import parse_multiple_files
def main() -> None:
"""Main function."""
filenames = sys.argv[1:]
bookings = parse_multiple_files(filenames)
print(yaml.dump(bookings, sort_keys=False))
if __name__ == "__main__":
main()

3
requirements.txt
View file

@ -9,6 +9,3 @@ dateutil
ephem
flask
requests
emoji
timezonefinder
playwright

9
run.fcgi
View file

@ -1,9 +1,8 @@
#!/usr/bin/python3
from flipflop import WSGIServer # type: ignore[import-untyped]
from flipflop import WSGIServer
import sys
sys.path.append('/home/edward/src/2021/agenda')
from web_view import app
sys.path.append("/home/edward/src/agenda") # isort:skip
from web_view import app # isort:skip
if __name__ == "__main__":
if __name__ == '__main__':
WSGIServer(app).run()

15
scripts/add-car-journeys-for-trip
View file

@ -1,15 +0,0 @@
#!/usr/bin/python3
import os
import sys
SCRIPT_PATH = os.path.realpath(__file__)
SCRIPT_DIR = os.path.dirname(SCRIPT_PATH)
REPO_ROOT = os.path.dirname(SCRIPT_DIR)
if REPO_ROOT not in sys.path:
sys.path.insert(0, REPO_ROOT)
from agenda.car_journey_yaml import main
if __name__ == "__main__":
raise SystemExit(main())

15
scripts/add-new-conference
View file

@ -1,15 +0,0 @@
#!/usr/bin/python3
import os
import sys
SCRIPT_PATH = os.path.realpath(__file__)
SCRIPT_DIR = os.path.dirname(SCRIPT_PATH)
REPO_ROOT = os.path.dirname(SCRIPT_DIR)
if REPO_ROOT not in sys.path:
sys.path.insert(0, REPO_ROOT)
from agenda.add_new_conference import main
if __name__ == "__main__":
raise SystemExit(main())

15
scripts/build_airport_yaml
View file

@ -1,15 +0,0 @@
#!/usr/bin/python3
import os
import sys
SCRIPT_PATH = os.path.realpath(__file__)
SCRIPT_DIR = os.path.dirname(SCRIPT_PATH)
REPO_ROOT = os.path.dirname(SCRIPT_DIR)
if REPO_ROOT not in sys.path:
sys.path.insert(0, REPO_ROOT)
from agenda.build_place_yaml import airport_main
if __name__ == "__main__":
raise SystemExit(airport_main())

15
scripts/build_station_yaml
View file

@ -1,15 +0,0 @@
#!/usr/bin/python3
import os
import sys
SCRIPT_PATH = os.path.realpath(__file__)
SCRIPT_DIR = os.path.dirname(SCRIPT_PATH)
REPO_ROOT = os.path.dirname(SCRIPT_DIR)
if REPO_ROOT not in sys.path:
sys.path.insert(0, REPO_ROOT)
from agenda.build_place_yaml import station_main
if __name__ == "__main__":
raise SystemExit(station_main())

15
scripts/generate-flight-booking-yaml
View file

@ -1,15 +0,0 @@
#!/usr/bin/python3
import os
import sys
SCRIPT_PATH = os.path.realpath(__file__)
SCRIPT_DIR = os.path.dirname(SCRIPT_PATH)
REPO_ROOT = os.path.dirname(SCRIPT_DIR)
if REPO_ROOT not in sys.path:
sys.path.insert(0, REPO_ROOT)
from agenda.generate_booking_yaml import flight_main
if __name__ == "__main__":
raise SystemExit(flight_main())

15
scripts/generate-train-booking-yaml
View file

@ -1,15 +0,0 @@
#!/usr/bin/python3
import os
import sys
SCRIPT_PATH = os.path.realpath(__file__)
SCRIPT_DIR = os.path.dirname(SCRIPT_PATH)
REPO_ROOT = os.path.dirname(SCRIPT_DIR)
if REPO_ROOT not in sys.path:
sys.path.insert(0, REPO_ROOT)
from agenda.generate_booking_yaml import train_main
if __name__ == "__main__":
raise SystemExit(train_main())

15
scripts/import-car-journeys-from-timeline
View file

@ -1,15 +0,0 @@
#!/usr/bin/python3
import os
import sys
SCRIPT_PATH = os.path.realpath(__file__)
SCRIPT_DIR = os.path.dirname(SCRIPT_PATH)
REPO_ROOT = os.path.dirname(SCRIPT_DIR)
if REPO_ROOT not in sys.path:
sys.path.insert(0, REPO_ROOT)
from agenda.car_journey_timeline import main
if __name__ == "__main__":
raise SystemExit(main())

381
static/css/trips.css
View file

@ -1,381 +0,0 @@
@import url('https://fonts.googleapis.com/css2?family=Playfair+Display:wght@600;700&family=Source+Sans+3:ital,wght@0,300;0,400;0,600;1,300&family=JetBrains+Mono:wght@400;500&display=swap');
:root {
--t-navy: #1e2d4a;
--t-slate: #374869;
--t-gold: #b8860b;
--t-amber: #e8a820;
--t-cream: #f9f7f3;
--t-white: #ffffff;
--t-muted: #7a8aa8;
--t-border: #dde3ed;
--t-text: #1e2533;
--t-shadow: rgba(30, 45, 74, 0.08);
}
/* Text pane background on list page */
.text-content {
background: var(--t-cream) !important;
padding-left: 12px;
padding-top: 12px;
}
/* ===========================
TRIP CARDS (list view)
=========================== */
.trip-card {
background: var(--t-white);
border-radius: 8px;
border: 1px solid var(--t-border);
border-left: 3px solid #8098c0;
box-shadow: 0 1px 4px var(--t-shadow);
padding: 14px 18px;
margin-bottom: 14px;
transition: box-shadow 0.15s ease, transform 0.15s ease;
}
.trip-card:hover {
box-shadow: 0 4px 16px rgba(30, 45, 74, 0.14);
transform: translateY(-1px);
}
.trip-card.trip-current {
border-left-color: var(--t-gold);
}
/* Trip name heading */
.trip-name {
margin-bottom: 2px;
font-size: 1.1rem;
line-height: 1.3;
}
.trip-name a {
font-family: 'Playfair Display', Georgia, 'Times New Roman', serif;
font-weight: 700;
color: var(--t-navy);
text-decoration: none;
}
.trip-name a:hover {
color: var(--t-gold);
}
.trip-name small {
font-family: 'JetBrains Mono', 'Courier New', monospace;
font-size: 0.7rem;
color: var(--t-muted);
font-weight: 400;
}
/* Countries as inline chips */
.trip-countries {
display: flex !important;
flex-wrap: wrap;
gap: 5px;
margin: 6px 0;
padding: 0;
}
.trip-countries li {
display: inline-flex;
align-items: center;
gap: 4px;
background: #eef2f8;
border: 1px solid #d5dce8;
border-radius: 20px;
padding: 1px 10px;
font-size: 0.8rem;
color: var(--t-text);
}
/* Dates */
.trip-dates {
font-size: 0.83rem;
color: var(--t-muted);
margin: 2px 0;
}
/* Stats row — pill chips */
.trip-stats {
display: flex;
flex-wrap: wrap;
gap: 4px;
margin: 7px 0;
}
.trip-stat {
display: inline-block;
background: #f3f6fa;
border: 1px solid #dde3ed;
border-radius: 20px;
padding: 2px 10px;
font-size: 0.74rem;
color: var(--t-slate);
white-space: nowrap;
font-family: 'JetBrains Mono', monospace;
}
/* School holiday info */
.school-holiday-info {
display: flex;
flex-wrap: wrap;
align-items: center;
gap: 6px;
margin: 4px 0 8px;
font-size: 0.82rem;
}
/* Inline weather chip in day headers */
.trip-weather-inline {
font-size: 0.78rem;
color: var(--t-muted);
font-weight: 400;
letter-spacing: 0;
text-transform: none;
display: inline-flex;
align-items: center;
gap: 3px;
vertical-align: middle;
}
/* Day sub-headers within a trip card */
.trip-day-header {
font-size: 0.75rem;
font-weight: 700;
text-transform: uppercase;
letter-spacing: 0.1em;
color: var(--t-navy);
margin: 18px 0 6px;
padding: 4px 0 4px 10px;
border-left: 3px solid var(--t-amber);
background: linear-gradient(to right, rgba(232, 168, 32, 0.07), transparent);
}
/* Condensed check-out line */
.trip-checkout {
font-size: 0.84rem;
padding: 4px 8px;
color: var(--t-muted);
border-left: 2px solid #aacde8;
margin: 3px 0 3px 1px;
}
/* Transport/accommodation element rows */
.trip-element {
font-size: 0.84rem;
padding: 2px 0;
color: var(--t-text);
line-height: 1.5;
}
/* ===========================
INLINE CONFERENCE CARDS
(within trip_item macro)
=========================== */
.trip-conference-card {
background: #fffef4;
border: 1px solid #e4d46c;
border-radius: 6px;
padding: 9px 13px;
margin: 5px 0;
}
.trip-conference-card .card-body {
padding: 0;
}
.trip-conference-card .card-title {
font-size: 0.9rem;
font-weight: 600;
margin-bottom: 3px;
color: var(--t-navy);
}
.trip-conference-card .card-text {
font-size: 0.82rem;
margin-bottom: 0;
color: var(--t-text);
}
/* ===========================
TRIP PAGE ACCOMMODATION
=========================== */
.trip-accommodation-card {
background: #f3faff;
border: 1px solid #aacde8;
border-radius: 6px;
padding: 9px 13px;
margin: 5px 0;
}
.trip-accommodation-card .card-body { padding: 0; }
.trip-accommodation-card .card-title {
font-size: 0.9rem;
font-weight: 600;
margin-bottom: 3px;
color: var(--t-navy);
}
.trip-accommodation-card .card-text {
font-size: 0.82rem;
margin-bottom: 0;
}
/* ===========================
TRIP PAGE TRANSPORT
=========================== */
.trip-transport-card {
background: #f7f9fc;
border: 1px solid #d0dbe8;
border-radius: 6px;
padding: 9px 13px;
margin: 5px 0;
}
.trip-transport-card .card-body { padding: 0; }
.trip-transport-card .card-title {
font-size: 0.9rem;
font-weight: 600;
margin-bottom: 3px;
color: var(--t-navy);
}
.trip-transport-card .card-text {
font-size: 0.82rem;
margin-bottom: 0;
}
/* ===========================
TRIP PAGE EVENTS
=========================== */
.trip-event-card {
background: #fdf5ff;
border: 1px solid #d4a8e8;
border-radius: 6px;
padding: 9px 13px;
margin: 5px 0;
}
.trip-event-card .card-body { padding: 0; }
.trip-event-card .card-title {
font-size: 0.9rem;
font-weight: 600;
margin-bottom: 3px;
color: var(--t-navy);
}
.trip-event-card .card-text {
font-size: 0.82rem;
margin-bottom: 0;
}
/* ===========================
TRIP PAGE HEADER & TYPOGRAPHY
=========================== */
.trip-page-title {
font-family: 'Playfair Display', Georgia, 'Times New Roman', serif;
font-weight: 700;
font-size: 1.8rem;
color: var(--t-navy);
margin-bottom: 4px;
}
/* Section divider headings on the trip detail page */
h3.trip-section-h,
h4.trip-section-h {
font-size: 0.72rem;
font-weight: 700;
text-transform: uppercase;
letter-spacing: 0.14em;
color: var(--t-muted);
margin-top: 22px;
margin-bottom: 8px;
padding-bottom: 4px;
border-bottom: 1px solid var(--t-border);
}
/* Prev/next nav */
.trip-prev-next {
font-size: 0.83rem;
color: var(--t-muted);
margin-bottom: 16px;
}
.trip-prev-next a {
color: var(--t-slate);
text-decoration: none;
font-weight: 500;
}
.trip-prev-next a:hover {
color: var(--t-gold);
}
/* ===========================
TRIP LIST PAGE SUMMARY BOX
=========================== */
.trip-list-summary {
background: var(--t-navy);
color: #c8d4e8;
border-radius: 8px;
padding: 14px 18px;
margin-bottom: 18px;
margin-top: 8px;
}
.trip-list-summary h2 {
color: #f0f4fa;
font-family: 'Playfair Display', Georgia, serif;
font-size: 1.25rem;
margin-bottom: 4px;
font-weight: 700;
}
.trip-list-summary a {
color: var(--t-amber);
text-decoration: none;
font-size: 0.82rem;
}
.trip-list-summary a:hover {
text-decoration: underline;
}
.summary-stats-row {
display: flex;
flex-wrap: wrap;
gap: 18px;
margin-top: 10px;
}
.summary-stat {
display: flex;
flex-direction: column;
}
.summary-stat-label {
font-size: 0.62rem;
text-transform: uppercase;
letter-spacing: 0.1em;
opacity: 0.55;
line-height: 1;
margin-bottom: 2px;
}
.summary-stat-value {
font-family: 'JetBrains Mono', monospace;
font-size: 0.82rem;
color: var(--t-amber);
font-weight: 500;
}

270
static/js/map.js
View file

@ -1,270 +0,0 @@
if (![].at) {
Array.prototype.at = function(pos) { return this.slice(pos, pos + 1)[0]; };
}
var emojiByType = {
"station": "🚉",
"airport": "✈️",
"ferry_terminal": "🚢",
"coach_station": "🚌",
"bus_stop": "🚏",
"home": "🏠",
"car_stop": "🚗",
"accommodation": "🏨",
"conference": "🖥️",
"event": "🍷"
};
function getIconMetrics(zoom) {
var outerSize;
if (zoom <= 3) {
outerSize = 20;
} else if (zoom <= 5) {
outerSize = 26;
} else if (zoom <= 8) {
outerSize = 32;
} else {
outerSize = 38;
}
var innerSize = outerSize - 6;
var fontSize = Math.max(12, Math.round(innerSize * 0.65));
return {
outerSize: outerSize,
fontSize: fontSize,
anchor: [outerSize / 2, outerSize / 2]
};
}
function emojiIcon(emoji, zoom) {
var symbol = emoji || "📍";
var metrics = getIconMetrics(zoom);
var iconStyle = [
"<div style=\"width:", metrics.outerSize, "px;height:", metrics.outerSize, "px;",
"border-radius:50%;border:1px solid rgba(0,0,0,0.2);background-color:rgba(255,255,255,0.9);",
"box-shadow:0 1px 3px rgba(0,0,0,0.2);display:flex;justify-content:center;align-items:center;\">",
"<div style=\"font-size:", metrics.fontSize, "px;line-height:1;\">",
symbol,
"</div></div>"
].join("");
return L.divIcon({
className: "custom-div-icon",
html: iconStyle,
iconSize: [metrics.outerSize, metrics.outerSize],
iconAnchor: metrics.anchor
});
}
function build_map(map_id, coordinates, routes) {
var bounds = coordinates.map(function(station) { return [station.latitude, station.longitude]; });
function addGeoJsonCoordinatesToBounds(geojsonCoordinates) {
if (!geojsonCoordinates) return;
if (typeof geojsonCoordinates[0] === "number" && typeof geojsonCoordinates[1] === "number") {
bounds.push([geojsonCoordinates[1], geojsonCoordinates[0]]);
return;
}
geojsonCoordinates.forEach(addGeoJsonCoordinatesToBounds);
}
function addGeoJsonGeometryToBounds(geometry) {
if (!geometry) return;
if (geometry.type === "GeometryCollection") {
geometry.geometries.forEach(addGeoJsonGeometryToBounds);
return;
}
addGeoJsonCoordinatesToBounds(geometry.coordinates);
}
function addGeoJsonToBounds(geojson) {
if (!geojson) return;
if (geojson.type === "FeatureCollection") {
geojson.features.forEach(function(feature) {
addGeoJsonGeometryToBounds(feature.geometry);
});
return;
}
if (geojson.type === "Feature") {
addGeoJsonGeometryToBounds(geojson.geometry);
return;
}
addGeoJsonGeometryToBounds(geojson);
}
routes.forEach(function(r) {
if (r.from) bounds.push(r.from);
if (r.to) bounds.push(r.to);
if (r.geojson) addGeoJsonToBounds(JSON.parse(r.geojson));
});
var map = bounds.length > 0
? L.map(map_id).fitBounds(bounds)
: L.map(map_id).setView([20, 0], 2);
L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
attribution: '&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors'
}).addTo(map);
var markers = [];
var offset_lines = [];
function getIconBounds(latlng, zoom) {
var iconSize = getIconMetrics(zoom).outerSize;
if (!latlng) return null;
var pixel = map.project(latlng, zoom);
var sw = map.unproject([pixel.x - iconSize / 2, pixel.y + iconSize / 2], zoom);
var ne = map.unproject([pixel.x + iconSize / 2, pixel.y - iconSize / 2], zoom);
return L.latLngBounds(sw, ne);
}
function calculateCentroid(markers) {
var latSum = 0, lngSum = 0, count = 0;
markers.forEach(function(marker) {
latSum += marker.getLatLng().lat;
lngSum += marker.getLatLng().lng;
count += 1;
});
return count > 0 ? L.latLng(latSum / count, lngSum / count) : null;
}
// Function to detect and group overlapping markers
function getOverlappingGroups(zoom) {
var groups = [];
var visited = new Set();
markers.forEach(function(marker) {
if (visited.has(marker)) {
return;
}
var group = [];
var markerBounds = getIconBounds(marker.getLatLng(), zoom);
markers.forEach(function(otherMarker) {
var otherBounds = getIconBounds(otherMarker.getLatLng(), zoom);
if (marker !== otherMarker && markerBounds && otherBounds && markerBounds.intersects(otherBounds)) {
group.push(otherMarker);
visited.add(otherMarker);
}
});
if (group.length > 0) {
group.push(marker); // Add the original marker to the group
groups.push(group);
visited.add(marker);
}
});
return groups;
}
function displaceMarkers(group, zoom) {
var markerPixelSize = Math.max(18, getIconMetrics(zoom).outerSize);
var mapRef = group[0]._map; // Assuming all markers are on the same map
var centroid = calculateCentroid(group);
if (!centroid) {
return;
}
var centroidPoint = mapRef.project(centroid, zoom);
var radius = markerPixelSize * 1.1;
var angleIncrement = (2 * Math.PI) / group.length;
group.forEach(function(marker, index) {
var angle = index * angleIncrement;
var newX = centroidPoint.x + radius * Math.cos(angle);
var newY = centroidPoint.y + radius * Math.sin(angle);
var newPoint = L.point(newX, newY);
var newLatLng = mapRef.unproject(newPoint, zoom);
var originalPos = marker.originalLatLng;
marker.setLatLng(newLatLng);
marker.polyline = L.polyline([originalPos, newLatLng], {color: "#909090", weight: 1, dashArray: "2,4"}).addTo(mapRef);
offset_lines.push(marker.polyline);
});
}
function updateMarkerIcons(zoom) {
markers.forEach(function(marker) {
marker.setIcon(emojiIcon(marker.emoji, zoom));
});
}
coordinates.forEach(function(item) {
var latlng = L.latLng(item.latitude, item.longitude);
var marker = L.marker(latlng, { icon: emojiIcon(emojiByType[item.type], map.getZoom()) }).addTo(map);
marker.bindPopup(item.name);
marker.originalLatLng = latlng;
marker.emoji = emojiByType[item.type];
markers.push(marker);
});
function resetMarkerPositions() {
markers.forEach(function(marker) {
marker.setLatLng(marker.originalLatLng);
if (marker.polyline) {
map.removeLayer(marker.polyline);
marker.polyline = null;
}
});
offset_lines.forEach(function(polyline) {
map.removeLayer(polyline);
});
offset_lines = [];
}
map.on('zoomend', function() {
resetMarkerPositions();
updateMarkerIcons(map.getZoom());
var overlappingGroups = getOverlappingGroups(map.getZoom());
overlappingGroups.forEach(function(group) { return displaceMarkers(group, map.getZoom()); });
});
updateMarkerIcons(map.getZoom());
var initialGroups = getOverlappingGroups(map.getZoom());
initialGroups.forEach(function(group) { return displaceMarkers(group, map.getZoom()); });
// Draw routes
routes.forEach(function(route) {
var color = {"train": "blue", "flight": "red", "unbooked_flight": "orange", "coach": "green", "bus": "purple", "car": "#555"}[route.type];
var style = { weight: 3, opacity: 0.5, color: color };
if (route.geojson) {
L.geoJSON(JSON.parse(route.geojson), {
style: function(feature) { return style; }
}).addTo(map);
} else if (route.type === "flight" || route.type === "unbooked_flight") {
var flightPath = new L.Geodesic([[route.from, route.to]], style).addTo(map);
} else {
L.polyline([route.from, route.to], style).addTo(map);
}
});
var mapElement = document.getElementById(map_id);
document.getElementById('toggleMapSize').addEventListener('click', function() {
var mapElement = document.getElementById(map_id);
var isFullWindow = mapElement.classList.contains('full-window-map');
if (isFullWindow) {
mapElement.classList.remove('full-window-map');
mapElement.classList.add('half-map');
mapElement.style.position = 'relative';
} else {
mapElement.classList.add('full-window-map');
mapElement.classList.remove('half-map');
mapElement.style.position = '';
}
// Ensure the map adjusts to the new container size
map.invalidateSize();
});
return map;
}

38
templates/accommodation.html
View file

@ -1,12 +1,9 @@
{% extends "base.html" %}
{% from "macros.html" import trip_link, accommodation_row with context %}
{% block title %}Accommodation - Edward Betts{% endblock %}
{% block style %}
{% set column_count = 9 %}
<style>
.grid-container {
display: grid;
grid-template-columns: repeat({{ column_count }}, auto);
grid-template-columns: repeat(6, auto);
gap: 10px;
justify-content: start;
}
@ -16,18 +13,24 @@
}
.heading {
grid-column: 1 / {{ column_count + 1 }}; /* Spans from the 1st line to the 7th line */
grid-column: 1 / 7; /* Spans from the 1st line to the 7th line */
}
</style>
{% endblock %}
{% macro row(item, badge) %}
<div class="grid-item text-end">{{ item.from.strftime("%a, %d %b %Y") }}</div>
<div class="grid-item text-end">{{ item.to.strftime("%a, %d %b") }}</div>
<div class="grid-item text-end">{{ (item.to.date() - item.from.date()).days }}</div>
<div class="grid-item">{{ item.name }}</div>
<div class="grid-item">{{ item.operator }}</div>
<div class="grid-item">{{ item.location }}</div>
{% endmacro %}
{% macro section(heading, item_list, badge) %}
{% if item_list %}
<div class="heading"><h2>{{heading}}</h2></div>
{% for item in item_list %}
{{ accommodation_row(item, badge) }}
<div class="grid-item">{% if item.linked_trip %} trip: {{ trip_link(item.linked_trip) }} {% endif %}</div>
{% endfor %}
{% for item in item_list %}{{ row(item, badge) }}{% endfor %}
{% endif %}
{% endmacro %}
@ -37,24 +40,13 @@
<h1>Accommodation</h1>
<h3>Statistics</h3>
<ul>
{% for year, stats in year_stats %}
{% set days_in_year = 366 if (year % 4 == 0 and year % 100 != 0) or (year % 400 == 0) else 365 %}
{% set total_percentage = (stats.total_nights / days_in_year) * 100 %}
{% set abroad_percentage = (stats.nights_abroad / days_in_year) * 100 %}
<li>
<strong>{{ year }}:</strong>
Total nights away: {{ stats.total_nights }} ({{ "%.1f"|format(total_percentage) }}%),
Total nights abroad: {{ stats.nights_abroad }} ({{ "%.1f"|format(abroad_percentage) }}%)
</li>
{% endfor %}
<li>Total nights away in 2024: {{ total_nights_2024 }}</li>
<li>Total nights abroad in 2024: {{ nights_abroad_2024 }}</li>
</ul>
<div class="grid-container">
{{ section("Current", current) }}
{{ section("Future", future) }}
{{ section("Past", past) }}
{{ section("Accommodation", items) }}
</div>
</div>

5
templates/base.html
View file

@ -7,7 +7,7 @@
<title>{% block title %}{% endblock %}</title>
<link rel="icon" href="data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 100 100%22><text y=%22.9em%22 font-size=%2290%22>📅</text></svg>">
<link href="{{ url_for("static", filename="bootstrap5/css/bootstrap.min.css") }}" rel="stylesheet">
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.2/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-T3c6CoIi6uLrA9TneNEoa7RxnatzjcDSCmG1MXxSR1GAsXEV/Dwwykc2MPK8M2HN" crossorigin="anonymous">
{% block style %}
{% endblock %}
@ -16,9 +16,8 @@
<body>
{% block nav %}{{ navbar() }}{% endblock %}
{% include "flash_messages.html" %}
{% block content %}{% endblock %}
{% block scripts %}{% endblock %}
<script src="{{ url_for("static", filename="bootstrap5/js/bootstrap.bundle.min.js") }}"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.2/dist/js/bootstrap.bundle.min.js" integrity="sha384-C6RzsynM9kWDrMNeT87bh95OGNyZPhcTNXj1NW7RuBCsyN/o0jlpcV8Qyq46cDfL" crossorigin="anonymous"></script>
</body>
</html>

27
templates/birthday_list.html
View file

@ -1,27 +0,0 @@
{% extends "base.html" %}
{% from "macros.html" import display_date %}
{% block title %}Birthdays - Edward Betts{% endblock %}
{% block content %}
<div class="container-fluid mt-2">
<h1>Birthdays</h1>
<table class="w-auto table">
<thead>
<tr>
<th class="text-end">Date</th>
<th>Event</th>
<th class="text-end">Days</th>
</tr>
</thead>
<tbody>
{% for event in items %}
<tr>
<td class="text-end">{{event.as_date.strftime("%a, %d, %b %Y")}}</td>
<td>{{ event.title }}</td>
<td class="text-end">{{ event.delta_days(today) }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endblock %}

266
templates/calendar.html
View file

@ -1,266 +0,0 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Agenda - Edward Betts</title>
<link href="{{ url_for("static", filename="bootstrap5/css/bootstrap.min.css") }}" rel="stylesheet">
<link rel="icon" href="data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 100 100%22><text y=%22.9em%22 font-size=%2290%22>📅</text></svg>">
<!-- TOAST UI Calendar CSS -->
<link rel="stylesheet" href="https://uicdn.toast.com/calendar/latest/toastui-calendar.min.css" />
<style>
/* Custom styles to better integrate with Bootstrap */
.toastui-calendar-layout {
border: 1px solid #dee2e6;
border-radius: 0.375rem;
}
#calendar-header {
margin-bottom: 1rem;
}
</style>
</head>
{% set event_labels = {
"economist": "📰 The Economist",
"mothers_day": "Mothers' day",
"fathers_day": "Fathers' day",
"uk_financial_year_end": "End of financial year",
"bank_holiday": "UK bank holiday",
"us_holiday": "US holiday",
"uk_clock_change": "UK clock change",
"us_clock_change": "US clock change",
"us_presidential_election": "US pres. election",
"xmas_last_second": "Christmas last posting 2nd class",
"xmas_last_first": "Christmas last posting 1st class",
"up_series": "Up documentary",
"waste_schedule": "Waste schedule",
"gwr_advance_tickets": "GWR advance tickets",
"critical_mass": "Critical Mass",
}
%}
{% from "navbar.html" import navbar with context %}
<body>
{{ navbar() }}
<div class="container-fluid mt-2">
<h1>Agenda</h1>
<p>
<a href="/tools">← personal tools</a>
</p>
{% if errors %}
{% for error in errors %}
<div class="alert alert-danger" role="alert">
Error: {{ error }}
</div>
{% endfor %}
{% endif %}
<div>
Markets:
<a href="{{ url_for(request.endpoint) }}">Hide while away</a>
| <a href="{{ url_for(request.endpoint, markets="show") }}">Show all</a>
| <a href="{{ url_for(request.endpoint, markets="hide") }}">Hide all</a>
</div>
<!-- Header for calendar controls -->
<div id="calendar-header" class="d-flex justify-content-between align-items-center mt-3">
<div class="btn-group" role="group">
<button type="button" class="btn btn-primary" id="prevBtn">Prev</button>
<button type="button" class="btn btn-primary" id="nextBtn">Next</button>
<button type="button" class="btn btn-outline-primary" id="todayBtn">Today</button>
</div>
<h3 id="calendar-title" class="mb-0"></h3>
<div class="btn-group" role="group">
<button type="button" class="btn btn-outline-primary" id="monthViewBtn">Month</button>
<button type="button" class="btn btn-outline-primary" id="weekViewBtn">Week</button>
<button type="button" class="btn btn-outline-primary" id="dayViewBtn">Day</button>
</div>
</div>
<div class="mb-3" id="calendar" style="height: 80vh;"></div>
{#
<div class="mt-2">
<h5>Page generation time</h5>
<ul>
<li>Data gather took {{ "%.1f" | format(data_gather_seconds) }} seconds</li>
<li>Stock market open/close took
{{ "%.1f" | format(stock_market_times_seconds) }} seconds</li>
{% for name, seconds in timings %}
<li>{{ name }} took {{ "%.1f" | format(seconds) }} seconds</li>
{% endfor %}
</ul>
</div>
#}
</div>
<!-- TOAST UI Calendar JS -->
<script src="https://uicdn.toast.com/calendar/latest/toastui-calendar.min.js"></script>
<script src="{{ url_for("static", filename="bootstrap5/js/bootstrap.bundle.min.js") }}"></script>
<script>
document.addEventListener('DOMContentLoaded', function() {
const Calendar = tui.Calendar;
const container = document.getElementById('calendar');
// --- Configuration ---
const calendars = {{ toastui_calendars | tojson }};
const events = {{ toastui_events | tojson }};
function getSavedView() {
return localStorage.getItem('toastUiCalendarDefaultView') || 'month';
}
function saveView(view) {
localStorage.setItem('toastUiCalendarDefaultView', view);
}
const options = {
defaultView: getSavedView(),
useDetailPopup: true,
useCreationPopup: false,
calendars: calendars,
week: {
startDayOfWeek: 1, // Monday
taskView: false,
eventView: ['time'],
showNowIndicator: true,
},
month: {
startDayOfWeek: 1, // Monday
},
timezone: {
zones: [{
timezoneName: Intl.DateTimeFormat().resolvedOptions().timeZone
}]
},
template: {
allday(event) {
return `<span title="${event.title}">${event.title}</span>`;
},
time(event) {
return `<span title="${event.title}">${event.title}</span>`;
},
},
};
const calendar = new Calendar(container, options);
calendar.createEvents(events);
// --- Event Handlers ---
calendar.on('clickEvent', ({ event }) => {
if (event.raw && event.raw.url) {
window.open(event.raw.url, '_blank');
}
});
calendar.on('afterRenderEvent', () => {
const currentView = calendar.getViewName();
saveView(currentView);
updateCalendarTitle();
});
// --- UI Control Logic ---
const prevBtn = document.getElementById('prevBtn');
const nextBtn = document.getElementById('nextBtn');
const todayBtn = document.getElementById('todayBtn');
const monthViewBtn = document.getElementById('monthViewBtn');
const weekViewBtn = document.getElementById('weekViewBtn');
const dayViewBtn = document.getElementById('dayViewBtn');
const calendarTitle = document.getElementById('calendar-title');
function updateCalendarTitle() {
const currentView = calendar.getViewName();
const currentDate = calendar.getDate().toDate();
let titleText = '';
if (currentView === 'month') {
const year = currentDate.getFullYear();
const monthName = currentDate.toLocaleDateString('en-GB', { month: 'long' });
titleText = `${monthName} ${year}`;
} else if (currentView === 'week') {
// Manually calculate the start and end of the week
const startDayOfWeek = options.week.startDayOfWeek; // 1 for Monday
const day = currentDate.getDay(); // 0 for Sunday, 1 for Monday...
// Calculate days to subtract to get to the start of the week
const offset = (day - startDayOfWeek + 7) % 7;
const startDate = new Date(currentDate);
startDate.setDate(currentDate.getDate() - offset);
const endDate = new Date(startDate);
endDate.setDate(startDate.getDate() + 6);
const startMonth = startDate.toLocaleDateString('en-GB', { month: 'short' });
const endMonth = endDate.toLocaleDateString('en-GB', { month: 'short' });
if (startMonth === endMonth) {
titleText = `${startDate.getDate()} - ${endDate.getDate()} ${startMonth} ${startDate.getFullYear()}`;
} else {
titleText = `${startDate.getDate()} ${startMonth} - ${endDate.getDate()} ${endMonth} ${startDate.getFullYear()}`;
}
} else { // Day view
titleText = currentDate.toLocaleDateString('en-GB', { dateStyle: 'full' });
}
calendarTitle.textContent = titleText;
}
// --- View-aware navigation logic (no changes here) ---
prevBtn.addEventListener('click', () => {
const currentView = calendar.getViewName();
if (currentView === 'month') {
const currentDate = calendar.getDate().toDate();
currentDate.setMonth(currentDate.getMonth() - 1);
currentDate.setDate(1);
calendar.setDate(currentDate);
} else {
calendar.prev();
}
updateCalendarTitle();
});
nextBtn.addEventListener('click', () => {
const currentView = calendar.getViewName();
if (currentView === 'month') {
const currentDate = calendar.getDate().toDate();
currentDate.setMonth(currentDate.getMonth() + 1);
currentDate.setDate(1);
calendar.setDate(currentDate);
} else {
calendar.next();
}
updateCalendarTitle();
});
todayBtn.addEventListener('click', () => {
calendar.today();
updateCalendarTitle();
});
monthViewBtn.addEventListener('click', () => {
calendar.changeView('month');
updateCalendarTitle();
});
weekViewBtn.addEventListener('click', () => {
calendar.changeView('week');
updateCalendarTitle();
});
dayViewBtn.addEventListener('click', () => {
calendar.changeView('day');
updateCalendarTitle();
});
// Initial setup
updateCalendarTitle();
});
</script>
</body>
</html>

152
templates/conference_detail.html
View file

@ -1,152 +0,0 @@
{% extends "base.html" %}
{% from "macros.html" import trip_link with context %}
{% block title %}{{ conf.name }} - Edward Betts{% endblock %}
{% block style %}
<style>
.conference-page h1 { overflow-wrap: anywhere; }
.conference-details dd { margin-bottom: 1rem; }
.flight-table td { vertical-align: middle; padding: .8rem .5rem; }
.flight-date { display: block; color: var(--bs-secondary-color, #6c757d); font-size: .85rem; white-space: nowrap; }
.flight-time { display: block; font-size: 1.1rem; font-weight: 600; }
.flight-price a { font-weight: 600; white-space: nowrap; }
@media (max-width: 575.98px) {
.conference-page { padding-left: 1rem; padding-right: 1rem; }
.conference-page h1 { font-size: 1.8rem; }
.conference-details { display: grid; grid-template-columns: minmax(0, 2fr) minmax(0, 3fr); gap: .5rem 1rem; margin: 0; }
.conference-details dt, .conference-details dd { width: auto; padding: 0; margin: 0; font-size: .9rem; }
#flight-search { flex-direction: column; align-items: stretch !important; gap: .75rem !important; }
#flight-search input { max-width: none !important; }
.flight-table, .flight-table tbody { display: block; width: 100%; }
.flight-table thead { display: none; }
.flight-table tr { display: grid; grid-template-columns: minmax(0, 1fr) minmax(0, 1fr); grid-template-areas: "route price" "departure arrival" "stops duration"; gap: .75rem 1rem; border: 1px solid #dee2e6; border-radius: .65rem; padding: 1rem; margin-bottom: .75rem; }
.flight-table td { display: block; min-width: 0; padding: 0; border: 0; box-shadow: none; background: transparent; overflow-wrap: anywhere; }
.flight-table td::before { content: attr(data-label); display: block; font-size: .75rem; color: var(--bs-secondary-color, #6c757d); margin-bottom: .2rem; }
.flight-departure { grid-area: departure; }
.flight-arrival { grid-area: arrival; }
.flight-route { grid-area: route; font-size: .9rem; }
.flight-stops { grid-area: stops; font-size: .85rem; }
.flight-duration { grid-area: duration; font-size: .85rem; }
.flight-price { grid-area: price; text-align: right; font-size: 1.15rem; }
.flight-price a { white-space: normal; }
.flight-route::before, .flight-price::before, .flight-stops::before, .flight-duration::before { display: none !important; }
.flight-date { font-size: .8rem; white-space: normal; }
}
</style>
{% endblock %}
{% block content %}
<div class="container py-4 conference-page">
<a href="{{ url_for('conference_list') }}">← Conferences</a>
<h1 class="mt-3">{{ conf.name }}</h1>
<p class="lead">{{ conf.display_date }}{% if not conf.has_exact_dates %} <span class="badge text-bg-secondary">{{ conf.date_status }}</span>{% endif %}</p>
{% if conf.url %}<p><a href="{{ conf.url }}" class="btn btn-outline-primary">Conference website ↗</a></p>{% endif %}
<div class="row g-4">
<div class="col-lg-6">
<h2 class="h4">Location and venue</h2>
<p>{{ conf.location }}{% if conf.country and get_country(conf.country) %} · {{ get_country(conf.country).flag }} {{ get_country(conf.country).name }}{% endif %}</p>
{% if conf.venue %}<p class="fw-semibold mb-1">{{ conf.venue }}</p>{% endif %}
{% if conf.address %}<p>{{ conf.address }}</p>{% endif %}
{% if conf.latitude is defined and conf.longitude is defined and conf.latitude is not none and conf.longitude is not none %}
<p><a href="https://www.openstreetmap.org/?mlat={{ conf.latitude }}&amp;mlon={{ conf.longitude }}#map=16/{{ conf.latitude }}/{{ conf.longitude }}">View venue on map</a> <span class="text-muted">({{ conf.latitude }}, {{ conf.longitude }})</span></p>
{% elif conf.venue or conf.address %}
<p><a href="https://www.google.com/maps/search/?api=1&amp;query={{ ((conf.venue or '') ~ ' ' ~ (conf.address or '') ~ ' ' ~ conf.location)|urlencode }}">Find venue on map</a></p>
{% endif %}
{% if conf.series_detail %}<p>Series: <a href="{{ url_for('conference_series_page', series_id=conf.series) }}">{{ conf.series_detail.name }}</a></p>{% endif %}
{% if conf.linked_trip %}<p>{{ trip_link(conf.linked_trip) }}</p>{% endif %}
</div>
<div class="col-lg-6">
<h2 class="h4">Conference details</h2>
<dl class="row conference-details">
{% for label, value in fields %}
<dt class="col-sm-4">{{ label }}</dt>
<dd class="col-sm-8 text-break">{% if value is sameas true %}Yes{% elif value is sameas false %}No{% elif value is string and (value.startswith('https://') or value.startswith('http://')) %}<a href="{{ value }}">{{ value }}</a>{% else %}{{ value|conference_date }}{% endif %}</dd>
{% endfor %}
</dl>
</div>
</div>
<section class="card mt-4"><div class="card-body">
<h2 class="h4">Flights</h2>
{% if cooldown %}<p class="alert alert-warning">{{ cooldown }}</p>{% endif %}
{% if eligible %}
{% if conf.date_status == 'tentative' %}<p class="text-muted">Using tentative conference dates: {{ conf.start_date|conference_date }} – {{ conf.end_date|conference_date }}.</p>{% endif %}
<p>{% if short_haul %}Direct flights from Bristol first. Start the day before and return the day after; if no flights are found, check progressively further dates, up to four days each side. Each direction stops at the nearest date with flights. London is checked if either direction has no Bristol flights.{% else %}Flights from London, departing two days before and returning the day after the conference. If none are found, try departing one day earlier. Direct or one-stop flights first.{% endif %}</p>
<form method="post" class="d-flex flex-wrap align-items-end gap-3" id="flight-search">
<input type="hidden" name="csrf_token" value="{{ csrf_token }}">
<div><label for="airport" class="form-label">Destination city or airport</label>
<input id="airport" name="airport" value="{{ airport }}" class="form-control" list="airport-suggestions" placeholder="Copenhagen, Denmark or CPH" maxlength="200" autocomplete="off" required style="max-width: 24rem" aria-describedby="airport-help"><datalist id="airport-suggestions"></datalist></div>
<button class="btn btn-primary" type="submit"{% if cooldown %} disabled{% endif %}>{{ 'Update flights' if cached else 'Look up flights' }}</button>
</form>
{% if resolved_airport %}<p class="small mt-2 mb-1">Destination: <strong>{{ resolved_name }}{% if resolved_name != resolved_airport %} ({{ resolved_airport }}){% endif %}</strong></p>{% endif %}
{% if default_google_flights and (not cached or not cached.searches) %}
<p class="mt-3"><a class="btn btn-outline-secondary" href="{{ default_google_flights.url }}" target="_blank" rel="noopener">Open in Google Flights ↗</a><span class="small text-muted d-block mt-1">Round trip: {{ default_google_flights.departure|conference_date }} – {{ default_google_flights.return|conference_date }}</span></p>
{% endif %}
{% if resolution_error %}<p class="text-danger small mt-2">{{ resolution_error }}</p>{% endif %}
<p id="airport-help" class="text-muted small mt-2">{% if suggested %}Suggested {{ suggested }}: {{ airport_reason }}. Enter a city, airport name or IATA code to choose another airport.{% else %}{{ airport_reason }}{% endif %}</p>
<p class="small text-muted">Direct flights first, with British Airways preferred within each stop count. Other airlines are included. Two-stop flights from London are checked only if no direct or one-stop options are found.</p>
<p class="small text-muted">Prices in GBP for one adult, economy. Each direction is priced separately. Results are cached for six hours; updates within that time reuse the cache.</p>
{% else %}
<p class="text-muted">Flight lookup is available for future conferences overseas or in Belfast with confirmed or tentative dates and an in-person venue.</p>
{% endif %}
{% if cached %}
<p>Last updated: <time class="cache-updated" datetime="{{ cached.updated_at }}">{{ cached.updated_at|conference_date }} UTC</time></p>
{% if cached.error %}<div class="alert alert-warning"><p class="mb-0">{{ cached.error }}{% if cached.searches %} Showing previously cached results.{% endif %}</p>{% if cached.error_details %}<p class="small text-break mt-2 mb-0">{{ cached.error_details }}</p>{% endif %}</div>{% endif %}
{% for search in cached.searches %}
<h3 class="h5 mt-4">{{ 'Bristol (BRS), direct only' if search.origin == 'BRS' else 'London (LON)' }} → {{ resolved_name or cached.destination }}{% if resolved_name and resolved_name != cached.destination %} ({{ cached.destination }}){% endif %}</h3>
<p><a class="btn btn-outline-secondary" href="{{ search.google_flights.url }}" target="_blank" rel="noopener">Open in Google Flights ↗</a><span class="small text-muted d-block mt-1">Round trip: {{ search.google_flights.departure|conference_date }} – {{ search.google_flights.return|conference_date }}</span></p>
{% for label, rows, days in [('Outbound', search.outbound, search.outbound_dates), ('Return', search.inbound, search.inbound_dates)] %}
<h4 class="h6">{{ label }} · one-way fares</h4>
{% if rows %}<p class="small text-muted mb-2">Departure and arrival times are local to each airport.</p>{% endif %}
{% if rows %}<div class="table-responsive"><table class="table table-sm flight-table">
<thead><tr><th>Departure</th><th>Arrival</th><th>Flight</th><th>Stops</th><th>Duration</th><th>Price</th></tr></thead><tbody>
{% for flight in rows %}<tr>
<td class="flight-departure" data-label="Departure"><time datetime="{{ flight.departure }}"><span class="flight-date">{{ flight.departure|conference_date('date') }}</span><span class="flight-time">{{ flight.departure|conference_date('time') }}</span></time></td>
<td class="flight-arrival" data-label="Arrival"><time datetime="{{ flight.arrival }}"><span class="flight-date">{{ flight.arrival|conference_date('date') }}</span><span class="flight-time">{{ flight.arrival|conference_date('time') }}</span></time></td>
<td class="flight-route" data-label="Flight">{% for leg in flight.legs %}{{ leg.airline }} {{ leg.number }} ({{ leg.origin }}–{{ leg.destination }}){% if not loop.last %}<br>{% endif %}{% endfor %}</td>
<td class="flight-stops" data-label="Stops">{{ 'Direct' if flight.stops == 0 else flight.stops|string ~ (' stop' if flight.stops == 1 else ' stops') }}</td>
<td class="flight-duration" data-label="Duration">{% if flight.duration is defined %}{{ flight.duration // 60 }}h {{ flight.duration % 60 }}m{% else %}—{% endif %}</td>
<td class="flight-price" data-label="Price"><a href="{{ flight.url }}">{% if flight.price is not none %}{{ '£' if flight.currency == 'GBP' else flight.currency ~ ' ' }}{{ '%.2f'|format(flight.price) }}{% else %}Check price{% endif %} ↗</a></td>
</tr>{% endfor %}
</tbody></table></div>{% else %}<p class="text-muted">{{ 'No direct flights found' if search.origin == 'BRS' else 'No suitable flights found' }}{% if days %} for departures on {{ days|map('conference_date')|join(', ') }}{% else %}: no eligible departure dates{% endif %}.{% if search.origin == 'BRS' and days %} Direct flights may not operate every day.{% endif %}</p>{% endif %}
{% endfor %}
{% endfor %}
{% endif %}
</div></section>
</div>
<script>
const airportInput = document.getElementById('airport');
const suggestions = document.getElementById('airport-suggestions');
let suggestionTimer;
let suggestionRequest;
if (airportInput) airportInput.addEventListener('input', function () {
clearTimeout(suggestionTimer);
if (suggestionRequest) suggestionRequest.abort();
suggestions.replaceChildren();
const query = airportInput.value.trim();
if (query.length < 2) return;
suggestionTimer = setTimeout(async function () {
suggestionRequest = new AbortController();
try {
const response = await fetch({{ url_for('conference.airport_search')|tojson }} + '?q=' + encodeURIComponent(query), {signal: suggestionRequest.signal});
const matches = await response.json();
if (airportInput.value.trim() !== query || !Array.isArray(matches)) return;
suggestions.replaceChildren(...matches.map(function (match) {
const option = document.createElement('option');
option.value = match.code;
option.label = match.name;
return option;
}));
} catch (error) {
// Typed names still resolve on submission if autocomplete is unavailable.
}
}, 250);
});
const form = document.getElementById('flight-search');
if (form) form.addEventListener('submit', function () {
const button = form.querySelector('button');
button.disabled = true;
button.textContent = 'Looking up flights…';
});
document.querySelectorAll('.cache-updated[datetime]').forEach(function (element) {
element.textContent = new Date(element.dateTime).toLocaleString('en-GB', {day: 'numeric', month: 'short', year: 'numeric', hour: '2-digit', minute: '2-digit', timeZoneName: 'short'});
});
</script>
{% endblock %}

368
templates/conference_list.html
View file

@ -1,359 +1,63 @@
{% extends "base.html" %}
{% from "macros.html" import trip_link with context %}
{% block title %}Conferences - Edward Betts{% endblock %}
{% block style %}
<style>
/* Timeline */
.conf-timeline {
position: relative;
background: #f8f9fa;
border: 1px solid #dee2e6;
border-radius: 4px;
overflow: hidden;
user-select: none;
}
.conf-tl-month {
position: absolute;
top: 0;
height: 100%;
border-left: 1px solid #dee2e6;
pointer-events: none;
}
.conf-tl-month-label {
position: absolute;
top: 3px;
left: 3px;
font-size: 0.68em;
color: #6c757d;
white-space: nowrap;
}
.conf-tl-today {
position: absolute;
top: 22px;
height: calc(100% - 22px);
border-left: 2px solid #dc3545;
pointer-events: none;
z-index: 10;
}
.conf-tl-bar {
position: absolute;
height: 26px;
border-radius: 3px;
overflow: hidden;
z-index: 5;
}
.conf-tl-bar a, .conf-tl-bar span {
display: block;
padding: 4px 6px;
font-size: 0.72em;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
line-height: 18px;
color: white;
text-decoration: none;
.grid-container {
display: grid;
grid-template-columns: repeat(6, auto); /* 7 columns for each piece of information */
gap: 10px;
justify-content: start;
}
/* Bidirectional hover highlight */
.conf-tl-bar.conf-hl {
filter: brightness(1.25);
box-shadow: inset 0 0 0 2px rgba(255,255,255,0.8);
z-index: 15;
}
tr.conf-hl > td {
background-color: rgba(255, 193, 7, 0.25) !important;
.grid-item {
/* Additional styling for grid items can go here */
}
/* Conference table */
.conf-section-row td {
background: #343a40 !important;
color: #fff;
font-weight: 700;
font-size: 0.85em;
padding-top: 0.5rem;
padding-bottom: 0.4rem;
border-bottom: none;
}
.conf-month-row td {
background: #e9ecef !important;
font-weight: 600;
font-size: 0.8em;
text-transform: uppercase;
letter-spacing: 0.06em;
color: #495057;
padding-top: 0.5rem;
padding-bottom: 0.3rem;
border-bottom: none;
}
.conf-going {
--bs-table-bg: rgba(25, 135, 84, 0.07);
}
@media (max-width: 767.98px) {
.conf-table,
.conf-table tbody {
display: block;
}
.conf-table colgroup,
.conf-table thead {
display: none;
}
.conf-table .conf-row {
display: flex;
flex-direction: column;
margin: 0.5rem 0 1rem;
padding: 0.35rem 0.75rem;
border: 1px solid #dee2e6;
border-radius: 0.5rem;
background: var(--bs-body-bg);
box-shadow: 0 0.125rem 0.25rem rgba(0, 0, 0, 0.075);
}
.conf-table .conf-row.conf-going {
border-left: 0.3rem solid #198754;
background: rgba(25, 135, 84, 0.07);
}
.conf-table .conf-row > td {
display: grid;
grid-template-columns: 5.25rem minmax(0, 1fr);
gap: 0.5rem;
padding: 0.35rem 0;
border: 0;
white-space: normal !important;
overflow-wrap: anywhere;
}
.conf-table .conf-row > td::before {
content: attr(data-label);
color: var(--bs-secondary-color);
font-size: 0.75rem;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.03em;
}
.conf-table .conf-name {
order: -2;
display: block !important;
padding-bottom: 0.5rem !important;
font-size: 1.05rem;
font-weight: 600;
border-bottom: 1px solid #dee2e6 !important;
}
.conf-table .conf-name::before {
display: none;
}
.conf-table .conf-dates {
order: -1;
}
.conf-table .conf-section-row,
.conf-table .conf-month-row {
display: block;
}
.conf-table .conf-section-row td,
.conf-table .conf-month-row td {
display: block;
width: 100%;
}
.conf-table .conf-section-row td {
margin-top: 1.25rem;
border-radius: 0.25rem;
}
.heading {
grid-column: 1 / 7; /* Spans from the 1st line to the 7th line */
}
</style>
{% endblock %}
{% set tl_colors = ["#0d6efd","#198754","#dc3545","#fd7e14","#6f42c1","#20c997","#0dcaf0","#d63384"] %}
{% macro render_timeline(timeline) %}
{% if timeline %}
{% set row_h = 32 %}
{% set header_h = 22 %}
{% set total_h = timeline.lane_count * row_h + header_h %}
<div class="mb-4">
<h3>Next 90 days</h3>
<div class="conf-timeline" style="height: {{ total_h }}px;">
{% for m in timeline.months %}
<div class="conf-tl-month" style="left: {{ m.left_pct }}%;">
<span class="conf-tl-month-label">{{ m.label }}</span>
</div>
{% endfor %}
<div class="conf-tl-today" style="left: 0;"></div>
{% for conf in timeline.confs %}
{% set color = tl_colors[conf.lane % tl_colors | length] %}
{% set top_px = conf.lane * row_h + header_h %}
<div class="conf-tl-bar"
style="left: {{ conf.left_pct }}%; width: {{ conf.width_pct }}%; top: {{ top_px }}px; background: {{ color }};"
title="{{ conf.label }}"
data-conf-key="{{ conf.key }}">
<a href="{{ url_for('conference.page', event_date=conf.event_date, slug=conf.slug) }}">{{ conf.name }}</a>
</div>
{% endfor %}
</div>
{% macro row(item, badge) %}
<div class="grid-item text-end">{{ item.start.strftime("%a, %d %b %Y") }}</div>
<div class="grid-item text-end">{{ item.end.strftime("%a, %d %b") }}</div>
<div class="grid-item">{{ item.name }}
{% if item.going and not (item.accommodation_booked or item.travel_booked) %}
<span class="badge text-bg-primary">
{{ badge }}
</span>
{% endif %}
{% if item.accommodation_booked %}
<span class="badge text-bg-success">accommodation</span>
{% endif %}
{% if item.transport_booked %}
<span class="badge text-bg-success">transport</span>
{% endif %}
</div>
{% endif %}
<div class="grid-item">{{ item.topic }}</div>
<div class="grid-item">{{ item.location }}</div>
<div class="grid-item"><a href="{{ item.url }}">{{ item.url }}</a></div>
{% endmacro %}
{% macro conf_rows(heading, item_list, badge) %}
{% macro section(heading, item_list, badge) %}
{% if item_list %}
{% set count = item_list | length %}
<tr class="conf-section-row">
<td colspan="7">{{ heading }} <span class="fw-normal opacity-75">{{ count }} conference{{ "" if count == 1 else "s" }}</span></td>
</tr>
{% set ns = namespace(prev_month="") %}
{% for item in item_list %}
{% set month_label = item.sort_date.strftime("%B %Y") %}
{% if month_label != ns.prev_month %}
{% set ns.prev_month = month_label %}
<tr class="conf-month-row">
<td colspan="7">{{ month_label }}</td>
</tr>
{% endif %}
<tr class="conf-row{% if item.going %} conf-going{% endif %}" data-conf-key="{{ item.sort_date.isoformat() }}|{{ item.name }}">
<td class="conf-dates text-nowrap text-muted small" data-label="Dates">
{{ item.display_date }}
{% if item.date_status == "tentative" %}
<span class="badge text-bg-warning ms-1">tentative</span>
{% elif item.date_status == "approximate" %}
<span class="badge text-bg-secondary ms-1">approximate</span>
{% endif %}
</td>
<td class="conf-name" data-label="Conference">
<a href="{{ url_for('conference.page', event_date=item.sort_date.isoformat(), slug=item.slug) }}">{{ item.name }}</a>
{% if item.going and not (item.accommodation_booked or item.travel_booked) %}
<span class="badge text-bg-primary ms-1">{{ badge }}</span>
{% endif %}
{% if item.accommodation_booked %}
<span class="badge text-bg-success ms-1">accommodation</span>
{% endif %}
{% if item.transport_booked %}
<span class="badge text-bg-success ms-1">transport</span>
{% endif %}
{% if item.linked_trip %}
{% set trip = item.linked_trip %}
<a href="{{ url_for('trip_page', start=trip.start.isoformat()) }}"
class="text-muted ms-1 text-decoration-none"
title="Trip: {{ trip.title }}">
🧳{% if trip.title != item.name %} {{ trip.title }}{% endif %}
</a>
{% endif %}
</td>
<td class="text-muted small" data-label="Topic">{{ item.topic }}</td>
<td class="text-nowrap text-muted small" data-label="Series">
{% if item.series and item.series_detail %}
<a href="{{ url_for('conference_series_page', series_id=item.series) }}">{{ item.series_detail.name }}</a>
{% elif item.series %}
{{ item.series }}
{% endif %}
</td>
<td class="text-nowrap" data-label="Location">
{% set country = get_country(item.country) if item.country else None %}
{% if country %}{{ country.flag }} {{ item.location }}
{% elif item.online %}💻 Online
{% else %}{{ item.location }}{% endif %}
</td>
<td class="text-nowrap text-muted small" data-label="CFP ends">
{% if item.cfp_end %}{{ item.cfp_end.strftime("%-d %b %Y") }}{% endif %}
</td>
<td data-label="Price">
{% if item.price and item.currency %}
<span class="badge bg-info text-nowrap">{{ "{:,d}".format(item.price | int) }} {{ item.currency }}</span>
{% if item.currency != "GBP" and item.currency in fx_rate %}
<span class="badge bg-info text-nowrap">{{ "{:,.0f}".format(item.price / fx_rate[item.currency]) }} GBP</span>
{% endif %}
{% elif item.free %}
<span class="badge bg-success text-nowrap">free</span>
{% endif %}
</td>
</tr>
{% endfor %}
<div class="heading"><h2>{{heading}}</h2></div>
{% for item in item_list %}{{ row(item, badge) }}{% endfor %}
{% endif %}
{% endmacro %}
{% block content %}
<div class="container-fluid mt-2">
<h1>Conferences</h1>
{% if country_options %}
<form method="get" class="row g-2 align-items-end mb-3">
<div class="col-auto">
<label for="country-filter" class="form-label mb-1">Country</label>
<select id="country-filter" name="country" class="form-select form-select-sm" onchange="this.form.submit()">
<option value="">All countries</option>
{% for option in country_options %}
<option value="{{ option.code }}"{% if option.code == selected_country %} selected{% endif %}>
{{ option.flag }} {{ option.name }}
</option>
{% endfor %}
</select>
</div>
<div class="col-auto">
<button type="submit" class="btn btn-sm btn-primary">Filter</button>
{% if selected_country %}
<a class="btn btn-sm btn-outline-secondary" href="{{ url_for(request.endpoint) }}">Clear</a>
{% endif %}
</div>
</form>
{% endif %}
{{ render_timeline(timeline) }}
<table class="conf-table table table-sm table-hover align-middle">
<caption class="visually-hidden">Conferences grouped by attendance and month</caption>
<colgroup>
<col style="width: 9rem">
<col>
<col style="width: 18rem">
<col style="width: 14rem">
<col style="width: 14rem">
<col style="width: 7rem">
<col style="width: 10rem">
</colgroup>
<thead class="table-light">
<tr>
<th>Dates</th>
<th>Conference</th>
<th>Topic</th>
<th>Series</th>
<th>Location</th>
<th>CFP ends</th>
<th>Price</th>
</tr>
</thead>
<tbody>
{{ conf_rows("Current", current, "attending") }}
{{ conf_rows("Future", future, "going") }}
{{ conf_rows("Past", past|reverse|list, "went") }}
</tbody>
</table>
<div class="grid-container">
{{ section("Current", current, "attending") }}
{{ section("Future", future, "going") }}
{{ section("Past", past|reverse, "went") }}
</div>
</div>
<script>
(function () {
// Build a map from conf-key → all matching elements (bars + rows)
const map = new Map();
document.querySelectorAll('[data-conf-key]').forEach(el => {
const k = el.dataset.confKey;
if (!map.has(k)) map.set(k, []);
map.get(k).push(el);
});
function setHighlight(key, on) {
(map.get(key) || []).forEach(el => el.classList.toggle('conf-hl', on));
}
map.forEach((els, key) => {
els.forEach(el => {
el.addEventListener('mouseenter', () => setHighlight(key, true));
el.addEventListener('mouseleave', () => setHighlight(key, false));
});
});
})();
</script>
{% endblock %}

84
templates/conference_series.html
View file

@ -1,84 +0,0 @@
{% extends "base.html" %}
{% block title %}{{ series.name }} - Edward Betts{% endblock %}
{% block content %}
<div class="container-fluid mt-2">
<h1>{{ series.name }}</h1>
<dl class="row">
{% if series.topic %}
<dt class="col-sm-2">Topic</dt>
<dd class="col-sm-10">{{ series.topic }}</dd>
{% endif %}
{% if series.usual_location or series.country %}
<dt class="col-sm-2">Usual location</dt>
<dd class="col-sm-10">
{% set country = get_country(series.country) if series.country else None %}
{% if country %}{{ country.flag }} {% endif %}{{ series.usual_location or "" }}
</dd>
{% endif %}
{% if series.cadence %}
<dt class="col-sm-2">Cadence</dt>
<dd class="col-sm-10">{{ series.cadence }}</dd>
{% endif %}
{% if series.url %}
<dt class="col-sm-2">Website</dt>
<dd class="col-sm-10"><a href="{{ series.url }}">{{ series.url }}</a></dd>
{% endif %}
{% if series.notes %}
<dt class="col-sm-2">Notes</dt>
<dd class="col-sm-10">{{ series.notes }}</dd>
{% endif %}
</dl>
<h2>Editions</h2>
<table class="table table-sm table-hover align-middle">
<thead class="table-light">
<tr>
<th>Dates</th>
<th>Conference</th>
<th>Location</th>
<th>Attendance</th>
</tr>
</thead>
<tbody>
{% for item in conferences %}
<tr>
<td class="text-nowrap text-muted small">
{{ item.display_date }}
{% if item.date_status == "tentative" %}
<span class="badge text-bg-warning ms-1">tentative</span>
{% elif item.date_status == "approximate" %}
<span class="badge text-bg-secondary ms-1">approximate</span>
{% endif %}
</td>
<td>
<a href="{{ url_for('conference.page', event_date=item.sort_date.isoformat(), slug=item.slug) }}">{{ item.name }}</a>
</td>
<td>
{% set country = get_country(item.country) if item.country else None %}
{% if country %}{{ country.flag }} {% endif %}{{ item.location }}
</td>
<td>
{% if item.going %}
<span class="badge text-bg-success">going</span>
{% endif %}
{% if item.registered %}
<span class="badge text-bg-primary">registered</span>
{% endif %}
{% if item.linked_trip %}
{% set trip = item.linked_trip %}
<a href="{{ url_for('trip_page', start=trip.start.isoformat()) }}"
class="ms-1"
title="Trip: {{ trip.title }}">
trip{% if trip.title != item.name %}: {{ trip.title }}{% endif %}
</a>
{% endif %}
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endblock %}

47
templates/conference_series_list.html
View file

@ -1,47 +0,0 @@
{% extends "base.html" %}
{% block title %}Conference Series - Edward Betts{% endblock %}
{% block content %}
<div class="container-fluid mt-2">
<h1>Conference Series</h1>
<table class="table table-sm table-hover align-middle">
<thead class="table-light">
<tr>
<th>Series</th>
<th>Topic</th>
<th>Usual location</th>
<th>Conferences</th>
<th>Next</th>
</tr>
</thead>
<tbody>
{% for item in series_list %}
<tr>
<td>
<a href="{{ url_for('conference_series_page', series_id=item.id) }}">{{ item.name }}</a>
{% if item.attended %}
<span class="badge text-bg-success ms-1">attended</span>
{% endif %}
{% if item.url %}
<a class="text-muted ms-1 text-decoration-none" href="{{ item.url }}">↗</a>
{% endif %}
</td>
<td class="text-muted small">{{ item.topic or "" }}</td>
<td>
{% set country = get_country(item.country) if item.country else None %}
{% if country %}{{ country.flag }} {% endif %}{{ item.usual_location or "" }}
</td>
<td>{{ item.count }}</td>
<td class="text-muted small">
{% if item.next_conf %}
{{ item.next_conf.display_date }} · {{ item.next_conf.name }}
{% endif %}
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endblock %}

46
templates/fixture/map.html
View file

@ -1,46 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Map fixture</title>
<link rel="icon" href="data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 100 100%22><text y=%22.9em%22 font-size=%2290%22>📅</text></svg>">
<link href="{{ url_for("static", filename="bootstrap5/css/bootstrap.min.css") }}" rel="stylesheet">
<link rel="stylesheet" href="{{ url_for("static", filename="leaflet/leaflet.css") }}">
<style>
body, html {
height: 100%;
margin: 0;
}
#map {
height: 100%;
width: 100%;
}
</style>
</head>
<body>
<div id="map" class="map"></div>
<script src="{{ url_for("static", filename="leaflet/leaflet.js") }}"></script>
<script src="{{ url_for("static", filename="leaflet-geodesic/leaflet.geodesic.umd.min.js") }}"></script>
<script src="{{ url_for("static", filename="js/map.js") }}"></script>
<script>
var coordinates = {{ coordinates | tojson }};
var routes = {{ routes | tojson }};
build_map("map", coordinates, routes);
</script>
<script src="{{ url_for("static", filename="bootstrap5/js/bootstrap.bundle.min.js") }}"></script>
</body>
</html>

12
templates/flash_messages.html
View file

@ -1,12 +0,0 @@
{% with messages = get_flashed_messages() %}
{% if messages %}
<div class="container-fluid mt-2">
{% for message in messages %}
<div class="alert alert-success alert-dismissible fade show" role="alert">
{{ message }}
<button type="button" class="btn-close" data-bs-dismiss="alert" aria-label="Close"></button>
</div>
{% endfor %}
</div>
{% endif %}
{% endwith %}

25
templates/gaps.html
View file

@ -1,5 +1,4 @@
{% extends "base.html" %}
{% block title %}Gaps - Edward Betts{% endblock %}
{% block content %}
<div class="p-2">
@ -18,31 +17,11 @@
<tbody>
{% for gap in gaps %}
<tr>
<td class="text-start">
{% for event in gap.before %}
<div>
{% if event.url %}
<a href="{{event.url}}">{{ event.title_with_emoji }}</a>
{% else %}
{{ event.title_with_emoji }}
{% endif %}
</div>
{% endfor %}
</td>
<td>{% for event in gap.before %}{% if not loop.first %}<br/>{% endif %}<span class="text-nowrap">{{ event.title or event.name }}</span>{% endfor %}</td>
<td class="text-end text-nowrap">{{ gap.start.strftime("%A, %-d %b %Y") }}</td>
<td class="text-end text-nowrap">{{ (gap.end - gap.start).days }} days</td>
<td class="text-end text-nowrap">{{ gap.end.strftime("%A, %-d %b %Y") }}</td>
<td class="text-start">
{% for event in gap.after %}
<div>
{% if event.url %}
<a href="{{event.url}}">{{ event.title_with_emoji }}</a>
{% else %}
{{ event.title_with_emoji }}
{% endif %}
</div>
{% endfor %}
</td>
<td>{% for event in gap.after %}{% if not loop.first %}<br/>{% endif %}<span class="text-nowrap">{{ event.title or event.name }}</span>{% endfor %}</td>
</tr>
{% endfor %}
</tbody>

39
templates/holiday_list.html
View file

@ -1,39 +0,0 @@
{% extends "base.html" %}
{% from "macros.html" import display_date %}
{% block title %}Holidays - Edward Betts{% endblock %}
{% block content %}
<div class="container-fluid mt-2">
<h1>Public holidays</h1>
<table class="table table-hover w-auto">
{% for item in items %}
{% set country = get_country(item.country) %}
<tr>
{% if loop.first or item.date != loop.previtem.date %}
<td class="text-end">{{ display_date(item.date) }}</td>
<td>in {{ (item.date - today).days }} days</td>
{% else %}
<td colspan="2"></td>
{% endif %}
<td>{{ country.flag }} {{ country.name }}</td>
<td>{{ item.display_name }}</td>
</tr>
{% endfor %}
</table>
<h2>
UK school holidays (Bristol)
<small class="fs-6 ms-2"><a href="{{ school_holiday_page_url }}">source</a></small>
</h2>
<table class="table table-hover w-auto">
{% for item in school_holidays %}
<tr>
<td class="text-end">{{ display_date(item.as_date) }}</td>
<td>to {{ display_date(item.end_as_date) }}</td>
<td>in {{ (item.as_date - today).days }} days</td>
<td>{{ item.title }}</td>
</tr>
{% endfor %}
</table>
</div>
{% endblock %}

133
templates/event_list.html → templates/index.html
View file

@ -3,11 +3,72 @@
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Agenda - Edward Betts</title>
<link href="{{ url_for("static", filename="bootstrap5/css/bootstrap.min.css") }}" rel="stylesheet">
<title>Agenda</title>
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.2/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-T3c6CoIi6uLrA9TneNEoa7RxnatzjcDSCmG1MXxSR1GAsXEV/Dwwykc2MPK8M2HN" crossorigin="anonymous">
<link rel="icon" href="data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 100 100%22><text y=%22.9em%22 font-size=%2290%22>📅</text></svg>">
<script async src="{{ url_for("static", filename="es-module-shims/es-module-shims.js") }}"></script>
<script async src="https://unpkg.com/es-module-shims@1.8.2/dist/es-module-shims.js"></script>
<script type='importmap'>
{
"imports": {
"@fullcalendar/core": "https://cdn.skypack.dev/@fullcalendar/core@6.1.9",
"@fullcalendar/daygrid": "https://cdn.skypack.dev/@fullcalendar/daygrid@6.1.9",
"@fullcalendar/timegrid": "https://cdn.skypack.dev/@fullcalendar/timegrid@6.1.9",
"@fullcalendar/list": "https://cdn.skypack.dev/@fullcalendar/list@6.1.9",
"@fullcalendar/core/locales/en-gb": "https://cdn.skypack.dev/@fullcalendar/core@6.1.9/locales/en-gb"
}
}
</script>
<script type='module'>
import { Calendar } from '@fullcalendar/core'
import dayGridPlugin from '@fullcalendar/daygrid'
import timeGridPlugin from '@fullcalendar/timegrid'
import listPlugin from '@fullcalendar/list'
import gbLocale from '@fullcalendar/core/locales/en-gb';
// Function to save the current view to local storage
function saveView(view) {
localStorage.setItem('fullCalendarDefaultView', view);
}
// Function to get the saved view from local storage
function getSavedView() {
return localStorage.getItem('fullCalendarDefaultView') || 'dayGridMonth';
}
document.addEventListener('DOMContentLoaded', function() {
const calendarEl = document.getElementById('calendar')
const calendar = new Calendar(calendarEl, {
locale: gbLocale,
plugins: [dayGridPlugin, timeGridPlugin, listPlugin ],
themeSystem: 'bootstrap5',
firstDay: 1,
initialView: getSavedView(),
viewDidMount: function(info) {
saveView(info.view.type);
},
headerToolbar: {
left: 'prev,next today',
center: 'title',
right: 'dayGridMonth,timeGridWeek,timeGridDay,listWeek'
},
nowIndicator: true,
weekNumbers: true,
eventTimeFormat: {
hour: '2-digit',
minute: '2-digit',
hour12: false,
},
events: {{ fullcalendar_events | tojson(indent=2) }},
eventDidMount: function(info) {
info.el.title = info.event.title;
},
})
calendar.render()
})
</script>
</head>
@ -27,7 +88,6 @@
"waste_schedule": "Waste schedule",
"gwr_advance_tickets": "GWR advance tickets",
"critical_mass": "Critical Mass",
"uk_school_holiday": "UK school holiday",
}
%}
@ -35,18 +95,15 @@
"bank_holiday": "bg-success-subtle",
"conference": "bg-primary-subtle",
"us_holiday": "bg-secondary-subtle",
"uk_school_holiday": "bg-warning-subtle",
"birthday": "bg-info-subtle",
"waste_schedule": "bg-danger-subtle",
} %}
{% from "macros.html" import trip_link, display_date_no_year, trip_item with context %}
{% from "navbar.html" import navbar with context %}
<body>
{{ navbar() }}
{% include "flash_messages.html" %}
<div class="container-fluid mt-2">
<h1>Agenda</h1>
@ -70,49 +127,16 @@
Sunset: {{ sunset.strftime("%H:%M:%S") }}</li>
</ul>
{% if home_weather %}
<div class="mb-2">
<strong>Bristol weather:</strong>
{% for day in home_weather %}
<span class="me-3 text-nowrap">
<small class="text-muted">{{ day.date_obj.strftime("%-d %b") }}</small>
<img src="https://openweathermap.org/img/wn/{{ day.icon }}.png" alt="{{ day.status }}" title="{{ day.detailed_status }}" width="25" height="25">
{{ day.temp_min }}–{{ day.temp_max }}°C
</span>
{% endfor %}
</div>
{% endif %}
{% if errors %}
{% for error in errors %}
<div class="alert alert-danger" role="alert">
Error: {{ error }}
</div>
{% endfor %}
{% endif %}
<h3>Stock markets</h3>
{% for market in stock_markets %}
<p>{{ market }}</p>
{% endfor %}
{% if current_trip %}
<div>
<h3>Current trip</h3>
{{ trip_item(current_trip) }}
</div>
{% endif %}
<div class="mb-3" id="calendar"></div>
<h3>Agenda</h3>
<div>
Markets:
<a href="{{ url_for(request.endpoint) }}">Hide while away</a>
| <a href="{{ url_for(request.endpoint, markets="show") }}">Show all</a>
| <a href="{{ url_for(request.endpoint, markets="hide") }}">Hide all</a>
</div>
{% for event in events if start_event_list <= event.as_date <= end_event_list %}
{% for event in events if event.as_date >= two_weeks_ago %}
{% if loop.first or event.date.year != loop.previtem.date.year or event.date.month != loop.previtem.date.month %}
<div class="row mt-2">
<div class="col">
@ -142,12 +166,11 @@
<div class="col-md-2{{ cell_bg }}">
{% if event.end_date %}
{% set duration = event.display_duration() %}
{% if duration %}
{% if event.end_as_date == event.as_date and event.has_time %}
end: {{event.end_date.strftime("%H:%M") }}
(duration: {{duration}})
(duration: {{event.end_date - event.date}})
{% elif event.end_date != event.date %}
to {{ event.end_as_date.strftime("%a, %d, %b") }}
{{event.end_date}}
{% endif %}
{% endif %}
</div>
@ -155,7 +178,7 @@
<div class="col-md-7 text-start">
{% if event.url %}<a href="{{ event.url }}">{% endif %}
{{ event_labels.get(event.name) or event.name }}
{%- if event.title -%}: {{ event.title_with_emoji }}{% endif %}
{%- if event.title -%}: {{ event.title }}{% endif %}
{% if event.url %}</a>{% endif %}
</div>
<div class="col-md-1{{ cell_bg }}">
@ -165,21 +188,7 @@
{% endif %}
{% endfor %}
<div class="mt-2">
<h5>Page generation time</h5>
<ul>
<li>Data gather took {{ "%.1f" | format(data_gather_seconds) }} seconds</li>
<li>Stock market open/close took
{{ "%.1f" | format(stock_market_times_seconds) }} seconds</li>
{% for name, seconds in timings %}
<li>{{ name }} took {{ "%.1f" | format(seconds) }} seconds</li>
{% endfor %}
<li>Render time: {{ "%.1f" | format(render_time) }} seconds</li>
</ul>
</div>
</div>
<script src="{{ url_for("static", filename="bootstrap5/js/bootstrap.bundle.min.js") }}"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.2/dist/js/bootstrap.bundle.min.js" integrity="sha384-C6RzsynM9kWDrMNeT87bh95OGNyZPhcTNXj1NW7RuBCsyN/o0jlpcV8Qyq46cDfL" crossorigin="anonymous"></script>
</body>
</html>

107
templates/launches.html
View file

@ -1,63 +1,11 @@
{% extends "base.html" %}
{% block title %}Space launches - Edward Betts{% endblock %}
{% block content %}
<div class="container-fluid mt-2">
<h1>Space launches</h1>
<h4>Filters</h4>
<p>Mission type:
{% if request.args.type %}<a href="{{ request.path }}">🗙</a>{% endif %}
{% for t in mission_types | sort %}
{% if t == request.args.type %}
<strong>{{ t }}</strong>
{% else %}
<a href="?type={{ t }}" class="text-nowrap">
{{ t }}
</a>
{% endif %}
{% if not loop.last %} | {% endif %}
{% endfor %}
</p>
<p>Vehicle:
{% if request.args.rocket %}<a href="{{ request.path }}">🗙</a>{% endif %}
{% for r in rockets | sort %}
{% if r == request.args.rockets %}
<strong>{{ r }}</strong>
{% else %}
<a href="?rocket={{ r }}" class="text-nowrap">
{{ r }}
</a>
{% endif %}
{% if not loop.last %} | {% endif %}
{% endfor %}
</p>
<p>Orbit:
{% if request.args.orbit %}<a href="{{ request.path }}">🗙</a>{% endif %}
{% for name, abbrev in orbits | sort %}
{% if abbrev == request.args.orbit %}
<strong>{{ name }}</strong>
{% else %}
<a href="?orbit={{ abbrev }}" class="text-nowrap">
{{ name }}
</a>
{% endif %}
{% if not loop.last %} | {% endif %}
{% endfor %}
</p>
{% for launch in launches %}
{% set highlight =" bg-primary-subtle" if launch.slug in config.FOLLOW_LAUNCHES else "" %}
{% set country = get_country(launch.country_code) %}
<div class="row{{highlight}}">
{% for launch in rockets %}
<div class="row">
<div class="col-md-1 text-nowrap text-md-end">{{ launch.t0_date }}
<br class="d-none d-md-block"/>
@ -68,20 +16,8 @@
<div class="col-md-1 text-md-nowrap">
<span class="d-md-none">launch status:</span>
<abbr title="{{ launch.status.name }}">{{ launch.status.abbrev }}</abbr>
{% if launch.is_active_crewed %}
<span class="badge text-bg-info">In space</span>
{% endif %}
{% if launch.is_future and launch.probability %}{{ launch.probability }}%{% endif %}
</div>
<div class="col">
<div>
{% if launch.image %}
<div>
<img src="{{ launch.image }}" class="img-fluid img-thumbnail" style="max-width: 350px; height: auto;" />
</div>
{% endif %}
<abbr title="{{ country.name }}">{{ country.flag }}</abbr>
{{ launch.rocket.full_name }}
<div class="col">{{ launch.rocket }}
&ndash;
<strong>{{launch.mission.name }}</strong>
&ndash;
@ -94,29 +30,14 @@
({{ launch.launch_provider_type }})
&mdash;
{{ launch.orbit.name }} ({{ launch.orbit.abbrev }})
&mdash;
{{ launch.mission.type }}
</div>
<div>
{% if launch.pad_wikipedia_url %}
<a href="{{ launch.pad_wikipedia_url }}">{{ launch.pad_name }}</a>
{% else %}
{{ launch.pad_name }} {% if launch.pad_name != "Unknown Pad" %}(no Wikipedia article){% endif %}
{% endif %}
&mdash; {{ launch.location }}
</div>
{% if launch.mission.agencies | count %}
<div>
{% for agency in launch.mission.agencies %}
{% set agency_country = get_country(agency.country_code) %}
{%- if not loop.first %}, {% endif %}
<a href="{{ agency.wiki_url }}">{{agency.name }}</a>
<abbr title="{{ agency_country.name }}">{{ agency_country.flag }}</abbr>
({{ agency.type }}) {# <img src="{{ agency.logo_url }}"/> #}
{% endfor %}
</div>
<br/>
{% if launch.pad_wikipedia_url %}
<a href="{{ launch.pad_wikipedia_url }}">{{ launch.pad_name }}</a>
{% else %}
{{ launch.pad_name }} {% if launch.pad_name != "Unknown Pad" %}(no Wikipedia article){% endif %}
{% endif %}
<div>
&mdash; {{ launch.location }}<br/>
{% if launch.mission %}
{% for line in launch.mission.description.splitlines() %}
<p>{{ line }}</p>
@ -124,13 +45,7 @@
{% else %}
<p>No description.</p>
{% endif %}
{% if launch.weather_concerns and launch.status.name != "Launch Successful" %}
<h4>Weather concerns</h4>
{% for line in launch.weather_concerns.splitlines() %}
<p>{{ line }}</p>
{% endfor %}
{% endif %}
</div>
</div>
</div>
{% endfor %}

534
templates/macros.html
View file

@ -1,534 +0,0 @@
{% macro display_datetime(dt) %}{{ dt.strftime("%a, %d, %b %Y %H:%M %z") }}{% endmacro %}
{% macro display_time(dt) %}
{% if dt %}{{ dt.strftime("%H:%M %z") }}{% endif %}
{% endmacro %}
{% macro display_date(dt) %}{{ dt.strftime("%a %-d %b %Y") }}{% endmacro %}
{% macro display_date_no_year(dt) %}{{ dt.strftime("%a %-d %b") }}{% endmacro %}
{% macro display_conf_date_no_year(dt) %}{%- if dt.hour is defined %}{{ dt.strftime("%a %-d %b %H:%M") }}{% else %}{{ dt.strftime("%a %-d %b") }}{% endif %}{% endmacro %}
{% macro format_distance(distance) %}
{{ "{:,.0f} km / {:,.0f} miles".format(distance, distance / 1.60934) }}
{% endmacro %}
{% macro trip_link(trip) %}
<a href="{{ url_for("trip_page", start=trip.start.isoformat()) }}">{{ trip.title }}</a>
{% endmacro %}
{% macro conference_row(item, badge, show_flags=True) %}
{% set country = get_country(item.country) if item.country else None %}
<div class="grid-item text-end text-nowrap">{{ item.start.strftime("%a, %d %b %Y") }}</div>
<div class="grid-item text-end text-nowrap">{{ item.end.strftime("%a, %d %b") }}</div>
<div class="grid-item">
{% if item.url %}
<a href="{{ item.url }}">{{ item.name }}</a>
{% else %}
{{ item.name }}
{% endif %}
{% if item.going and not (item.accommodation_booked or item.travel_booked) %}
<span class="badge text-bg-primary">
{{ badge }}
</span>
{% endif %}
{% if item.accommodation_booked %}
<span class="badge text-bg-success">accommodation</span>
{% endif %}
{% if item.transport_booked %}
<span class="badge text-bg-success">transport</span>
{% endif %}
</div>
<div class="grid-item text-end">
{% if item.price and item.currency %}
<span class="badge bg-info text-nowrap">{{ "{:,d}".format(item.price | int) }} {{ item.currency }}</span>
{% if item.currency != "GBP" and item.currency in fx_rate %}
<span class="badge bg-info text-nowrap">{{ "{:,.2f}".format(item.price / fx_rate[item.currency]) }} GBP</span>
{% endif %}
{% elif item.free %}
<span class="badge bg-success text-nowrap">free to attend</span>
{% endif %}
</div>
<div class="grid-item">{{ item.topic }}</div>
<div class="grid-item">{{ item.location }}</div>
<div class="grid-item text-end text-nowrap">{{ display_date(item.cfp_end) if item.cfp_end else "" }}</div>
<div class="grid-item">
{% if country %}
{% if show_flags %}{{ country.flag }}{% endif %} {{ country.name }}
{% elif item.online %}
💻 Online
{% else %}
<span class="text-bg-danger p-2">
country code <strong>{{ item.country }}</strong> not found
</span>
{% endif %}
</div>
{% endmacro %}
{% macro accommodation_row(item, badge, show_flags=True) %}
{% set country = get_country(item.country) %}
{% set nights = (item.to.date() - item.from.date()).days %}
<div class="grid-item text-end">{{ item.from.strftime("%a, %d %b %Y") }}</div>
<div class="grid-item text-end">{{ item.to.strftime("%a, %d %b") }}</div>
<div class="grid-item text-end">{% if nights == 1 %}1 night{% else %}{{ nights }} nights{% endif %}</div>
<div class="grid-item">{{ item.operator }}</div>
<div class="grid-item">{{ item.location }}</div>
<div class="grid-item">
{% if country %}
{% if show_flags %}{{ country.flag }}{% endif %} {{ country.name }}
{% else %}
<span class="text-bg-danger p-2">
country code <strong>{{ item.country }}</strong> not found
</span>
{% endif %}
</div>
<div class="grid-item">
{% if g.user.is_authenticated and item.url %}
<a href="{{ item.url }}">{{ item.name }}</a>
{% else %}
{{ item.name }}
{% endif %}
</div>
<div class="grid-item text-end">
{% if g.user.is_authenticated and item.price and item.currency %}
<span class="badge bg-info text-nowrap">{{ "{:,f}".format(item.price) }} {{ item.currency }}</span>
{% if item.currency != "GBP" %}
<span class="badge bg-info text-nowrap">{{ "{:,.2f}".format(item.price / fx_rate[item.currency]) }} GBP</span>
{% endif %}
{% endif %}
</div>
{% endmacro %}
{% macro flightradar24_url(flight) -%}
https://www.flightradar24.com/data/flights/{{ flight.airline_detail.iata | lower + flight.flight_number }}
{%- endmacro %}
{% macro flight_booking_row(booking, show_flags=True) %}
<div class="grid-item">
{% if g.user.is_authenticated %}
{{ booking.booking_reference or "reference missing" }}
{% else %}
<em>redacted</em>
{% endif %}
</div>
<div class="grid-item text-end">
{% if g.user.is_authenticated and booking.price and booking.currency %}
<span class="badge bg-info text-nowrap">{{ "{:,f}".format(booking.price) }} {{ booking.currency }}</span>
{% if booking.currency != "GBP" %}
<span class="badge bg-info text-nowrap">{{ "{:,.2f}".format(booking.price / fx_rate[booking.currency]) }} GBP</span>
{% endif %}
{% endif %}
</div>
{% for i in range(9) %}
<div class="grid-item"></div>
{% endfor %}
{% for item in booking.flights %}
{% set full_flight_number = item.airline_code + item.flight_number %}
{% set radarbox_url = "https://www.radarbox.com/data/flights/" + full_flight_number %}
<div class="grid-item"></div>
<div class="grid-item"></div>
<div class="grid-item text-end">{{ item.depart.strftime("%a, %d %b %Y") }}</div>
<div class="grid-item">{{ item.from }} → {{ item.to }}</div>
<div class="grid-item">{{ item.depart.strftime("%H:%M") }}</div>
<div class="grid-item">
{% if item.arrive %}
{{ item.arrive.strftime("%H:%M") }}
{% if item.arrive.date() != item.depart.date() %}+1 day{% endif %}
{% endif %}
</div>
<div class="grid-item">{{ item.duration }}</div>
<div class="grid-item">{{ full_flight_number }}</div>
<div class="grid-item">
<a href="{{ flightradar24_url(item) }}">flightradar24</a>
| <a href="https://uk.flightaware.com/live/flight/{{ full_flight_number }}">FlightAware</a>
| <a href="{{ radarbox_url }}">radarbox</a>
</div>
<div class="grid-item text-end">
{% if item.distance %}
{{ "{:,.0f} km / {:,.0f} miles".format(item.distance, item.distance / 1.60934) }}
{% endif %}
</div>
<div class="grid-item text-end">{{ "{:,.1f}".format(item.co2_kg) }} kg</div>
{% endfor %}
{% endmacro %}
{% macro flight_row(item) %}
{% set full_flight_number = item.airline_code + item.flight_number %}
{% set radarbox_url = "https://www.radarbox.com/data/flights/" + full_flight_number %}
<div class="grid-item text-end">{{ item.depart.strftime("%a, %d %b %Y") }}</div>
<div class="grid-item">{{ item.from }} → {{ item.to }}</div>
<div class="grid-item">{{ item.depart.strftime("%H:%M") }}</div>
<div class="grid-item">
{% if item.arrive %}
{{ item.arrive.strftime("%H:%M") }}
{% if item.arrive.date() != item.depart.date() %}+1 day{% endif %}
{% endif %}
</div>
<div class="grid-item">{{ item.duration }}</div>
<div class="grid-item">{{ full_flight_number }}</div>
<div class="grid-item">
{% if g.user.is_authenticated %}
{{ item.booking_reference }}
{% else %}
<em>redacted</em>
{% endif %}
</div>
<div class="grid-item">
<a href="{{ flightradar24_url(item) }}">flightradar24</a>
| <a href="https://uk.flightaware.com/live/flight/{{ full_flight_number }}">FlightAware</a>
| <a href="{{ radarbox_url }}">radarbox</a>
</div>
<div class="grid-item text-end">
{% if item.distance %}
{{ "{:,.0f} km / {:,.0f} miles".format(item.distance, item.distance / 1.60934) }}
{% endif %}
</div>
<div class="grid-item text-end">
{% if g.user.is_authenticated and item.price and item.currency %}
<span class="badge bg-info text-nowrap">{{ "{:,f}".format(item.price) }} {{ item.currency }}</span>
{% if item.currency != "GBP" %}
<span class="badge bg-info text-nowrap">{{ "{:,.2f}".format(item.price / fx_rate[item.currency]) }} GBP</span>
{% endif %}
{% endif %}
</div>
{% endmacro %}
{% macro train_row(item) %}
{% set url = item.url %}
<div class="grid-item text-end">{{ item.depart.strftime("%a, %d %b %Y") }}</div>
<div class="grid-item">
{% if g.user.is_authenticated and item.url %}<a href="{{ url }}">{% endif %}
{{ item.from }} → {{ item.to }}
{% if g.user.is_authenticated and item.url %}</a>{% endif %}
</div>
<div class="grid-item">{{ item.depart.strftime("%H:%M") }}</div>
<div class="grid-item">
{% if item.arrive %}
{{ item.arrive.strftime("%H:%M") }}
{% if item.depart != item.arrive and item.arrive.date() != item.depart.date() %}+1 day{% endif %}
{% endif %}
</div>
<div class="grid-item">{{ ((item.arrive - item.depart).total_seconds() // 60) | int }} mins</div>
<div class="grid-item">{{ item.operator }}</div>
<div class="grid-item">
{% if g.user.is_authenticated %}
{{ item.booking_reference }}
{% else %}
<em>redacted</em>
{% endif %}
</div>
<div class="grid-item">
{% for leg in item.legs %}
{% if leg.url %}
<a href="{{ leg.url }}">[{{ loop.index }}]</a>
{% endif %}
{% endfor %}
</div>
<div class="grid-item text-end">
{% if item.distance %}
{{ "{:,.0f} km / {:,.0f} miles".format(item.distance, item.distance / 1.60934) }}
{% endif %}
</div>
<div class="grid-item text-end">
{% if g.user.is_authenticated and item.price and item.currency %}
<span class="badge bg-info text-nowrap">{{ "{:,f}".format(item.price) }} {{ item.currency }}</span>
{% if item.currency != "GBP" and item.currency in fx_rate %}
<span class="badge bg-info text-nowrap">{{ "{:,.2f}".format(item.price / fx_rate[item.currency]) }} GBP</span>
{% endif %}
{% endif %}
</div>
{% endmacro %}
{% macro coach_row(item) %}
{% set url = item.url %}
<div class="grid-item text-end">{{ item.depart.strftime("%a, %d %b %Y") }}</div>
<div class="grid-item">
{% if g.user.is_authenticated and item.url %}<a href="{{ url }}">{% endif %}
{{ item.from }} → {{ item.to }}
{% if g.user.is_authenticated and item.url %}</a>{% endif %}
</div>
<div class="grid-item">{{ item.depart.strftime("%H:%M") }}</div>
<div class="grid-item">
{% if item.arrive %}
{{ item.arrive.strftime("%H:%M") }}
{% if item.depart != item.arrive and item.arrive.date() != item.depart.date() %}+1 day{% endif %}
{% endif %}
</div>
<div class="grid-item">{{ ((item.arrive - item.depart).total_seconds() // 60) | int }} mins</div>
<div class="grid-item">{{ item.operator }}</div>
<div class="grid-item">
{% if g.user.is_authenticated %}
{{ item.booking_reference }}
{% else %}
<em>redacted</em>
{% endif %}
</div>
<div class="grid-item">
</div>
<div class="grid-item text-end">
{% if item.distance %}
{{ "{:,.0f} km / {:,.0f} miles".format(item.distance, item.distance / 1.60934) }}
{% endif %}
</div>
<div class="grid-item text-end">
{% if g.user.is_authenticated and item.price and item.currency %}
<span class="badge bg-info text-nowrap">{{ "{:,f}".format(item.price) }} {{ item.currency }}</span>
{% if item.currency != "GBP" and item.currency in fx_rate %}
<span class="badge bg-info text-nowrap">{{ "{:,.2f}".format(item.price / fx_rate[item.currency]) }} GBP</span>
{% endif %}
{% endif %}
</div>
{% endmacro %}
{% macro bus_row(item) %}
{% set url = item.url %}
<div class="grid-item text-end">{{ item.depart.strftime("%a, %d %b %Y") }}</div>
<div class="grid-item">
{% if g.user.is_authenticated and item.url %}<a href="{{ url }}">{% endif %}
{{ item.from }} → {{ item.to }}
{% if g.user.is_authenticated and item.url %}</a>{% endif %}
</div>
<div class="grid-item">{{ item.depart.strftime("%H:%M") }}</div>
<div class="grid-item">
{% if item.arrive %}
{{ item.arrive.strftime("%H:%M") }}
{% if item.depart != item.arrive and item.arrive.date() != item.depart.date() %}+1 day{% endif %}
{% endif %}
</div>
<div class="grid-item">{{ ((item.arrive - item.depart).total_seconds() // 60) | int }} mins</div>
<div class="grid-item">{{ item.operator }}</div>
<div class="grid-item">
{% if g.user.is_authenticated %}
{{ item.booking_reference }}
{% else %}
<em>redacted</em>
{% endif %}
</div>
<div class="grid-item">
</div>
<div class="grid-item text-end">
{% if item.distance %}
{{ "{:,.0f} km / {:,.0f} miles".format(item.distance, item.distance / 1.60934) }}
{% endif %}
</div>
<div class="grid-item text-end">
{% if g.user.is_authenticated and item.price and item.currency %}
<span class="badge bg-info text-nowrap">{{ "{:,f}".format(item.price) }} {{ item.currency }}</span>
{% if item.currency != "GBP" and item.currency in fx_rate %}
<span class="badge bg-info text-nowrap">{{ "{:,.2f}".format(item.price / fx_rate[item.currency]) }} GBP</span>
{% endif %}
{% endif %}
</div>
{% endmacro %}
{% macro ferry_row(item) %}
<div class="grid-item text-end">{{ item.depart.strftime("%a, %d %b %Y") }}</div>
<div class="grid-item">
{{ item.from }} → {{ item.to }}
</div>
<div class="grid-item">{{ item.depart.strftime("%H:%M") }}</div>
<div class="grid-item">
{% if item.arrive %}
{{ item.arrive.strftime("%H:%M") }}
{% if item.depart != item.arrive and item.arrive.date() != item.depart.date() %}+1 day{% endif %}
{% endif %}
</div>
<div class="grid-item"></div>
<div class="grid-item">{{ item.operator }}</div>
<div class="grid-item"></div>
<div class="grid-item"></div>
<div class="grid-item"></div>
<div class="grid-item text-end">
{% if g.user.is_authenticated and item.price and item.currency %}
<span class="badge bg-info text-nowrap">{{ "{:,f}".format(item.price) }} {{ item.currency }}</span>
{% if item.currency != "GBP" %}
<span class="badge bg-info text-nowrap">{{ "{:,.2f}".format(item.price / fx_rate[item.currency]) }} GBP</span>
{% endif %}
{% endif %}
</div>
{# <div class="grid-item">{{ item | pprint }}</div> #}
{% endmacro %}
{% macro flag(trip, flag) %}{% if trip.show_flags %}{{ flag }}{% endif %}{% endmacro %}
{% macro conference_list(trip) %}
{% for item in trip.conferences %}
{% set country = get_country(item.country) if item.country else None %}
<div class="trip-conference-card my-1">
<div class="card-body">
<h5 class="card-title">
<a href="{{ item.url }}">{{ item.name }}</a>
<small class="text-muted">
{{ display_conf_date_no_year(item.attend_start if item.attend_start else item.start) }} to {{ display_conf_date_no_year(item.attend_end if item.attend_end else item.end) }}
{% if item.attend_start or item.attend_end %}
(full conference: {{ display_date_no_year(item.start) }} to {{ display_date_no_year(item.end) }})
{% endif %}
</small>
</h5>
<p class="card-text">
Topic: {{ item.topic }}
| Venue: {{ item.venue }}
| Location: {{ item.location }}
{% if country %}
{{ flag(trip, country.flag) }}
{% elif item.online %}
💻 Online
{% else %}
<span class="text-bg-danger p-2">
country code <strong>{{ item.country }}</strong> not found
</span>
{% endif %}
{% if item.free %}
| <span class="badge bg-success text-nowrap">free to attend</span>
{% elif item.price and item.currency %}
| <span class="badge bg-info text-nowrap">price: {{ item.price }} {{ item.currency }}</span>
{% endif %}
</p>
</div>
</div>
{% endfor %}
{% endmacro %}
{% macro render_trip_element(e, trip) %}
{% set item = e.detail %}
{% if e.element_type == "check-in" %}
{% set nights = (item.to.date() - item.from.date()).days %}
<div class="trip-element">
{{ e.get_emoji() }} <strong>{{ item.name }}</strong>
{% if item.operator and item.operator != item.name %}<small class="text-muted">{{ item.operator }}</small>{% endif %}
<small class="text-muted">({% if nights == 1 %}1 night{% else %}{{ nights }} nights{% endif %})</small>
</div>
{% elif e.element_type == "check-out" %}
<div class="trip-checkout">
{{ e.get_emoji() }} Check out: {{ item.name }}
{% if item.operator and item.operator != item.name %}<span class="text-muted small">{{ item.operator }}</span>{% endif %}
</div>
{% elif e.element_type != "conference" %}
{# Transport: flight, train, ferry, coach, bus, car #}
{% set has_arrive = item.arrive is defined and item.arrive %}
{# item.depart may be a date (no .date() method) or a datetime (has .date()) #}
{% set has_time = item.depart is defined and item.depart and item.depart.hour is defined %}
{% set depart_date = item.depart.date() if has_time else item.depart %}
{% set arrive_date = item.arrive.date() if (has_arrive and item.arrive.hour is defined) else item.arrive %}
{% set is_overnight = has_arrive and depart_date != arrive_date %}
{% set dur_mins = ((item.arrive - item.depart).total_seconds() // 60) | int if (has_time and has_arrive) else none %}
<div class="trip-element">
{% if is_overnight %}🌙{% else %}{{ e.get_emoji() }}{% endif %}
{{ e.start_loc }} → {{ e.end_loc }}
{% if has_time %}
· {{ item.depart.strftime("%H:%M") }}{% if has_arrive and item.arrive.hour is defined %} → {{ item.arrive.strftime("%H:%M") }}{% if is_overnight %} <span class="text-muted small">+1 day</span>{% endif %}{% endif %}
{% endif %}
{% if dur_mins %}
{%- set h = dur_mins // 60 %}{%- set m = dur_mins % 60 %}
<span class="text-muted">🕒{% if h %}{{ h }}h {% endif %}{% if m %}{{ m }}m{% endif %}</span>
{% endif %}
{% if e.element_type == "flight" %}
<small class="text-muted">· {{ item.airline_name }} {{ item.airline_code }}{{ item.flight_number }}</small>
{% elif item.operator %}
<small class="text-muted">· {{ item.operator }}</small>
{% endif %}
{% if item.distance %}
<span class="text-muted small">· {{ "{:,.0f} km".format(item.distance) }}</span>
{% endif %}
{% if item.co2_kg is defined and item.co2_kg is not none %}
<span class="text-muted small">CO₂ {{ "{:,.1f}".format(item.co2_kg) }} kg</span>
{% endif %}
</div>
{% endif %}
{% endmacro %}
{% macro trip_item(trip) %}
{% set distances_by_transport_type = trip.distances_by_transport_type() %}
{% set total_distance = trip.total_distance() %}
{% set total_co2_kg = trip.total_co2_kg() %}
{% set end = trip.end %}
{% set trip_end = end or trip.start %}
{% set is_current = trip.start <= today and trip_end >= today %}
<div class="trip-card{% if is_current %} trip-current{% endif %}">
<h3 class="trip-name">
{{ trip_link(trip) }}
<small>({{ display_date(trip.start) }})</small></h3>
{% set school_holidays = trip_school_holiday_map.get(trip.start.isoformat(), []) if trip_school_holiday_map is defined else [] %}
{% if school_holidays %}
<div class="school-holiday-info">
<span class="badge bg-warning text-dark">UK school holiday</span>
{% for item in school_holidays %}
<span class="text-muted">{{ item.title }} ({{ display_date_no_year(item.as_date) }} to {{ display_date_no_year(item.end_as_date) }})</span>
{% endfor %}
</div>
{% endif %}
<ul class="list-unstyled trip-countries">
{% for c in trip.countries %}
<li>{{ c.flag }} {{ c.name }}</li>
{% endfor %}
</ul>
{% if end %}
<div class="trip-dates">Dates: {{ display_date_no_year(trip.start) }} to {{ display_date_no_year(end) }}
{% if g.user.is_authenticated and trip.start <= today %}
<a href="https://photos.4angle.com/search?query=%7B%22takenAfter%22%3A%22{{trip.start}}T00%3A00%3A00.000Z%22%2C%22takenBefore%22%3A%22{{end}}T23%3A59%3A59.999Z%22%7D">photos</a>
{% endif %}
</div>
{% else %}
<div>Start: {{ display_date_no_year(trip.start) }} (end date missing)</div>
{% endif %}
<div class="trip-stats">
{% if total_distance %}
<span class="trip-stat">{{ format_distance(total_distance) }}</span>
{% endif %}
{% if distances_by_transport_type %}
{% for transport_type, distance in distances_by_transport_type %}
<span class="trip-stat">{{ transport_type | title }}: {{format_distance(distance) }}</span>
{% endfor %}
{% endif %}
{% if total_co2_kg %}
<span class="trip-stat">CO₂ {{ "{:,.1f}".format(total_co2_kg) }} kg</span>
{% endif %}
{% set co2_by_transport = trip.co2_by_transport_type() %}
{% if co2_by_transport %}
{% for transport_type, co2_kg in co2_by_transport %}
<span class="trip-stat">{{ transport_type | title }} CO₂ {{ "{:,.1f}".format(co2_kg) }} kg</span>
{% endfor %}
{% endif %}
</div>
{% if trip.schengen_compliance %}
<div>
<strong>Schengen:</strong>
{% if trip.schengen_compliance.is_compliant %}
<span class="badge bg-success">✅ Compliant</span>
{% else %}
<span class="badge bg-danger">❌ Non-compliant</span>
{% endif %}
<span class="text-muted small">({{ trip.schengen_compliance.total_days_used }}/90 days used)</span>
</div>
{% endif %}
{{ conference_list(trip) }}
{% set trip_weather = trip_weather_map.get(trip.start.isoformat(), {}) if trip_weather_map is defined else {} %}
{% for day, elements in trip.elements_grouped_by_day() %}
{% set weather = trip_weather.get(day.isoformat()) %}
<h4 class="trip-day-header">{{ display_date_no_year(day) }}
{% if weather %}
<span class="trip-weather-inline">
<img src="https://openweathermap.org/img/wn/{{ weather.icon }}.png" alt="{{ weather.status }}" title="{{ weather.detailed_status }}" width="18" height="18">
{{ weather.temp_min }}–{{ weather.temp_max }}°C
<span class="fw-normal fst-italic">{{ weather.detailed_status }}</span>
</span>
{% endif %}
{% if g.user.is_authenticated and day <= today %}
<a class="ms-2 small fw-normal" href="https://photos.4angle.com/search?query=%7B%22takenAfter%22%3A%22{{day}}T00%3A00%3A00.000Z%22%2C%22takenBefore%22%3A%22{{day}}T23%3A59%3A59.999Z%22%7D">photos</a>
{% endif %}
</h4>
{% for e in elements %}
{{ render_trip_element(e, trip) }}
{% endfor %}
{% endfor %}
</div>
{% endmacro %}

54
templates/meteors.html
View file

@ -1,54 +0,0 @@
{% extends "base.html" %}
{% block title %}Meteor Showers 2025 - Edward Betts{% endblock %}
{% block content %}
<div class="container-fluid mt-2">
<h1>Meteor Showers 2025</h1>
<p>Major meteor showers visible throughout 2025. All times are approximate and viewing conditions depend on light pollution, weather, and moon phase.</p>
{% for meteor in meteors %}
<div class="row mb-4">
<div class="col-md-2 text-nowrap text-md-end">
<strong>{{ meteor.peak }}</strong>
<br class="d-none d-md-block"/>
<small class="text-muted">{{ meteor.active }}</small>
</div>
<div class="col-md-2 text-md-nowrap">
<span class="d-md-none">Rate:</span>
{{ meteor.rate }}
<br class="d-none d-md-block"/>
<small class="text-muted">{{ meteor.radiant }}</small>
</div>
<div class="col">
<div>
<h4>{{ meteor.name }}</h4>
<p><strong>Moon Phase:</strong> {{ meteor.moon_phase }}</p>
<p><strong>Best Visibility:</strong> {{ meteor.visibility }}</p>
<p>{{ meteor.description }}</p>
</div>
</div>
</div>
{% endfor %}
<div class="mt-4">
<h3>Viewing Tips</h3>
<ul>
<li>Find a dark location away from city lights</li>
<li>Allow your eyes to adjust to darkness for 20-30 minutes</li>
<li>Look northeast for most showers, but meteors can appear anywhere</li>
<li>Best viewing is typically after midnight</li>
<li>No telescope needed - use naked eye viewing</li>
<li>Check weather conditions and moon phase before planning</li>
</ul>
</div>
<div class="mt-4">
<h3>About Meteor Showers</h3>
<p>Meteor showers occur when Earth passes through debris trails left by comets or asteroids. The debris burns up in our atmosphere, creating the streaks of light we see as "shooting stars."</p>
<p><strong>Data Sources:</strong> Information compiled from NASA, American Meteor Society, and astronomical observations for 2025.</p>
</div>
</div>
{% endblock %}

72
templates/navbar.html
View file

@ -1,34 +1,12 @@
{% macro navbar() %}
{% set pages_before_dropdowns = [
{% set pages = [
{"endpoint": "index", "label": "Home" },
{"endpoint": "recent", "label": "Recent" },
{"endpoint": "calendar_page", "label": "Calendar" },
] %}
{% set pages_after_dropdowns = [
{"endpoint": "conference_list", "label": "Conference" },
{"endpoint": "travel_list", "label": "Travel" },
{"endpoint": "accommodation_list", "label": "Accommodation" },
{"endpoint": "gaps_page", "label": "Gaps" },
{"endpoint": "weekends", "label": "Weekends" },
{"endpoint": "launch_list", "label": "Space launches" },
{"endpoint": "meteor_list", "label": "Meteor showers" },
{"endpoint": "holiday_list", "label": "Holidays" },
{"endpoint": "schengen_report", "label": "Schengen" },
] + ([{"endpoint": "birthday_list", "label": "Birthdays" }]
if g.user.is_authenticated else [])
%}
{% set trip_pages = [
{"endpoint": "trip_future_list", "label": "Future trips" },
{"endpoint": "trip_past_list", "label": "Past trips" },
{"endpoint": "trip_stats", "label": "Trip statistics" },
] %}
{% set conference_pages = [
{"endpoint": "conference_list", "label": "Conferences" },
{"endpoint": "past_conference_list", "label": "Past conferences" },
{"endpoint": "conference_series_list", "label": "Conference series" },
] %}
@ -40,7 +18,7 @@
</button>
<div class="collapse navbar-collapse" id="navbarSupportedContent">
<ul class="navbar-nav me-auto mb-2 mb-lg-0">
{% for page in pages_before_dropdowns %}
{% for page in pages %}
{% set is_active = request.endpoint == page.endpoint %}
<li class="nav-item">
<a class="nav-link{% if is_active %} border border-white border-2 active{% endif %}" {% if is_active %} aria-current="page"{% endif %} href="{{ url_for(page.endpoint) }}">
@ -48,50 +26,6 @@
</a>
</li>
{% endfor %}
<!-- Trip dropdown -->
{% set trip_active = request.endpoint in ['trip_future_list', 'trip_past_list', 'trip_stats'] %}
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle{% if trip_active %} border border-white border-2 active{% endif %}" href="#" role="button" data-bs-toggle="dropdown" aria-expanded="false">
Trips
</a>
<ul class="dropdown-menu">
{% for page in trip_pages %}
{% set is_active = request.endpoint == page.endpoint %}
<li><a class="dropdown-item{% if is_active %} active{% endif %}" href="{{ url_for(page.endpoint) }}">{{ page.label }}</a></li>
{% endfor %}
</ul>
</li>
<!-- Conference dropdown -->
{% set conference_active = request.endpoint in ['conference_list', 'past_conference_list', 'conference_series_list', 'conference_series_page'] %}
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle{% if conference_active %} border border-white border-2 active{% endif %}" href="#" role="button" data-bs-toggle="dropdown" aria-expanded="false">
Conferences
</a>
<ul class="dropdown-menu">
{% for page in conference_pages %}
{% set is_active = request.endpoint == page.endpoint %}
<li><a class="dropdown-item{% if is_active %} active{% endif %}" href="{{ url_for(page.endpoint) }}">{{ page.label }}</a></li>
{% endfor %}
</ul>
</li>
{% for page in pages_after_dropdowns %}
{% set is_active = request.endpoint == page.endpoint %}
<li class="nav-item">
<a class="nav-link{% if is_active %} border border-white border-2 active{% endif %}" {% if is_active %} aria-current="page"{% endif %} href="{{ url_for(page.endpoint) }}">
{{ page.label }}
</a>
</li>
{% endfor %}
</ul>
<ul class="navbar-nav ms-auto">
{% if g.user.is_authenticated %}
<li class="nav-item"><a class="nav-link" href="{{ url_for("logout") }}">Logout</a></li>
{% else %}
<li class="nav-item"><a class="nav-link" href="{{ url_for("login", next=request.url) }}">Login</a></li>
{% endif %}
</ul>
</div>
</div>

230
templates/schengen_report.html
View file

@ -1,230 +0,0 @@
{% extends "base.html" %}
{% set heading = "Schengen area compliance report" %}
{% block title %}{{ heading }} - Edward Betts{% endblock %}
{% block content %}
<div class="container-fluid mt-2">
<h1>{{ heading }}</h1>
<div class="row">
<div class="col-md-8">
<div class="card mb-4">
<div class="card-header">
<h5 class="card-title">Current Status</h5>
</div>
<div class="card-body">
<div class="row">
<div class="col-md-6">
<h6>Compliance Status</h6>
{% if current_compliance.is_compliant %}
<span class="badge bg-success fs-6">✅ COMPLIANT</span>
{% else %}
<span class="badge bg-danger fs-6">❌ NON-COMPLIANT</span>
{% endif %}
</div>
<div class="col-md-6">
<h6>Days Used</h6>
<div class="fs-4">{{ current_compliance.total_days_used }}/90</div>
</div>
</div>
<div class="row mt-3">
<div class="col-md-6">
{% if current_compliance.is_compliant %}
<h6>Days Remaining</h6>
<div class="fs-5 text-success">{{ current_compliance.days_remaining }}</div>
{% else %}
<h6>Days Over Limit</h6>
<div class="fs-5 text-danger">{{ current_compliance.days_over_limit }}</div>
{% endif %}
</div>
<div class="col-md-6">
{% if current_compliance.next_reset_date %}
<h6>Next Reset Date</h6>
<div class="fs-6">{{ current_compliance.next_reset_date.strftime('%Y-%m-%d') }}</div>
{% endif %}
</div>
</div>
<div class="mt-3">
<h6>Current 180-day Period</h6>
<div class="text-muted">
{{ current_compliance.current_180_day_period[0].strftime('%Y-%m-%d') }} to
{{ current_compliance.current_180_day_period[1].strftime('%Y-%m-%d') }}
</div>
</div>
</div>
</div>
{% if current_compliance.stays_in_period %}
<div class="card mb-4">
<div class="card-header">
<h5 class="card-title">Stays in Current 180-day Period</h5>
</div>
<div class="card-body">
<div class="table-responsive">
<table class="table table-striped">
<thead>
<tr>
<th>Entry Date</th>
<th>Exit Date</th>
<th>Country</th>
<th>Days</th>
<th>Trip</th>
</tr>
</thead>
<tbody>
{% for stay in current_compliance.stays_in_period %}
<tr>
<td>{{ stay.entry_date.strftime('%Y-%m-%d') }}</td>
<td>
{% if stay.exit_date %}
{{ stay.exit_date.strftime('%Y-%m-%d') }}
{% else %}
<span class="text-muted">ongoing</span>
{% endif %}
</td>
{% set country = get_country(stay.country) %}
<td>{{ country.flag }} {{ country.name }}</td>
<td>{{ stay.days }}</td>
<td>
{% if stay.trip_date and stay.trip_name %}
<a href="{{ url_for('trip_page', start=stay.trip_date.strftime('%Y-%m-%d')) }}" class="text-decoration-none">
{{ stay.trip_name }}
</a>
{% else %}
<span class="text-muted">-</span>
{% endif %}
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
</div>
</div>
{% endif %}
</div>
<div class="col-md-4">
{% if warnings %}
<div class="card mb-4">
<div class="card-header">
<h5 class="card-title">Warnings</h5>
</div>
<div class="card-body">
{% for warning in warnings %}
<div class="alert alert-{{ 'danger' if warning.severity == 'error' else 'warning' }} alert-dismissible fade show" role="alert">
<strong>{{ warning.date.strftime('%Y-%m-%d') }}</strong><br>
{{ warning.message }}
<button type="button" class="btn-close" data-bs-dismiss="alert" aria-label="Close"></button>
</div>
{% endfor %}
</div>
</div>
{% endif %}
<div class="card">
<div class="card-header">
<h5 class="card-title">Compliance History</h5>
</div>
<div class="card-body">
<div class="chart-container" style="height: 300px;">
<canvas id="complianceChart"></canvas>
</div>
</div>
</div>
</div>
</div>
{% if trips_with_compliance %}
<div class="card mt-4">
<div class="card-header">
<h5 class="card-title">Trip Compliance History</h5>
</div>
<div class="card-body">
<div class="table-responsive">
<table class="table table-striped">
<thead>
<tr>
<th>Trip Date</th>
<th>Trip Name</th>
<th>Days Used</th>
<th>Days Remaining</th>
<th>Status</th>
</tr>
</thead>
<tbody>
{% for trip_date, calculation in trips_with_compliance.items() %}
<tr>
<td>{{ trip_date.strftime('%Y-%m-%d') }}</td>
<td>
<a href="{{ url_for('trip_page', start=trip_date.strftime('%Y-%m-%d')) }}" class="text-decoration-none">
{% for trip in trip_list if trip.start == trip_date %}
{{ trip.title }} {{ trip.country_flags }}
{% endfor %}
</a>
</td>
<td>{{ calculation.total_days_used }}/90</td>
<td>{{ calculation.days_remaining }}</td>
<td>
{% if calculation.is_compliant %}
<span class="badge bg-success">Compliant</span>
{% else %}
<span class="badge bg-danger">Non-compliant</span>
{% endif %}
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
</div>
</div>
{% endif %}
</div>
{% endblock %}
{% block scripts %}
<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
<script>
const ctx = document.getElementById('complianceChart').getContext('2d');
const complianceHistory = {{ compliance_history | tojson }};
const chart = new Chart(ctx, {
type: 'line',
data: {
labels: complianceHistory.map(item => item.date),
datasets: [{
label: 'Days Used',
data: complianceHistory.map(item => item.days_used),
borderColor: 'rgb(75, 192, 192)',
backgroundColor: 'rgba(75, 192, 192, 0.2)',
tension: 0.1
}]
},
options: {
responsive: true,
maintainAspectRatio: false,
scales: {
y: {
beginAtZero: true,
max: 90,
ticks: {
callback: function(value) {
return value + '/90';
}
}
}
},
plugins: {
legend: {
display: true
}
}
}
});
</script>
{% endblock %}

2
templates/show_error.html
View file

@ -1,7 +1,5 @@
{% extends "base.html" %}
{% block title %}Agenda error - Edward Betts{% endblock %}
{% block style %}
<link rel="stylesheet" href="{{url_for('static', filename='css/exception.css')}}" />
{% endblock %}

51
templates/travel.html
View file

@ -1,23 +1,23 @@
{% extends "base.html" %}
{% from "macros.html" import flight_booking_row, train_row with context %}
{% block title %}Travel - Edward Betts{% endblock %}
{% block travel %}
{% endblock %}
{% set flight_column_count = 11 %}
{% set column_count = 10 %}
{% macro display_datetime(dt) %}{{ dt.strftime("%a, %d, %b %Y %H:%M %z") }}{% endmacro %}
{% macro display_time(dt) %}{{ dt.strftime("%H:%M %z") }}{% endmacro %}
{% block style %}
<style>
.grid-container {
display: grid;
grid-template-columns: repeat({{ flight_column_count }}, auto);
grid-template-columns: repeat(7, auto); /* 7 columns for each piece of information */
gap: 10px;
justify-content: start;
}
.train-grid-container {
display: grid;
grid-template-columns: repeat({{ column_count }}, auto);
grid-template-columns: repeat(6, auto); /* 7 columns for each piece of information */
gap: 10px;
justify-content: start;
}
@ -37,20 +37,27 @@
<h3>flights</h3>
<div class="grid-container">
<div class="grid-item">reference</div>
<div class="grid-item">price</div>
<div class="grid-item text-end">date</div>
<div class="grid-item">route</div>
<div class="grid-item">take-off</div>
<div class="grid-item">land</div>
<div class="grid-item">duration</div>
<div class="grid-item">flight</div>
<div class="grid-item">tracking</div>
<div class="grid-item">distance</div>
<div class="grid-item">CO₂ (kg)</div>
<div class="grid-item">reference</div>
{% for item in flights %}
{{ flight_booking_row(item) }}
{% for item in flights | sort(attribute="depart") if item.arrive %}
<div class="grid-item text-end">{{ item.depart.strftime("%a, %d %b %Y") }}</div>
<div class="grid-item">{{ item.from }} &rarr; {{ item.to }}</div>
<div class="grid-item">{{ item.depart.strftime("%H:%M") }}</div>
<div class="grid-item">
{% if item.arrive %}
{{ item.arrive.strftime("%H:%M") }}
{% if item.arrive.date() != item.depart.date() %}+1 day{% endif %}
{% endif %}
</div>
<div class="grid-item">{{ item.duration }}</div>
<div class="grid-item">{{ item.airline }}{{ item.flight_number }}</div>
<div class="grid-item">{{ item.booking_reference }}</div>
{% endfor %}
</div>
@ -61,15 +68,21 @@
<div class="grid-item">route</div>
<div class="grid-item">depart</div>
<div class="grid-item">arrive</div>
<div class="grid-item">duration</div>
<div class="grid-item">operator</div>
<div class="grid-item">reference</div>
<div class="grid-item"></div>
<div class="grid-item"></div>
<div class="grid-item"></div>
{% for item in trains | sort(attribute="depart") %}
{{ train_row(item) }}
{% for item in trains | sort(attribute="depart") if item.arrive %}
<div class="grid-item text-end">{{ item.depart.strftime("%a, %d %b %Y") }}</div>
<div class="grid-item">{{ item.from }} &rarr; {{ item.to }}</div>
<div class="grid-item">{{ item.depart.strftime("%H:%M") }}</div>
<div class="grid-item">
{% if item.arrive %}
{{ item.arrive.strftime("%H:%M") }}
{% if item.arrive.date() != item.depart.date() %}+1 day{% endif %}
{% endif %}
</div>
<div class="grid-item">{{ item.operator }}</div>
<div class="grid-item">{{ item.booking_reference }}</div>
{% endfor %}
</div>

113
templates/trip/list.html
View file

@ -1,113 +0,0 @@
{% extends "base.html" %}
{% from "macros.html" import trip_link, display_date_no_year, display_date, display_datetime, display_time, format_distance, trip_item with context %}
{% block title %}{{ heading }} - Edward Betts{% endblock %}
{% block style %}
<link rel="stylesheet" href="{{ url_for("static", filename="leaflet/leaflet.css") }}">
<link rel="stylesheet" href="{{ url_for("static", filename="css/trips.css") }}">
<style>
body, html {
height: 100%;
margin: 0;
}
.container-fluid {
height: calc(100% - 56px); /* Subtracting the height of the navbar */
}
.text-content {
overflow-y: scroll;
height: 100%;
}
.map-container {
position: sticky;
top: 56px; /* Adjust to be below the navbar */
height: calc(100vh - 56px); /* Subtracting the height of the navbar */
}
#map {
height: 100%;
}
@media (max-width: 767.98px) {
.container-fluid {
display: block;
height: auto;
}
.map-container {
position: relative;
top: 0;
height: 40vh;
}
.text-content {
height: auto;
overflow-y: auto;
}
}
</style>
{% endblock %}
{% macro section(heading, item_list) %}
{% if item_list %}
{% set items = item_list | list %}
<div class="trip-list-summary">
<h2>{{ heading }}</h2>
<p class="mb-2"><a href="{{ url_for("trip_stats") }}">Trip statistics</a> &middot; {{ items | count }} trips</p>
<div class="summary-stats-row">
<div class="summary-stat">
<span class="summary-stat-label">Total distance</span>
<span class="summary-stat-value">{{ format_distance(total_distance) }}</span>
</div>
{% for transport_type, distance in distances_by_transport_type %}
<div class="summary-stat">
<span class="summary-stat-label">{{ transport_type | title }}</span>
<span class="summary-stat-value">{{ format_distance(distance) }}</span>
</div>
{% endfor %}
<div class="summary-stat">
<span class="summary-stat-label">Total CO₂</span>
<span class="summary-stat-value">{{ "{:,.1f}".format(total_co2_kg / 1000.0) }} t</span>
</div>
{% for transport_type, co2_kg in co2_by_transport_type %}
<div class="summary-stat">
<span class="summary-stat-label">{{ transport_type | title }} CO₂</span>
<span class="summary-stat-value">{{ "{:,.1f}".format(co2_kg) }} kg</span>
</div>
{% endfor %}
</div>
</div>
{% for trip in items %}
{{ trip_item(trip) }}
{% endfor %}
{% endif %}
{% endmacro %}
{% block content %}
<div class="container-fluid d-flex flex-column flex-md-row">
<div class="text-content col-12 col-md-6 pe-3">
{{ section(heading, trips) }}
</div>
<div class="map-container col-12 col-md-6">
<div id="map" class="map"></div>
</div>
</div>
{% endblock %}
{% block scripts %}
<script src="{{ url_for("static", filename="leaflet/leaflet.js") }}"></script>
<script src="{{ url_for("static", filename="leaflet-geodesic/leaflet.geodesic.umd.min.js") }}"></script>
<script src="{{ url_for("static", filename="js/map.js") }}"></script>
<script>
var coordinates = {{ coordinates | tojson }};
var routes = {{ routes | tojson }};
build_map("map", coordinates, routes);
</script>
{% endblock %}

217
templates/trip/stats.html
View file

@ -1,217 +0,0 @@
{% extends "base.html" %}
{% from "macros.html" import format_distance with context %}
{% set heading = "Trip statistics" %}
{% block title %}{{ heading }} - Edward Betts{% endblock %}
{% block style %}
<style>
.co2-chart {
height: 240px;
}
</style>
{% endblock %}
{% macro stat_list(id, label, counter, show_top=5) %}
<div class="mb-2">
<strong>{{ label }}:</strong> {{ counter | count }}
{% if counter | count > 0 %}
<a class="btn btn-sm btn-outline-secondary ms-2" data-bs-toggle="collapse" href="#{{ id }}" role="button" aria-expanded="false">
show/hide
</a>
<div class="collapse mt-2" id="{{ id }}">
<div class="d-flex flex-wrap gap-1">
{% for item, count in counter.most_common() %}
<span class="badge text-bg-secondary">{{ item }} <span class="badge text-bg-light text-dark">{{ count }}</span></span>
{% endfor %}
</div>
</div>
{% endif %}
</div>
{% endmacro %}
{% macro format_co2(co2_kg) %}
{% if co2_kg >= 1000 %}
{{ "{:,.2f}".format(co2_kg / 1000.0) }} tonnes
{% else %}
{{ "{:,.0f}".format(co2_kg) }} kg
{% endif %}
{% endmacro %}
{% block content %}
<div class="container-fluid">
<h1>Trip statistics</h1>
<div class="card mb-4">
<div class="card-header"><strong>Overall Summary</strong></div>
<div class="card-body">
<div class="row">
<div class="col-md-6">
<div>Trips: {{ count }}</div>
<div>Conferences: {{ conferences }}</div>
<div>Total distance: {{ format_distance(total_distance) }}</div>
{% for transport_type, distance in distances_by_transport_type %}
<div class="ms-3">{{ transport_type | title }}: {{ format_distance(distance) }}</div>
{% endfor %}
{% if total_co2_kg %}
<div class="mt-2">CO₂: {{ format_co2(total_co2_kg) }}</div>
{% endif %}
{% for transport_type, co2_kg in co2_by_transport_type %}
<div class="ms-3">{{ transport_type | title }} CO₂: {{ format_co2(co2_kg) }}</div>
{% endfor %}
</div>
<div class="col-md-6">
<div>Flight segments: {{ overall_stats.flight_count }}</div>
<div>Train segments: {{ overall_stats.train_count }}</div>
{% if co2_by_transport_type %}
<div class="co2-chart mt-3">
<canvas id="overall-co2-chart"></canvas>
</div>
{% endif %}
</div>
</div>
<hr>
{{ stat_list("overall-airlines", "Airlines", overall_stats.airlines) }}
{{ stat_list("overall-airports", "Airports", overall_stats.airports) }}
{{ stat_list("overall-stations", "Stations", overall_stats.stations) }}
</div>
</div>
{% for year, year_stats in yearly_stats | dictsort(reverse=True) %}
{% set countries = year_stats.countries | default([]) | sort(attribute="name") %}
{% set new_countries = year_stats.new_countries | default([]) %}
<div class="card mb-3">
<div class="card-header">
<strong>{{ year }}</strong>
<span class="text-muted ms-2">{{ year_stats.count }} trips</span>
</div>
<div class="card-body">
<div class="row mb-2">
<div class="col-md-6">
<div>Trips: {{ year_stats.count }}</div>
<div>Conferences: {{ year_stats.conferences }}</div>
<div>Distance: {{ format_distance(year_stats.total_distance or 0) }}</div>
{% if year_stats.distances_by_transport_type %}
{% for transport_type, distance in year_stats.distances_by_transport_type.items() %}
<div class="ms-3">{{ transport_type | title }}: {{ format_distance(distance) }}</div>
{% endfor %}
{% endif %}
</div>
<div class="col-md-6">
<div>Flight segments: {{ year_stats.flight_count or 0 }}</div>
<div>Train segments: {{ year_stats.train_count or 0 }}</div>
{% if year_stats.co2_kg %}
<div>CO₂: {{ format_co2(year_stats.co2_kg) }}</div>
{% for transport_type, co2_kg in year_stats.co2_by_transport_type.items() %}
<div class="ms-3">{{ transport_type | title }} CO₂: {{ format_co2(co2_kg) }}</div>
{% endfor %}
<div class="co2-chart mt-3">
<canvas id="co2-chart-{{ year }}"></canvas>
</div>
{% endif %}
</div>
</div>
<div class="mb-2">
<strong>Countries:</strong> {{ countries | count }}
{% if new_countries %}({{ new_countries | count }} new){% endif %}
<div class="d-flex flex-wrap gap-1 mt-1">
{% for c in countries %}
<span class="badge text-bg-light text-dark border">
{{ c.flag }} {{ c.name }}
{% if c in new_countries %}
<span class="badge text-bg-info">new</span>
{% endif %}
</span>
{% endfor %}
</div>
</div>
{% if year_stats.airlines %}
{{ stat_list("airlines-" ~ year, "Airlines", year_stats.airlines) }}
{% endif %}
{% if year_stats.airports %}
{{ stat_list("airports-" ~ year, "Airports", year_stats.airports) }}
{% endif %}
{% if year_stats.stations %}
{{ stat_list("stations-" ~ year, "Stations", year_stats.stations) }}
{% endif %}
</div>
</div>
{% endfor %}
</div>
{% endblock %}
{% block scripts %}
<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
<script>
const co2ChartColors = [
"#4e79a7",
"#f28e2b",
"#e15759",
"#76b7b2",
"#59a14f",
"#edc948",
"#b07aa1",
"#ff9da7",
"#9c755f",
"#bab0ac",
];
function titleCase(value) {
return value.replace(/\b\w/g, char => char.toUpperCase());
}
function co2TooltipLabel(context) {
const label = context.label || "";
const value = context.parsed || 0;
return `${label}: ${value.toLocaleString(undefined, { maximumFractionDigits: 1 })} kg`;
}
function renderCo2Chart(id, values) {
const canvas = document.getElementById(id);
if (!canvas || !values || values.length === 0) {
return;
}
new Chart(canvas.getContext("2d"), {
type: "pie",
data: {
labels: values.map(item => titleCase(item[0])),
datasets: [{
data: values.map(item => item[1]),
backgroundColor: co2ChartColors,
}],
},
options: {
responsive: true,
maintainAspectRatio: false,
plugins: {
legend: {
position: "right",
},
tooltip: {
callbacks: {
label: co2TooltipLabel,
},
},
},
},
});
}
renderCo2Chart("overall-co2-chart", {{ co2_by_transport_type | tojson }});
{% for year, year_stats in yearly_stats | dictsort(reverse=True) %}
{% set year_co2_by_transport_type = year_stats.co2_by_transport_type | default({}) %}
renderCo2Chart(
"co2-chart-{{ year }}",
{{ year_co2_by_transport_type.items() | list | tojson }}
);
{% endfor %}
</script>
{% endblock %}

Some files were not shown because too many files have changed in this diff Show more