edward/agenda
1
0
Fork 0
Code Issues 9 Pull requests Projects Releases Packages Wiki Activity
agenda/docs/personal-data-yaml.md

1002 lines
32 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Personal Data YAML Formats
This document describes the YAML files read from `../personal-data/`. It is intended for humans and LLMs generating new entries.
## General Rules
- Use YAML lists for most files. `airports.yaml` is a mapping keyed by IATA code.
- Use ISO-like YAML dates and datetimes:
- Date: `2026-03-14`
- Datetime with timezone: `2026-03-14 09:30:00+01:00`
- Use lowercase ISO 3166-1 alpha-2 country codes, for example `gb`, `be`, `us`.
- Use quoted strings for prices and identifiers that might otherwise be parsed as numbers: `'154.34'`, `'06525269'`, `'0042'`.
- Currencies must be in `config.CURRENCIES` or `GBP`.
- Travel and trip-related entries are grouped by the `trip` date. That date should match an entry in `trips.yaml` when a named trip is needed, but trip groups can also be created from travel/accommodation/conference entries.
- Keep chronological files sorted by their natural start field. `validate_yaml.py` checks ordering for trips, flights, trains, ferries, conferences, and accommodation.
- Preserve the existing whitespace style. Long top-level list files such as `accommodation.yaml`, `buses.yaml`, `car_journeys.yaml`, `coaches.yaml`, `conferences.yaml`, `ferries.yaml`, `flights.yaml`, `stations.yaml`, `trains.yaml`, and `trips.yaml` use one blank line between top-level items. Mapping files such as `airports.yaml` do not use this list-item spacing.
- Coordinates are `latitude` then `longitude`, both numeric.
## Cross-File References
- `flights.yaml` flight `airline` values reference `airlines.yaml` `iata`.
- `flights.yaml` flight `from` and `to` values reference `airports.yaml` keys.
- `trains.yaml` journey and leg `from` and `to` values reference `stations.yaml` `name`.
- `ferries.yaml` `from` and `to` values reference `ferry_terminals.yaml` `name`.
- `buses.yaml` `from` and `to` values reference `bus_stops.yaml` `name`.
- `coaches.yaml` `from` and `to` values reference `coach_stations.yaml` `name`.
- `car_journeys.yaml` `route` values name files in `car_routes/`. The `.geojson` extension is optional.
- Station, stop, and terminal `routes` values name GeoJSON files without the `.geojson` extension.
## `accommodation.yaml`
Top-level shape: list of accommodation stays.
Used by: agenda events, trip pages, trip maps, busy/location logic.
Required fields:
- `type`: accommodation category such as `hotel`, `apartment`, `airbnb`.
- `name`: property name.
- `country`: lowercase country code.
- `location`: city or place name.
- `trip`: trip start date.
- `from`: check-in datetime.
- `to`: check-out datetime.
Common optional fields:
- Booking: `operator`, `booking_reference`, `confirmation_code`, `booking_url`, `url`, `email`, `phone`.
- Money: `price`, `currency`, `room_rate`, `estimated_taxes`, `estimated_additional_fees`.
- Room/stay: `address`, `room_type`, `room_name`, `room_number`, `number_of_adults`, `breakfast_included`, `breakfast`, `cancellation_policy`, `free_cancellation`, `refundable`.
- Coordinates/IDs: `latitude`, `longitude`, `timezone`, `osm_node`, `wikidata`.
- Loyalty: `rewards`, `radisson_rewards_number`.
Example:
```yaml
- type: hotel
operator: Example Hotels
name: Example Central Hotel
location: Brussels
country: be
trip: 2026-02-06
from: 2026-02-06 15:00:00+01:00
to: 2026-02-09 11:00:00+01:00
address: 1 Example Street, Brussels
confirmation_code: ABC123
price: '312.50'
currency: EUR
number_of_adults: 1
room_type: Standard double
breakfast_included: true
latitude: 50.8466
longitude: 4.3528
```
## `airlines.yaml`
Top-level shape: list of airlines.
Used by: flight loading and display.
Required fields:
- `iata`: two-character IATA airline code.
- `icao`: three-character ICAO airline code.
- `name`: airline name.
Optional fields:
- `flight_number_prefer_icao`: boolean. When true, display flight numbers with the ICAO code instead of the IATA code.
Example:
```yaml
- iata: BA
icao: BAW
name: British Airways
- iata: U2
icao: EZY
name: easyJet
flight_number_prefer_icao: true
```
## `airports.yaml`
Top-level shape: mapping keyed by IATA airport code.
Used by: flight loading, distance calculation, maps, unbooked route hints.
Required fields for each airport:
- `iata`: IATA code. Should match the mapping key.
- `name`: airport name.
- `city`: city or main served place.
- `country`: lowercase country code.
- `latitude`, `longitude`: numeric coordinates.
- `qid`: Wikidata QID.
Optional fields:
- `alt_name`: display name override used in labels.
- `elevation`: metres.
- `website`, `url`.
Example:
```yaml
BRU:
iata: BRU
name: Brussels Airport
city: Brussels
country: be
qid: Q220613
latitude: 50.9014
longitude: 4.4844
elevation: 56
website: https://www.brusselsairport.be/
```
## `bus_stops.yaml`
Top-level shape: list of bus stops.
Used by: bus trip loading, maps, route rendering.
Required fields:
- `name`: stop name referenced by `buses.yaml`.
- `city`: city or place.
- `country`: lowercase country code.
- `latitude`, `longitude`: numeric coordinates.
- `routes`: mapping from destination stop name to GeoJSON filename without `.geojson`.
Optional fields:
- `Atco`: UK ATCO stop code.
- `osm_node`.
Example:
```yaml
- name: West Street
city: Bristol
country: gb
Atco: '0100BRA10073'
osm_node: 485403178
latitude: 51.4393854
longitude: -2.6017977
routes:
Bristol Airport: West_Street_to_Bristol_Airport
```
## `buses.yaml`
Top-level shape: list of bus journeys.
Used by: trip loading, maps, trip timeline. Bus journeys are not counted for Schengen tracking.
Required fields:
- `trip`: trip start date.
- `depart`: departure datetime.
- `arrive`: arrival datetime. `validate_yaml.py` requires arrival after departure and duration no more than 12 hours.
- `from`, `to`: names from `bus_stops.yaml`.
Optional fields:
- `operator`, `price`, `currency`.
Example:
```yaml
- trip: 2026-03-14
depart: 2026-03-14 08:20:00+00:00
arrive: 2026-03-14 08:55:00+00:00
from: West Street
to: Bristol Airport
operator: First Bus
price: '2.00'
currency: GBP
```
## `coach_stations.yaml`
Top-level shape: list of coach stations.
Used by: coach trip loading, maps, route rendering.
Fields are the same pattern as `bus_stops.yaml`, except entries describe coach stations.
Example:
```yaml
- name: Example Coach Station
city: Example City
country: gb
latitude: 51.4500
longitude: -2.5800
routes:
Other Coach Station: example_city_to_other_city
```
## `coaches.yaml`
Top-level shape: list of coach journeys.
Used by: trip loading, maps, trip timeline. Coach journeys are not counted for Schengen tracking.
Required fields:
- `trip`, `depart`, `arrive`, `from`, `to`.
- `from` and `to` must be names from `coach_stations.yaml`.
Optional fields:
- `operator`, `class`, `booking_reference`, `price`, `currency`, `price_details`.
Example:
```yaml
- booking_reference: ABC123
trip: 2026-05-25
price: '55.00'
currency: GBP
depart: 2026-05-26 14:45:00+01:00
arrive: 2026-05-26 18:30:00+01:00
from: Example Coach Station
to: Other Coach Station
operator: Example Coaches
class: Standard
price_details:
base_fare: '55.00'
```
## `car_journeys.yaml`
Top-level shape: list of car journeys.
Used by: trip loading, maps, trip timeline. Car journeys are for driving your own car or a rental car.
Required fields:
- `trip`: trip start date.
- `depart`: departure date or datetime.
- `arrive`: arrival date or datetime.
- `route`: GeoJSON filename in `car_routes/`, with or without the `.geojson` extension.
Optional fields:
- `from`, `to`: endpoint labels. If omitted and the route filename uses `A_to_B`, labels are inferred from the filename.
- `show_markers`: boolean. When true, render both car endpoint markers on maps. Defaults to false because car endpoints often duplicate airport, accommodation, ferry terminal, or conference pins. Home endpoints still render by default.
- `from_show_marker`, `to_show_marker`: booleans. Per-endpoint marker overrides.
- `from_type`, `to_type`: marker type override used when that endpoint marker is shown. Use `home` to render a house icon. Otherwise car endpoints render as car markers.
- `operator`, `vehicle`, `price`, `currency`, `distance`.
The route distance is calculated from the GeoJSON when `distance` is not present. Car routes render on maps by default. Non-home endpoint pins are opt-in. If an endpoint label is `home`, `PCH`, or `Picture House Court`, the marker is rendered as a house by default.
Example:
```yaml
- trip: 2026-07-16
depart: 2026-07-16
arrive: 2026-07-16
route: PCH_to_EMF.geojson
- trip: 2026-07-16
depart: 2026-07-19
arrive: 2026-07-19
route: EMF_to_PCH.geojson
```
## `conferences.yaml`
Top-level shape: list of conferences and conference-like events.
Used by: agenda events, trip pages, trip maps, conference list, CFP reminders.
Required fields:
- `name`: event name.
- `topic`: topic/category.
- `location`: city or location label.
- Date information, either as legacy top-level `start` and `end`, or preferred nested `dates`.
Preferred `dates` fields:
- `status`: one of `exact`, `tentative`, or `approximate`.
- For `exact` and `tentative`: `start` and `end` dates/datetimes. `end` must be no earlier than `start`, and duration must be under 20 days.
- For `approximate`: `earliest` and `latest` dates for sorting/past-future filtering.
- `label`: optional human-readable date text. Recommended for `tentative` and `approximate`, for example `likely first weekend of February 2027` or `March 2027`.
- `basis`: optional explanation of why a tentative date is expected.
Date status behavior:
- `exact`: confirmed dates. These create agenda events, iCalendar entries, and timeline bars.
- `tentative`: guessed or unconfirmed exact dates. These appear on the conference list with a status badge, but do not create agenda/iCalendar events or timeline bars.
- `approximate`: only a broad date range is known. These appear on the conference list with a status badge, but do not create agenda/iCalendar events or timeline bars.
Legacy fields:
- Existing top-level `start` and `end` are still supported and are treated as `exact` unless `date_status` says otherwise.
Common optional fields:
- Series: `series`, a key from `conference_series.yaml`.
- Trip/location: `trip`, `country`, `venue`, `address`, `latitude`, `longitude`.
- Attendance: `going`, `registered`, `speaking`, `online`, `accommodation_booked`, `transport_booked`.
- Partial attendance: `attend_start`, `attend_end`. These may be dates or timezone-aware datetimes and are used on trip pages instead of official dates.
- Web/CFP: `url`, `cfp_end`, `cfp_url`, `hashtag`, `description`.
- Money/tickets: `free`, `price`, `currency`, `ticket_type`.
- Other flags: `hackathon`, `attendees`.
Exact example:
```yaml
- name: FOSDEM
series: fosdem
topic: FOSDEM
location: Brussels
country: be
trip: 2026-02-06
dates:
status: exact
start: 2026-02-07
end: 2026-02-08
attend_start: 2026-02-07 14:00:00+01:00
attend_end: 2026-02-08
going: true
registered: true
accommodation_booked: true
transport_booked: true
url: https://fosdem.org/2026/
venue: Universite Libre de Bruxelles
address: Av. Franklin Roosevelt 50, 1050 Bruxelles, Belgium
latitude: 50.8132
longitude: 4.3822
```
Tentative example:
```yaml
- name: FOSDEM
series: fosdem
topic: FOSDEM
location: Brussels
country: be
dates:
status: tentative
start: 2027-01-30
end: 2027-01-31
label: likely first weekend of February 2027
basis: FOSDEM is usually on the weekend where Sunday is the first Sunday in February
url: https://fosdem.org/2027/
```
Approximate examples:
```yaml
- name: Wikimedia Hackathon 2027
series: wikimedia-hackathon
topic: Wikimedia
location: Albania
country: al
dates:
status: approximate
label: mid-April 2027
earliest: 2027-04-11
latest: 2027-04-20
hackathon: true
- name: PyCascades 2027
series: pycascades
topic: Python
location: TBC
dates:
status: approximate
label: March 2027
earliest: 2027-03-01
latest: 2027-03-31
```
## `conference_series.yaml`
Top-level shape: mapping from stable series ID to series metadata.
Used by: conference list pages, conference series index/detail pages, and validation of `conferences.yaml` `series` references.
Required fields for each series:
- `name`: display name for the series.
Common optional fields:
- `topic`: default topic/category.
- `cadence`: for example `annual` or `recurring`.
- `usual_location`: common city/place when the event usually stays in one place.
- `country`: common lowercase country code when stable.
- `url`: series homepage.
- `notes`: free-text generation or scheduling notes.
Example:
```yaml
fosdem:
name: FOSDEM
topic: FOSDEM
cadence: annual
usual_location: Brussels
country: be
url: https://fosdem.org/
notes: Usually the weekend where Sunday is the first Sunday in February.
geomob-london:
name: Geomob London
topic: Maps
cadence: recurring
usual_location: London
country: gb
url: https://thegeomob.com/
```
## `entities.yaml`
Top-level shape: list of people/entities.
Used by: birthday events.
Required fields for birthday support:
- `name`: full name.
- `label`: display name.
- `type`: for example `human`.
- `birthday`: mapping with `day`, `month`, and optionally `year`.
Optional fields:
- `relation`, `email`.
If `birthday.year` is omitted, age is shown as unknown.
Example:
```yaml
- name: Ada Example
label: Ada
type: human
relation: friend
birthday:
day: 10
month: 12
year: 1990
```
## `events.yaml`
Top-level shape: list of general events.
Used by: agenda events and trip pages.
Required fields:
- `name`: event type.
- One date source:
- `date`: single event date/datetime, or
- `start_date`: used for events with a separate `end_date`, or
- `rrule`: recurrence rule string.
Optional fields:
- `title`: display title.
- `end_date`: explicit end date/datetime.
- `duration`: ISO 8601 duration such as `PT2H`, `P1D`.
- `url`.
- Trip/map fields: `trip`, `location`, `country`, `venue`, `address`, `latitude`, `longitude`.
Special cases:
- For `name: travel_insurance`, the event date field is `end_date`; no `end_date` is attached to the generated event.
- For recurring events, if the `rrule` has no `BYHOUR`, `BYMINUTE`, or `BYSECOND`, generated events are all-day dates. Otherwise generated datetimes are localized to UK time.
- `skip_trips=True` consumers ignore entries with `trip`.
Examples:
```yaml
- name: travel_insurance
start_date: 2026-05-04
end_date: 2027-05-03
- name: meetup
title: Example Geo Meetup
date: 2026-06-18 18:30:00+01:00
duration: PT2H
url: https://example.org/meetup
location: Bristol
country: gb
latitude: 51.4545
longitude: -2.5879
- name: market
title: Monthly Example Market
rrule: FREQ=MONTHLY;BYDAY=1SA
```
## `ferries.yaml`
Top-level shape: list of ferry journeys.
Used by: trip loading, maps, trip timeline, Schengen tracking.
Required fields:
- `trip`: trip start date.
- `depart`, `arrive`: datetimes. Ferry `arrive` is required.
- `from`, `to`: names from `ferry_terminals.yaml`.
Common optional fields:
- `operator`, `ferry`, `direction`, `class`, `booking_reference`, `price`, `currency`.
- `price_details`: free-form mapping of fare components.
- `vehicle`: mapping with fields such as `type`, `registration`, `height`, `length`, `extras`.
Example:
```yaml
- booking_reference: ABC123
trip: 2026-05-04
price: '302.00'
currency: GBP
depart: 2026-05-04 23:00:00+01:00
arrive: 2026-05-05 08:00:00+02:00
from: Portsmouth
to: Cherbourg
operator: Brittany Ferries
class: Commodore cabin
price_details:
base_fare: '153.00'
cabin: '149.00'
vehicle:
type: Example car
registration: AB12CDE
height: 1.63m
length: 4.15m
```
## `ferry_terminals.yaml`
Top-level shape: list of ferry terminals.
Used by: ferry loading and route rendering.
Required fields:
- `name`: terminal name referenced by `ferries.yaml`.
- `city`, `country`.
- `latitude`, `longitude`.
- `routes`: mapping from destination terminal name to GeoJSON filename without `.geojson`. Ferry route rendering expects a GeoJSON route.
Optional fields:
- `osm_node`, `osm_way`.
Example:
```yaml
- name: Portsmouth
city: Portsmouth
country: gb
osm_way: 123456
latitude: 50.8120
longitude: -1.0880
routes:
Cherbourg: portsmouth_cherbourg
```
## `flight_destinations.yaml`
Top-level shape: list of origin rules for unbooked conference flight route hints.
Used by: trip maps when a trip has conferences but no booked travel.
Required fields:
- `origin`: origin airport IATA code.
- `airline`: airline IATA code. Currently loaded for validation/description but not used in origin selection.
- `destinations`: list of destination airport IATA codes.
Example:
```yaml
- origin: BRS
airline: U2
destinations:
- AMS
- BCN
- CDG
```
## `flights.yaml`
Top-level shape: list of flight bookings. Each booking contains one or more flight legs.
Used by: agenda transport events, trip loading, maps, distance calculation.
Required booking fields:
- `trip`: trip start date.
- `flights`: list of flight leg mappings.
Common optional booking fields:
- `booking_reference`, `price`, `currency`.
Required flight leg fields:
- `depart`: departure datetime.
- `from`, `to`: airport IATA codes from `airports.yaml`.
- `flight_number`: numeric/string flight number without airline prefix.
- `airline`: airline IATA code from `airlines.yaml`.
Common optional flight leg fields:
- Time/location: `arrive`, `from_terminal`, `to_terminal`, `duration`.
- Seat/cabin: `seat`, `seat_type`, `class`, `cabin`.
- Aircraft: `plane`, `registration`.
- Tracking: `distance`, `co2_kg`, `openflights_trip`, `reason`.
- Ticket/passenger: `e_ticket_number`, `ticket_number`, `frequent_flyer_number`, `passenger_name`, `passengers`, `baggage`, `payment_details`.
`validate_yaml.py` checks that every booking has `trip`, all flight airlines exist in `airlines.yaml`, bookings are sorted by first departure, and currencies are configured. It reports flights missing `co2_kg`.
Example:
```yaml
- booking_reference: ABC123
trip: 2026-04-22
price: '62.50'
currency: GBP
flights:
- depart: 2026-04-22 17:20:00+01:00
arrive: 2026-04-22 20:20:00+02:00
from: LHR
to: BRU
flight_number: '1234'
airline: BA
duration: 01:00
seat: 5F
seat_type: W
class: C
cabin: business
plane: Airbus A320
registration: G-ABCD
co2_kg: 154
```
## `follow_launches.yaml`
Top-level shape: list of SpaceDevs launch slugs.
Used by: no current in-repo reader was found, but the file appears intended as a watch list for launch update tooling.
Example:
```yaml
- starship-integrated-flight-test-5
- artemis-ii
```
## `stations.yaml`
Top-level shape: list of railway stations.
Used by: train loading, maps, route rendering.
Required fields:
- `name`: station name referenced by `trains.yaml`.
- `country`: lowercase country code.
- `latitude`, `longitude`.
- `routes`: mapping from destination station name to GeoJSON filename without `.geojson`.
Common optional fields:
- `uic`, `alpha3`, `wikidata`, `osm_node`.
Note: the code reads `routes`, not `rotues`; `rotues` appears to be a typo in existing data and should not be used for new entries.
Example:
```yaml
- name: London St Pancras
uic: 7015400
alpha3: STP
wikidata: Q720102
latitude: 51.531921
longitude: -0.126361
country: gb
routes:
Brussels Midi: london_brussels_eurostar
```
## `subscriptions.yaml`
Top-level shape: list of subscriptions.
Used by: subscription renewal agenda events when `renewal_date` is present.
Required fields:
- `name`: subscription name.
Common optional fields:
- Dates: `start`, `start_date`, `renewal_date`.
- `price`: mapping with `amount` and `currency`.
- `term`: mapping with `duration` and `unit` or `term_unit`.
- Account: `email`, `account_url`, `account_number`.
Only items with `renewal_date` create agenda events.
Example:
```yaml
- name: Example Magazine
start_date: 2026-01-01
renewal_date: 2027-01-01
price:
amount: 99
currency: GBP
term:
duration: 1
unit: year
email: me@example.com
account_url: https://example.com/account
account_number: '001234'
```
## `trains.yaml`
Top-level shape: list of train journeys. Each journey contains one or more legs.
Used by: agenda transport events, trip loading, trip timeline, maps, stats.
Required journey fields:
- `operator`: booking/operator label.
- `from`, `to`: station names from `stations.yaml`.
- `trip`: trip start date.
- `depart`, `arrive`: journey datetimes or dates.
- `legs`: list of leg mappings.
Common optional journey fields:
- `class`, `number`, `tickets`, `ticket_code`, `total_price`, `co2_kg`.
Required leg fields:
- `from`, `to`: station names from `stations.yaml`.
- `depart`, `arrive`.
- `operator`.
Common optional leg fields:
- `train`, `number`, `service`, `service_number`, `service_numbers`, `reporting_number`, `mode`.
- Seat/reservation: `coach`, `seat`, `seat_type`, `seat_features`, `reservation_number`, `platform`.
- `class`, `trip`, `url`.
Ticket fields are free-form but commonly include `booking_reference`, `url`, `price`, `currency`, `booking_date`, `ticket`, `ticket_code`, `ticket_type`, `from`, `to`, `class`, `validity`, `route`, `fare`, `quantity`, `seat_reservation`.
Example:
```yaml
- operator: eurostar
from: London St Pancras
to: Brussels Midi
trip: 2026-02-06
depart: 2026-02-06 15:04:00+00:00
arrive: 2026-02-06 18:12:00+01:00
class: Standard Premier
tickets:
- booking_reference: ABCDEF
url: https://example.com/booking/ABCDEF
price: '89.00'
currency: GBP
legs:
- from: London St Pancras
to: Brussels Midi
depart: 2026-02-06 15:04:00+00:00
arrive: 2026-02-06 18:12:00+01:00
coach: 1
seat: 41
operator: Eurostar
```
## `travel_rewards.yaml`
Top-level shape: list of travel loyalty accounts.
Used by: no current in-repo reader was found. The file is structured as account metadata.
Common fields:
- `name`: programme name.
- `type`: category such as `hotel`, `airline`, `rail`.
- `member_number`: membership identifier.
- `balance`: current points/miles balance.
- `expiry`: expiry date or null.
- `url`: account URL.
- `person`: account holder key/name.
- `email`, `note`.
Example:
```yaml
- name: Example Rewards
type: hotel
member_number: '123456789'
balance: 3665
expiry: 2027-08-15
url: https://example.com/rewards
person: edward
```
## `trips.yaml`
Top-level shape: list of trip metadata.
Used by: trip grouping and trip titles.
Required fields:
- `trip`: trip start date. This is the grouping key used by travel, accommodation, conferences, and trip events.
Optional fields:
- `name`: explicit trip title.
- `private`: boolean. Private trips are hidden from unauthenticated users.
Example:
```yaml
- trip: 2026-02-06
name: Brussels for FOSDEM
private: false
```
### Conference detail pages and airport suggestions
Conference titles in the list and series pages link to
`/conference/YYYY-MM-DD/series-or-title-location`. The date is the conference's
start date (or the earliest date for an approximate event). The detail page keeps
an external website link and displays the venue, address, coordinates, attendance
fields and remaining conference metadata.
Airport suggestions start with `agenda/conference_airports.json`. You can override
or extend these in `personal-data/conference_airports.yaml`, using lowercase
country codes and casefolded location names:
```yaml
"be:brussels": CRL
"us:pasadena, california": LAX
"se:malmö": [MMX, CPH]
"ch:bern": [BRN, BSL]
"de:bonn": [CGN, DUS]
"dk:funen": [BLL, CPH]
"it:bologna": [BLQ, VRN, VCE]
```
Mappings may name several airports. Malmö (also entered as Malmo or Malmö,
Sweden) searches MMX and CPH together, including cross-border airports explicitly
listed in the mapping. Bern searches BRN and Basel (BSL) together; Bonn searches Cologne/Bonn (CGN)
and Düsseldorf (DUS) together. Funen includes Billund (BLL) and Copenhagen (CPH). Bologna includes BLQ, Verona
(VRN) and Venice Marco Polo (VCE). You can also enter comma-separated IATA codes such as
`MMX,CPH`. Each route/date uses one combined Google search, so Bristol's direct
flights to CPH are considered without doubling the number of requests. Results
show the actual airport for each flight leg.
Otherwise the page matches cities in `airports.yaml`, then suggests the nearest
known airport within 150 km of the venue, limited to the same country when known.
The flight lookup form accepts IATA codes, cities and airport names, with local
autocomplete from the bundled public-domain OurAirports index
(`agenda/airport_index.json`, downloaded from https://ourairports.com/data/).
Names from `PERSONAL_DATA/airports.yaml` (normally
`~/src/personal-data/airports.yaml`) override bundled labels for individual airports,
combined destinations and autocomplete. Missing or blank names fall back to the
bundled index, and edits take effect on the next request. Both personal and
bundled names remain searchable.
Scheduled-service airports are preferred; three-letter inputs are treated as
IATA codes. London resolves to LON (all London airports). The highest-ranked match is
used and its name/code shown; pick an autocomplete result for a specific airport.
Country suffixes narrow the matches; US state abbreviations such as CA are
recognized too. For example,
`Copenhagen, Denmark` resolves to CPH. A nearby airport may
still require a substantial ground transfer. Unknown locations need an IATA code
or city/airport name entered manually.
“Open in Google Flights” opens a prefilled round trip in a new tab, with the
current airport group, outbound/return dates, stops, one adult in economy and
GBP/en-GB settings. Before a lookup it uses the initial Bristol or London dates;
for cached results each origin has its own link using the displayed flights'
departure dates (or the last checked dates when no flights were found). Links
are generated locally and work during the shared server cooldown.
Only the flight lookup button contacts Google Flights from the server. Searches use Python
Playwright with installed Chromium, one adult in economy, language
`en-GB`, country `GB`
and currency `GBP`. Overseas European destinations within 3,500 km of London
prefer direct Bristol flights. Start with the day before and the day after the
conference, expanding each direction independently to at most four days away
only when no suitable flights are found. Stop searching a direction at the first
date with flights; when both nearest dates work, this needs just two searches. If either direction has no suitable results, London is
searched too. Other destinations start with London. London means LHR, LGW, STN,
LTN, LCY and SEN together, initially with at most one stop; departures are two days
before the conference to allow overnight travel, trying three days before if
no suitable outbound flights are found. Returns are the day after. Outbound
flights must arrive before the opening date. Prices are **one-way fares**, shown
separately for outbound and return travel. Searches use Google’s best-flight group first,
then prefer direct flights and BA within each stop count (all BA legs first, then
some BA legs, then other airlines). Operating airline is used when available,
otherwise the advertised airline. Google’s order is preserved within each group.
If a London search has no direct or one-stop results, it retries with at most two
stops. Bristol always remains direct-only.
Results live in `DATA_DIR/conference-flights` (normally
`/home/edward/lib/data/conference-flights`), keyed by dates, destination and search
policy. Updates reuse results for six hours. Empty results are also cached;
errors have a 15-minute retry cooldown and preserve any previous successful
results. Failures show the route/date and underlying exception type/message;
full tracebacks are logged on the server. Explicit Google date-range rejections
ask you to retry closer to departure, rather than after 15 minutes. Confirmed
“no flights found” and “no non-stop flights found” pages count as empty results;
Bristol searches can then continue to nearby dates and show “No direct flights
found” with the checked dates if none work; other unusable responses remain
errors and include the requested date. No fixed airline booking horizon is assumed. Legacy cached failures without details
can be retried immediately. A shared process lock prevents simultaneous conference flight searches.
The page shows when results were last fetched, the destination airport names,
and the departure dates checked when no suitable flights were found. Belfast is an exception to the UK exclusion: it searches Belfast International
(BFS) and Belfast City (BHD) together, starting with direct flights from Bristol.
Conferences with tentative start/end dates can also search for flights; the flight
section displays those provisional dates. Other domestic, past, online and
approximate-date conferences display details but do not offer flight lookup.
Individual route/date searches (including empty results) are cached for six hours
under `conference-flights/days`, so overlapping conferences reuse the same Google
lookup. Browser searches are spaced at least three seconds apart using shared disk state
across Apache workers. Chromium loads the page’s assets normally. HTTP 429 is not immediately
retried: it pauses all uncached Google flight requests for 30 minutes. The page
shows the shared cooldown and continues to display cached results. Failed navigations are not automatically retried.
Search URL encoding and result parsing follow the format documented by
https://github.com/AWeirdDev/flights (MIT attribution in
`licenses/fast-flights-MIT.txt`). `flights` is no longer an application dependency. `run.fcgi` uses `/usr/bin/python3`; a project virtualenv is no longer
needed. System Python must provide Playwright and the existing Flask dependencies,
and Chromium must be installed. On Debian we launch
`/usr/lib/chromium/chromium` directly: Apache's `ProcSubset=pid` hides
`/proc/cpuinfo`, which causes `/usr/bin/chromium`'s shell-wrapper CPU check to
fail incorrectly. Other installations fall back to `chromium` on PATH. The browser is started only on a cache miss,
reused for that lookup’s dates, and closed after the lookup. Its persistent profile
in `conference-flights/browser-profile` preserves consent choices. The shared
search lock prevents multiple workers opening that profile simultaneously.
Google’s result format is undocumented, and using a browser does not guarantee
avoidance of rate limits or verification challenges. Existing successful cached
fares remain usable; old empty results and failures can be refreshed using the
browser, subject to the existing shared cooldown.
Failed browser searches save compressed JSON diagnostics in
`DATA_DIR/conference-flights/errors/*.json.gz`. Each contains the UTC timestamp,
route, requested date, stop limit, locale/currency, request and final page URLs,
HTTP status, exception details, raw flight-data script, visible page text, page
HTML, and up to five Google HTTP error response bodies. Capture uses the loaded
page and received responses; it makes no extra Google requests. Capture failures
are logged and cannot replace the original lookup error. Local cooldown refusals
and successful searches do not create snapshots.
Snapshots are retained for 30 days, with at most 100 files; cleanup runs when a
new snapshot is written. Captures are capped at 2 MiB of HTML, 1 MiB of flight
script, 64 KiB of visible text and 256 KiB per HTTP error body, with any truncation
recorded. Snapshot files have mode 0640. Previous failures cannot be recovered
from these snapshots; recording begins with this change. Use Python's `gzip.open`
to read the JSON for analysis or replay the saved `flight_script` through
`agenda.google_flights.parse_results`.
Reference in a new issue View git blame Copy permalink