Add conference detail pages and cached flight searches
This commit is contained in:
parent
6b253f2d04
commit
dcaf723336
24 changed files with 12198 additions and 12 deletions
|
|
@ -861,3 +861,142 @@ Example:
|
|||
name: Brussels for FOSDEM
|
||||
private: false
|
||||
```
|
||||
|
||||
### Conference detail pages and airport suggestions
|
||||
|
||||
Conference titles in the list and series pages link to
|
||||
`/conference/YYYY-MM-DD/series-or-title-location`. The date is the conference's
|
||||
start date (or the earliest date for an approximate event). The detail page keeps
|
||||
an external website link and displays the venue, address, coordinates, attendance
|
||||
fields and remaining conference metadata.
|
||||
|
||||
Airport suggestions start with `agenda/conference_airports.json`. You can override
|
||||
or extend these in `personal-data/conference_airports.yaml`, using lowercase
|
||||
country codes and casefolded location names:
|
||||
|
||||
```yaml
|
||||
"be:brussels": CRL
|
||||
"us:pasadena, california": LAX
|
||||
"se:malmö": [MMX, CPH]
|
||||
"ch:bern": [BRN, BSL]
|
||||
"de:bonn": [CGN, DUS]
|
||||
"dk:funen": [BLL, CPH]
|
||||
"it:bologna": [BLQ, VRN, VCE]
|
||||
```
|
||||
|
||||
Mappings may name several airports. Malmö (also entered as Malmo or Malmö,
|
||||
Sweden) searches MMX and CPH together, including cross-border airports explicitly
|
||||
listed in the mapping. Bern searches BRN and Basel (BSL) together; Bonn searches Cologne/Bonn (CGN)
|
||||
and Düsseldorf (DUS) together. Funen includes Billund (BLL) and Copenhagen (CPH). Bologna includes BLQ, Verona
|
||||
(VRN) and Venice Marco Polo (VCE). You can also enter comma-separated IATA codes such as
|
||||
`MMX,CPH`. Each route/date uses one combined Google search, so Bristol's direct
|
||||
flights to CPH are considered without doubling the number of requests. Results
|
||||
show the actual airport for each flight leg.
|
||||
|
||||
Otherwise the page matches cities in `airports.yaml`, then suggests the nearest
|
||||
known airport within 150 km of the venue, limited to the same country when known.
|
||||
The flight lookup form accepts IATA codes, cities and airport names, with local
|
||||
autocomplete from the bundled public-domain OurAirports index
|
||||
(`agenda/airport_index.json`, downloaded from https://ourairports.com/data/).
|
||||
Names from `PERSONAL_DATA/airports.yaml` (normally
|
||||
`~/src/personal-data/airports.yaml`) override bundled labels for individual airports,
|
||||
combined destinations and autocomplete. Missing or blank names fall back to the
|
||||
bundled index, and edits take effect on the next request. Both personal and
|
||||
bundled names remain searchable.
|
||||
Scheduled-service airports are preferred; three-letter inputs are treated as
|
||||
IATA codes. London resolves to LON (all London airports). The highest-ranked match is
|
||||
used and its name/code shown; pick an autocomplete result for a specific airport.
|
||||
Country suffixes narrow the matches; US state abbreviations such as CA are
|
||||
recognized too. For example,
|
||||
`Copenhagen, Denmark` resolves to CPH. A nearby airport may
|
||||
still require a substantial ground transfer. Unknown locations need an IATA code
|
||||
or city/airport name entered manually.
|
||||
|
||||
“Open in Google Flights” opens a prefilled round trip in a new tab, with the
|
||||
current airport group, outbound/return dates, stops, one adult in economy and
|
||||
GBP/en-GB settings. Before a lookup it uses the initial Bristol or London dates;
|
||||
for cached results each origin has its own link using the displayed flights'
|
||||
departure dates (or the last checked dates when no flights were found). Links
|
||||
are generated locally and work during the shared server cooldown.
|
||||
|
||||
Only the flight lookup button contacts Google Flights from the server. Searches use Python
|
||||
Playwright with installed Chromium, one adult in economy, language
|
||||
`en-GB`, country `GB`
|
||||
and currency `GBP`. Overseas European destinations within 3,500 km of London
|
||||
prefer direct Bristol flights. Start with the day before and the day after the
|
||||
conference, expanding each direction independently to at most four days away
|
||||
only when no suitable flights are found. Stop searching a direction at the first
|
||||
date with flights; when both nearest dates work, this needs just two searches. If either direction has no suitable results, London is
|
||||
searched too. Other destinations start with London. London means LHR, LGW, STN,
|
||||
LTN, LCY and SEN together, initially with at most one stop; departures are two days
|
||||
before the conference to allow overnight travel, trying three days before if
|
||||
no suitable outbound flights are found. Returns are the day after. Outbound
|
||||
flights must arrive before the opening date. Prices are **one-way fares**, shown
|
||||
separately for outbound and return travel. Searches use Google’s best-flight group first,
|
||||
then prefer direct flights and BA within each stop count (all BA legs first, then
|
||||
some BA legs, then other airlines). Operating airline is used when available,
|
||||
otherwise the advertised airline. Google’s order is preserved within each group.
|
||||
If a London search has no direct or one-stop results, it retries with at most two
|
||||
stops. Bristol always remains direct-only.
|
||||
|
||||
Results live in `DATA_DIR/conference-flights` (normally
|
||||
`/home/edward/lib/data/conference-flights`), keyed by dates, destination and search
|
||||
policy. Updates reuse results for six hours. Empty results are also cached;
|
||||
errors have a 15-minute retry cooldown and preserve any previous successful
|
||||
results. Failures show the route/date and underlying exception type/message;
|
||||
full tracebacks are logged on the server. Explicit Google date-range rejections
|
||||
ask you to retry closer to departure, rather than after 15 minutes. Confirmed
|
||||
“no flights found” and “no non-stop flights found” pages count as empty results;
|
||||
Bristol searches can then continue to nearby dates and show “No direct flights
|
||||
found” with the checked dates if none work; other unusable responses remain
|
||||
errors and include the requested date. No fixed airline booking horizon is assumed. Legacy cached failures without details
|
||||
can be retried immediately. A shared process lock prevents simultaneous conference flight searches.
|
||||
The page shows when results were last fetched, the destination airport names,
|
||||
and the departure dates checked when no suitable flights were found. Belfast is an exception to the UK exclusion: it searches Belfast International
|
||||
(BFS) and Belfast City (BHD) together, starting with direct flights from Bristol.
|
||||
Conferences with tentative start/end dates can also search for flights; the flight
|
||||
section displays those provisional dates. Other domestic, past, online and
|
||||
approximate-date conferences display details but do not offer flight lookup.
|
||||
|
||||
|
||||
Individual route/date searches (including empty results) are cached for six hours
|
||||
under `conference-flights/days`, so overlapping conferences reuse the same Google
|
||||
lookup. Browser searches are spaced at least three seconds apart using shared disk state
|
||||
across Apache workers. Chromium loads the page’s assets normally. HTTP 429 is not immediately
|
||||
retried: it pauses all uncached Google flight requests for 30 minutes. The page
|
||||
shows the shared cooldown and continues to display cached results. Failed navigations are not automatically retried.
|
||||
|
||||
|
||||
Search URL encoding and result parsing follow the format documented by
|
||||
https://github.com/AWeirdDev/flights (MIT attribution in
|
||||
`licenses/fast-flights-MIT.txt`). `flights` is no longer an application dependency. `run.fcgi` uses `/usr/bin/python3`; a project virtualenv is no longer
|
||||
needed. System Python must provide Playwright and the existing Flask dependencies,
|
||||
and Chromium must be installed. On Debian we launch
|
||||
`/usr/lib/chromium/chromium` directly: Apache's `ProcSubset=pid` hides
|
||||
`/proc/cpuinfo`, which causes `/usr/bin/chromium`'s shell-wrapper CPU check to
|
||||
fail incorrectly. Other installations fall back to `chromium` on PATH. The browser is started only on a cache miss,
|
||||
reused for that lookup’s dates, and closed after the lookup. Its persistent profile
|
||||
in `conference-flights/browser-profile` preserves consent choices. The shared
|
||||
search lock prevents multiple workers opening that profile simultaneously.
|
||||
|
||||
Google’s result format is undocumented, and using a browser does not guarantee
|
||||
avoidance of rate limits or verification challenges. Existing successful cached
|
||||
fares remain usable; old empty results and failures can be refreshed using the
|
||||
browser, subject to the existing shared cooldown.
|
||||
|
||||
Failed browser searches save compressed JSON diagnostics in
|
||||
`DATA_DIR/conference-flights/errors/*.json.gz`. Each contains the UTC timestamp,
|
||||
route, requested date, stop limit, locale/currency, request and final page URLs,
|
||||
HTTP status, exception details, raw flight-data script, visible page text, page
|
||||
HTML, and up to five Google HTTP error response bodies. Capture uses the loaded
|
||||
page and received responses; it makes no extra Google requests. Capture failures
|
||||
are logged and cannot replace the original lookup error. Local cooldown refusals
|
||||
and successful searches do not create snapshots.
|
||||
|
||||
Snapshots are retained for 30 days, with at most 100 files; cleanup runs when a
|
||||
new snapshot is written. Captures are capped at 2 MiB of HTML, 1 MiB of flight
|
||||
script, 64 KiB of visible text and 256 KiB per HTTP error body, with any truncation
|
||||
recorded. Snapshot files have mode 0640. Previous failures cannot be recovered
|
||||
from these snapshots; recording begins with this change. Use Python's `gzip.open`
|
||||
to read the JSON for analysis or replay the saved `flight_script` through
|
||||
`agenda.google_flights.parse_results`.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue