tetrak_authority.snapshot

Build a dated snapshot: the subset, the fetch, the files.

A snapshot is a directory holding snapshot.json, the manifest (when it was made, from which endpoint, with which queries, how many of each kind, how many requests), and entities.jsonl, one Entity per line. The manifest is what a measurement cites, beside the library version: the same queries on another day give another snapshot.

The subset is a Scope:

  • People: items that are human (P31 Q5) with a label or alias in the scope’s languages, or with one of the scope’s ethnicities (P172) or citizenships (P27), so that royalty and saints with a label but no modern tag come in through the label and a diaspora figure with no Armenian label comes in through the tag.

  • Organisations: items of the scope’s organisation classes with a label in the languages.

  • Places: items with coordinates (P625) inside the scope’s box and a label or alias in the languages. The default box is an approximation of historic Armenia and the lands around it, where a place is likeliest to carry names in several languages: the far west of Turkey to Tehran, Jerusalem to the northern edge of Georgia.

Enumeration needs an endpoint that can filter by label language over the whole of Wikidata, which the Query Service cannot within its limit and the QLever mirror can; the fetch of names and facts, in batches of item ids, works on either.

Module Attributes

ORGANISATION_CLASSES

organisation and the kinds a catalogue meets as a creator or a subject.

FORMAT

2 stores each name's fold key beside it.

FACT_PROPERTIES

property → the name it is kept under.

HISTORIC_ARMENIA

the far west of modern Turkey to Tehran, Jerusalem to the northern edge of Georgia.

ARMENIAN

Armenian and Western Armenian labels, Armenians, Armenia, and the historic region.

Functions

assemble(id, kind, names, facts[, script])

One record from a batch's rows.

batches(items, size)

Consecutive slices of items, each at most size long.

build(client, out[, scope, kinds, batch, ...])

Enumerate, fetch and write a snapshot; return its manifest.

count(client[, scope, kinds])

How many items each kind of the scope holds, one query per kind.

enumerate_ids(client, scope, kind[, limit])

The item ids of one kind, in a stable order.

facts_query(ids)

The facts in FACT_PROPERTIES and the English description of the given items.

fetch(client, ids, kind[, script])

Fetch names and facts for one batch of ids and assemble their records.

fetch_resilient(client, ids, kind[, script, ...])

Fetch a batch, halving it when the endpoint refuses it, and skipping an item it will not answer.

names_query(ids)

Every label and alias, in every language, of the given items.

point(value)

(latitude, longitude) from a WKT point (Point(44.5 40.18)), or None.

rekey(path[, script, log])

Recompute every key in a snapshot with the installed tetrak-translit, in place.

scope_json(scope)

The scope as the manifest records it.

year(value)

The year of a Wikidata time value (1869-02-19T00:00:00Z → 1869, -0500-… → -500).

Classes

Box(west, south, east, north)

A bounding box in degrees: west and east longitudes, south and north latitudes.

Scope([script, languages, ethnicities, ...])

What a snapshot covers.