Flâneur is a read-only atlas for Curius, a place where people save links.
The whole thing starts with one edge:
`text person -> saved -> link `
If you crawl enough of those edges, you get two useful indices. One is a people index: who tends to save the same things? The other is a content index: which pages mean similar things?
The crawler keeps a queue of public profiles, fetches many at once, extracts saved links, normalizes URLs, and writes the edges. It also keeps follow edges when they are visible. Everything else in Flâneur is built from those tables.
The Save Graph
The people index is just a self-join over saves.
`text if two people saved the same link, connect them if they saved many of the same links, make the edge stronger `
This answers a different question than followers. It says: whose libraries overlap? If you ask who has the most "friends" here, friend just means "shared a bunch of saved links," not a declared social relationship.
This is useful before any ML. A link can be important because many different people saved it. A person can be central because their saves overlap with many other people. A recommendation can start from this graph instead of from global popularity.
The Semantic Map
The content index starts from the links.
For each link, Flâneur fetches the page, extracts readable text, embeds it, runs UMAP, and draws it with deck.gl. Each dot is one link. Nearby dots should be nearby in meaning.
The important detail is chunking. Long pages are split into passages, each passage is embedded, and the passage vectors are pooled back into one vector for the link. The UI still gets one dot per link, but the vector is based on the whole page, not just the first part that fit in the model.
The Runtime
The map only feels good if search and neighbors are fast.
The first version rebuilt too much on every request: load vectors from SQLite, parse JSON, normalize, then compare. The comparison was cheap. Preparing the vectors was slow.
Now the offline job writes one Float32 matrix of link vectors. The worker loads it once and scans it in memory. Clicking a dot is vector against matrix. Searching is query vector against matrix. Repeated query vectors are cached.
At this scale, brute force is simpler and fast enough. No ANN index, no vector database, no per-request JSON parsing. The live path is one static object and a loop.
Flâneur
The pipeline is:
`text profiles -> saves/follows saves -> co-save graph links -> text -> chunks -> pooled vectors vectors -> UMAP map vectors -> static matrix -> search + neighbors `
That is the basic idea: crawl once, build better offline artifacts, keep the interactive path small.