Wine scores from tasting notes#

wine.py predicts a wine critic's score (80–100) from the tasting note with a Venn regression family. It follows the data and split of TypeSafe's Autoresearch feature discovery cookbook: 2,000 winemag reviews, the first 1,200 after a seed-0 shuffle to fit and the other 800 held out.

pip install -e ".[embedding]"
venn download-embedder
python examples/wine/wine.py                    # the API's default reader, on CPU
python examples/wine/wine.py --device cuda:0    # the same reader on a GPU, bf16
python examples/wine/wine.py --form kernel_ridge  # SentenceSimilarity + length inputs
python examples/wine/wine.py --form kernel_ridge --device cuda:0

The first run downloads the Venn decision head automatically. Set VENN_HOME to choose where the embedding cache and fitted family are saved.

The script downloads the pinned CSV from Hugging Face (GroNLP/ik-nlp-22_winemag, revision 90eb39f3…, CC BY-SA 4.0, derived from Wine Enthusiast reviews) each time it runs. No notes, vectors or predictions are stored in this repository. It embeds the 2,000 notes with the venn-1 reader, registers a regression family on the 1,200 fitting notes, scores the 800 held-out notes and prints the held-out RMSE.

The default family is a ridge regression on the note's 768-dimensional vector. Its penalty is chosen by 5-fold cross-validation repeated 3 times on the fitting notes, the same protocol the cookbook uses to select its features. A Score question that names the family returns a score and an 80% prediction interval:

service = DecisionService(model_dir=model_dir)
vectors = service.embed(notes, prompt="Classification")
examples = [{"state": {"vector": vector.tolist()}, "points": score}
            for vector, score in zip(vectors[:1200], points[:1200])]
service.register_family("wine_points", examples, objective="regress", target="points",
                        split={"train": list(range(1200))})
service.decide({"state": {"vectors": vectors[1200:1712].tolist()},
                "questions": {"points": {"type": "Score", "family": "wine_points"}}})

Here notes and points are the 2,000 tasting notes and scores, and DecisionService comes from venn.

Results on the 800 held-out notes#

TypeSafe's numbers are copied from the cookbook page. Their five-round pipeline is more accurate than the default ridge baseline. The kernel configuration reaches similar accuracy in one registration call. It uses the reader's SentenceSimilarity prompt and the note's log word and character counts. The script caches that prompt separately, fits only on the first 1,200 notes, and sends the remaining notes as text for prediction. Its JSON API returns both the predicted score and an 80% interval. On 3,000 additional notes, the recorded kernel head scored 1.803 RMSE versus 1.794 for the 38-question pipeline. See Venn and Jev and the result record.

Recorded ridge timings#

The CPU column is the API's default reader in float32 on an AMD EPYC 7V12 host with 2 sockets of 48 cores. A batch of 32 or more texts raises PyTorch to 24 threads, or to VENN_EMBED_THREADS if set, and restores the one-thread pin used for serving afterwards. On this host one thread took 722 s to embed the notes and all 96 took 1,017 s, because threads spanning both sockets stall each other.

The table records the earlier ridge-only script. The current script prints download, embedding, registration and prediction timings for the selected configuration, plus total elapsed time including model startup. Kernel fitting does more work than ridge. Repeat runs reuse the embedding cache.

View this page on GitHub