Getting started#

Get a first CPU result without downloading a reader or head, then choose a workflow for real vectors or text. The first example uses authored coordinates; it demonstrates the interface, not semantic accuracy.

Install#

Venn requires Python 3.12. CPU workflows run on Linux and Windows, including ARM64. On Linux, clone the repository and enter its root directory:

git clone https://github.com/markrussinovich/venn-vdm.git venn
cd venn
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install torch --index-url https://download.pytorch.org/whl/cpu   # CPU-only PyTorch
python -m pip install .
python -m pip check

If you already have a checkout, start with the virtual-environment command from its root directory.

Skip the pip install torch line if you want CUDA PyTorch; the default wheel on Linux includes the CUDA libraries and is several times larger. Neither install downloads a model. Contributors use python -m pip install -e '.[dev]'.

ARM64 and Windows#

Use the dedicated ARM64 and Windows setup for native Python and matching CPU PyTorch wheels. The installation guide also lists optional extras and download requirements.

Your first result#

Run the first example:

venn quickstart

The JSON response names account as the answer. Its options entries show mean outcomes of 0.9 for account and 0.4 for billing, three cases each, and the IDs of the supporting cases. Account wins because its nearby cases had better recorded outcomes. Your application supplies what “good” means.

This example uses hand-written vectors to demonstrate the feedback-and-lookup mechanism without downloading a model. To decide from actual ticket text, continue to Serve text questions.

Optional: measure a synthetic vector sweep#

venn sweep --rows 1000 --batch-size 256 generates normalized synthetic 768-dimensional vectors, scores four centered options for every row and consumes all output. It reports rows, per-option counts that sum to rows, and timings for generation and scoring. Use it as a CPU throughput probe; it loads no model. Rows go up to one million, using about 1.5 KB of temporary disk per row.

Ways to run Venn#

Decide from an outcome store#

OutcomeService chooses among options using the outcomes recorded in your own vector store. It needs no trained head, and for already-tagged vectors it needs no reader or GPU:

python examples/outcome_store.py

It prints support. The script creates a FAISS store, records one resolved case, and asks which queue a new ticket should go to. The outcome API guide explains tags, adapters for other stores, image batches and how your application closes the feedback loop.

Serve text questions#

Text questions need the reader and the venn-1 head. The reader is EmbeddingGemma 2, an Apache-2.0 model. The download contains about 1.49 GB of multimodal weights. Text serving loads only the approximately 270M-parameter text components: about 1.08 GB of weights in the default CPU float32 mode. See model footprint for sizes and precision.

python -m pip install '.[embedding]'
venn download-embedder
venn serve --host 127.0.0.1 --port 8000

venn serve downloads the head from markrussinovich/venn-1 at the pinned revision on first use. To use a local copy instead, pass --model-dir or set VENN_MODEL_DIR to a directory holding config.json, manifest.json and universal.safetensors. Server state (the embedding cache and registered families) lives under VENN_HOME, which defaults to ~/.cache/venn. venn serve starts loading the reader in the background; pass --no-preload to skip that on low-memory or vector-only servers. See server options.

In another terminal:

python examples/route_ticket.py --text "I cannot sign in"

The response contains answers.team.choice, the probabilities and Venn's abstention metadata. The first text request loads the reader and can take several seconds; GET /health reports reader_loaded once it is ready. Add the header X-Venn-Return-Vector: 1 if you want the 768-dimensional state vector back in the response. Including a criterion with the value null, as the example does with no_match, gives the model an explicit way to say none of the above. Vector-only requests do not need the reader, but every vector must come from the exact venn-1 profile.

Reproduce the measurements#

The benchmark, the semantic boundary suite and the authored demos embed text with the pinned reader and currently require at least one CUDA GPU. Install .[embedding,benchmark,dev] in a clean environment, run venn download-embedder, then follow the reproduction guide. The reference run used four A100 GPUs; requirements for smaller hardware have not been measured.

Troubleshooting#

The server has no authentication and binds to loopback by default. Keep it local.

Continue with a workflow#

Read concepts to understand readers, profiles, stores and families. Choose a runnable path in the example catalog, follow a task recipe, or look up request fields in the API reference.

View this page on GitHub