1. Build MemVanta
git clone https://github.com/sauravsingla/MemVanta.git
cd MemVanta
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j
ctest --test-dir build --output-on-failureThe default build produces the real-inference executable build/memvanta_real together with benchmark, inspection, tokenizer, and test targets.
2. Run text generation
./build/memvanta_real \
--model /path/to/model.gguf \
--prompt "Hello from MemVanta" \
--n 64 \
--threads 4 \
--ctx 2048 \
--temperature 0Use ./build/memvanta_real --help to print the current CLI syntax. A temperature of 0 selects deterministic greedy generation in the current CLI path.
3. Know which executable you are using
memvanta_real is the trained-model GGUF text-generation CLI. The separate memvanta run <file> command exercises the mapped streaming/cache runtime and reports memory/streaming telemetry; it is not the text-generation command.
Keeping those two paths distinct avoids a common first-run mistake and makes benchmark tooling separate from model generation.
Current model scope
Trained-model execution currently supports GGUF files with general.architecture=llama. The parser also validates pinned Qwen2 GGUF files, but Qwen2 inference is not implemented. Parser compatibility is therefore broader than execution support.
MemVanta remains an active research / engineering prototype. It is designed around memory-constrained CPU inference and does not claim throughput leadership over mature general-purpose runtimes.
Main generation options
| Option | Purpose |
|---|---|
--model | Path to the GGUF model file. |
--prompt | Prompt text. Defaults to Hello. |
--n | Number of tokens to generate. Default: 64. |
--threads | CPU worker count. Defaults to available hardware concurrency. |
--ctx | Context size override. |
--temperature | Sampling temperature. Default: 0. |
Next steps
Once generation works, use the benchmark and reproduction pages to understand MemVanta's measured memory/performance trade-off rather than extrapolating from a single local run.