Run with llama.cpp

This page shows how to serve the recommended models with llama.cpp. To use OpenVINO instead, see Run with OpenVINO.

Install llama.cpp so that llama-server is available in your terminal, then start the server for your chosen model. Adapt the model paths and settings to your environment.

Start the server

Qwen3 Coder 30B A3B Instruct

llama-server `
    -m "Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf" `
    --alias "Qwen3-Coder-30B-A3B" `
    --host 127.0.0.1 `
    --port 8081 `
    -c 90000 `
    --log-file "Qwen3-Coder-30B-A3B_llama_log.txt" `
    -ngl 100 `
    -e `
    -s 0 `
    -mg 0

Qwen3.6 35B A3B

llama-server `
    --hf-repo "unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_M" `
    --alias "Qwen3.6-35B-A3B" `
    --reasoning-budget 0 `
    --host 127.0.0.1 `
    --port 8081 `
    -c 90000 `
    --log-file "Qwen3.6-35B-A3B_llama_log.txt" `
    -ngl 100 `
    -e `
    -s 0 `
    -mg 0

Both examples serve the model at http://127.0.0.1:8081/. You will enter this address in Artemis Desktop when you connect the model.

Keep the server running

The model server must keep running while you use Artemis Desktop. Closing its terminal window stops it.

Adjust the settings

The examples use a context length of 90,000 tokens (-c) and offload all model layers to the GPU (-ngl). If the model does not fit in your GPU or system memory, reduce these values. See Troubleshooting.

Parameter reference

Parameter	Description
-m	Path to a local GGUF model file.
--hf-repo	Hugging Face repository and quantisation to load.
--alias	Model identifier exposed by the server.
--host	Network interface on which the server listens.
--port	Port used by the API server.
-c	Context length.
--log-file	File used to record server logs.
-ngl	Number of model layers to offload to the GPU.
-e	Enables escape sequence handling.
-s	Sets the random seed.
-mg	Configures the main GPU for model placement.
--reasoning-budget	Configures the reasoning token budget where supported by the build.

Next step

Connect the model to Artemis Desktop.