Strix With Local LLMs: My MacBook Test

Strix can now connect to local models through Ollama, which sounds like the answer to the biggest problem in my first hands-on Strix review: cost. That earlier quick scan consumed 5.7 million input tokens, cost roughly $17, and triggered an automatic provider-key suspension. A local model promises zero API fees and keeps assessment data on your own machine.

I tried it on an M5 MacBook Pro with 24 GB of unified memory. Strix launched, Ollama used the GPU, and the deliberately vulnerable target was reachable. I tested models from 7B through 24B parameters. The agent still never made its first useful tool call.

That is not the same as saying local Strix does not work. It means models that fit on consumer hardware can pass a simple tool-use demo and still fail under Strix's real agent prompt. Here is the complete setup, the failed runs, and what I would test next.

The short result

The successful synthetic tests matter, but they were not predictive. Strix 1.0.4 generated a 55,058-character root prompt and exposed 27 tools before asking the model to act. That is a very different workload from asking a model to call one tiny test function.

Why I used an isolated vulnerable target

Strix is an autonomous penetration-testing agent. It does not just inspect headers. It can browse, send crafted requests, run code, and attempt to validate vulnerabilities. Only run it against systems you own or are explicitly authorized to test.

For this experiment I used OWASP Juice Shop, an intentionally vulnerable training application. I published it only on the Mac's loopback interface:

docker pull bkimminich/juice-shop
docker run --rm -d \
  --name strix-juice-shop-lab \
  -p 127.0.0.1:3000:3000 \
  bkimminich/juice-shop

Strix runs its tools inside a Docker sandbox, so the agent target was http://host.docker.internal:3000. I verified that both the host and sandbox received HTTP 200 before starting a scan. No public system was in scope.

Hardware and software

The test machine was an Apple M5 MacBook Pro with 10 CPU cores, 10 GPU cores, and 24 GB unified memory. The software stack was:

• Strix 1.0.4
• Ollama 0.32.14
• Docker Engine 29.4.0
• Strix sandbox image 1.0.0, about 10.9 GB
• Qwen3-VL 8B, a 6.1 GB Ollama download
• Devstral Small 2 24B, a 15 GB Ollama download

The model ran fully on the Apple GPU. At a 32K context, Ollama reported roughly 10 GB resident for Qwen3-VL.

The first failure: tool calls returned as text

Before involving Strix, I sent Qwen2.5-Coder 7B a minimal request with one function definition. The model understood the request, but returned a JSON-looking function call as ordinary assistant text. Ollama's response contained no structured tool_calls field.

That distinction is critical. Strix is tool-driven. A function call printed as prose looks plausible to a human, but the agent runtime cannot execute it. Strix's own local-model guide calls this out as a common failure mode.

Qwen3-VL passed the same synthetic check. It returned a native tool call in 11.47 seconds on a cold request, so I moved it into the real scan.

The second failure: Ollama silently used a 4K context

The first real Qwen3-VL run started normally. Strix created its sandbox and registered 27 tools. Ollama, however, loaded the model with a 4,096-token context.

Strix's rendered root prompt was 55,058 characters before conversation history or tool output. The model responded with reasoning but no tool call. Strix forced another attempt. After five retries, the run had consumed 24.2K input tokens and 1.2K output tokens, made no target request, and reported zero findings.

The fix was a custom Ollama model profile:

FROM qwen3-vl:8b
PARAMETER num_ctx 32768
PARAMETER temperature 0.2

ollama create qwen3-vl-strix -f Modelfile.qwen3-vl-strix

ollama ps then confirmed a 32,768-token context and 100 percent GPU execution.

The third failure: thinking never produced an action

With truncation addressed, Qwen3-VL still did not complete its first Strix turn within about 149 seconds. The likely problem is the thinking path. Ollama's native API accepts think: false, but Strix reaches Ollama through LiteLLM and did not disable thinking in this run.

I tested Gemma 4 as a second family. It also produced a correct native tool call in a small test. I created a 32K profile with a 1,024-token output cap to prevent an unlimited reasoning loop. Under the full Strix prompt, it still did not finish the first turn within about 184 seconds.

These were bounded tests, not endurance benchmarks. I interrupted both runs after it was clear that the first action alone was taking minutes. Neither model reached Juice Shop, so zero findings means no assessment occurred.

Devstral Small 2 fit, but still did not run Strix

Devstral Small 2 was the most serious local candidate in this Mac test. It is a 24B agentic model, and its Ollama package was 15 GB. I began at 16K context to preserve memory headroom:

FROM devstral-small-2:latest
PARAMETER num_ctx 16384
PARAMETER num_predict 1024
PARAMETER temperature 0.2

The model fit, but only just. Ollama reported 17 GB resident with 92 percent GPU and 8 percent CPU execution. A synthetic native tool call completed in 17.0 seconds on a cold request, including 10.7 seconds to load the model.

Then I reproduced a current Ollama parser defect. A tool whose argument was named query worked. Changing only the argument name to the lowercase word name caused Ollama to return empty content and no tool call. Strix's base create_agent tool uses a required name argument, so the defect directly affects its normal delegation path.

The first real Devstral Strix run did not complete its opening turn in 6 minutes 26 seconds. I then added an instruction prohibiting child-agent creation and requiring direct browser, shell, or Python tools. The warm-model retry still produced no completed turn in about six minutes. Juice Shop received no agent request in either run.

This separates two problems. The name collision can break at least one important Strix tool, but avoiding that tool did not make the full agent prompt practical on this machine.

A config-file trap worth knowing

I also found an operational security issue in my old setup. My existing Strix config still contained a provider key in plaintext. When I supplied a different config path for the local run, Strix 1.0.4 rewrote that file and copied the stale key into the workspace.

I deleted the generated file and switched to process-scoped local settings with dummy credentials:

env \
  STRIX_LLM=ollama/qwen3-vl-strix \
  LLM_API_BASE=http://localhost:11434 \
  OLLAMA_API_BASE=http://localhost:11434 \
  LLM_API_KEY=ollama \
  OPENAI_API_KEY=ollama \
  strix \
    --target http://host.docker.internal:3000 \
    --instruction-file ./instructions.md \
    --non-interactive \
    --scan-mode quick

If you previously used Strix with a cloud provider, inspect ~/.strix/cli-config.json before publishing logs or copying configuration. Revoke old keys rather than assuming a local model means no secret can leak.

MacBook or RTX 4060: which is the better Strix host?

For model capacity, the 24 GB Apple Silicon Mac is the better machine in this specific comparison. An RTX 4060 desktop card typically has 8 GB of VRAM, although the exact Windows machine must be checked with nvidia-smi. Eight gigabytes is comfortable for many 7B or 8B quantized models, but those are exactly the sizes Strix warns may struggle with multi-step tool use.

The NVIDIA machine may generate tokens faster for a model that fits entirely in VRAM. It does not automatically solve the capability problem. The Mac's unified memory did run Devstral Small 2, but the 16K profile occupied 17 GB, used partial CPU offload, and still failed to complete Strix's first turn.

What I would test next

The Mac phase stops here. A 32K Devstral run would increase memory pressure while retaining the same Ollama parser defect, and the 16K run was already unusably slow.

The next phase will be a Windows PC comparison added to this article as a dated update. Before running it, I will record the exact GPU, VRAM, system RAM, driver, and Windows Subsystem for Linux version. Then I will repeat the same synthetic tool schemas and localhost-only Juice Shop workflow. Keeping those gates identical matters more than forcing a successful result.

Is local Strix worth trying?

Yes, if privacy and experimentation matter more than immediate results. The software ran, the local model cost was zero, and no assessment data went to a model provider. But "supports Ollama" does not mean every Ollama model can operate Strix.

My result was simple: local models from 7B to 24B passed some isolated function-call tests and failed the real agent workload. Context size, native tool-call formatting, parser compatibility, memory pressure, and first-turn latency were the practical blockers. Consumer hardware turns this into a systems experiment rather than a free drop-in replacement for a frontier model.

That is still a useful outcome. The honest benchmark is not whether the terminal starts. It is whether the model completes a valid first tool call, reaches the authorized target, and proves a finding.

Frequently asked questions

Can Strix run with Ollama?

Yes. Strix supports an Ollama endpoint, and all three serious candidates in this test were served locally through Ollama. Compatibility at the connection layer does not guarantee that a particular model can complete Strix's full agent workflow.

Which local model worked best with Strix?

Devstral Small 2 was the strongest candidate that fit on this 24 GB MacBook, but it still did not finish the first real Strix turn. Qwen3-VL and Gemma 4 passed small tool-call checks but also stalled under the full agent prompt.

How much memory does Devstral Small 2 need?

In this test, the 16K-context Ollama profile occupied about 17 GB and ran with 92 percent GPU and 8 percent CPU execution. That leaves little headroom on a 24 GB unified-memory Mac once Strix, Docker, and the operating system are included.

Will an RTX 4060 run Strix better than a MacBook?

That remains untested here. An RTX 4060 may generate faster when the complete model fits in VRAM, but an 8 GB card will usually require a smaller or more aggressively quantized model than a 24 GB unified-memory Mac. I will add the controlled Windows result to this page.

Does a local Strix run cost nothing?

It avoids per-token model API fees, but it still uses your electricity, storage, compute time, and hardware. The Strix software may report zero model cost while a slow or failed run still consumes substantial local resources.

Can I run this test against any website?

No. Only test systems you own or have explicit authorization to assess. This experiment used OWASP Juice Shop bound to localhost, with Strix reaching it through the Docker host bridge.