Skip to content

Performance Tuning

Whittl is intended to feel responsive on modest hardware. If it doesn't, there's usually a specific thing to tune. This page covers the common bottlenecks.

Whittl's own performance

Slow startup

Whittl's startup time depends on:

  • Disk speed. Loading the Python runtime from SSD: ~1-2s. From HDD: 5-10s.
  • Project count. 50+ projects in the sidebar takes longer to render than 5. Workaround: archive old projects (right-click → Archive).
  • First-launch setup. First launch after install runs a one-time "ensure Python is bundled" check that can take 30-60s on Windows.

If steady-state startup is slow (not just first launch), check:

  • ~/.whittl/settings.json file size — should be under 10KB
  • ~/.whittl/projects/ directory count — should be manageable
  • ~/.whittl/crash.log — if multi-megabyte, truncate or delete

Editor feels laggy

Common causes:

  • Very large file (10K+ lines). Qt's QPlainTextEdit gets slow around this size. Split the file into smaller modules.
  • Syntax highlighting running on every keystroke. If the file is both large and has complex syntax (f-strings with embedded expressions, heavy decorators), the tokenizer can't keep up. Since v2.5.0 files past roughly 60,000 characters defer their lint and fold passes briefly after opening, which keeps the switch responsive, but highlighting itself is still per-keystroke.
  • Autocomplete triggering too often. Completions fire on . and on identifier characters, and there is no toggle for them. If they are the problem on a huge file, split the file.

Generation takes forever

This is usually about the backend, not Whittl:

  • Cloud backend. 10-60 seconds is normal. If it's taking minutes, check the provider's status page (Anthropic, Google, OpenRouter, etc.).
  • Ollama. First generation on a fresh session is slow (model loading). Subsequent ones are fast. If even subsequent ones are slow, see Ollama section below.
  • Agent Mode. Multi-round agent tasks can run 1-5 minutes legitimately. Check the round counter in the status bar.

UI freezes during generation

Shouldn't happen. Auto-fix runs on a background thread and the generation pipeline is fully async. If you see a frozen UI:

  1. Check the terminal panel — is Whittl still working?
  2. If genuinely frozen (not "thinking but UI responsive"), force-quit and restart. Report via Reporting Bugs — include what you were doing.

Ollama performance

Ollama's local generation speed varies dramatically by hardware and model. Tuning areas:

Model size vs RAM

The biggest factor. A model that doesn't fit in RAM swaps to disk and becomes unusably slow:

  • 6-10 GB RAM available: Run a 7B model (qwen2.5-coder:7b, deepseek-coder:6.7b)
  • 14-16 GB RAM available: 13B-14B models (qwen2.5-coder:14b)
  • 32 GB+: 30B-34B models
  • Less than 6 GB: skip local models entirely; use a cloud backend

If Ollama says your model loaded but generations are running at 1-2 tokens/sec, it's swapping. Switch to a smaller model.

GPU utilization

Check ollama ps to see if your GPU is being used:

ollama ps

Look at the "PROCESSOR" column. 100% GPU is ideal. 50/50 or 30/70 means the model is partially on CPU (too big for VRAM). 100% CPU means the GPU isn't being detected.

To force GPU usage:

  • NVIDIA: ensure recent CUDA drivers
  • AMD: ROCm support on Linux only; consumer cards are hit-or-miss

Keep-alive

Ollama evicts unused models after 5 minutes of inactivity. First generation after eviction has to reload — 5-30 second delay.

To keep models resident longer:

# Linux
OLLAMA_KEEP_ALIVE=2h ollama serve

# Or set permanently via systemd environment

Costs RAM the whole time, so only do this if you're actively iterating.

Batching and parallelism

Ollama doesn't batch across Whittl sessions — each generation runs sequentially. On a single-GPU setup that's fine. If you have multiple GPUs, you'd need to configure Ollama's parallelism separately; Whittl's integration doesn't expose this.

Generated-app performance

Built .exe is slow to start

Typical for PyInstaller one-file builds: 2-5 seconds of startup is normal. The executable has to extract itself to a temp directory on every launch.

Alternative: Build as a folder (not one-file). In Build dialog → Advanced → uncheck "Single file." You get a folder of files that starts faster but is messier to distribute.

Built .exe is too large

See Building an Installer → Output size. Short version: remove unused imports from your code so PyInstaller stops bundling packages you don't actually need.

Generated app is slow to run

This is about the generated code, not Whittl:

  • Profiling: add cProfile instrumentation or use Python's built-in profiler. Let the AI help ("profile this function and suggest optimizations").
  • UI frame drops: usually because long-running work is blocking the UI thread. Ask Whittl to move heavy work to a QThread or threading.Thread.
  • Database queries slow: add indexes, check for N+1 query patterns. The AI is decent at diagnosing these from symptoms.

Large project feels slow

For projects with 30+ files where smart routing is active:

  • Check the status bar. It should name the files sent and the total when routing is active. If it always sends everything, the project may be under the 8-file threshold; Preferences → File Context → Smart Routing for All Projects lowers it to 2.
  • The first Test Run of a session is the slow one. Whittl's pre-launch checks are cached per file after that, in memory and on disk, so later runs on an unchanged project are near-instant. A 67-file project measured 16 seconds on the first Play and under a second after.
  • Very large files (hundreds of thousands of characters) defer their lint and fold passes for a moment after opening so the switch stays responsive.
  • Keep build output out of the project folder. PyInstaller's dist/ and build/ are ignored as source since v2.5.0, but a project folder full of them still slows every disk walk.
  • Organize into subfolders. Smart routing is smarter when files are grouped by concern (ui/, core/, etc.) rather than flat.

Network performance

OpenRouter feels slower than direct

OpenRouter adds a small latency tax (~100-200ms) for routing. Usually imperceptible. If it's consistently noticeable, switch to the direct provider (Anthropic, Google, etc.) for that task.

VPN / proxy makes everything slow

Whittl's API calls route through whatever network config you have. If you're on a VPN that's slow or proxies that compress badly, you'll feel it. Try disconnecting the VPN to confirm it's the cause.

Rate-limit backoff stalls

When you hit a provider's rate limit, Whittl backs off with exponential delay (2s, 4s, 8s, 16s). If you're hitting rate limits constantly:

  • Upgrade your tier on the provider (pay for higher rate limits)
  • Switch to a different model or different backend temporarily
  • Space out your requests — iterate slower

Disk usage

If ~/.whittl/ is using a lot of disk:

  • archives/ inside each project — version history snapshots. Prune old ones manually or delete entire archive folders of projects you're done iterating on.
  • bundled_python/ — Python runtime + pip packages. Grows as you pip install things. Reset by uninstalling and reinstalling Whittl.
  • update_cache/ (v2.4+) — downloaded installers pending apply. Clears after installs complete.

What's next