Skip to content

Performance Tuning

Whittl is intended to feel responsive on modest hardware. If it doesn't, there's usually a specific thing to tune. This page covers the common bottlenecks.

Whittl's own performance

Slow startup

Whittl's startup time depends on:

  • Disk speed. Loading the Python runtime from SSD: ~1-2s. From HDD: 5-10s.
  • Project count. 50+ projects in the sidebar takes longer to render than 5. Workaround: archive old projects (right-click → Archive).
  • First-launch setup. First launch after install runs a one-time "ensure Python is bundled" check that can take 30-60s on Windows.

If steady-state startup is slow (not just first launch), check:

  • ~/.whittl/settings.json file size — should be under 10KB
  • ~/.whittl/projects/ directory count — should be manageable
  • ~/.whittl/crash.log — if multi-megabyte, truncate or delete

Editor feels laggy

Common causes:

  • Very large file (10K+ lines). Qt's QPlainTextEdit gets slow around this size. Split the file into smaller modules.
  • Syntax highlighting running on every keystroke. If the file is both large and has complex syntax (f-strings with embedded expressions, heavy decorators), the tokenizer can't keep up. Pending improvement to v2.4.
  • Autocomplete triggering too often. Turn off via Edit → Preferences → Editor → Autocomplete.

Generation takes forever

This is usually about the backend, not Whittl:

  • Cloud backend. 10-60 seconds is normal. If it's taking minutes, check the provider's status page (Anthropic, Google, OpenRouter, etc.).
  • Ollama. First generation on a fresh session is slow (model loading). Subsequent ones are fast. If even subsequent ones are slow, see Ollama section below.
  • Agent Mode. Multi-round agent tasks can run 1-5 minutes legitimately. Check the round counter in the status bar.

UI freezes during generation

Shouldn't happen in v2.3+. Auto-fix moved to a background thread in v2.2 and the generation pipeline is fully async. If you see a frozen UI:

  1. Check the terminal panel — is Whittl still working?
  2. If genuinely frozen (not "thinking but UI responsive"), force-quit and restart. Report via Reporting Bugs — include what you were doing.

Ollama performance

Ollama's local generation speed varies dramatically by hardware and model. Tuning areas:

Model size vs RAM

The biggest factor. A model that doesn't fit in RAM swaps to disk and becomes unusably slow:

  • 6-10 GB RAM available: Run a 7B model (qwen2.5-coder:7b, deepseek-coder:6.7b)
  • 14-16 GB RAM available: 13B-14B models (qwen2.5-coder:14b)
  • 32 GB+: 30B-34B models
  • Less than 6 GB: skip local models entirely; use a cloud backend

If Ollama says your model loaded but generations are running at 1-2 tokens/sec, it's swapping. Switch to a smaller model.

GPU utilization

Check ollama ps to see if your GPU is being used:

ollama ps

Look at the "PROCESSOR" column. 100% GPU is ideal. 50/50 or 30/70 means the model is partially on CPU (too big for VRAM). 100% CPU means the GPU isn't being detected.

To force GPU usage:

  • NVIDIA: ensure recent CUDA drivers
  • AMD: ROCm support on Linux only; consumer cards are hit-or-miss

Keep-alive

Ollama evicts unused models after 5 minutes of inactivity. First generation after eviction has to reload — 5-30 second delay.

To keep models resident longer:

# Linux
OLLAMA_KEEP_ALIVE=2h ollama serve

# Or set permanently via systemd environment

Costs RAM the whole time, so only do this if you're actively iterating.

Batching and parallelism

Ollama doesn't batch across Whittl sessions — each generation runs sequentially. On a single-GPU setup that's fine. If you have multiple GPUs, you'd need to configure Ollama's parallelism separately; Whittl's integration doesn't expose this.

Generated-app performance

Built .exe is slow to start

Typical for PyInstaller one-file builds: 2-5 seconds of startup is normal. The executable has to extract itself to a temp directory on every launch.

Alternative: Build as a folder (not one-file). In Build dialog → Advanced → uncheck "Single file." You get a folder of files that starts faster but is messier to distribute.

Built .exe is too large

See Building an Installer → Output size. Short version: remove unused imports from your code so PyInstaller stops bundling packages you don't actually need.

Generated app is slow to run

This is about the generated code, not Whittl:

  • Profiling: add cProfile instrumentation or use Python's built-in profiler. Let the AI help ("profile this function and suggest optimizations").
  • UI frame drops: usually because long-running work is blocking the UI thread. Ask Whittl to move heavy work to a QThread or threading.Thread.
  • Database queries slow: add indexes, check for N+1 query patterns. The AI is decent at diagnosing these from symptoms.

Large project feels slow

For projects with 30+ files where smart routing is active:

  • Check the status bar. Should say "Smart routing: X of Y files" when active. If it's always sending all files, check Preferences → AI Generation → Send All Files (should be off).
  • Organize into subfolders. Smart routing is smarter when files are grouped by concern (ui/, core/, etc.) rather than flat.

Network performance

OpenRouter feels slower than direct

OpenRouter adds a small latency tax (~100-200ms) for routing. Usually imperceptible. If it's consistently noticeable, switch to the direct provider (Anthropic, Google, etc.) for that task.

VPN / proxy makes everything slow

Whittl's API calls route through whatever network config you have. If you're on a VPN that's slow or proxies that compress badly, you'll feel it. Try disconnecting the VPN to confirm it's the cause.

Rate-limit backoff stalls

When you hit a provider's rate limit, Whittl backs off with exponential delay (2s, 4s, 8s, 16s). If you're hitting rate limits constantly:

  • Upgrade your tier on the provider (pay for higher rate limits)
  • Switch to a different model or different backend temporarily
  • Space out your requests — iterate slower

Disk usage

If ~/.whittl/ is using a lot of disk:

  • archives/ inside each project — version history snapshots. Prune old ones manually or delete entire archive folders of projects you're done iterating on.
  • bundled_python/ — Python runtime + pip packages. Grows as you pip install things. Reset by uninstalling and reinstalling Whittl.
  • update_cache/ (v2.4+) — downloaded installers pending apply. Clears after installs complete.

What's next