Skip to content

Screenshot to App

Drop a screenshot of any UI into the chat and the AI rebuilds it as a native Python app. This is the thing most other AI coding tools cannot do, and the single biggest reason to use Whittl over a browser-based alternative.

What you can drop in

  • Your own app screenshots. Tired of a command-line tool? Drop a picture of the idea, get a GUI.
  • Reference UIs. A login screen, a settings panel, a notification view. The AI takes the layout and the feel, not a pixel-perfect copy.
  • Design mock-ups and sketches. Export a PNG, drop it in, iterate in the chat.
  • Error screenshots. The AI reads the error text and debugs the code.
  • Photos of paper sketches. Low fidelity works; the AI picks up where the controls go.

Privacy by default

A picture goes only to the backend you picked (Anthropic, Google, OpenRouter, DeepSeek, or your own machine with Ollama). There is no Whittl server in between and no telemetry.

Attaching pictures

  • Drag and drop image files onto the message box. Drop several at once and each one is attached.
  • Paste with Ctrl+V: a screenshot you just took (Windows: Win+Shift+S), a picture copied from a browser, or image files copied in your file manager.

Each picture appears above the message box as a thumbnail. Hover over it for the file's name and size, click it to see it full size, and use its Remove button to take it off. A message can carry up to ten pictures; drop more and the chat says how many were left out.

Then send a prompt like:

  • "Rebuild this as a PySide6 desktop app"
  • "Make me an app that looks like this"
  • "These are the three screens of my app: match their colours across the whole project"

Your message appears in the chat with its pictures under it, side by side when there are several. Click any of them to see it full size. They are kept with the project, so they are still there after a restart.

Which models read images

Direct backends

Backend Models Reads images
Claude Haiku 4.5, Sonnet 5, Sonnet 5.5, Opus 5, Opus 5.5, Fable 5, Fable 5.1 Yes, every tier
Gemini Flash 3, Flash 2.5, Pro 3.1 Yes
DeepSeek V4 Flash, V4 Pro, V4 Flash Vision Only the Vision (images) tier, which DeepSeek still marks experimental
Ollama (local) Whatever you have pulled When the model can: Whittl asks Ollama which models read images (gemma4, llava, moondream and others say so)

OpenRouter

Any model in OpenRouter's catalog that takes images works. The Models dialog (the button beside the backend dropdown) marks them with a [Vision] chip; filter it to Vision to see the current list.

When your model cannot read images

You do not have to switch models. When the model you are using does not read images, Whittl has a model that does describe each picture first, and sends the description with your message; with several pictures each gets its own numbered description. The chat says when this happens.

Which model describes them is set in Preferences > Vision (Image Input). Auto uses the first that works among the providers you have set up; or pick one yourself, from a free one up to Claude Sonnet 5 (the list shows each one's price per million tokens).

On the Ollama backend your pictures stay on your machine: only an installed local model that reads images may describe them, never a cloud model, whatever keys you have saved. If none is installed, the pictures are not sent, and the chat says so and suggests pulling one (llava, for example).

What to expect from the output

Vision-model code generation is good but not magic.

What works well

  • Overall layout. Sidebar and content, toolbar and content, a grid of tiles, dialogs.
  • Control types. Buttons, dropdowns, checkboxes, tab bars, tree views.
  • Text. Headings, labels, placeholder text and button labels, usually transcribed correctly.
  • The palette. Not exact hex codes, but the overall feel: dark or light, warm or cool, the accent colour.

What does not work as well

  • Pixel-perfect spacing. Padding, margins and alignment need a follow-up or two: "make the left margin wider".
  • Icons. The AI sees an icon's shape but cannot name the file. Ask for real icons in a follow-up ("use the Phosphor icons, grab them online" with Agent Mode on), or put your own in assets/.
  • Unusual controls. A custom-drawn progress ring or a one-off layout comes back as a close, ordinary approximation.
  • Whole flows. A picture is one screen. Several pictures in one message help, but a multi-screen app is still best built a screen or two at a time.

A workflow that works

  1. First prompt: the overall shape. "Rebuild this as a PySide6 desktop app with the same layout." Do not over-specify.
  2. Run it and compare. The gap between the picture and the running app is the next prompt.
  3. Fix the structure. "The sidebar needs to be wider and the content should scroll."
  4. Polish. "Make the buttons the tan from the second screenshot."
  5. Add assets. Put logos or icons in assets/ and say "use my logo at assets/logo.png in the header."

Troubleshooting

The AI says it cannot see a screenshot

Check that the thumbnail appeared above the message box before you sent. If the chat said the picture was described or not sent, that explains it: your model does not read images (see above). On OpenRouter, choose a model with the [Vision] chip.

The app looks nothing like the screenshot

Likely causes, roughly in order:

  1. The model saw a description, not the picture. A description loses detail; use a model that reads images for close copies.
  2. The prompt was vague. "Make an app like this" leaves room to improvise. Try "Rebuild this exact layout as a PySide6 app."
  3. The picture was small. Native-resolution screenshots work better than thumbnails; tiny text cannot be read.

What's next