Auto Caption — overview

Open 🤖 Auto-Caption, pick a Processing Mode and press ▶ Start Batch Process. There are four modes: 🏷️ Tagger (ONNX), 🖥️ Natural Language (Offline VLMs), 🦙 Ollama and 🔥 PyTorch / ONNX (local server) — the last one exists only on localhost.

📁🔁 Recursive Mode: enable the checkbox above the settings to caption a whole franchise folder in one go — it scans every subfolder (one per character, for example) starting from the folder you currently have loaded, instead of just the folder/subfolder shown in the dropdowns. Each image is tagged and saved right next to itself, inside its own character subfolder. The app's own folders (_trash, _archive, _rename_cache) are skipped. Note: Review Mode (ghost suggestions) is not available together with Recursive Mode — tags are written directly to disk in this mode.

💡 Review Mode writes nothing to disk: the results appear as ghost tags that you accept one by one. Second Model Review (Tagger mode) runs another ONNX tagger afterwards and only adds the tags the first one missed.

🏷️ Tagger (ONNX)

Runs WD14 / SmilingWolf style taggers directly in the browser (WebGPU or CPU). You need the model file and its tag list:

1. Download a WD14 ONNX model and its CSV tags file (model.onnx + selected_tags.csv).
   SmilingWolf Official Links:
   🔗 HuggingFace Models | 🔗 GitHub Repository
2. Keep both files in the same folder. The CSV is matched in this order: <model name>.csv, <model name>_tags.csv, selected_tags.csv, tags.csv.
3. Localhost: put them anywhere inside models/onnx (one sub-folder per model works great) — they appear in the list by themselves.
4. Any other place: click "📁 Set Directory" and select the folder; it is scanned recursively and remembered.

Threshold is the minimum confidence for a tag. Additional Prefix adds tags to the beginning of every image; Exclude Tags removes the ones you list.

🖥️ Natural Language (Offline VLMs)

Generates natural English descriptions with Florence-2 or SmolVLM, fully inside your browser (WebGPU, falling back to CPU automatically):

• Model list: ✅ already on your disk, ⬇️ will be downloaded on first use. On localhost the files are saved in the models/ folder and found automatically.
• Storage: Auto (the models/ folder on localhost, the browser cache elsewhere), Browser cache, or Choose a folder (Chrome / Edge) — the same layouts as in the Prompt Helper.
• Florence-2: pick a Task (Caption, Detailed, More detailed, OCR, Object labels). SmolVLM: write the instruction in the prompt box. WebGPU can be unstable with some drivers — the app then retries without graph optimization and finally on the CPU.
• ⏏ Unload from memory frees RAM / VRAM (files stay on disk). 🧹 Delete downloaded model removes the files the app downloaded (it is dimmed for models it does not own). ⬇️ Load model now only prepares the model; the batch loads it automatically. ⏹ Cancel appears while a download / load is running.
• Save Result As: Automatic saves captions as an NL: tag and tag-like results (taggers, "Generate tags", object labels) as normal tags.

🦙 Ollama (Auto-detected Vision Models)

It asks your local Ollama server which installed models support vision and lists only those, so you never have to type model names. Works with Qwen2.5-VL / Qwen3-VL, LLaVA, Gemma 3, Llama 3.2 Vision, MiniCPM-V, JoyCaption (ollama pull user-v4/joycaption-beta), etc. Choose Natural Language for a caption or Booru-style tags to get a comma-separated tag list merged into the existing tags. If the list is empty or the connection fails, make sure Ollama is running (and, if needed, started with OLLAMA_ORIGINS=*).

🔥 PyTorch (local server)

The model runs in Python instead of the browser: the page talks to server_max.py (port 8100), which uses florence_backend.py and PyTorch (CUDA if you have an NVIDIA GPU, otherwise CPU). It can run bigger models, Florence-2 PromptGen v2.0 (MiaoshouAI), any model saved in a folder on your PC (safetensors / pytorch_model.bin), other vision-language models, and ONNX taggers through Python. Start the app with the localhost .bat file; the mode is hidden on the online version.

Install (Python 3.10+, use the same Python that starts the server, then restart the server):

python -m pip install torch --index-url https://download.pytorch.org/whl/cu124
python -m pip install transformers pillow huggingface_hub
python -m pip install einops timm

• The first line is PyTorch with CUDA 12.4 — copy the exact command for your system from pytorch.org. CPU only: python -m pip install torch.
• einops timm are needed by PromptGen and the microsoft/ Florence models. For the newest native Florence-2 (florence-community/) update with python -m pip install -U transformers.
• Check the GPU: python -c "import torch; print(torch.cuda.is_available())" should print True. The badge at the top of the panel shows 🔥 CUDA, 🖥️ CPU only, or what is missing.
• Models: PromptGen v2.0 (tasks: Generate tags, Caption, Detailed, More detailed, Analyze, Mixed, Mixed +), Florence-2 Base / Large, plus everything found in models/ and in folders added with 📁 Add model folder (📁 local models). Downloads go to models/ and then work offline. Raise Max tokens (512–1024) for the long PromptGen tasks.
• Only Hugging Face repositories microsoft/Florence-2*, florence-community/Florence-2* and MiaoshouAI/Florence-2* are downloaded automatically (they need remote code). Local Florence-2 folders that ship their own Python code (auto_map in config.json) run that code too — only add folders you trust.
• The local server only accepts requests from pages it serves itself (localhost / 127.0.0.1); other websites open in the browser cannot use it.

⚠️ Performance & Execution Mode

• Auto Captioning (Heavy Processing): Always run this tool via Localhost (e.g., using the provided .bat file or Live Server) or use the hosted Online version. This unlocks your CPU multi-threading and WebGPU acceleration.
• Direct File Execution (file://): If you open the HTML file directly with a double-click, security restrictions will cripple the processing speed. Only use this mode if you just want to manually edit or review already existing captions.