Verification baseline: 2026-06-30 / The original document (as of Qwen2.5 / early Qwen3) has been updated to the current lineup and hardware baseline, with references appended. Core principle: VRAM (or unified memory) is a hard constraint — if the weights don't fit, the model simply won't run. Always calculate quantization, context, and KV cache together. [R7][R10]

Overview of Qwen

Qwen (Chinese: 千问, pronounced "Qwen") is a family of large language models (LLMs) and large multimodal models developed by Alibaba Group. The first version was released in beta form in April 2023, and it has since grown into one of the largest open-source AI ecosystems in the world. In March 2026, Alibaba unified its AI brand under "Qwen Large Model," resolving prior naming confusion.

Key Capabilities

Qwen can perform a wide range of multimodal and language tasks.

Natural language understanding and text generation

Visual understanding (images, video)

Audio understanding and processing

Tool calling and AI agent capabilities

Role-play and multi-turn dialogue

The models are pretrained on large-scale multilingual, multimodal data and fine-tuned on high-quality data to align with human preferences.

Technical Architecture and Characteristics

Dual-mode architecture (thinking mode / non-thinking mode) Qwen3's core innovation is equipping a single model with both a "thinking mode" and a "non-thinking mode" simultaneously to improve efficiency.

Mode | Applicable situation Thinking mode | Tasks requiring deep reasoning: complex logic, math, programming, etc. Non-thinking mode | General conversation requiring a fast response The model dynamically assesses problem complexity and allocates compute automatically via a "thinking budget" mechanism.

Model architecture Dense models and MoE (Mixture of Experts) models coexist: available in sizes ranging from 0.6B to 235B.

Qwen3-Next uses a highly sparse MoE architecture: total parameters of 80 billion, but only about 3 billion (roughly 3.75%) activated at inference time, dramatically improving efficiency.

Qwen3-Max has over 1 trillion (1T) total parameters, pretrained on 36T tokens.

Training strategy 3-stage pretraining: build language foundations → strengthen reasoning ability → extend long-text capability.

4-stage post-training: including long chain-of-thought cold start, reasoning reinforcement learning, and mode fusion.

A "large model raises small models" distillation approach: training small models with data generated by large models.

Major Versions and Models

Qwen 3.5 February 2026 — flagship model Total 397B (17B active) 262K native context (expandable to 1M) Supports 201 languages, Apache 2.0 open source

Qwen3.7-Max May 2026 — agent-specialized Capable of fully autonomous operation on complex tasks lasting over 35 hours

Qwen3-Max September 2025 — over 1T total parameters, ranked #3 globally on LMArena

Open-source models include the Qwen, Qwen1.5, Qwen2, and Qwen2.5 series, ranging from 0.5B to 110B. There are also dedicated vision-language models (Qwen-VL), audio models (Qwen-Audio), coding models (Qwen-Coder), and reasoning models (QwQ).

Application Domains

Qwen can be used across a wide range of domains.

Software development: code generation, debugging, review, support for 50+ programming languages

Content creation: long-form writing, SEO, social media, translation (201 languages)

Research and data analysis: literature review, chart interpretation, scientific reasoning, medical analysis

Enterprise operations: customer-service chatbots, document processing, knowledge-base Q&A

On benchmark tests, Qwen 3.5 scored 76.4 on SWE-Bench Verified, a level comparable to GPT-5.2 and Claude Opus 4.5.

How to Use

Open-source models: free to use under the Apache 2.0 license, supports local execution via Ollama, llama.cpp, LM Studio, etc.

API service: accessible via Alibaba Cloud's DashScope API

Free trial: chat available at the official Qwen Chat website (chat.qwen.ai)

Enterprise service: offered to over 290,000 enterprise customers worldwide

0. Version Lineup You Should Know First (Document Update)

The original document only covers qwen2.5, qwen3, and sporadically qwen3.5:4b, but as of 2026-06 the lineup has expanded as follows.

Generation Release Composition Multimodal Languages License Local-capable
Qwen3 2025-04 dense 0.6B-32B + MoE 30B-A3B, 235B-A22B text 119 Apache-2.0 Yes [R2][R7]
Qwen3.5 2026-02 Small 0.8B/2B/4B/9B + 27B / 35B-A3B / 122B-A10B / 397B-A17B vision (native) 201 Apache-2.0 Yes [R1][R9]
Qwen3.6 2026-04~05 27B (dense) / 35B-A3B (MoE), enhanced coding/agent vision — Apache-2.0 Yes [R11][R12]
Qwen3.7-Max 2026-05-20 proprietary API only, open weights not released — — closed No (local not possible) [R7]

Note:

  • This guide covers open-weight models only. API-only models like *-Max and *-Plus are not local install targets. [R7][R12]
  • In the "2507" refresh of 2025-07~08, Qwen3 split from the hybrid (toggle) approach into separate thinking/instruct checkpoints. Confirm which variant you're downloading. [R7]

1. Choosing an Installation Method

Method Characteristics Recommended for
Ollama One-command install/run, easiest LLM beginners, quick testing
LM Studio GUI-based, a few clicks. Supports MLX/GGUF Those who want to use it without coding
Hugging Face Transformers Fine-grained control via Python code Developers, customization

2. Installing Ollama (Simplest)

2.1 Installing Ollama [R8]

Windows

irm https://ollama.com/install.ps1 | iex

Or download the installer from the official site (ollama.com).

macOS

brew install ollama

Linux

curl -fsSL https://ollama.com/install.sh | sh

2.2 Running

ollama serve   # manual start (usually runs automatically after install)

2.3 Downloading and Running Models [R8][R16]

# Current recommendation — Qwen3 family
ollama run qwen3:8b            # baseline/entry-level, about 5GB
ollama run qwen3:4b            # lightweight
ollama run qwen3:14b           # about 9GB
ollama run qwen3:32b           # higher-end dense
ollama run qwen3:30b-a3b       # MoE: 30B quality at ~8B-class speed (still needs full VRAM)

# Qwen3.5 family (multimodal)
ollama run qwen3.5:9b          # recommended baseline for 8GB cards, about 6.6GB
ollama run qwen3.5:4b          # lightweight, about 3.4GB (file size)

# Older version
ollama run qwen2.5:7b

Available sizes (examples): qwen2.5 0.5/1.5/3/7/14/32/72B, qwen3 0.6-32B + 30B-A3B/235B-A22B, qwen3.5 0.8/2/4/9/27/35/122/397B family. [R1][R2][R10]

Caution:

  • Many Ollama tags without instruct are still chat models, but since Qwen3 2507, thinking/instruct variants are distributed separately, so check the tag directly. (The original document's blanket claim that "all are Instruct" is inaccurate under the current lineup.) [R7]
  • The "3.4GB" of qwen3.5:4b is the GGUF file size, not runtime RAM. You must add KV cache + context overhead on top of that in practice. [R7][R14]

2.4 Using Your Own GGUF File (Modelfile) [R8]

FROM qwen2.5-7b-instruct-q5_0.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.8
PARAMETER repeat_penalty 1.05
PARAMETER top_k 20

ollama create my-qwen -f Modelfile
ollama run my-qwen

Caution: multimodal GGUFs like Qwen3.5 may have limited direct Ollama import due to the separate vision (mmproj) file issue. In that case, a llama.cpp-based backend or LM Studio is recommended. (Pulling via an Ollama library tag works normally.) [R9]

3. Installing LM Studio (GUI)

  1. Download and install the macOS/Windows/Linux installer from the official site.
  2. Launch the app → search for models with Cmd+Shift+M (Mac) / Ctrl+Shift+M (PC).
  3. Search "Qwen" → download the model matching your hardware.
  4. You can also connect directly via "Use this model → LM Studio" on a Hugging Face model card.
  5. Multimodal/Thinking toggles may require per-model YAML settings, with CLI support for manual import via lms import <path>. [R9]

On Apple Silicon, using LM Studio with MLX format support offers a significant performance advantage. [R9][R11]

4. Hugging Face Transformers (for Developers)

4.1 Prerequisites

pip install transformers torch accelerate
pip install modelscope   # faster downloads from within China

4.2 Loading and Running

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Qwen/Qwen3-30B-A3B"   # or Qwen/Qwen3-8B, etc.

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",   # auto GPU/CPU allocation, CPU offload if insufficient
)

messages = [{"role": "user", "content": "Qwen, please introduce yourself in Korean."}]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,   # thinking mode (Qwen3 and above)
)

outputs = model.generate(
    **tokenizer([text], return_tensors="pt").to(model.device),
    max_new_tokens=200,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

enable_thinking=True is advantageous for complex reasoning but slows down responses. False is recommended for everyday conversation. [R2][R8]

Tip for checking VRAM suitability before installing — estimate memory by reading only the header, without downloading: [R18]

uvx hf-mem --model-id Qwen/Qwen3-8B --experimental --max-model-len 8192

5. Model Selection Guide (by Hardware, Q4_K_M Baseline)

VRAM figures are based on Q4_K_M GGUF; add +1-2GB for 4K context KV cache. If VRAM is insufficient, Ollama/llama.cpp automatically offloads to system RAM, but speed drops sharply (10-20x slower memory bandwidth). [R13][R18]

Hardware Recommended model Approx. VRAM/RAM Notes
Older laptop (4-8GB RAM) qwen3.5:4b / qwen2.5:3b 3-4GB (file) CPU-only possible, slow
Standard desktop (16GB RAM) qwen3:8b / qwen3.5:9b about 5-6.6GB CPU-only possible [R13][R14]
8GB GPU VRAM qwen3.5:9b (Q4) / qwen3:7-8b about 5.5-6.6GB RTX 3060/4060 class [R13][R14]
12GB GPU VRAM qwen3:14b (Q4) about 9-9.5GB room for Q6_K [R13][R14]
16-24GB GPU VRAM qwen3:32b / qwen3.6:27b (about 17GB) 17-20GB comfortable at Q4_K_M with 24GB [R12][R14]
24GB GPU VRAM qwen3.6:35b-a3b (UD-Q4 about 22GB) about 22GB don't push context too far [R12][R14]
Multi-GPU / server qwen2.5:72b / 235B-A22B / 397B-A17B tens to hundreds of GB requires tensor-parallel [R16][R19]

MoE trap: models like 30B-A3B, 35B-A3B, 80B-A3B must load the entire weight set into VRAM even though only 3B parameters are active. "3B active" does not mean "3B of memory." Example: Qwen3-Next-80B-A3B requires about 160GB VRAM unquantized. [R14][R19]

6. Cautions and Tips

6.1 Storage Space

  • 7B ≈ 5GB, 14B ≈ 9GB, 27B ≈ 17GB, 32B ≈ 20GB, 35B-A3B ≈ 22GB (Q4 baseline). [R13][R14]
  • Ollama's default storage path: ~/.ollama/models. Ensure sufficient free space. [R8]

6.2 Quantization [R13][R16][R18]

  • Q4_K_M: the default for most cases. About 75% VRAM savings, minimal quality loss.
  • Q5_K_M: ~60% savings, slightly higher quality than Q4.
  • Q8_0: highest-quality quantization. Use when VRAM headroom is available.
  • NVFP4: native on Blackwell (RTX 5060 Ti/5090), more efficient than Q4.
  • Explicit tag example: ollama run qwen3:8b-q4_K_M
  • Warning: Q2_K noticeably degrades output quality for CJK languages like Korean/Chinese. Use Q4_K_M as the minimum for CJK work. [R13]

6.3 GPU Usage

  • NVIDIA: requires CUDA drivers.
  • Apple Silicon (M1-M4): advantageous for large models via unified memory. MLX format + LM Studio recommended (e.g., M3 Max 64GB gets ~22 tok/s on qwen3:32b). [R13]
  • If VRAM is insufficient, device_map="auto" offloads to CPU, but perceived speed drops sharply beyond 10-20% offload — a smaller model is recommended. [R18]

6.4 Korean-Language Use

  • The Qwen3 family supports 119 languages, and Qwen3.5 onward supports 201. Korean performance is strong, and Korean prompts can be used as-is. [R2][R9]

6.5 Thinking Mode (Qwen3 and Above)

  • CLI: ollama run qwen3 --think / --no-think [R8]
  • API: {"model":"qwen3","think":false,...} or set a reasoning budget cap via {"thinking":{"budget_tokens":1024}} [R8]
  • In-prompt /think, /no_think toggles are also supported by some variants.
  • Recommendation: use no-think for summaries/drafts, think for code debugging, math, and logic.

6.6 Context / KV Cache Cautions (Gap in the Original, Now Supplemented)

  • Qwen3.5/3.6 have a native 256K (262,144) context, expandable to ~1M via YaRN. [R12][R17]
  • The longer the context, the more heavily KV cache consumes VRAM. The "minimum VRAM" table above is based on short/medium context — actually using 128K-256K requires substantially more headroom. [R12]

6.7 Network

  • An internet connection is required for the initial download (a few GB to tens of GB). Fully offline use is possible afterward. [R8]

6.8 License

  • Qwen3/3.5/3.6 open weights are Apache-2.0 — commercial use is permitted. (Note: *-Max API models have separate terms.) [R7]

7. Troubleshooting

Problem Solution
"CUDA out of memory" Use a smaller model or lower quantization (Q4→Q3) / reduce context [R12][R14]
Ollama can't find the model Download explicitly with ollama pull qwen3:8b [R8]
Responses are too slow On CPU-only, use a smaller model (0.6-4B), or check the offload ratio (>10-20% means insufficient GPU) [R18]
Model doesn't appear in LM Studio Manually import with lms import <model path> [R9]
Multimodal GGUF is broken in Ollama mmproj separation issue — use llama.cpp/LM Studio [R9]
Checking VRAM suitability before download uvx hf-mem --model-id <repo> --experimental [R18]

References

Disclaimer: model lineups, tags, and file sizes are updated frequently upstream (Ollama/HF/Unsloth). Recheck the latest tags and file sizes on the respective library pages before actually installing.