Guide · Local AI

Choose a GPU for local AI

Fresh price samples unavailable · Hardware advice reviewed: 2026-10-04 · Methodology

Choose software and memory together

Check the model architecture, quantization, operating system and driver supported by your runtime. Ollama lists different AMD support on Linux and Windows; NVIDIA and Apple have their own requirements. See Ollama hardware documentation.

Estimate weights plus context cache and runtime working memory using our VRAM guide. More capacity helps only when the software can use it. We have not benchmarked the cards below; this is a buying shortlist, not a speed ranking.

Used versus new

Compare the total checkout cost with alternatives that meet your memory and software requirements. An older editorial estimate is context, not a price you can assume is available. Check the condition, return policy, and any warranty offered for each listing.

Macs and the rest of the PC

Mac unified memory is shared with the operating system and applications. Compare available memory and matched workload results, rather than assuming a Mac memory figure equals dedicated VRAM. See Mac vs GPU.

For a PC, size system RAM and storage for the actual artifacts and loading or offloading behavior. Include the host system, PSU, cooling and case clearance in the budget. Follow the board manufacturer's power and connector requirements.

Buying checklist

• Confirm full model name (avoid laptop / mobile variants).
• Ask for photos of the card, ports, and fan condition.
• Prefer local meetup or returns-friendly sellers for $800+ cards.
• PSU: 3090/4090/5090 need serious wattage and enough PCIe power cables.
• Compare dated asking-price samples with recent sold listings and the full checkout cost.

Sources and review

Hardware advice reviewed October 4, 2026. These are documentation-based recommendations; we have not benchmarked these configurations.

• Ollama hardware and operating-system support
• Ollama context and memory guidance
• MLX LM model support and memory controls
• llama.cpp multi-GPU configuration

FAQ

What is the best used GPU for local AI in 2026?

There is no single best used GPU for every local AI workload. Start with the memory your model and context need, check software support, then compare dated asking prices and the cost of a compatible build. The RTX 3090 and RTX 4090 both offer 24GB; the RTX 5090 offers 32GB; workstation options include 48GB cards and the 96GB RTX PRO 6000 Blackwell.

What local LLM hardware do I need to get started?

Start with the exact model artifact, runtime, context and concurrency. Check available GPU and system memory, storage for the artifacts, and power and cooling requirements. Small models can run on hardware you already own; there is no universal minimum build.

How much VRAM do I need for local LLMs?

Weights are only part of the requirement. Add context cache and runtime working memory, and confirm support for the exact quantization. Use our VRAM guide to estimate capacity rather than treating parameter counts as guaranteed fits.

Always-on agents that call hosted models

A different job than local inference: bots and automations that stay up and call an API. For that, a cheap Linux VPS is the usual box. VPSDime is the referral we use when that is the workload. (Affiliate link — see disclosure.)

See RTX 3090 price guideVRAM guideMuse Glimmer hardwareAffiliate disclosure