Purchases and Costs

Already own an RTX 3090? Check this before upgrading for local AI

A new GPU benchmark can make your current card feel old overnight. But an owner faces a different decision from a first-time buyer: not which card offers the best value in isolation, but whether the additional spending removes a problem that actually matters.

Check placement before replacing hardware

Slow output can come from partial CPU placement or a runtime that is not using the expected acceleration path. With Ollama, run the model and inspect the PROCESSOR column in ollama ps. Mixed CPU/GPU placement is not the same condition as a fully GPU-resident benchmark. Check other applications using VRAM too. Establish normal GPU operation with a small model and short input before shopping; the remedy may be a configuration change rather than a replacement.

Describe the delay precisely

Is it slow while writing code, or before the first character appears after a long document? One generation-rate number cannot describe both. Keep an example that already feels satisfactory as well. Compare upgrades on the task you want to improve, using the same model, quantization and input length. Switch between full-response and decode-only comparisons. A card that wins one workload will not necessarily improve every task by the same proportion.

Capacity is a different upgrade

If your current model works well but an essential alternative does not fit, faster hardware with the same capacity may miss the point. Establish the file and context requirements first. If you chose that model merely because it is new, test a smaller alternative on your work. Whether a bigger model improves your answers and whether a bigger machine can hold it are separate questions. Answering them in that order helps avoid buying capacity without gaining useful results.

Resale value is not a guaranteed discount

Subtracting an online asking price from the cost of a new card can make an upgrade look cheap. The eventual sale price and timing are uncertain. Check the power supply, case clearance, cables and cooling as well. Do not mix the cost of a component swap with that of a complete replacement PC. Write down the cash outlay you can handle even if the old card does not sell immediately, then treat resale separately. The TCO tool is useful only when its prices reflect the configuration you actually choose.

Keeping the card is a valid decision

If daily tasks already finish well and you can do something else during the wait, a better number alone is a weak reason to rush. Write down a replacement trigger instead: an essential model that will not fit, or repeated delays affecting your work. Keep the current device on the left of the comparison so the decision retains its baseline. Use estimated speed to understand the likely direction of a change, then seek reproducible evidence for the same setup before a major purchase.