I Have an RTX 3090—Should I Upgrade My Graphics Card for Local AI?

When downloading Qwen3.8-27B, start with the smaller Q4 file, which uses quantization² to reduce its size. It is about 3.3GB smaller than Q5, leaving more room to process documents and continue the conversation.

Qwen3.8-27B file list · Unsloth

Within the 24GB, the runtime³ and KV cache⁴ use the space not occupied by the model file. The KV cache grows as documents get longer and conversation history accumulates. If Q4 summarizes short documents well but stops on long ones, check whether the runtime and KV cache have enough room.

Before switching to Q5, ask the same question about the same document. Compare whether the summary omits important points, whether dates and numbers are correct, and whether follow-up answers are grounded in the source. If Q4 already gives you what you need, there is no need to switch to a larger file.

The Q8 file alone is about 29GB, which exceeds the VRAM⁵ on a single RTX 3090. To run Q8 fully loaded on the GPU, you would need to add another GPU or replace the card with one that has more VRAM.

After sending a prompt⁶ in Ollama, open another terminal and enter the command below.

Check Ollama's runtime status

If part of the model is in system RAM⁷, the CPU⁸ also performs computation and exchanges data with the GPU. Close models or apps you are not using; if more of the model then fits on the GPU and answers get faster, you have reduced the wait without buying a new card.

If it stops only on long documents, check the context-length setting too. If it is set far above the length needed for the document, try reducing it. Keep enough room for the document and follow-up questions. Consider adding or replacing a GPU only if the required document still will not fit after adjusting the setting.

For Qwen3.8-27B Q4_K_M on an RTX 3090, the site's calculator estimates a generation rate of 42.3~46.9 tok/s. Generating 500 tokens⁹ takes about 11~12 seconds, with document processing time on top.

Matching input and output lengths makes it easier to compare how much waiting an upgrade could save. The example below processes a 4,096-token input and then generates a 500-token answer. The model is already loaded, requests are sent one at a time, and cached prompt computation is not reused.

Calculator estimate · 4,096 input tokens / 500 output tokens · completion time = input-processing time + generation time for the remaining 499 tokens. RTX 3090 settings · RTX 5090 settings

Using representative speeds, the estimates are about 17 seconds and 3 seconds. If you wait for an answer to finish before sending the next question 30 times a day, that is about 14 seconds saved each time—or roughly 7 minutes per day. It may feel minor if you leave a summary running while doing other work, but noticeable if you edit a document as soon as each answer arrives.

If the first character appears slowly after a long document is submitted, compare prompt-processing (prefill¹⁰) speed. If the answer starts quickly but the text streams slowly, compare decode¹¹ speed, which produces the answer tokens. To finish document summaries and follow-up questions faster, compare both the wait to the first response and the time until the full answer is complete. Prefill and decode

When upgrading to an RTX 5090, consider the work of moving to a different runtime along with the card price. The fast configuration used above runs an NVFP4¹² model with MTP-2 in vLLM. You will need model files compatible with vLLM's NVFP4 and MTP¹³ setup instead of the GGUF¹⁴ file you currently use with Ollama.

Published RTX 5090 run

Published measurements · 2026-09-05 · 256-token output budget · vLLM 0.28.0 / NVFP4 / FP8¹⁵ KV cache / MTP-2 / driver 610.43.02 · GPU-only execution · 1 concurrent request · prompt cache disabled · median of 3 runs per condition. Benchmark and runtime configuration

With roughly 8,000 tokens, the answer finished in about 2 seconds; with roughly 30,000 tokens, it took about 3.6 seconds just to begin the first response. If you often process long documents, check whether the wait also decreases at the lengths you use.

The RTX 5090 has 32GB of VRAM, 8GB more than the 3090. Rather than choosing Q8 based only on file size, check whether enough room remains for documents and conversation after loading the model. If you want the fast NVFP4·MTP configuration, first read the published installation steps and list what you need to prepare for your operating system. RTX 5090 specifications · Compare both GPUs with different input lengths

If you keep the existing 3090 and add another NVIDIA GPU, llama.cpp can distribute the model across both cards. Adding a 16GB GPU gives you 40GB of physical VRAM in total; adding a 24GB GPU gives you 48GB. That creates an option for splitting the roughly 29GB Q8 file across the cards.

When splitting a model, leave room on each card for the runtime and KV cache. Use llama.cpp's --split-mode layer and --tensor-split to distribute the model according to each card's available memory. The connection and split ratio also affect answer speed. llama.cpp GPU-splitting settings

Before adding a GPU, check the slots, cooling, and power supply.

1. Check that the motherboard has an available PCIe slot and how many lanes the second slot uses.

2. Check that both cards fit in the case without blocking each other's fans.

3. Check that the power supply has enough capacity and the connectors needed to power both cards and the CPU.

Adding a GPU can help when the model file is too large to run. By contrast, if Q4 already fits entirely on the GPU and you want faster answers, compare a faster single GPU first. Before buying an additional GPU, also check whether your app can be configured to use both cards together.

Let's compare scenarios that reuse the existing RTX 3090 PC and assume $0 for additional parts and installation. GPU purchase costs use the lowest listed price among verified new products; shipping and personal discounts are excluded. Prices for both options were checked on 2026-10-12.

“Add” means adding one 16GB RTX 5060 Ti; “replace” means switching to a 32GB RTX 5090.

Looking only at the card purchase prices, the difference between the two options is ₩7,995,080. If your goal is to run a model split across 40GB, adding a GPU is worth considering. If you want faster answers from your current model, look at the wait-time difference discussed earlier rather than VRAM capacity.

For the final estimate, add the new GPU price plus the cost of parts and installation you actually need to change. If you sell the 3090 as part of a replacement, subtract the amount you actually receive after the sale is complete. For an added GPU, include any parts needed for slots and cooling; for a replacement, include only parts that must change to fit the new card and meet its power requirements.

Check the power supply too. Rated GPU power is 350W for the RTX 3090, 180W for the 5060 Ti, and 575W for the 5090 Founders Edition. The rated total for a 3090 plus a 5060 Ti is 530W; two 3090s total 700W. When choosing a PSU, account for the CPU and other components as well, and check the power requirements of the specific card you plan to buy. RTX 3090 specifications · RTX 5060 Ti specifications · RTX 5090 specifications

If electricity use and fan noise are a concern, you can also try a power limit. This setting lowers the maximum power the GPU can draw. In a published comparison running Qwen3.8 27B on an RTX 3090, lowering the limit from 350W to 300W retained about 95% of generation speed while measured GPU power fell by about 14%. The power-by-power results, model, and runtime settings are in the power-limit guide.

A power limit can reduce heat and power draw while leaving the card's VRAM available. Compare how much slower answers become and choose a limit that works for you. Size the PSU and cables for the card's rated power requirements, since a reboot or cleared setting can restore the default power limit.

If you plan to run the system for long periods, estimate electricity and cooling costs as well as the purchase price. Enter the actual power draw after applying the limit and daily hours of use to compare the operating costs of keeping, adding, or replacing a GPU. How to apply and restore the setting

First, use a document and prompt you normally work with. Load the model once, run the same task three times, and record the wait to the first response, time to completion, and actual output-token count. Sort the three results and compare the middle value.

Also judge whether the summary captures the document and whether the answer helps with your work. When comparing Q4 and Q5, use the same document and record input length, output length, and whether prompt caching is used. Record initial model-loading time separately from the wait for each prompt.

To save runtime measurements to a file, you can use the site's local measurement tool. It requires Node.js 20 or later and an OpenAI-compatible server running the model. If you use LM Studio or a similar app, start its local server and note the displayed address and model name.

1. Save the measurement tool and metadata file in the same folder.

2. Replace the null values in the metadata file with your hardware, model, and runtime details. Set outputTokens to 500 and enter the context length¹⁶ configured on the server.

3. Validate the metadata file with the first command, then run the measurement. Replace 1234 and SERVED_MODEL in the example with the port and model name shown by your server.

The tool runs a fixed sample once as a warm-up, then measures three runs. It saves runtime conditions, actual input/output token counts, and timings to measurement.json. Requests go only to the server on your computer; results are not automatically submitted to the site. Use it to record baseline response speed, separately from tests with your own documents.

If Q4 can summarize the documents you need and handle follow-up questions, keep the RTX 3090. If waiting for the next answer is not disruptive, you do not need to spend money on an upgrade.

If you must run Q8 or a larger model and VRAM is the limitation, start by pricing an additional GPU. First check the slots, power, and cooling in your existing PC, and confirm that your runtime can split execution across two GPUs.

If your desired model already runs well but waiting for answers repeatedly interrupts your work, a faster GPU may be worth considering. Compare the wait-time reduction for documents of your usual length with a real quote. Whether the new card solves the problem you have matters more than its product tier.