read first

A computer for local coding AI: autocomplete and agents need different tests

Compare coding AI on real tasks, from autocomplete to repository edits

Local coding AI can suggest the next code in an editor, revise a function, or work across multiple files. Decide which task you need, then run the same real task on your current device and record test results along with review time to narrow down the model and memory you need.

Producing code quickly is not the same as finishing an edit and its review. The model's context and execution permissions differ depending on whether you need a short suggestion, a one-function change, or coordinated edits across repository files. Choosing a repository agent from autocomplete scores or chat response speed alone can misrepresent the real workflow.

Requirements and key details
  • For autocomplete, check whether suggestions arrive in time and fit the editing flow.
  • For function edits, inspect boundary-case tests and the diff.
  • Give an agent only the context and file/command permissions it needs, starting with the narrowest scope.

Check whether autocomplete suggestions are useful while typing

While writing a TypeScript form, autocomplete proposes a short continuation after the cursor. Even a correct suggestion helps less if it arrives after you have already typed the next line. In the same file, record when suggestions arrive, whether you accept them, and what you edit afterward. Generating a complete function in chat means reviewing and inserting the whole result; using autocomplete means judging whether a short suggestion fits the current line and variable flow.

First check that your editor supports the model, then see whether suggestions follow the project's variable names and code style. Pasting a complete function and then renaming variables has different review needs from receiving a bracket or conditional one line at a time. Open a typical file in your actual editor, accept suggestions, and record how much you change to see whether autocomplete helps your workflow.

A choice between a small code suggestion and a task needing more context
For autocomplete, check whether a short suggestion fits the current cursor and code flow.

Pair each function edit with its input and expected result

For an invitation-code validator, define the expected behavior: reject an empty string and three spaces, but accept AB-12 unchanged. A length-only check could accept three spaces, so trim surrounding whitespace before checking whether the value is empty. The normalized value used for this check does not have to replace the stored value. Changing the case of a valid code or its storage format would go beyond the request.

Give the model the reproducing inputs, current behavior, expected behavior, and only the relevant function and caller. In the diff, check which files changed and how broad the edit is; in tests, check the actual outcome for all three cases. If there are no automated tests, create a small reproduction with those inputs. One passing case does not validate the whole form: check the validator separately from the screen's submit and error-message behavior. Confirm that the screen prevents submission and clears its error when appropriate so users experience the policy correctly.

When a test fails, distinguish a validator bug from a problem in how the screen passes the value. If both layers need changes, decide which layer owns the rule and limit edits to the necessary files. This reduces the risk that handling an exception in one screen changes another form's behavior.

Give an agent relevant context instead of the whole repository

An agent editing multiple files uses the request, code, earlier responses, and tool results as context. If the invitation-code rule spans a screen and tests, locate and provide those relevant files together. Sending the entire repository can add unrelated code and make the needed locations harder to identify. A longer context can hold more material, but it does not automatically find the right files.

Locate the function containing the rule, the screen that calls it, and the existing tests. If the agent edited a different form with a similar name, correct the search path first. If it read the right files but missed how the function and screen connect, break the plan into steps: edit the validator, inspect the caller, and check the error display. Existing tests may capture exception policies that are not obvious from a function name, so read them before editing and verify that the agent's explanation matches the files it actually inspected.

A diagram selecting relevant files between a repository and an agent
Give the agent the function, its calling screen, and relevant tests.

Grant file-edit and command permissions separately

Reading files, applying edits, and running tests or package-install commands have different effects on the repository and environment. Do not enable the entire repository, terminal, and installation permissions at once. Start with read access to locate the necessary files. Review the diff before applying edits, and run tests only after understanding what the command affects. Using a local model does not mean search tools or external APIs also operate locally.

Test an invitation-code change in a copy or separate branch, and do not provide secrets or real customer data. If tests require network access or installation, inspect the runtime1 environment separately from file-edit permissions. For sensitive repositories, exclude tools that may transmit data externally and test execution permissions only in a reversible copy.

Measure completion time from the start of work through testing

Compare candidates on the same TypeScript form task. For autocomplete, check whether suggestions fit the typing flow; for a function edit, confirm that empty and whitespace-only input is rejected while AB-12 is preserved; for an agent, inspect edit scope and test results. If you plan to use multiple roles, test the model combination and concurrency you will actually use. Short-suggestion speed does not predict completion time for a multi-file task.

Consider two hypothetical edits. A takes 15 seconds to generate and another 90 seconds of human editing, for 105 seconds total. B takes 30 seconds to generate and 20 seconds of editing, for 50 seconds total. The slower generator finishes the overall task first. These illustrative numbers are not measurements of specific models; tests and review effort must also be comparable for a fair comparison. Use speed comparisons to narrow candidates, then complete the real form task through testing. If editing and review work well on your current device, you can postpone a purchase.

A diagram comparing results and completion time across two task paths
Include tests and human edits in addition to model response time.

Terminology notes

  1. Runtime — The software environment that provides facilities needed while a program runs. In local AI, it can also refer to a model execution engine.

    Back to the text

Frequently asked questions

What should you check first in this article?
Local coding AI can suggest the next code in an editor, revise a function, or work across multiple files. Decide which task you need, then run the same real task on your current device and record test results along with review time to narrow down the model and memory you need.