GLM coding API evaluation for builders: test one repository patch with diff, tests, and rollback

Last checked: 2026-07-15. Z.AI currently lists GLM-5.2 as available in its model API at $1.40 input / $0.26 cached input / $4.40 output per million tokens. GLM-5 is $1.00 / $0.20 / $3.20, GLM-4.7 is $0.60 / $0.11 / $2.20, and GLM-4.7-Flash is listed as free. The separate Coding Plan starts at $18/month and publishes prompt and MCP quotas rather than token billing.

Current GLM API models and prices

The current Z.AI pricing table also lists GLM-5.1, GLM-4.6, GLM-4.5, AirX and multimodal models. For a 100K-input, 10K-output run, GLM-5.2 costs $0.184 before cache and retries; GLM-4.7 costs $0.082; GLM-4.5-Air costs $0.031. That makes GLM-5.2 roughly 2.24× the listed GLM-4.7 cost and nearly 5.94× the Air example for the same token volume.

Coding Plan quotas and verified public price

The Coding Plan overview says the 5-hour quota resets dynamically and the weekly quota resets every seven days. Web Search MCP, Web Reader MCP and Zread share the monthly MCP limit. The Z.AI quick start separates Coding Plan configuration from the general model API, while the chat completion API covers pay-as-you-go application requests.

Choose the model API when the product owns requests and token economics. Choose Coding Plan when developers use supported coding clients and prompt quotas are the relevant constraint. The free Flash row is useful for smoke tests, but it is not evidence that it matches GLM-5.2 on repository work.

Minimal API smoke test

The official quick start provides cURL and SDK examples. A controlled request should use the documented base URL and a verified current model:

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ["ZAI_API_KEY"],
    base_url="https://api.z.ai/api/paas/v4/",
)

response = client.chat.completions.create(
    model="<CURRENT_VERIFIED_GLM_MODEL>",
    messages=[
        {"role": "system", "content": "Return a patch plan. Do not claim files were changed."},
        {"role": "user", "content": "Add --json output to the read-only status command. List files and tests first."},
    ],
)
print(response.choices[0].message.content)

Use the live documentation if the base URL or model surface changes. The first test asks for a plan, not an autonomous edit. This confirms access and lets the team inspect whether the model understands scope before tools are enabled.

Repository test card

Use one issue that is valuable but reversible:

Repository: internal TypeScript CLI, 42 files
Task: add --json to the read-only `status` command
Allowed writes: src/commands/status.ts, src/output/status.ts, and their tests
Allowed commands: npm test -- status, npm run typecheck, git diff --check
Forbidden: package installation, network access, secret reads, git push, destructive commands
Baseline: 18 tests pass on a clean branch
New fixtures: 2 JSON-output cases
Rollback: reset the test branch to the recorded baseline commit

This is better than asking the model to "improve the repo." The files, command, expected output and stop conditions are inspectable.

Execution sequence

1. Create a clean branch and record the baseline commit.
2. Run the 18 existing tests and save the result.
3. Give the model the issue, file map and allowed commands.
4. Require a plan listing files and tests before any write.
5. Approve only the bounded plan.
6. Apply or allow the patch inside the low-risk workspace.
7. Inspect git diff --stat, full diff and new fixtures.
8. Run the targeted tests, full relevant tests, typecheck and git diff --check.
9. Record model/API usage or Coding Plan session details.
10. Accept, revise or reset to baseline.

If the tool cannot separate planning from editing, reduce its permission boundary. A coding evaluation does not need production credentials, deployment access or write access outside the test repository.

Acceptance table

Quality is not the number of lines generated. A small, well-scoped diff with clear tests is better than a broad refactor that happens to include the feature.

Worked failure sample

The six-person team requests --json output. The model edits the status command, moves a shared formatter, updates a package, and rewrites snapshots. Targeted tests pass, but the full CLI suite is never run. The output looks complete in the coding client.

This is a failed evaluation. The tool exceeded file scope, introduced a dependency change and left regression evidence missing. The team resets to the baseline commit, narrows allowed writes and asks for a patch limited to the two implementation files and two test files. The failure is useful because it measures permission discipline, not just code fluency.

What to record for a fair comparison

Save wall-clock time, human review minutes, API input/output usage or plan session consumption, number of repair prompts, files changed, tests executed and final result. Also record whether the model detected uncertainty and asked a useful question. A comparison with another coding model should use the same baseline commit, issue, permissions and fixtures.

Do not quote a vendor benchmark as the result of this test. Benchmarks can justify looking at a model; they cannot replace the diff and tests from your repository.

Pause conditions

Stop immediately when the model requests secrets or production access, uses a destructive command, edits unrelated files, changes dependencies without approval, suppresses or skips failing tests, cannot show a diff, or loses the baseline rollback point. Also stop if the account cannot distinguish Coding Plan usage from API billing well enough for the intended budget.

FAQ

Is GLM Coding Plan the same as the general model API?

No. Z.AI documents a separate Coding Plan setup and dedicated configuration. Verify the purchase and endpoint surface you are actually testing.

What repository task should I test first?

Choose a read-only feature or small bug with a clear file boundary, existing tests, two new fixtures and a one-command rollback.

Should the agent be allowed to run shell commands?

Only an explicit allowlist in a low-risk workspace. Test, typecheck and diff commands are useful; network, secret, deployment and destructive commands are unnecessary for this trial.

What is a pass?

A bounded diff, all required tests, no unauthorized dependency or permission changes, reproducible usage/cost records and a clean rollback path.

Related reading

• Qwen API pricing and access for builders: verify the model, region, and real test cost
• MiniMax long-context API evaluation: test evidence retrieval before trusting the window size
• Cloudflare Pay Per Crawl test guide: what publishers and AI crawlers can verify in private beta
• AI model router selection for developers: compare fallback, logs, cost controls, and self-hosting

RadarAI helps builders track AI updates, compare source-backed signals, and decide which changes are worth acting on.