MiniMax long-context API evaluation: test evidence retrieval before trusting the window size

Last checked: 2026-07-15. MiniMax-M3 is now available as the current long-context option in MiniMax's official model table: a multimodal coding model with a 1M-token context window. The standard API price is $0.30 input / $1.20 output per million tokens up to 512K input, then $0.60 input / $2.40 output above 512K. A 500K-input, 10K-output request costs $0.162 at the listed standard rate; a 650K-input, 10K-output request crosses the tier and costs $0.414 before retries and cache-write charges.

Current MiniMax models, tiers, and prices

The MiniMax model introduction also lists M2.5, M2.5-highspeed, M2.1, M2.1-highspeed and M2. M2 is documented with 200K context and up to 128K output including chain of thought. The pay-as-you-go pricing page supplies the current M3 and M2.7 prices above; the text API reference supplies the request surface.

For a long document job, M3 standard is the direct fit because the official table exposes the 1M context and its 512K price boundary. M2.7 is cheaper than its highspeed sibling at the same listed input/output rates as M3's lower standard tier, but it is positioned around coding/agent work rather than the 1M M3 path.

Actual M3 cost at 500K and 650K input

The tier boundary more than doubles this example's standard total from $0.162 to $0.414 even though input grows by only 30%. Splitting or retrieval can therefore change both quality and the price tier, not only token count.

Build a packet that can expose real failure

A useful long-context test needs structure. Use 120 files totalling about 650,000 tokens. Include contracts, product specifications, security reports, meeting notes and financial appendices. Give every file a stable source ID such as CONTRACT-014 or SECURITY-007. Keep a manifest with file name, source ID, date, type, token count and relevant clauses.

Create 30 questions:

Twelve questions should require evidence from at least two files. That is the hard part of the evaluation. A model can find one memorable sentence and still fail the workflow by omitting the amendment, later meeting decision, or exception in another file.

Define the output contract

Require JSON with these fields:

{
  "answer": "short answer",
  "source_ids": ["CONTRACT-014", "MEETING-022"],
  "evidence": [
    {"source_id": "CONTRACT-014", "quote": "..."}
  ],
  "missing_information": [],
  "confidence": "high|medium|low"
}

The quotes let a reviewer verify support without rereading the whole packet. Source IDs prevent the model from hiding behind a vague phrase such as "according to the documents." For unanswerable questions, the correct result includes missing information rather than an invented citation.

Score quality and operations separately

Use this rubric:

Do not combine these into one flattering score. A system that answers well but costs too much for the product is not ready. A cheap system that silently omits amendments is not ready either.

Worked case: due-diligence packet

A due-diligence product team receives 120 files for a software acquisition. The packet contains a main customer agreement, two amendments, a security audit, remediation notes, a board memo and meeting minutes. One question asks whether the target must notify a customer after a security incident and whether a later amendment changed the deadline.

The correct answer requires the original contract and the amendment. In the failure run, the model cites only the original agreement and reports the old deadline. The prose sounds confident and the citation is real, but the answer is operationally wrong. That is the failure this evaluation is designed to catch.

The team runs two variants. Variant A sends the normalized packet in one long request. Variant B retrieves a smaller candidate set and sends those documents to the same model. Both use the same output schema and question set. The comparison is not "long context versus RAG" in general; it shows which configuration works for this packet.

Acceptance requires at least 27 supported answers, all required sources on the 12 cross-file questions, correct refusal on all four unanswerable questions, and cost/latency inside the predeclared budget. A missed amendment is a failed question even if the rest of the answer is polished.

Prepare input so the test is fair

Normalize scanned files with OCR and inspect a sample for missing pages. Preserve headings and page numbers. Insert clear boundaries between documents. Do not repeat large boilerplate sections unless they are part of the real workload. Count tokens after normalization, not from file size. Keep system instructions short enough that they do not consume a meaningful share of the window.

Run a small three-file smoke test first. Verify that the API accepts the model identifier, returns usage, respects the output schema and preserves source IDs. Then scale to 30, 60 and 120 files. Incremental runs make silent truncation and sudden cost changes easier to identify.

Cost record

For every run, save exact input tokens, output tokens, retries, latency and the dated price row. Use:

run cost = input_tokens / 1,000,000 × input_price
         + output_tokens / 1,000,000 × output_price
         + retry usage

If the API reports usage that differs materially from your tokenizer estimate, investigate before expanding. Document extraction, hidden formatting, or retry behavior can change the bill. Do not claim a per-document cost until it is derived from recorded API usage.

Pause conditions

Pause when the input appears silently cut off, source IDs disappear, citations do not support claims, cross-file evidence is consistently incomplete, unanswerable questions receive confident answers, token usage cannot be reconciled, or the model/API status changes during the test. Also pause if the full packet contains sensitive data that the selected deployment and account agreement are not allowed to process.

A context window is capacity, not an accuracy warranty. The decision should come from the evidence packet and error log.

FAQ

Does a larger context window remove the need for retrieval?

No. Test a long-packet and a retrieval-assisted variant on the same questions. The better architecture depends on completeness, citation quality, latency, cost and update frequency.

Is a needle-in-a-haystack test enough?

No. It tests locating a planted fact. Production work also needs cross-file synthesis, contradiction handling, refusal and accurate citations.

What is the most important failure sample?

A supported but incomplete answer: the model cites a real source while missing a later amendment or another required file. It looks trustworthy and can still cause a bad decision.

When is the MiniMax configuration ready?

When it meets the predeclared support, cross-file completeness, abstention, latency and cost thresholds on your packet, with current official model and pricing facts recorded.

Related reading

• MiniMax entity page
• Chinese AI models list
• Chinese open-source AI models