A production workflow addresses a measurable classification problem
DiDi International Business Group and Amazon Web Services published a production implementation report on September 8, 2026, describing an automated quality-assurance system for Spanish and Portuguese customer-service conversations. The system covers ride-hailing, food delivery and financial services across live chat and transcribed phone calls. It replaces a third-party system that DiDi characterized as opaque and difficult to update. The new design separates three jobs: verifying contact-reason labels, checking service-compliance criteria and identifying trends across batches of conversations.
DiDi and AWS report that intent-verification accuracy increased from 38 percent to 86 percent during production validation. They also report average compliance-scoring accuracy above 90 percent and say a trend-analysis task that previously took hours could be completed in minutes. Those are meaningful operational measurements, but the report does not publish the validation-set size, sampling method, class distribution, annotation protocol, uncertainty intervals or results by language and business line. The figures have not been independently reproduced and do not measure customer satisfaction, representative performance or issue resolution directly.
Separating verification from reclassification reduced overcorrection
The original intent workflow gave a language model the full hierarchy of possible contact reasons and asked it to evaluate the label assigned by a representative. According to the report, the model tended to find a more specific alternative even when the existing label was reasonable. Prompt revisions did not solve that behavior. DiDi instead divided the task. The first call sees only the assigned label and conversation, asking whether the label is acceptable. The complete classification tree appears only after that verification fails.
This isolation changes the decision from an open search for the best conceivable label into a narrower check of the label already chosen. Tickets marked Other follow a separate path that first searches nearby categories, then the complete hierarchy, and finally identifies a possible coverage gap. DiDi attributes the rise from 38 percent to 86 percent to this two-stage design. The report does not disclose comparable results for alternative architectures, individual classes or false-pass and false-fail rates, so it cannot show where the remaining errors concentrate or what consequences they carry.
External rules make multilingual evaluation easier to change
The compliance pipeline uses one prompt template assembled from external configurations. Language, business-line context, criterion definitions and pass-or-fail rules are inserted when a ticket is evaluated. That structure addresses a practical maintenance problem. A company operating several services in several languages would otherwise need many separate prompts, each of which could drift when standards change. Keeping rules outside the template allows a new criterion or market variation to be introduced through configuration rather than a separate prompt rewrite.
Centralized configuration can make revisions easier to trace, but consistency still depends on translation quality, rule clarity and representative testing. A criterion that works in one market may not capture the same conversational norm in another. DiDi reports average compliance accuracy above 90 percent without publishing category-level performance, denominators or error costs. Contact-center QA also measures adherence to defined policies. It does not by itself establish that customers received helpful answers, that workers exercised good judgment in unusual cases or that a scored interaction produced a satisfactory outcome.
Structured output improves handling without guaranteeing correctness
DiDi's simplified example forces selection of an Amazon Bedrock tool intended to return a machine-readable score and generated rationale. It does not show the documented strict: true setting or a Converse outputConfig.textFormat schema, and its outer schema does not require every criterion. The example therefore demonstrates a structured-output design, not guaranteed schema compliance. AWS documentation says that guarantee requires strict tool use or its dedicated structured-output mode. DiDi does not report whether either mechanism was enabled in the production configuration.
DiDi also reports programmatic checks where software can calculate a rule directly. Spelling errors are checked against the representative's messages, while response-wait times are calculated in code and supplied to the model instead of being inferred from prose. These controls reduce reliance on generation for facts that deterministic software can establish. Even when schema compliance is enforced, however, it does not establish semantic validity. A correctly formed JSON object can contain an incorrect score. Validation must therefore cover both the response format and the underlying judgment.
Generated rationales aid review but do not expose model cognition
Each score is paired with a generated explanation intended to help reviewers understand why the system assigned it. This can be operationally useful. A quality analyst can compare the cited conversation evidence with the applicable rule, challenge an unsupported score and identify a criterion that needs revision. Explanations also give representatives more information than an unexplained pass or fail. Their value comes from making a claim inspectable, not from proving that the automated judgment is correct.
A generated rationale is not a faithful transcript of a model's internal computation. It may present a plausible explanation after producing a score, omit influential information or cite evidence selectively. Calling such text a reasoning chain can overstate what it reveals. The safer interpretation is that it is a review artifact generated alongside the judgment. DiDi's deterministic checks provide stronger evidence for calculable facts, while subjective evaluations still require testing against human annotations and examination of disagreements.
Trend analysis combines extraction, clustering and generation
The Voice of Customer pipeline processes batches in three stages. A model first extracts fields such as issue type, sentiment, resolution status and proposed root cause from each conversation. An embedding model then groups semantically similar issue labels and ranks clusters by frequency. Finally, a language model produces a report describing prominent complaints and possible actions. Splitting the work prevents one model call from receiving thousands of conversations at once and producing categories too broad to guide operations.
The report describes a surge in cancellation-fee complaints across Latin American markets. DiDi says the pipeline identified frequent triggers and likely root causes within minutes, compared with hours of manual reading and summarization. That is a reported processing-time improvement for one described analysis, not an independently verified labor-saving rate. Extraction errors can distort later clusters, and a generated root-cause statement remains a hypothesis until checked against operational records. Faster detection is valuable when it directs human attention, but the publication does not report whether the analysis improved resolution rates, retention or customer satisfaction.
The strongest result remains narrow and actionable
The implementation shows useful engineering progress in how an organization can structure a multilingual quality-assurance workload. Isolating label verification reduced a reported tendency to overcorrect. External rules made changing standards easier. Deterministic calculations removed some tasks from the model, and staged trend analysis turned a large review job into a more manageable pipeline. These choices are more informative than a generic claim that a newer model performs better because they describe where the system receives information and where software or people can check its output.
The evidence supports a production-validation result, not a general finding that AI improves customer service or worker quality. The reported accuracy figures need fuller methods, independent replication and outcome measures closer to customers. Generated rationales can make scores easier to challenge, but they cannot establish the hidden basis of a model's decision. DiDi's design is most persuasive where it narrows tasks, exposes rules and preserves review. Its broader public benefit will depend on whether those controls produce fairer evaluations, faster correction of systemic problems and better service after deployment.