TRV-2026-3097Version 1 · Published

Saved · see the current record

Why this version exists

Certified into the record

The exact text we saved

This is the version exactly as it was saved, character for character. The SHA-256 hash below is a 64-character code worked out from that text. Run the same text through SHA-256 and you get the same code; change a single letter and the code comes out completely different. Each version also stores the previous version’s hash, which chains them together so no version can be swapped out quietly.

TRUVACE RECORD VERSION
record: TRV-2026-3097
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-11-22T12:02:48.313657Z
status: published
lens: g_space
sector: business
headline: Efficiency Without Quality? A Six Sigma DMAIC (Define, Measure, Analyze, Improve, Control) Field Study of Sub-Intent Library Curation in a Live Airline Chatbot
dek: This study asks whether the Six Sigma DMAIC (Define, Measure, Analyze, Improve, Control) cycle can improve utterance capture in a live enterprise chatbot, and how much of the change is attributable to the intervention during a concurrent migration to large language model (LLM) components. Turkish WhatsApp utterances of Turkish Airlines’ TK Assistant (80,295 utterances, January–July 2025) was analyzed; utterance capture scores (success threshold 0.95) were graded on a five-level scale, with 0.95 as the success th…
gain_title: In a field study of Turkish Airlines' TK Assistant WhatsApp chatbot, 17 of 22 recurring added texts were captured as exact matches at first recurrence, versus none of eight recurring non-added controls.
problem_title: (none)
trace_subject: (none)
gain_reading: In a field study of Turkish Airlines' TK Assistant WhatsApp chatbot, 17 of 22 recurring added texts were captured as exact matches at first recurrence, versus none of eight recurring non-added controls.
gain_evidence: 17 of 22 at first recurrence, against none of eight recurring controls
problem_reading: (none)
problem_evidence: (none)
quick_read: Researchers analysed 80,295 Turkish WhatsApp utterances from Turkish Airlines' TK Assistant chatbot, collected from January to July 2025. After a Pareto analysis, they added 349 high-scoring utterances that had no verbatim counterpart to ten sub-intent libraries.

Added texts that came back were captured as exact matches far more often: 17 of 22 at first recurrence, against none of eight recurring controls, a 77-point difference. The effect did not spread beyond near-identical forms of the added text.

Overall, the successful utterance capture rate fell from 21.04% to 16.11% over the period. The authors say that decline cannot be separated from the chatbot's concurrent migration to large language model components. Blind annotators judged 87% of exact matches correctly routed versus 98% of Strong-band matches, leading the authors to conclude that verbatim curation buys exact-match efficiency, not understanding.
limitation: The drop in successful capture rate from 21.04% to 16.11% cannot be separated from the concurrent migration to large language model components, so the intervention's effect on overall performance is uncertain.
tag: Evidence-backed gain
key_points: The dataset was 80,295 Turkish WhatsApp utterances from Turkish Airlines' TK Assistant between January and July 2025, scored against a 0.95 success threshold. | 349 Strong-band utterances without a verbatim counterpart were added to ten sub-intent libraries after a bidirectional Pareto analysis. | 17 of 22 recurring added texts became exact matches at first recurrence, versus none of eight recurring controls (difference 77 points, 95% confidence interval 39–90). | Added texts were not promoted organically beyond near-identical forms. | Under stated assumptions, the intervention was followed by 69 additional exact matches, while the successful capture rate fell from 21.04% to 16.11%. | In a blind validation of 300 utterances, exact matches were judged correctly routed 87% of the time versus 98% for Strong-band matches.
rundown: The team applied the Six Sigma DMAIC cycle to utterance capture in a live enterprise chatbot. Assessment combined utterance-level re-capture analysis against non-added controls, expected-versus-actual attribution, Laney p′ charts and blind human validation.

The curation worked narrowly: recurring added texts were matched exactly, but the library did not generalise to similar phrasings. Under the stated assumptions, 69 additional exact matches followed the intervention.

The headline capture rate moved the other way, falling from 21.04% to 16.11%, during a concurrent migration to large language model components. The authors say confidence-based metrics should be secondary improvement targets, since exact-match gains did not mean better routing.
sources:
- peer_reviewed | Applied Sciences | https://doi.org/10.3390/app162211506 | 2026-11-20
prev: 0000000000000000000000000000000000000000000000000000000000000000
SHA-256 hash
9c47d17dc1ef796cac86ca718a0f406d801e52800f01c418c5d84a34d0f747b1
Previous version’s hash
0000000000000000000000000000000000000000000000000000000000000000
Check this record hasn’t been altered
How to check it yourself, without trusting this page

Download any version from /api/record/TRV-2026-3097 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.