Truace
Subscribe

Pilot study finds tailored prompts improve AI-written business market research

Study title: Advancing value creation in B2B market research – a human–AI collaboration framework for large language model

Researchers tested ChatGPT, Gemini and Perplexity on three B2B market research tasks using basic, context-augmented and role-specific prompts, producing 81 outputs. A 27-output subset was scored with automated metrics and blinded human raters.

Read the study abstract

Purpose This study aims to examine how prompt design influences the quality of large language model (LLM)-generated business-to-business (B2B) market research outputs. It also proposes a switch-point governance mechanism for converting LLM-generated drafts into decision-ready market intelligence artefacts. Design/methodology/approach The study uses an exploratory pilot evaluation of three market-facing LLM-based tools: ChatGPT, Gemini and Perplexity AI. The tools are compared across three benchmark B2B market re…

By the Truvace desk1 min read1 sourceEvidence: Emerging · 48/100 — Benchmark or lab evaluation · n = 81 · 1 independent sourcePeer-reviewedTRV-2026-2305· cite & verify
Advancing value creation in B2B market research – a human–AI collaboration framework for large language model
Biofuels Bioprod Bioref - 2024 - Elias - Market analysis of coupled biochar and carbon credit production from wildfire fuel reduction projects in the western USA by Micah Elias, Daniel L. Sanchez, Phil Saksa, Josiah Hunt, Jonathan Remucal. CC BY 4.0 · https://creativecommons.org/licenses/by/4.0

In brief

The work matters because companies increasingly draft market intelligence with chatbots, and the study offers a way to screen drafts before they inform decisions. The evidence is preliminary: the task set is tiny, and the statistical tests were exploratory rather than confirmatory.

Main points

  1. 81 outputs were generated across three tools, three tasks and three prompt types.
  2. A role-specific subset of 27 outputs received automated and blinded human scoring.
  3. ChatGPT drafts were relevant but longer and less readable; Perplexity's were concise but less aligned with the brief.
  4. Cross-run semantic stability aligned directionally with higher decision-readiness ratings.
  5. A switch-point matrix maps confidence signals and human judgement to accept, judge, review/redo or rewrite.

The gain

In a small pilot, role-specific prompts produced the strongest descriptive scores for AI-drafted B2B market research, and Gemini showed the best observed profile on relevance, stability and speed.

The rundown

The tools were compared on relevance via cosine similarity, reference overlap via ROUGE-F1, cross-run stability, latency, token length and readability. Human raters judged relevance, completeness, factuality, instruction-following and insightfulness, and their ratings followed a similar descriptive pattern.

Beyond the comparison, the authors propose a governance mechanism in which LLM outputs are treated as candidate value propositions rather than finished knowledge. Teams would combine automated checks with human evaluation and use transparent escalation rules to allocate decision rights.

The authors say the framework could reduce the risk that persuasive but weakly grounded claims reach managerial decisions, and they call for human oversight, evidence checks and traceable governance.

Sources

  1. Peer-reviewedJournal of Business & Industrial Marketing2026-10-28

The debate