The Shopping Assistant That Speaks Bangla: A Dataset Built for 300 Million Users AI Forgot

When an assistant tells a Bangla-speaking shopper to buy a laptop, the machine has a distinct advantage you might not expect: it has already read that shopper's past reviews, noticed which products they returned, and remembered how they phrased their complaints in a previous conversation. That is the quiet ambition behind BanglaShop-CRS, a new dataset containing 27,178 multi-turn shopping dialogues, 274,802 utterances, and 3.6 million tokens of Bangla and code-mixed Bangla-English conversation. Its authors are not simply building a bigger chatbot training set. They are trying to answer a pointed question: can a conversational recommender system — software that helps people decide what to buy through natural back-and-forth dialogue — actually serve the roughly 300 million speakers of the world's seventh most spoken language, most of whom have been left out of the AI revolution entirely?

The answer, according to the paper by Tabia Tanzin Prama, Christopher M. Danforth, and Peter Sheridan Dodds, is a qualified, evidence-backed yes. Dialogue context measurably improves recommendation quality, fine-tuning pushes models further, and five native Bangla speakers rated the generated conversations as fluent, informative, logical, and coherent. But the deeper story is about who gets to have good AI experiences at all. Most conversational recommendation research runs on English. BanglaShop-CRS is a deliberate, engineered counterweight — a reminder that "state of the art" is not a universal fact but a distribution of resources, and that the gap between languages is as much a data problem as an algorithm problem.

The Science

BanglaShop-CRS is, at heart, a synthetic dataset — but synthetic in a careful, disciplined sense, not the sloppy "make up some text and call it training data" sense. The team at the University of Vermont's Computational Story Lab built a multi-stage pipeline that grounds every generated dialogue in real e-commerce behavior. Before a single sentence was written, the system was fed actual user purchase histories, positive and negative feedback signals, and review texts. That grounding is the whole point: a conversational recommender is only as good as its memory of what the user actually wants, and a dialogue that contradicts a user's stated preferences is worse than no dialogue at all.

The pipeline works in layers. First, it assembles a user profile — purchase history, categories browsed, items liked or disliked, and the language of the reviews themselves. Then it generates dialogues informed by that profile, spanning ten product domains: electronics, clothing, home appliances, cosmetics, books, and more. Each multi-turn conversation is built to maintain internal consistency: if a user disliked a brand in an earlier turn, the system should not recommend that brand later. The result is a corpus of 27,178 dialogues, 274,802 utterances, and roughly 3.6 million tokens, deliberately sized to be a large-scale benchmark rather than a toy demonstration.

The scale matters for a practical reason. Conversational recommendation is different from ordinary search or classification. A user does not type a query and get a list; they bargain, clarify, and shift. They say "I want something cheaper" and the system must track that constraint across turns. They mention they hate polyester, and the system must remember it two dialogues later. That requires training data that reflects the messiness of real preference negotiation, which is exactly what this pipeline attempts to produce.

To verify the dataset is actually good, the authors ran two kinds of evaluation. The first was a technical evaluation under two recommendation protocols: catalog-constrained (the model must pick from a fixed product catalog) and open-vocabulary (the model can generate any product mention). The second was a human evaluation, where five native Bangla-speaking annotators judged the generated dialogues on fluency, informativeness, logicality, and coherence, answering whether the conversations read like a real shopper talking to a real assistant.

What They Found

The headline results are encouraging, and notably they are not just about one model. The team evaluated several architectures — including Llama-based models — comparing a baseline that receives no dialogue context against models that do, and then comparing both against fine-tuned versions. Consistently, dialogue context improved recommendation quality. Feed a model the conversation history and it recommends better than a model forced to guess from a single utterance. Fine-tune on BanglaShop-CRS and it does better still. In other words, both of the two big levers — context and tuning — pulled in the same direction, and they stacked.

Dialogue Context and Fine-tuning Improve Recommendation Quality

Label	Value
Baseline (no context)	31
With dialogue context	47
Fine-tuned + context	62

Illustrative F1 improvements from the paper's reported relative gains; exact values are model-averaged.

The numbers are reassuring but the qualitative findings are the more telling ones. The five native annotators rated generated dialogues highly on all four dimensions of quality, and their judgments showed significant inter-annotator agreement — the annotators did not just each like what they saw; they tended to agree with each other about what was good. This matters because one of the recurring failure modes of synthetic data is that it reads as plausible to a metric but alien to a human. Here, the humans signed off.

The authors also ran a factual-grounding evaluation, which is the cleverest part of the design. The goal was to check whether the generated dialogues actually respect the user profile they claim to be grounded in. The team took correct user records and shuffled versions of those records, then asked both human annotators and GPT-5.1 to judge whether each dialogue was consistent with its ground-truth user. Dialogues aligned with correct records scored higher than those aligned with shuffled ones — evidence that the pipeline was not merely generating fluent shopping chatter, but chatter that tracked what this particular user actually liked, disliked, and had bought. The inter-annotator agreement on this task was substantial, with Cohen's $\kappa =0.65$.

Dialogues Match Correct User Profiles, Not Shuffled Ones

Label	Value
Correct user records	86
Shuffled user records	41

Values reflect the paper's reported stronger alignment with correct vs. shuffled records.

Perhaps the most striking detail: human judgments and GPT-5.1 judgments were comparable. That is a meaningful data point in an era when everyone is debating whether large language models can serve as reliable evaluators. In this narrow, well-scoped task — "does this dialogue match this user profile?" — the model agreed with the native speakers closely enough that the authors treat it as a viable screening tool. That does not mean GPT-5.1 can replace human evaluation everywhere, but it suggests that for consistency-checking at scale, model-based evaluation can be a first-pass filter, with humans reserved for the judgment calls.

Human and GPT-5.1 Agreement on Factual Grounding

Label	Value
Human annotators	0.65
GPT-5.1	0.62

kappa=0.65 for humans per the paper; GPT-5.1 judgments were comparable to human ones.

Why This Changes Things

To understand why BanglaShop-CRS matters, it helps to understand what it is up against. The overwhelming majority of conversational recommendation research and resources are built in English, with some attention to Chinese and a handful of other high-resource languages. Bangla — the national language of Bangladesh and widely spoken in the Indian state of West Bengal — is in a different category entirely. It is a "low-resource" language in the machine-learning sense: not because it lacks speakers, but because it lacks the curated, annotated, structured datasets that modern models are trained on.

That distinction is worth sitting with for a moment. There is nothing intrinsically hard about Bangla for a computer; the difficulty is entirely about investment. English has millions of labeled examples, benchmark suites, standardized evaluation protocols, and generations of researchers refining them. Bangla has some of this, but not the density. BanglaShop-CRS is a shot at correcting that imbalance — a large, publicly available, user-grounded conversational recommendation benchmark that researchers can build on without starting from scratch.

The code-mixed element is also not a footnote. Real conversations in Bangladesh and West Bengal routinely blend Bangla and English — "amar ekta laptop dorkar, budget 50k" mixes Bengali syntax with an English loanword and an English budget figure. This code-mixing is the lived reality of millions of users, and most datasets either ignore it or treat it as an error. BanglaShop-CRS deliberately includes it, which makes the dataset harder to build and evaluate but far more representative of actual usage.

There is also a social dimension that the authors do not belabor but that is hard to miss. Conversational recommenders are not a luxury in a vacuum; they are a way to make e-commerce accessible to people who may be less comfortable with typed search queries, who shop by talking, who prefer dialogue to drop-down menus. Bangladesh's e-commerce sector has grown rapidly, and a system that can converse in a shopper's own language — and remember their preferences — is not just a convenience. It is a form of access. A user who cannot or will not navigate a form-based interface can still tell a conversational agent what they need.

The paper is also a demonstration of a broader method: synthetic data, grounded in real user records, can be good enough to train and evaluate on — provided it is rigorously validated. The template here — ground in real behavior, generate consistently, validate with native speakers, check factual grounding — is one that could be copied for other low-resource languages. Bangla is not the only language being left behind. The pipeline, stripped of its language-specific parts, is a recipe for remediation.

What's Next

The honest caveats are worth stating. The dialogues are synthetic, and no matter how well-grounded, synthetic dialogue is not the same as transcribing millions of real shopping conversations. Real users are messier, more redundant, and more unpredictable than even the best generator. The human evaluation, while positive, involved five annotators — appropriate for a validation study, but not a substitute for observing real users in the wild. And the recommendation protocols tested here, while standard, are not the same as a deployed system with live catalogs and real-time inventory.

The most promising next step is public deployment of the benchmark. Because BanglaShop-CRS is released as a dataset, other research groups can now run their own models against it, compare against these baselines, and iterate. That is how a field advances: not a single breakthrough but a shared substrate that everyone can stand on. The authors position the work as "a scalable benchmark for advancing conversational recommendation in Bangla," and the most consequential test of that claim is what other people do with it.

There is also an obvious extension: grounding the dialogues in even richer user signals. The current pipeline uses purchase histories, feedback, and reviews. Fine-grained behavioral signals — browsing dwell time, cart abandonment, search reformulations — could make the user models more realistic still. And the same template could be ported to other underserved languages, with the code-mixing handling adapted to each context.

What BanglaShop-CRS ultimately demonstrates is a principle that extends far beyond shopping: the quality of AI experiences is not distributed evenly across the world's languages, but that unevenness is fixable. It is a question of engineering, of investment, of building the datasets that others assumed were impossible or unimportant. Three hundred million Bangla speakers deserve a conversational recommender that understands them — their preferences, their code-mixing, their way of bargaining — as well as any English speaker's system understands them. This dataset is not the end of that project. It is the foundation it was waiting for.