Architecting TonuAI: High-Throughput E-Commerce RAG & Conversational Bangla AI Engine

When building conversational AI for specialized retail domains, traditional vector embedding search often suffers from high latency, vocabulary misalignment, and poor handling of regional dialects. TonuAI stands as Bangladesh's first specialized E-Commerce RAG (Retrieval-Augmented Generation) engine and AI skincare advisor, designed specifically for Tonu's Shop (tonusshop.com), a leading skincare platform in Bangladesh.

In this technical breakdown, we explore how TonuAI achieves sub-second context retrieval, multi-intent user prompt resolution, automated catalog parsing from MySQL dumps, and natural conversational Bangla interaction through its AI persona, Ira (ঈরা).

1. High-Level Architecture Overview

TonuAI consists of two primary operational layers:

1. Python Ingestion & Matching Pipeline (Data Layer):
   • Parses unstructured markdown product dossiers (Products Details.md, ENGLISH SEO BLOG CONTENT.md).
   • Scans and extracts live pricing, stock, and SKU data directly from MySQL database dumps (tonusshop_web.sql) via a custom character-by-character scanner.
   • Executes multi-source greedy pair matching to generate a unified, idempotent JSON product database (products_db.json).
   • Runs image-to-catalog matching (build_gallery.py) to map real stock photographs to products with form-guard rules.
2. Next.js RAG Engine & Conversational Interface (App Layer):
   • Implements multi-intent prompt segmentation (splitIntoSegments).
   • Performs instant, non-embedding multi-lingual keyword scoring (Bangla, Banglish, English) across product categories, active ingredients, skin concerns, and brand names.
   • Stream-renders responses with a resilient, custom Markdown renderer that handles raw URLs, inline product images, lightbox zoom, and inline customer profile capture markers.

2. Multi-Source Ingestion & Data Pipeline

E-commerce catalogs are notoriously fragmented across structured SQL tables and unstructured marketing content. parse_data.py merges three distinct data sources without manual intervention:

Hardened Greedy Pair Matching

To combine unstructured markdown with MySQL rows without duplicate assignment or cross-brand contamination, TonuAI utilizes a scored greedy matching algorithm combining slug similarity, token overlap, and brand veto protection:

• Brand Guard: get_brand() enforces a strict brand veto to prevent cross-brand pairing (e.g., matching a COSRX product description with a Beauty of Joseon SQL record).
• Slug Normalization: normalize_key() strips units, pack sizes, and non-alphanumeric noise to build resilient matching keys.

Form-Guarded Product Gallery Builder (build_gallery.py)

Photographs of actual inventory are automatically matched to catalog items while avoiding form mismatches:

• Form Guard Enforcement: Prevents assigning a Power Moisture Serum photo to a Daily Moisture Cream despite high token overlap.
• Deduplication: Computes SHA-1 hashes across image files to drop byte-identical duplicates automatically.

3. High-Performance Keyword RAG Engine

Rather than incurring 200ms+ vector embedding lookup latencies, TonuAI executes a specialized multi-lingual keyword scoring algorithm directly in Node.js:

// Multi-lingual Keyword Scoring Table (Bangla, Banglish & English)
const SCORE_WEIGHTS = {
  BRAND_MATCH: 35,
  SKIN_TYPE_MATCH: 30,
  CATEGORY_MATCH: 25,
  CONCERN_MATCH: 20,
  NAME_WORD_MATCH: 15,
  INGREDIENT_MATCH: 5
};

Multi-Intent Prompt Segmentation

Customers frequently submit compound queries in a single message (e.g., "Which cleanser is good for oily skin, what is the price, and do you deliver to Chittagong?").

TonuAI's splitIntoSegments() engine:

1. Deconstructs the message into distinct intent segments (Product Advice vs. Price Check vs. Shipping Policy).
2. Executes keyword retrieval independently per segment.
3. Unions the candidate sets to select the top 3–4 most relevant products dynamically.

4. AI Persona & Conversational Intelligence: Ira (ঈরা)

As Bangladesh's first specialized skincare AI, **Ira (ঈরা)** acts as an empathetic, expert Bangladeshi skincare advisor.

Key System Prompt Rules:

• Language: Speaks in warm, conversational Bangla by default (e.g., using "আপনার ত্বক কেমন?" rather than medical jargon).
• Strict Scope Constraint: Off-topic queries (general programming, politics, non-skincare items) are politely declined.
• Model-Driven Profile Capture: Ira organically extracts the customer's name and phone number during conversation by appending unobtrusive state markers:

  [[NAME: Sadia Rahman]] [[PHONE: 01700000000]]

  The API route interceptor extracts these markers to update live admin support sessions in real time before stripping them from the user-facing render stream.

5. Summary & System Metrics

TonuAI demonstrates how domain-specific, first-principles software architecture can pioneer regional AI innovation:

• Milestone: Bangladesh's First Conversational Bangla AI Skincare Advisor & RAG Engine.
• Retrieved Product Database: 55+ Verified Active Products & 30+ Mapped Real-Stock Photos.
• Retrieval Latency: < 5ms (Keyword RAG Engine).
• Supported Languages: Bangla, Banglish, English.
• Admin Dashboard: Real-Time Live Support Session Tracking (/admin).