Card-Sense
Overview
A chat-based RAG-powered credit card intelligence assistant - because 100 million Indians shouldn't need a law degree to understand their own rewards.
The Problem
India has 100M+ active credit card users. Reward structures are buried in 40-page MITC PDFs written in legal language. Someone with 3 cards from 3 different banks has no single place to ask: "Which card should I use for this purchase?"
Every existing solution - Reddit, bank websites, financial influencers - is either outdated, biased, or leaves interpretation to the user.
Complexity overload
3 cards × 6 categories = 18 reward rules. Impossible to memorize at the point of purchase.
Generic, ungrounded results
Searches return advice not grounded in the user's specific card portfolio or current T&C.
No natural language interface
MITC PDFs are 40+ pages of legal language with no way to ask a plain question.
Asterisks misread
"Up to 5X points", on what exactly? Conditional clauses routinely get lost.
Decision fatigue
When clarity fails, users default-swipe. Value is lost, not recovered.
Current Workflow
The primary persona is Arjun - a salaried professional in his mid-20s with 3 credit cards across 3 banks. He uses them interchangeably without a clear strategy, and suspects he's leaving reward value on the table every month.
- A purchase moment arises - booking flights, ordering food
- Fails to recall which card is best for this category
- Searches Google, Reddit, bank websites - gets generic, outdated, or biased results
- Attempts to find the actual MITC PDF
- Gets a partial answer buried in legal language, misinterprets an asterisk
- Gives up and swipes the default card - unresolved, value lost not recovered
Reward structures are too complex to memorize, results are generic rather than grounded in the user's portfolio, and decision fatigue ends in a default swipe every time.
Why RAG Fits
Each product requirement maps directly to a RAG capability.
| Requirement | Why RAG fits |
|---|---|
| Answers must cite official source | RAG retrieves from actual MITC documents, not model memory |
| Multi-card comparison | Cross-document retrieval across banks within a single query |
| T&C language is complex | LLM interprets dense legal text; retrieval grounds it in the source |
| Documents update periodically | Corpus can be re-indexed without retraining the model |
| Can't hallucinate reward rates | Retrieval-first architecture prevents ungrounded claims |
Use Cases
Use cases are sequenced by frequency of need and risk of getting wrong. Negative cases are zero-tolerance from day one and not a later phase.
P0 - Category Optimization
As a salaried employee with multiple cards across different banks, I want to know which card gives the best return on a specific spend category, so that I stop using cards interchangeably and actually maximize rewards.
Happy path: "Which of my cards is best for dining?" → Retrieves reward chunks for dining from each MITC, normalizes to effective cashback %, returns a ranked answer with inline citations.
Edge case: Cards use different reward units - "5X reward points" vs "2% cashback." System normalizes to a common unit before ranking. Incompatible figures are never presented side by side.
P0 - Benefit Discovery
As a salaried employee unsure of what my cards actually offer, I want to ask plain-language questions about my existing card benefits, so that I stop missing entitlements I am already paying an annual fee for.
Happy path: "Do I get lounge access with my HDFC Millennia?" → Retrieves the lounge access section, returns a direct yes/no with entitlement details and conditions cited.
Edge case: Benefit is conditional - "yes, but only if you spend ₹1L in the previous quarter." The system surfaces the condition alongside the benefit. It never returns just the headline.
P1 - Cross-Card Comparison
As a salaried employee with cards across 3 banks, I want a side-by-side comparison of my cards for a specific use case, so that I make an informed decision instead of guessing at the point of purchase.
Happy path: "Compare Regalia vs Axis Atlas for international travel." → Parallel retrieval for both cards surfaces forex markup, lounge access, and travel rewards from each MITC in a structured comparison.
Edge case: One card is not in the corpus. System presents what it has, explicitly flags the missing card. Incomplete data is never compared silently.
Zero Tolerance - Account Data Queries
As a user, I should NOT get answers about my credit limit, balance, credit score, or transaction history, so that the product never creates the impression it has access to my bank account.
Response: "CardSense only answers questions based on official card documents and does not have access to your account. Please contact your bank directly."
Zero Tolerance - Financial Advice & Eligibility Queries
As a user, I should NOT receive personalized financial advice ("should I close this card," "will I get approved for Diners Black"), so that the product never enters regulated advisory territory.
Response: The system presents what the MITC says about eligibility, then frames the decision back: "Here is what the official document says. The decision of whether to apply is yours to make."
Technical Architecture
Every component is chosen for the specific constraints of MITC documents - dense tables, image-rendered fee grids, conditional clauses that span multiple chunks.
Data sources & ingestion. MITC PDFs from multiple banks (HDFC, Axis, SBI, ICICI), 30–50 pages each, covering fee schedules, reward structures, eligibility clauses, and benefit tables. Manually downloaded from official bank websites, versioned and date-stamped at ingestion. Reward rate tables and fee grids are frequently image-rendered in Indian bank PDFs, so AWS Textract's table extraction API preserves row-column structure.
Chunking Strategy
| Content type | Strategy | Chunk size |
|---|---|---|
| Prose clauses (T&C, definitions) | Fixed-size | 512 tokens, 50 overlap |
| Short definitions & FAQs | Sentence-level | 1–3 sentences |
| Tables (fee & reward) | Table-aware | Whole table as one chunk |
Splitting a table mid-row destroys meaning, so fee and reward tables are always treated as atomic units (Textract's table API runs at roughly ₹0.60/document extra).
Embedding & LLM Choice
| Component | Choice |
|---|---|
| Embedding | text-embedding-3-small |
| Retrieval | Hybrid semantic + BM25 |
| Reranker | Cohere Rerank |
| LLM (simple) | GPT-4.1 Nano |
| LLM (complex) | GPT-5.4 |
Context Window Budget
(5 × 512)
(2 turns)
Observability Plan
| Layer | What is logged |
|---|---|
| Query | Raw query, timestamp, session ID |
| Retrieval | Top-K chunk IDs, similarity scores, reranker scores, metadata filters |
| Generation | Final prompt, token count, response, latency |
| Response | Citation accuracy flag, user feedback |
| Alerts | Low retrieval score, empty chunk returns, LLM timeout |
RAG Failure Analysis
Access control is not a failure point for Card Sense. The entire corpus is publicly available in MITC documents published under RBI's mandatory disclosure requirements. Card Sense holds no user-specific financial data, no account access, no transaction history, no personal information beyond what the user self-discloses in chat.
| Failure layer | Mitigation |
|---|---|
| Data quality / OCR | Remove repeated headers before chunking |
| Chunking strategy | Semantic chunking with clause-boundary preservation |
| Embedding quality | Query expansion using domain synonyms |
| Search / retrieval | Dynamic top-K (higher for comparisons) |
| Re-ranking | Enforce per-card result quota |
| Prompt / augmentation | Prioritize most relevant chunk in prompt |
| Model usage | Two-tier model routing |
| Data drift | Monthly refresh with change detection |
| User behavior | Clarification prompts for short queries |
Guardrails & Escalation
Low retrieval confidence doesn't produce a hedged answer. It produces a hard stop and a direct link to the source. CardSense never compares incomplete data silently or serves stale documents without warning.
| Trigger | Response |
|---|---|
| Retrieval confidence < 0.55 | No answer attempted. Direct link to the relevant bank's MITC page. |
| Out-of-scope query | Matter-of-fact rejection. Bank customer care link appended. |
| Conflicting document versions | Both versions surfaced with dates. Conflict flagged explicitly. |
| Citation older than 180 days | Staleness warning renders in amber on every citation tag. |
CardSense answers are based on official MITC documents. Benefits may have changed. Always verify with your bank before making a financial decision.
Source Citation Format
Every factual claim is cited inline: card name, document section, and document date are always shown. Citations are expandable to reveal the exact retrieved chunk. Example: "HDFC Regalia charges a 2% forex markup [HDFC Regalia MITC · Fees & Charges · Jan 2025], while Axis Atlas charges 2% as well [Axis Atlas MITC · Foreign Currency Transactions · Nov 2024]."
Success Metrics
| Retrieval Quality | Target |
|---|---|
| Precision@5 | ≥80% |
| Recall@5 | ≥70% |
| Mean Reciprocal Rank | ≥0.75 |
| Card coverage rate | 100% |
| Generation Quality | Target |
|---|---|
| Faithfulness | ≥95% |
| Answer relevance (1–5) | ≥4.0 |
| Citation accuracy | ≥90% |
| Hallucination rate | ≤5% |
| Business Metrics | Target |
|---|---|
| Session completion rate | ≥75% |
| Query resolution rate | ≥70% |
| Time-to-answer (p50) | ≤4s |
| User satisfaction (thumbs up) | ≥80% |
| D7 retention | ≥30% |
| Escalation rate | ≤15% |
Out-of-scope responses (credit score, loans, eligibility) 0%, fabricated card information 0%, hallucinated reward rate 0%, stale data served without warning ≤1%.
Evaluation Method
LLM-as-a-Judge runs on 100% of responses in staging and 10% in production, scoring faithfulness, relevance, and citation accuracy with structured JSON output. A weekly human audit covers the 20 lowest-rated responses and a random 20 high-confidence retrievals, involving credit card experts and Card Sense users. The combination provides scale from automation and ground-truth from human review.
Token Economics & Cost
Two-tier model routing is a deliberate cost architecture. Simple lookups - 60% of all queries - go to GPT-4.1 Nano at ₹0.13 per query. Only complex synthesis goes to GPT-5.4.
| Path | Cost per query | Use case |
|---|---|---|
| Simple (60%) - GPT-4.1 Nano | ~$0.0016 (₹0.13) | Fee lookups, yes/no benefits, exact-match rates |
| Complex (40%) - GPT-5.4 | ~$0.027 (₹2.29) | Multi-card comparison, milestone synthesis, reward normalization |
Cost Per User Per Month (Blended)
| User type | Queries/mo | Blended cost |
|---|---|---|
| Casual | 6 | ₹5.96 |
| Regular | 25 | ₹24.85 |
| Power | 80 | ₹79.52 |
Pricing Model
The free tier is an acquisition cost, sized to demonstrate value without covering infrastructure. Standard margin of ~₹74/month leaves room for customer support and iteration.
| Tier | Price/month | Query limit |
|---|---|---|
| Free | ₹0 | 15 queries - acquisition cost |
| Standard | ₹99 | 100 queries - ~₹74 margin |
| Pro | ₹249 | Unlimited - ~₹170 margin at avg usage |
UX & Feedback Loop
Chat interface was chosen because queries are conversational and multi-part, follow-up questions flow naturally without re-entering context, and inline citation design works natively in chat.
Feedback Mechanism
Thumbs up signals high retrieval accuracy and correct synthesis. Thumbs down opens a mandatory 3-option reason picker: Wrong Information (factually incorrect), Incomplete (missing key details, e.g. forgot to mention the cap), or Misunderstood (failed to interpret intent).
When thumbs-down feedback clusters, the system triggers specific reviews:
By specific card
Negative feedback spike on one card flags that MITC PDF for re-ingestion review - checks for outdated data or parsing errors.
By simple queries
Short questions failing triggers a clarification threshold adjustment to improve the retriever's confidence scores.
By comparisons
Complex queries failing initiates a card coverage metric review to check for missing cross-bank context.
Launch Phases
| Phase | Scope | Exit criteria |
|---|---|---|
| Alpha (Weeks 1–4) | 4 banks, 20–30 invited users, all responses manually reviewed | Hallucination < 5%, Precision@5 ≥ 75%, zero policy violations |
| Beta (Weeks 5–10) | 6 banks, 200–500 waitlist users, LLM-as-a-Judge on 100% of responses | Session completion ≥ 70%, satisfaction ≥ 75%, escalation ≤ 20%, two-tier routing live |
| GA (Week 11+) | Full corpus, freemium pricing active, full observability stack, monthly freshness pipeline | D7 retention ≥ 30%, Standard conversion ≥ 10%, hallucination < 5%, query resolution ≥ 70% |
Launch Risks
Zero-tolerance guardrail + LLM-as-a-Judge on all Alpha/Beta responses + inline citations make ungrounded claims immediately visible. Response protocol: public correction post with specific query, failure reason, and fix applied.
Document date shown on every citation. Staleness warning at 180 days. Weekly re-ingestion in Beta catches this early.
System prompt instructs the LLM to present information, not make recommendations. Framing hardcoded: "Here's what each card offers - the decision is yours."
Persistent footer on every response reminds users that benefits may have changed and to verify with their bank before making a financial decision.
Future Scope: Fine-Tuning & Agentic RAG
Two signals from production data would trigger a fine-tuning evaluation: persistent retrieval misses on Indian fintech terminology ("milestone benefit," "forex markup waiver") despite query expansion, and consistent reward unit normalization errors across banks. Both are repeatable domain-specific tasks where a fine-tuned model outperforms prompt engineering at scale.
Approach: A hybrid RAG architecture using PEFT/LoRA to fine-tune a Mistral 7B or Llama 3 8B model, trained on a distillation dataset of GPT-5's high-benchmark responses and 500 synthetic Card Sense queries, to handle the high-volume "Simple Query" tier. Hosting the fine-tuned adapter on Together AI achieves sub-second latency and cost reduction, leaving GPT-5.4 to focus exclusively on complex, multi-card synthesis.
Agentic RAG Roadmap
| Version | Capability |
|---|---|
| V2 | Live document fetching |
| V3 | Personal spend context |
| V4 | Action layer |