Computer Vision / OCR Fintech 2 Semesters

🧾 Receipt OCR Expense Tracker — Benchmarking OCR Engines

Snap a receipt, get your spending tracked automatically. Malaysian receipts are a brutal OCR test — thermal print that fades, bilingual item names, crumpled paper. Which OCR engine survives that? That comparison is your research.

1The Pitch — Product + Research in One

🛠️ What you build (the demo)

A web/mobile app: photograph a receipt → merchant, date, items, and total extracted automatically → monthly spending dashboard with categories (food, transport, groceries).

🔬 What you research (the grade)

Benchmark three OCR engines on real Malaysian receipts, measure how badly real-world photo conditions hurt accuracy, and quantify how much a preprocessing pipeline recovers.

2Problem Statement & Objectives

Pernyataan masalah: Manual expense tracking fails because typing every receipt is tedious; existing OCR tools are tuned for clean documents, not faded bilingual thermal receipts photographed with a phone in bad lighting.

Build a receipt-scanning expense tracker with automatic field extraction (the MVP).
Benchmark three OCR engines — Tesseract, PaddleOCR, and a cloud vision API — on a self-collected Malaysian receipt dataset.
Evaluate the impact of image preprocessing on extraction accuracy for the best-performing engine.

3Benchmark Design — the 3 Experiments

Experiment	Setup	What it proves
Eksperimen 1 Clean scans	All 3 engines on flat, well-lit receipt scans — the controlled ceiling.	Best-case accuracy per engine; the baseline comparison table.
Eksperimen 2 Real-world photos	Same engines on phone photos: crumpled, faded, angled, dim lighting.	The robustness gap — how much real conditions degrade each engine.
Eksperimen 3 Preprocessing pipeline	Best engine from E2, with vs without deskew + binarization + contrast enhancement.	Whether classic CV preprocessing recovers the lost accuracy — an actionable engineering finding.

📦 Dataset plan

Self-collect 200–400 Malaysian receipts (mamak, convenience stores, petrol, groceries) under a documented photo protocol; annotate ground truth for merchant, date, total, and line items. The bilingual (Malay/English) item names give the dataset a local-contribution angle no public dataset has.

4Evaluation Metrics

Metric	Why it matters here
Character Error Rate (CER)	Standard OCR quality measure — the per-engine headline number.
Field extraction accuracy	What users feel: did merchant, date, and total come out right? Scored per field.
Total-amount exact match	The money field is the one that must be perfect — reported separately.
Processing time per receipt	Cloud API vs local engine trade-off: accuracy vs latency vs cost per 1,000 receipts.

5Report Mapping (Bab 1–5)

Chapter	What goes in it (from this package)
Bab 1 — Pengenalan	Problem statement, objectives, scope (receipt types, 3 engines, extraction fields), Gantt chart.
Bab 2 — Sorotan Kesusasteraan	Comparison tables: OCR engine studies, receipt/document understanding papers, preprocessing technique comparisons.
Bab 3 — Metodologi	Receipt collection protocol, annotation guideline, preprocessing pipeline design, app architecture (camera → OCR → parser → dashboard).
Bab 4 — Pembangunan & Pengujian	The 3 experiments, CER/field-accuracy tables, degradation analysis, before/after preprocessing comparison, app screenshots.
Bab 5 — Dapatan Kajian	Recommended engine + pipeline with evidence, limitations (receipt variety, handwriting), future work (line-item categorization with an LLM).

6Tech Stack

PythonTesseractPaddleOCR OpenCVFlask / FastAPIChart.js

🔎 Related works — Perplexity API placeholder

Live literature search (receipt OCR and document understanding papers, 2020–2026) will be generated here via the Perplexity API once the subscription is active.

⏳ Generate Literature Review (coming soon)

← Back to all FYP ideas · Assignment Helper