mitakshgea1000159 / india-data-formats

Name: mitakshgea1000159/india-data-formats
Author: mitakshgea1000159

Validates, normalizes, formats, masks and generates Indian data with offline Python scripts (PAN, Aadhaar with Verhoeff, GSTIN with checksum, IFSC, UPI IDs, mobile numbers, PIN codes, vehicle numbers, rupee lakh/crore formatting and amount in words, DD/MM/YYYY dates, financial vs assessment year, PII masking). Use when asked to validate a GSTIN or PAN, clean a customer CSV, write an amount in words, find the FY for a date, or mask a ticket before sharing, and for any KYC, invoice or GST work.

Version

SKILL.md

10.9 KB

India Data Formats

Indian identifiers have real structure and checksums. Agents routinely get them wrong: they accept any 12 digits as Aadhaar, write ₹1,234,567 instead of ₹12,34,567, read 05/03/2024 as 3 May, confuse FY with AY, invent GSTIN check characters, and paste full Aadhaar numbers into logs. This skill replaces guessing with deterministic scripts plus a short set of rules.

Core rule: never validate, format or invent an Indian identifier by eye. Run the script. All scripts are pure Python 3.8+, offline, and in scripts/.

When to use

Validating or cleaning user input / CSV / JSON containing PAN, Aadhaar, GSTIN, IFSC, UPI, mobile, PIN code, vehicle number
Writing regex or validation logic for an Indian app (copy the rules below instead of inventing them)
Formatting money for Indian users (invoices, dashboards, cheques, "amount in words")
Anything with dates in Indian context, financial years, assessment years, GST/ITR quarters
Sharing, logging or displaying data that contains Indian PII → mask it first
Creating test data / fixtures / seed rows for an Indian product → generate checksum-valid synthetic values

Workflow

Identify which Indian types are present. If unsure, run scripts/validate.py detect "<text or file>" — it finds and validates every identifier in free text.
Validate each value: scripts/validate.py <type> <value> --json. Trust the script's valid, normalized, errors and details; relay the error reasons to the user verbatim — they are written to be user-facing. For a whole CSV, use scripts/clean_csv.py <file> instead — one pass validates and normalizes every typed column and writes <file>.clean.csv with __status/__note columns (--mask writes a separate shareable <file>.clean.masked.csv, --map col=type for odd headers, --json for counts).
Normalize to the script's normalized form before storing or comparing (upper-case PAN/GSTIN/IFSC, digits-only Aadhaar, E.164 mobile +91XXXXXXXXXX, lower-case UPI handle).
Format money/dates with scripts/inr.py and scripts/dates.py, never by hand.
Mask before anything leaves the system (logs, chat replies, screenshots, exports): scripts/mask.py <file>. Default output shows only the last 4 characters.
Report with the output format below. State clearly what is format-valid vs verified to exist — scripts prove format and checksum only; existence needs an authoritative lookup (NSDL for PAN, GST portal for GSTIN, RBI list for IFSC, India Post for PIN).

Quick reference (what the scripts enforce)

Type	Shape	Hard rules	Normalized
PAN	`AAAAA9999A` (10)	4th char = holder type: P individual, C company, H HUF, F firm/LLP, A AOP, T trust, B BOI, L local authority, J juridical person, G government. 5th = first letter of surname (individuals) or entity name.	upper-case
Aadhaar	12 digits	Never starts with 0 or 1. Last digit is a Verhoeff check digit. Displayed as `XXXX XXXX 1234` only.	digits only
Aadhaar VID	16 digits	Verhoeff; starts 2-9	digits only
GSTIN	15 chars	`SS` state code (01-38, 97, 99) + 10-char PAN + entity no. (1-9/A-Z) + `Z` + mod-36 check char	upper-case
IFSC	11 chars	4-letter bank code + `0` + 6 alnum branch code	upper-case
UPI ID (VPA)	`user@handle`	user: letters/digits/`.`/`_`/`-`; handle alphanumeric (e.g. okaxis, ybl, paytm, upi). No dots in handle (that's an email).	handle lower-case
Mobile	10 digits	Starts 6-9. Accept `+91`, `91`, `0`, spaces, hyphens.	`+91XXXXXXXXXX`
PIN code	6 digits	First digit 1-9 (postal zone); first 2-3 digits map to region	digits
Vehicle	`MH12AB1234` or BH-series `22BH1234AB`	State/UT code must be real; RTO code 2 digits	no spaces, upper

Money: Indian grouping is 3-then-2: 12,34,567.89. 1 lakh = 1,00,000 = 10⁵; 1 crore = 1,00,00,000 = 10⁷; 10 lakh = 1 million; 100 crore = 1 billion. Write ₹ or Rs. before the number. Amount in words uses Lakh/Crore, never Million/Billion, and ends with "Only" on cheques/invoices.

Dates: DD/MM/YYYY. Flag ambiguity when both day and month ≤ 12. Financial year = 1 April → 31 March; FY 2024-25 and FY25 both mean Apr 2024–Mar 2025. Assessment year = FY + 1 (FY 2024-25 → AY 2025-26). FY quarters: Q1 Apr-Jun, Q2 Jul-Sep, Q3 Oct-Dec, Q4 Jan-Mar. Time zone: IST = UTC+05:30, no DST, Asia/Kolkata.

Full structure, state-code tables and edge cases: [references/identifiers.md](references/identifiers.md), [references/currency-and-dates.md](references/currency-and-dates.md).

Scripts

python3 scripts/validate.py pan ABCPE1234F --json        # one value, JSON result
python3 scripts/validate.py gstin "27 AAPFU 0939F 1ZV"   # spaces/case tolerated
python3 scripts/validate.py gstin-check 36AABCT1332L1Z   # compute the 15th (check) character → F
python3 scripts/validate.py detect customers.csv         # find + validate everything in a file
python3 scripts/clean_csv.py customers.csv --mask        # whole-CSV validate+normalize → customers.clean.csv + counts
python3 scripts/inr.py format 1234567.89 --symbol        # ₹12,34,567.89
python3 scripts/inr.py convert "2.5 million" 83          # foreign amount × rate → ₹20,75,00,000 = 20.75 crore
python3 scripts/inr.py parse "Rs. 1.5 Cr"                # 15000000 + words
python3 scripts/inr.py words 250000                      # Two Lakh Fifty Thousand Rupees Only
python3 scripts/inr.py short 123456789                   # ₹12.35 Cr
python3 scripts/dates.py parse 05/03/2024                # 2024-03-05, ambiguity warning
python3 scripts/dates.py fy 31/03/2025                   # FY 2024-25, AY 2025-26, Q4
python3 scripts/dates.py fy-range FY25                   # 2024-04-01 → 2025-03-31
python3 scripts/mask.py export.csv                       # writes export.masked.csv
python3 scripts/mask.py "text with PAN ABCPE1234F"       # masked text
python3 scripts/generate.py row 20 --seed 42             # 20 synthetic customer rows (CSV)
python3 scripts/generate.py gstin 5 --json               # 5 checksum-valid synthetic GSTINs

Exit codes for validate.py: 0 valid, 1 invalid, 2 usage error — usable in CI and shell pipelines.

Output format when reporting to the user

For a single value:

GSTIN 27AAPFU0939F1ZV — format valid ✔
  State 27 Maharashtra · PAN AAPFU0939F (Firm/LLP) · entity 1 · check char V ✔
  Not verified against the GST portal — do that before invoicing.

For a dataset: a table with columns row · field · value (masked) · status · reason, followed by counts (valid / invalid / fixable), the normalized output file path, and any privacy warnings. Invalid PAN/mobile/UPI values may be shown in full when the user needs them to fix the data; Aadhaar is always masked in chat, valid or not — the privacy rules win over convenience, and the full value is in the output file anyway.

Privacy rules (non-negotiable)

Aadhaar: never print, log, store or echo the full 12 digits unless the user explicitly needs it for a legitimate KYC step. Default to XXXX XXXX 1234. The Aadhaar Act (s. 29) and UIDAI circulars restrict display and storage; most businesses have no legal basis to store it.
PAN, mobile, UPI ID, bank account: mask to last 4 in logs, summaries, screenshots, commit messages, test output.
Never fabricate "real-looking" identifiers for examples — use generate.py and label them synthetic.
Never attempt to look up or de-anonymise a person from an identifier.
Under the Digital Personal Data Protection Act 2023, collect only what the purpose needs; if the user is designing a form, ask "what purpose requires this field?" for every identifier (does a newsletter sign-up need PAN?) and recommend dropping the ones with no answer.
Before sharing any dataset/output, run the 5-point checklist at the end of [references/privacy.md](references/privacy.md).

Common agent mistakes this skill prevents

Accepting 123456789012 as Aadhaar (starts with 1, fails Verhoeff).
Inventing the 15th GSTIN character or using ABCDE1234F inside a GSTIN (4th char D is not a holder type).
₹1,234,567 / "1.2 million rupees" in Indian-facing output → should be ₹12,34,567 / "12.35 lakh".
Parsing 05/03/2024 as 3 May.
Writing "FY 2025" when the user means FY 2024-25, or treating AY as the income year.
Calendar quarters (Q1 = Jan-Mar) instead of FY quarters (Q1 = Apr-Jun).
Storing mobile as 9876543210, 09876543210, +919876543210 and 91-98765-43210 in the same column — normalize to E.164.
Regex ^[A-Z]{4}0\d{6}$ for IFSC (branch part may contain letters, e.g. HDFC0CABC01).
Treating name@gmail.com as a UPI ID, or name@okaxis as an email.
Pasting full Aadhaar/PAN into chat output, logs or test fixtures.

Examples

User: "Clean this customer CSV, columns pan, aadhaar, mobile, pincode."
Agent: runs clean_csv.py customers.csv --json, reports the per-column valid/invalid/empty counts and the specific reasons for invalid rows, points to customers.clean.csv (and a --mask copy for sharing), shows Aadhaar/PAN masked in the chat summary, and notes that PIN codes/IFSC/GSTIN are format-checked only.

User: "Convert $2.5 million to crore at ₹83."
Agent: runs inr.py convert "2.5 million" 83 → ₹20,75,00,000 = 20.75 crore. Never assumes an exchange rate — asks or looks it up.

User: "Show revenue of 45600000 in the dashboard."
Agent: runs inr.py short 45600000 → ₹4.56 Cr; full form ₹4,56,00,000.

User: "What's the due date for Q2 GSTR-3B this FY?"
Agent: runs dates.py fy <today> to pin down the FY and FY quarter (Q2 = Jul-Sep), then answers with the filing rule; does not assume Q2 = Apr-Jun.

User: "Give me 10 test users for our KYC form."
Agent: runs generate.py row 10 --seed 1, labels the output as synthetic, never reuses real-looking numbers from the conversation.

See [examples/](examples/) for a messy input file and the expected cleaned output.