1 line
22 KiB
JSON
1 line
22 KiB
JSON
{"repo": "000shreeharish000/audit-analytics-system", "files": {"README.md": {"9b72b4396897c5059a4d2ceeb5929d0a930d121c": {"t": 1785596458.2924278, "n": 20415, "h": "c70b55ec5fa925fd", "c": "# Decision & Financial Digital Twin Platform\n\n> **Enterprise-grade AI-powered fraud detection and compliance governance system** — built on graph intelligence, multi-agent orchestration, and cryptographic audit trails.\n\n---\n\n## 📌 Image Reference Guide\n\n| Image | Where to Place | Why |\n|-------|---------------|-----|\n| `Delliote_mindmap_2.png` — *Audit Analytics Architecture with GenW.AI* | **After the Architecture Overview section** | Best technical diagram — shows full 3-layer stack (AppMaker → GenW Agent Builder → Visualization) cleanly |\n| `Delliote_mindmap_1.png` — *POC Development Approach* | **After the Multi-Agent System section** | Shows agent coordination and data flow between all agents including Coordinating Agent and Realm AI |\n| `Agent_Builder.png` — *Intelligence Layer — GenW Agent Builder* | **After the Agent Pipeline section** | Detailed internals of the 5-agent sequence with orchestration logic |\n| `App_Maker.png` — *Data Ingestion Layer — AppMaker* | **After the Data Ingestion section** | Shows ERP → SQL → Data Connectors → Rule-Based Agents flow |\n\n> **Recommendation**: Use `Delliote_mindmap_2.png` as the **hero/primary architecture diagram** right below the intro — it's the clearest, most technical, and most appropriate for a README.\n\n---\n\n## Table of Contents\n\n- [Overview](#overview)\n- [Architecture](#architecture)\n- [Technology Stack](#technology-stack)\n- [Core Modules](#core-modules)\n - [Data Ingestion & Validation](#data-ingestion--validation)\n - [Graph Intelligence Engine](#graph-intelligence-engine)\n - [Governance Rule Engine](#governance-rule-engine)\n - [Risk Scoring & Trust Engine](#risk-scoring--trust-engine)\n - [Multi-Agent AI Orchestration](#multi-agent-ai-orchestration)\n - [Case Management & Investigation](#case-management--investigation)\n - [Auditor Integrity & Conflict Detection](#auditor-integrity--conflict-detection)\n - [Policy & Compliance Governance](#policy--compliance-governance)\n - [Security Infrastructure](#security-infrastructure)\n - [Persistence & Disaster Recovery](#persistence--disaster-recovery)\n- [Frontend Modules](#frontend-modules)\n- [API Reference](#api-reference)\n- [Authentication & RBAC](#authentication--rbac)\n- [Deployment](#deployment)\n- [Environment Variables](#environment-variables)\n- [Feature Summary](#feature-summary)\n\n---\n\n## Overview\n\nThe **Decision & Financial Digital Twin Platform** is a production-grade system for detecting financial fraud, governance violations, and compliance breaches in enterprise environments. It ingests ERP data (vendors, invoices, approvals, payments), constructs a live graph-based digital twin of financial relationships, and applies deterministic AI reasoning to surface fraud patterns, generate audit cases, and support investigator workflows.\n\n**Key capabilities:**\n- Real-time graph construction from enterprise ERP data (employees, vendors, invoices, approvals, payments)\n- 6 built-in fraud detection rules with mathematical confidence scoring\n- 4-agent AI analysis panel with air-gapped and hybrid modes\n- Blockchain-style append-only audit chain with HMAC signatures\n- Role-based dashboards for admin, auditor, and risk analyst personas\n- 3D interactive graph explorer with risk-highlighted nodes\n- Zero-randomness deterministic AI engine — no external calls required\n\n---\n\n## Architecture\n\n\n\n*Audit Analytics Architecture with GenW.AI - Three-layer system: AppMaker Data Ingestion -> GenW Agent Builder -> Visualization Layer.*\n\nThe platform is structured across three layers:\n\n```\n┌─────────────────────────────────────────────────────────────┐\n│ LAYER 1: AppMaker Data Ingestion │\n│ ERP Databases / Logs & Records / REST APIs │\n│ → Data Validation · SQL Queries · JS Bindings │\n└──────────────────────────┬──────────────────────────────────┘\n │\n┌──────────────────────────▼──────────────────────────────────┐\n│ LAYER 2: GenW Agent Builder │\n│ Start Node │\n│ → Data Validation Agent │\n│ → Relationship Graph Agent │\n│ → Compliance & Anomaly Agents │\n│ → Orchestration Layer │\n│ → Risk Scoring & Condition Nodes │\n│ → Decision Pathway Analysis Agent │\n└──────────────────────────┬──────────────────────────────────┘\n │\n┌──────────────────────────▼──────────────────────────────────┐\n│ LAYER 3: Visualization Layer │\n│ GenW Playground Dashboards · Realm AI LLM Assistant │\n│ → Audit Report Generation │\n└─────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## Technology Stack\n\n| Layer | Technology |\n|-------|-----------|\n| Backend API | Python · FastAPI |\n| Graph Engine | NetworkX MultiDiGraph |\n| Database | SQLite with WAL mode (14-table schema) |\n| Encryption | Fernet (AES-128 CBC + HMAC) · PBKDF2-HMAC-SHA256 |\n| Audit Chain | SHA256 chained hashing · HMAC-SHA256 signatures |\n| Frontend | Next.js · React · Zustand |\n| 3D Visualization | Three.js |\n| Animation | Framer Motion |\n| PDF Export | jsPDF + html2canvas |\n| Auth | JWT (HS256) with JTI, 60-min expiry |\n| External AI (optional) | OpenAI-compatible API (air-gapped by default) |\n\n---\n\n## Core Modules\n\n### Data Ingestion & Validation\n\n\n\n*AppMaker Data Ingestion Layer - SQL queries fetch ERP data through Data Connectors into Rule-Based Agents.*\n\nEnterprise ERP data is ingested as a JSON payload containing employees, vendors, invoices, approvals, payments, and social relationships.\n\n**AnomalyGuard Validation** runs an 8-step pipeline on every ingestion:\n\n1. Company ID check\n2. Size limits (50,000 records per entity type)\n3. Duplicate ID detection (Counter-based)\n4. Referential integrity: vendors → employees\n5. Referential integrity: invoices → vendors\n6. Referential integrity: approvals → targets\n7. Referential integrity: payments → invoices/vendors/employees\n8. Relationship nodes → known nodes\n\nData sources are tracked (`api_upload` vs `local_dataset`). A pre-loaded simulated enterprise dataset is available for instant demos.\n\n---\n\n### Graph Intelligence Engine\n\nA **NetworkX MultiDiGraph** is constructed from the validated dataset.\n\n**Node types:** Employee · Vendor · Invoice · Approval · Payment · Decision\n\n**Edge types:**\n\n| Edge | Meaning |\n|------|---------|\n| `CREATED_VENDOR` | Employee created vendor record |\n| `APPROVED_VENDOR` | Employee approved vendor |\n| `ISSUED_INVOICE` | Vendor issued invoice |\n| `EXECUTED_PAYMENT` | Payment executed against invoice |\n| `DECISION_LINK` | Connects decision audit trail nodes |\n| `TRIGGERED_RULE` | Rule triggered on entity |\n| `PART_OF_CASE` | Entity belongs to investigation case |\n\nSocial relationship edges (`KNOWS`, `REFERRED_BY`, `FAMILY`, `EX_COLLEAGUE`, `SOCIAL`) are mapped bidirectionally with confidence scores to enable collusion path detection.\n\n**Control Bypass Detection** (via `GraphReasoner`):\n- **Single-actor bypass** — set intersection finds same person across all control steps\n- **Collusive chain** — Counter frequency finds 2–3 actors sharing multiple roles\n\nGraph density is computed as `2 × edges / (nodes × (nodes − 1))` and used as a system health indicator.\n\n---\n\n### Governance Rule Engine\n\nSix deterministic fraud detection rules are built in:\n\n| Rule ID | Detection Pattern | Confidence | Severity |\n|---------|------------------|-----------|---------|\n| `RULE_VENDOR_SELF_APPROVAL` | `approved_by == created_by` | 0.90 | 0.95 |\n| `RULE_JUST_BELOW_THRESHOLD` | ≥2 approvals in 90–100% of threshold | 0.82 | — |\n| `RULE_HIGH_VALUE_LOW_APPROVAL` | Invoice ≥ threshold with fewer approvals than required | 0.86 | — |\n| `RULE_RAPID_APPROVAL_CHAIN` | Invoice→payment ≤ 8 hours, ≥2 rapid events | 0.77 | — |\n| `RULE_INVOICE_SPLITTING` | Small invoices from same vendor within 2 days totaling ≥150% of threshold | 0.84 | — |\n| `RULE_ROLE_COLLISION` | ≤2 unique actors control creation + approval + payment | 0.83 | — |\n\nAdditional rules are extracted automatically from uploaded company/government policy documents.\n\n**What-If Simulation:** The `/rules/simulate` endpoint lets you change thresholds hypothetically and observe impact on rule results without committing changes.\n\n---\n\n### Risk Scoring & Trust Engine\n\n**Rule-level risk score formula:**\n\n```\nscore = base_risk\n + (confidence × 20)\n + (severity × 20)\n + amount_factor # capped at 15pts ($2M)\n + repetition_factor # capped at 20pts (5 actors)\n```\n\n**Case-level trust score** (max 100) weighted across 5 components:\n\n| Component | Weight |\n|-----------|--------|\n| Confidence | 30 |\n| Path length | 20 |\n| Actor repetition | 20 |\n| Transaction amount | 15 |\n| Severity | 15 |\n\n**Risk bands:**\n\n| Band | Score Range |\n|------|------------|\n| LOW | < 40 |\n| MEDIUM | 40 – 64 |\n| HIGH | 65 – 84 |\n| CRITICAL | ≥ 85 |\n\nThe `SecureAIInferenceEngine` is entirely deterministic — zero external calls, zero randomness, pure mathematical computation with template-based explanations.\n\n---\n\n### Multi-Agent AI Orchestration\n\n\n\n*GenW Agent Builder Intelligence Layer - 5-agent pipeline with central orchestration and decision pathway analysis.*\n\nThe `MultiAgentOrchestrator` runs four specialized agents in parallel per case:\n\n| Agent | Responsibility |\n|-------|---------------|\n| `graph_analyst` | Graph topology and relationship analysis |\n| `policy_critic` | Policy compliance evaluation |\n| `counterfactual_agent` | Alternative non-malicious interpretations |\n| `remediation_agent` | Recommended corrective actions |\n\n**Consensus Engine** averages confidence across all agents, deduplicates recommendations, and computes a `conflict_score` (spread between max/min confidence × 100). Returns top 6 recommendations.\n\n**Operating modes:**\n\n- **Air-Gapped Mode** (default): Entirely local, zero internet dependency, no data leaves the system\n- **Hybrid Mode**: Local deterministic analysis + optional call to OpenAI-compatible external API\n\nWhen external AI is enabled, `ModelGovernor` sanitizes outbound payloads by redacting entity IDs (E/V/I/P/A patterns), CASE-IDs, and 6+ digit numbers. Payload capped at 5,000 characters.\n\n---\n\n### Case Management & Investigation\n\n\n\n*POC Development Approach - Agent pipeline with Coordinating Agent, GenW Playground, and Realm AI investigation assistant.*\n\nCases are automatically generated by `PathwayDetector`, which matches detected bypass paths with triggered rules, computes trust scores, and assigns sequential IDs (`CASE-001`, `CASE-002`, ...). Cases are sorted by risk score (highest first).\n\nEach investigation report contains:\n- Executive summary\n- Sequence of events (narrative)\n- Rule evidence with confidence values\n- Risk explanation\n- Counterfactual analysis (\"What if this wasn't fraud?\")\n- Recommended remediation actions\n- Event timeline\n- Full traceability dictionary\n\n**Case lifecycle:** `open` → `investigating` → `escalated` → `closed`\n\nCases involving flagged auditors are automatically marked `admin_only` — the implicated auditor cannot see their own conflict alerts.\n\nEvidence bundles can be exported as a streaming ZIP download via the API.\n\n---\n\n### Auditor Integrity & Conflict Detection\n\n`AuditorGuard` builds a NetworkX graph of auditor relationships and detects:\n- Auditors with ≥2 approvals on the same vendor\n- Auditors with cumulative approval value ≥ $300K\n- Social proximity via shortest-path hop distance\n\n**Favoritism severity formula:**\n```\nseverity = 0.45 + (count × 0.1) + ((max_hops − hops + 1) × 0.08)\n```\n\n**Least-conflicted auditor assignment** scores all available auditors:\n```\nconflict_score = max(0.1, 1/(hops+1)) + workload × 0.05\n```\nThe auditor with the lowest total score is assigned to the case.\n\n---\n\n### Policy & Compliance Governance\n\nPolicy documents (PDF, DOCX, CSV, TXT, JSON, MD) can be uploaded and processed through automated rule extraction. The system scans document text for compliance keywords (`must`, `shall`, `required`, `prohibited`, `compliance`) and creates up to 25 rules per document with automatic severity scoring.\n\nPolicy versioning follows a **Draft → Published** workflow with full version history, approval tracking, and compliance tags.\n\nConfigurable policy thresholds per company:\n- `invoice_approval_threshold`\n- `high_value_payment_threshold`\n- `required_approvals`\n- `max_connection_hops`\n\n---\n\n### Security Infrastructure\n\n| Control | Implementation |\n|---------|---------------|\n| Encryption at rest | Fernet (AES-128 CBC + HMAC), key derived via SHA256 |\n| Integrity verification | SHA256 hash stored per record, recomputed on retrieval |\n| Audit chain | Append-only JSONL; each event hash = `SHA256(prev_hash + event_body)` |\n| Password hashing | PBKDF2-HMAC-SHA256, 390K iterations, 16-byte random salt, constant-time compare |\n| Rate limiting | 120 requests/minute per IP+route, sliding window |\n| Request size limit | 5MB max via Content-Length check |\n| Security headers | X-Content-Type-Options, X-Frame-Options: DENY, Referrer-Policy, Permissions-Policy, Cache-Control: no-store |\n| Sensitive data masking | Auto-redacts `password`, `token`, `secret`, `api_key`, `authorization` fields before audit logging |\n| Request tracing | UUID-based `X-Request-ID` injected per request |\n| Host validation | TrustedHostMiddleware blocks spoofed Host headers |\n\n---\n\n### Persistence & Disaster Recovery\n\n- **SQLite with WAL mode** — 14-table schema, thread-safe with `Lock`, `PRAGMA synchronous=FULL`, foreign keys enforced\n- **Timestamped backups** — copies DB + audit log, retains last N (default 20) backups with auto-deletion of oldest\n- **Backup restore** — two modes: `preview` (temp copy) and `inplace` (overwrites originals), admin-only\n- **State recovery on startup** — `_load_latest_state()` restores dataset, decisions, rules, cases, and investigations from encrypted SQLite on boot\n\n---\n\n## Frontend Modules\n\n### Admin Dashboard\n\n- **3D Graph Explorer** — Three.js canvas with type-based lane layout (switches to degree-based ring at >62% node homogeneity); risk nodes glow with additive blending; active case paths highlighted with dashed lines; hover tooltips\n- **Event Stream** — Live color-coded feed (ingest=cyan, rule=blue, pathway=red, security=amber); click to expand full JSON; max 40 events\n- **Investigation Panel** — Case selector with confidence ring, risk bullets, manual audit records, rules/actors pills, event timeline\n- **Policy Compliance Updater** — Company workspace loader, threshold config, PDF/DOCX upload, version history\n- **Telemetry Dashboard** — 4 KPI sparkline cards (30-sample rolling, 5s sampling), agent load gauges, 6-agent activity table, 20s auto-refresh\n- **AI Search Bar** — Keyword routing (\"investigate\" → investigation tab, \"graph\" → graph tab); chat history popup\n\n### Auditor Dashboard\n\n- **Pipeline Console** — SVG flow diagram of 12 backend stages; click any stage for subprocess drill-down with audit traces and evidence references\n- **Auditor Report Writer** — ContentEditable rich text editor with 16-button toolbar; auto-saves every 3s to localStorage + server; PDF export via jsPDF + html2canvas\n- **Vendor Graph Workspace** — Debounced vendor search (220ms), multi-filter panel, active filter chips, subgraph visualization\n- **Manual Audit Panel** — Vendor selector, case multi-select, severity buttons, notes/findings/recommendations form\n\n### Landing Page\n\n- **3D Compliance Brain** — Three.js scene with 170 nodes in brain + ring layout; morphs on scroll; risk nodes highlighted red; transmission material shell with chromatic aberration\n- **Scroll-Linked Animations** — Framer Motion scroll-triggered reveals with staggered delays across 6 section panels\n\n---\n\n## API Reference\n\n### Authentication\n\n```\nPOST /auth/login # Returns JWT token\nPOST /auth/users # Create user (admin only)\n```\n\n### Data\n\n```\nPOST /ingest # Upload enterprise dataset\nGET /dataset/load # Load simulated demo dataset\nGET /export/neo4j # Export graph to Neo4j-compatible format\n```\n\n### Analysis\n\n```\nGET /pipeline/run # Run full analysis pipeline\nGET /cases # List all investigation cases\nGET /cases/{id} # Get case details + investigation report\nGET /cases/{id}/evidence # Download evidence ZIP bundle\n```\n\n### Rules\n\n```\nGET /rules # List all active rules\nPOST /rules/simulate # What-if threshold simulation\n```\n\n### Audit\n\n```\nGET /audit/chain/verify # Verify full blockchain-style audit chain\nGET /audit/alerts # List auditor conflict alerts\nPOST /audit/findings # Submit manual audit finding\n```\n\n### Policies\n\n```\nGET /policies # List company policies\nPOST /policies # Create policy\nPOST /policies/{id}/documents # Upload policy document\nPOST /policies/{id}/publish # Publish policy version\n```\n\n### System\n\n```\nGET /system/health # System health snapshot\nGET /system/metrics # Metrics registry (counters + gauges)\nGET /system/pipeline # Pipeline deep-dive staged summary\nPOST /backup # Trigger backup (admin only)\nPOST /backup/restore # Restore from backup (admin only)\n```\n\n---\n\n## Authentication & RBAC\n\nThree roles with gated endpoint access via `require_roles()` FastAPI dependency:\n\n| Role | Permissions |\n|------|------------|\n| `admin` | Full access including user provisioning, backup/restore, admin-only cases |\n| `auditor` | Own findings, pipeline console, report writer, vendor workspace |\n| `risk_analyst` | Case viewing, investigation reports, rule simulation |\n\n**Account lockout:** 5 failed login attempts → 15-minute lockout (tracked via `locked_until` in SQLite)\n\n**Bootstrap users** are auto-created from `BOOTSTRAP_USERS_JSON` on startup.\n\n---\n\n## Deployment\n\n### Prerequisites\n\n- Python 3.10+\n- Node.js 18+\n- SQLite3\n\n### Backend\n\n```bash\n# Install dependencies\npip install -r requirements.txt\n\n# Configure environment\ncp .env.example .env\n\n# Start the API server\nuvicorn main:app --host 0.0.0.0 --port 8000\n```\n\n### Frontend\n\n```bash\ncd frontend\nnpm install\nnpm run build\nnpm start\n```\n\n### Docker (recommended)\n\n```bash\ndocker-compose up --build\n```\n\n---\n\n## Environment Variables\n\n| Variable | Required | Description |\n|----------|----------|-------------|\n| `SECRET_KEY` | Yes | JWT signing key (HS256) |\n| `FERNET_KEY` | Yes | Encryption key for SQLite data at rest |\n| `BOOTSTRAP_USERS_JSON` | Yes | JSON array of initial users with roles |\n| `ENABLE_EXTERNAL_AI` | No | `true` to enable hybrid AI mode (default: `false`) |\n| `EXTERNAL_AI_API_KEY` | No | API key for OpenAI-compatible external AI |\n| `EXTERNAL_AI_MODEL` | No | Model name (must be in governance whitelist) |\n| `NEXT_PUBLIC_API_URL` | Yes | Backend API base URL for Next.js frontend |\n| `NEXT_PUBLIC_DEMO_USERNAME` | No | Auto-fill demo credential on login page |\n| `NEXT_PUBLIC_DEMO_PASSWORD` | No | Auto-fill demo credential on login page |\n\n---\n\n## Feature Summary\n\nThe platform ships **99 distinct features** across 18 subsystems:\n\n| Subsystem | Features |\n|-----------|---------|\n| Authentication & Access Control | 7 |\n| Data Ingestion & Validation | 3 |\n| Graph Intelligence Engine | 8 |\n| Governance Rule Engine | 8 |\n| Risk Scoring & Trust Engine | 4 |\n| Case Management & Investigation | 9 |\n| Multi-Agent AI Orchestration | 6 |\n| Auditor Integrity & Conflict Detection | 4 |\n| Policy & Compliance Governance | 5 |\n| Security Infrastructure | 9 |\n| Persistence & Disaster Recovery | 4 |\n| Observability & Monitoring | 5 |\n| Frontend — Landing & Marketing | 5 |\n| Frontend — Authentication | 3 |\n| Frontend — Admin Dashboard | 8 |\n| Frontend — Auditor Dashboard | 4 |\n| Frontend — State & API | 5 |\n| Report & Export | 3 |\n\n---\n\n## License\n\nProprietary. All rights reserved.\n\n---\n\n*Built with GenW.AI infrastructure — AppMaker · Agent Builder · Playground · Realm AI*"}}}} |