1 line
28 KiB
JSON
1 line
28 KiB
JSON
{"repo": "abzdevs18/gabay-docs", "files": {"gabay-form/ai-feedback-system.md": {"d3e454a95df563f63f3969aae55983575d0842e4": {"t": 1785746444.2392492, "n": 13224, "h": "bd49acde9ffb8553", "c": "# Gabay Form - AI Feedback System\n\n> Automatic personalized feedback for every student submission\n\n---\n\n## Overview\n\nThe AI Feedback System automatically generates personalized, encouraging feedback for students after they complete assessments. Using advanced GPT models, it analyzes their responses and provides constructive guidance.\n\n---\n\n## How It Works\n\n### 1. Student Submits Form\n\n```\nStudent fills out exam → Clicks Submit\n ↓\n API saves response\n ↓\n Job queued for processing\n ↓\n Student sees confirmation\n```\n\n**Time:** <200ms (synchronous)\n\n### 2. Background Processing\n\n```\nWorker picks up job → Fetches form data\n ↓\n Analyzes answers\n ↓\n Generates AI prompt\n ↓\n Calls OpenAI API\n ↓\n Receives feedback\n ↓\n Creates email HTML\n ↓\n Sends email\n```\n\n**Time:** 5-8 seconds (asynchronous)\n\n### 3. Student Receives Email\n\nBeautiful HTML email with:\n- Personal greeting\n- Completion statistics\n- AI-generated feedback\n- Study recommendations\n- Encouragement\n\n---\n\n## AI Models\n\n### Supported Models\n\n| Model | Provider | Cost | Speed | Quality |\n|-------|----------|------|-------|---------|\n| `gpt-4o-mini` | OpenAI | $0.15/1M | Fast | High |\n| `gpt-5-nano` | OpenAI | $0.10/1M | Fastest | Good |\n| `deepseek-chat` | DeepSeek | Varies | Fast | Good |\n\n### Configuration\n\n```bash\n# .env\nOPENAI_API_KEY=sk-...\nOPENAI_CHAT_MODEL=gpt-4o-mini\nOPENAI_FEEDBACK_MODEL=gpt-4o-mini # Override for feedback\nDEEPSEEK_API_KEY=sk-... # Fallback\n```\n\n---\n\n## Automatic Scoring\n\nBefore AI feedback generation, the system automatically scores student responses:\n\n### Supported Question Types\n\n| Question Type | Answer Field | Scoring Method |\n|--------------|-------------|----------------|\n| **Multiple Choice** | `correctAnswerIds` | Checks if student's answer ID is in array |\n| **True/False** | `correctAnswerIds` | Same as multiple choice |\n| **Checkbox** | `correctAnswerIds` | All selected IDs must match (exact match) |\n| **Dropdown** | `correctAnswerIds` | Same as multiple choice |\n| **Fill-in-Blank** | `correctAnswers` | Case-insensitive string matching |\n| **Short Answer** | `correctAnswers` | Multiple acceptable answers supported |\n| **Identification** | `correctAnswers` | Trimmed, case-insensitive comparison |\n| **Essay** | N/A | Manual grading (no auto-scoring) |\n\n### Scoring Logic\n\n**Choice-Based Questions** (Multiple Choice, True/False, Dropdown):\n```typescript\n// Student answer: \"choice_1\"\n// Correct answers: [\"choice_1\"]\nisCorrect = correctAnswerIds.includes(studentAnswer);\n```\n\n**Multi-Select (Checkbox)**:\n```typescript\n// Student answer: [\"choice_1\", \"choice_3\"]\n// Correct answers: [\"choice_1\", \"choice_3\"]\nisCorrect = studentAnswer.length === correctAnswerIds.length && \n studentAnswer.every(ans => correctAnswerIds.includes(ans));\n```\n\n**Text-Based Questions**:\n```typescript\n// Student answer: \"Photosynthesis\"\n// Correct answers: [\"photosynthesis\", \"photo synthesis\"]\nnormalizedStudent = studentAnswer.trim().toLowerCase();\nisCorrect = correctAnswers.some(ca => \n ca.trim().toLowerCase() === normalizedStudent\n);\n```\n\n### Correct Answer Detection\n\nThe system intelligently finds correct answers from multiple possible locations:\n\n1. **Primary Fields:**\n - `correctAnswerIds` - For choice-based questions\n - `correctAnswers` - For text-based questions\n\n2. **Legacy Fields:**\n - `correctAnswer`, `answer`, `correct_answer`\n\n3. **Embedded in Options:**\n - `options[].isCorrect = true`\n - `choices[].isCorrect = true`\n\n4. **Validation Objects:**\n - `validation.correctAnswer`\n - `settings.correctAnswer`\n\nThis multi-path detection ensures compatibility with forms created by different builders and versions.\n\n### Score Calculation\n\n```\ncorrectCount = number of correct answers\ntotalQuestions = number of questions with defined correct answers\nscorePercentage = (correctCount / totalQuestions) × 100\n```\n\n**Example:**\n- Total Questions: 10\n- Correct Answers: 7\n- Score: 70%\n\n---\n\n## Feedback Generation\n\n### AI Prompt\n\nThe system uses a carefully crafted prompt:\n\n```\nYou are an encouraging and supportive K-12 teacher providing \npersonalized feedback to students after completing an exam.\n\nYour feedback should:\n1. Be warm, encouraging, and age-appropriate\n2. Acknowledge effort and completion\n3. Identify 2-3 specific strengths based on their answers\n4. Suggest 1-2 areas for improvement (gently and constructively)\n5. Provide specific study tips related to the exam content\n6. End with an encouraging message\n\nKeep the tone positive and motivating.\nLength: 250-350 words.\n```\n\n### Context Provided\n\n```\nExam: Math Quiz\n\nACTUAL SCORE:\n- Correct Answers: 7 out of 10\n- Incorrect Answers: 3\n- Points Earned: 70 out of 100\n- Score Percentage: 70%\n\nStatistics:\n- Total Questions: 10\n- Questions Answered: 10\n- Total Possible Points: 100\n- Time Taken: 5 minutes\n\nQuestions and Answers (with correctness marked):\nQuestion 1: What is 2+2?\nType: MULTIPLE_CHOICE\nPoints: 10\nStudent's Answer: choice_1\nCorrect Answer: choice_1\n✅ CORRECT\n\nQuestion 2: What is the capital of France?\nType: MULTIPLE_CHOICE\nPoints: 10\nStudent's Answer: choice_2\nCorrect Answer: choice_3\n❌ INCORRECT\n\n...\n```\n\n### Example Output\n\n```\nGreat job on completing the Math Quiz, John! 🎉\n\nI'm impressed by your effort and dedication. You answered all \n10 questions and showed a strong understanding of the material.\n\nYour Strengths:\n✅ Excellent work on arithmetic problems! Your calculations \n were accurate and well-thought-out.\n✅ Strong grasp of geography concepts, especially with \n European capitals.\n✅ Great time management - you completed the quiz efficiently \n without rushing.\n\nAreas for Growth:\n💡 Review fraction operations, particularly when working with \n mixed numbers.\n💡 Practice identifying prime numbers - a quick review of \n factors will help!\n\nStudy Tips:\n- Create flashcards for key math formulas\n- Try online practice quizzes for geography\n- Review your notes on number theory\n\nKeep up the fantastic work! Your dedication to learning shows, \nand I'm confident you'll continue to improve. Remember, every \nchallenge is an opportunity to grow stronger! 🌟\n\nWell done!\n```\n\n---\n\n## Email Template\n\n### Structure\n\n```html\n<!DOCTYPE html>\n<html>\n<head>\n <style>\n /* Modern, responsive CSS */\n /* Mobile-optimized */\n /* Professional appearance */\n </style>\n</head>\n<body>\n <div class=\"container\">\n <!-- Header -->\n <h1>🎓 Exam Completed!</h1>\n <p>Hi John Doe,</p>\n \n <!-- Stats Cards -->\n <div class=\"stats\">\n <div class=\"stat-card\">\n <strong>Questions:</strong> 10/10\n </div>\n <div class=\"stat-card\">\n <strong>Time:</strong> 5 minutes\n </div>\n <div class=\"stat-card\">\n <strong>Completion:</strong> 100%\n </div>\n </div>\n \n <!-- AI Feedback -->\n <div class=\"feedback-section\">\n <h2>📝 Personalized Feedback</h2>\n <div class=\"feedback-content\">\n [AI-generated feedback here]\n </div>\n </div>\n \n <!-- Footer -->\n <div class=\"footer\">\n <p>This is an automated email from Gabay Assessment Platform.</p>\n <p>© 2025 Gabay. All rights reserved.</p>\n </div>\n </div>\n</body>\n</html>\n```\n\n### Features\n\n- ✅ Mobile-responsive\n- ✅ Beautiful typography\n- ✅ Clear hierarchy\n- ✅ Professional branding\n- ✅ Multi-tenant customization\n\n---\n\n## Performance & Cost\n\n### Processing Metrics\n\n| Metric | Value |\n|--------|-------|\n| Queue → Worker | <50ms |\n| Fetch Data | ~100ms |\n| AI Generation | 3-5s |\n| Email Send | 1-2s |\n| **Total Time** | **5-8s** |\n\n### Throughput\n\n- **Rate Limit**: 10 feedbacks/minute\n- **Concurrency**: 3 simultaneous\n- **Daily Capacity**: ~14,400/day\n\n### Cost Analysis\n\n**Per Feedback (gpt-4o-mini):**\n- Input tokens: ~800 (~$0.0001)\n- Output tokens: ~400 (~$0.0001)\n- **Total: ~$0.0002**\n\n**Monthly Estimates:**\n- 1,000 submissions: $0.20\n- 10,000 submissions: $2.00\n- 100,000 submissions: $20.00\n\n💰 **Extremely cost-effective!**\n\n---\n\n## Multi-Tenant Support\n\n### Tenant Isolation\n\nThe system maintains complete data isolation:\n\n```\nStudent submits (Tenant A)\n ↓\nJob includes tenantId + tenantToken\n ↓\nWorker uses Tenant A's Prisma client\n ↓\nFetches from tenant_a.gabay_forms\n ↓\nEmail sent with Tenant A branding\n```\n\n**Guarantees:**\n- ✅ Correct database schema\n- ✅ Proper data isolation\n- ✅ Tenant-specific emails\n- ✅ No cross-tenant leakage\n\n---\n\n## Configuration Options\n\n### Disable AI Feedback\n\n```typescript\n// worker-manager.service.ts\nconst workerManager = WorkerManager.getInstance({\n enableFormResponseWorker: false\n});\n```\n\n### Adjust Concurrency\n\n```typescript\n// form-response-worker.service.ts\nconcurrency: 5 // Process 5 jobs at once\n```\n\n### Change Rate Limit\n\n```typescript\n// form-response-queue.service.ts\nlimiter: {\n max: 20, // 20 jobs\n duration: 60000 // per minute\n}\n```\n\n### Switch AI Provider\n\n```bash\n# Use DeepSeek instead\nOPENAI_FEEDBACK_MODEL=deepseek-chat\n```\n\n---\n\n## Monitoring\n\n### Log Output\n\n```bash\n📧 Form Response - Active: 2, Waiting: 5, Completed: 1,234, Failed: 0\n🔄 [FormResponse] Processing response abc-123\n📧 [FormResponse] Student email: student@school.com\n🏢 [FormResponse] Tenant: school_alpha\n📚 [FormResponse] Analyzing 10 questions\n\n[SCORING] Summary: {\n totalQuestions: 10,\n correctCount: 7,\n incorrectCount: 3,\n pointsEarned: 70,\n totalPoints: 100,\n scorePercentage: 70,\n hasCorrectAnswers: 10\n}\n\n[FormResponse] Using openai provider with model: gpt-4o-mini\n[FormResponse] AI feedback generated (1,958 chars)\n📤 [FormResponse] Sending email to student@school.com...\n✅ [FormResponse] Feedback email sent\n✅ [FormResponse] Response processed successfully\n```\n\n**Scoring Warnings** (only shown if issues detected):\n```bash\n[SCORING] 2/10 questions missing correct answers. Sample: {\n id: 'q_1',\n type: 'MULTIPLE_CHOICE',\n hasCorrectAnswerIds: false,\n fields: ['id', 'type', 'title', 'choices']\n}\n\n[SCORING] Question \"What is X?\" (MULTIPLE_CHOICE) missing correct answer. \nAvailable fields: ['id', 'type', 'title', 'choices', 'required']\n```\n\n### Queue Stats\n\n```bash\nredis-cli LLEN bull:form-response-processing:wait\nredis-cli LLEN bull:form-response-processing:active\nredis-cli LLEN bull:form-response-processing:completed\n```\n\n---\n\n## Troubleshooting\n\n### No Email Received\n\n**Possible Causes:**\n1. Missing `respondentEmail` in form metadata\n2. Invalid email address\n3. Brevo API key not configured\n4. Worker not running\n\n**Solutions:**\n```bash\n# Check worker status\nnpm run start:workers\n\n# Verify email in metadata\n# Form must collect respondentEmail field\n\n# Check Brevo API key\necho $BREVO_API_KEY\n\n# View logs\ntail -f logs/worker.log\n```\n\n### Generic Feedback\n\n**Cause:** AI API key invalid or quota exceeded\n\n**Solution:**\n```bash\n# Verify OpenAI API key\ncurl https://api.openai.com/v1/models \\\n -H \"Authorization: Bearer $OPENAI_API_KEY\"\n\n# Check quota\n# https://platform.openai.com/usage\n```\n\n### Score Shows 0% Despite Correct Answers\n\n**Cause:** Correct answers not properly configured in form questions\n\n**Solution:**\n1. **Check logs** for scoring details:\n ```bash\n [SCORING] Summary: {\n correctCount: 0,\n hasCorrectAnswers: 0 # ← Should match totalQuestions\n }\n ```\n\n2. **Verify question structure** - ensure questions have:\n - Multiple Choice: `correctAnswerIds` field (array of choice IDs)\n - Text Questions: `correctAnswers` field (array of strings)\n\n3. **Example correct format:**\n ```json\n {\n \"type\": \"MULTIPLE_CHOICE\",\n \"correctAnswerIds\": [\"choice_1\"],\n \"choices\": [...]\n }\n ```\n\n4. **Check warning logs:**\n ```bash\n [SCORING] Question \"...\" missing correct answer\n ```\n This shows which questions lack correct answer definitions.\n\n### Worker Not Processing\n\n**Cause:** Redis connection issue\n\n**Solution:**\n```bash\n# Check Redis\nredis-cli ping\n\n# Restart Redis (if needed)\nredis-server\n\n# Restart workers\nnpm run start:workers\n```\n\n---\n\n## Security & Privacy\n\n### Data Protection\n\n- ✅ Email addresses validated\n- ✅ Student data not logged externally\n- ✅ Minimal context sent to AI\n- ✅ Tenant isolation maintained\n- ✅ GDPR/FERPA compliant\n\n### AI Safety\n\n- No PII sent to AI (only answers)\n- No student names in prompts (anonymized)\n- Feedback reviewed for appropriateness\n- Family-friendly content only\n\n---\n\n## Future Enhancements\n\n**Planned Features:**\n- [ ] Customizable feedback templates\n- [ ] Teacher review before sending\n- [ ] Multilingual support\n- [ ] Voice feedback (audio)\n- [ ] Video feedback\n- [ ] Parent CC option\n- [ ] Feedback history tracking\n- [ ] A/B testing for prompts\n\n---\n\n## Related Documentation\n\n- [Worker System](./worker-system.md)\n- [Multi-Tenant Support](../../MULTI_TENANT_WORKER_SUPPORT.md)\n- [Form Response Worker](../../FORM_RESPONSE_WORKER_SYSTEM.md)\n- [API Cost Estimate](../../AI_FEEDBACK_COST_ESTIMATE.md)\n"}, "904b1d0804b7773c0cf3dccc8d0f82f77e2dca57": {"t": 1785764309.159624, "n": 13224, "h": "bd49acde9ffb8553", "c": "# Gabay Form - AI Feedback System\n\n> Automatic personalized feedback for every student submission\n\n---\n\n## Overview\n\nThe AI Feedback System automatically generates personalized, encouraging feedback for students after they complete assessments. Using advanced GPT models, it analyzes their responses and provides constructive guidance.\n\n---\n\n## How It Works\n\n### 1. Student Submits Form\n\n```\nStudent fills out exam → Clicks Submit\n ↓\n API saves response\n ↓\n Job queued for processing\n ↓\n Student sees confirmation\n```\n\n**Time:** <200ms (synchronous)\n\n### 2. Background Processing\n\n```\nWorker picks up job → Fetches form data\n ↓\n Analyzes answers\n ↓\n Generates AI prompt\n ↓\n Calls OpenAI API\n ↓\n Receives feedback\n ↓\n Creates email HTML\n ↓\n Sends email\n```\n\n**Time:** 5-8 seconds (asynchronous)\n\n### 3. Student Receives Email\n\nBeautiful HTML email with:\n- Personal greeting\n- Completion statistics\n- AI-generated feedback\n- Study recommendations\n- Encouragement\n\n---\n\n## AI Models\n\n### Supported Models\n\n| Model | Provider | Cost | Speed | Quality |\n|-------|----------|------|-------|---------|\n| `gpt-4o-mini` | OpenAI | $0.15/1M | Fast | High |\n| `gpt-5-nano` | OpenAI | $0.10/1M | Fastest | Good |\n| `deepseek-chat` | DeepSeek | Varies | Fast | Good |\n\n### Configuration\n\n```bash\n# .env\nOPENAI_API_KEY=sk-...\nOPENAI_CHAT_MODEL=gpt-4o-mini\nOPENAI_FEEDBACK_MODEL=gpt-4o-mini # Override for feedback\nDEEPSEEK_API_KEY=sk-... # Fallback\n```\n\n---\n\n## Automatic Scoring\n\nBefore AI feedback generation, the system automatically scores student responses:\n\n### Supported Question Types\n\n| Question Type | Answer Field | Scoring Method |\n|--------------|-------------|----------------|\n| **Multiple Choice** | `correctAnswerIds` | Checks if student's answer ID is in array |\n| **True/False** | `correctAnswerIds` | Same as multiple choice |\n| **Checkbox** | `correctAnswerIds` | All selected IDs must match (exact match) |\n| **Dropdown** | `correctAnswerIds` | Same as multiple choice |\n| **Fill-in-Blank** | `correctAnswers` | Case-insensitive string matching |\n| **Short Answer** | `correctAnswers` | Multiple acceptable answers supported |\n| **Identification** | `correctAnswers` | Trimmed, case-insensitive comparison |\n| **Essay** | N/A | Manual grading (no auto-scoring) |\n\n### Scoring Logic\n\n**Choice-Based Questions** (Multiple Choice, True/False, Dropdown):\n```typescript\n// Student answer: \"choice_1\"\n// Correct answers: [\"choice_1\"]\nisCorrect = correctAnswerIds.includes(studentAnswer);\n```\n\n**Multi-Select (Checkbox)**:\n```typescript\n// Student answer: [\"choice_1\", \"choice_3\"]\n// Correct answers: [\"choice_1\", \"choice_3\"]\nisCorrect = studentAnswer.length === correctAnswerIds.length && \n studentAnswer.every(ans => correctAnswerIds.includes(ans));\n```\n\n**Text-Based Questions**:\n```typescript\n// Student answer: \"Photosynthesis\"\n// Correct answers: [\"photosynthesis\", \"photo synthesis\"]\nnormalizedStudent = studentAnswer.trim().toLowerCase();\nisCorrect = correctAnswers.some(ca => \n ca.trim().toLowerCase() === normalizedStudent\n);\n```\n\n### Correct Answer Detection\n\nThe system intelligently finds correct answers from multiple possible locations:\n\n1. **Primary Fields:**\n - `correctAnswerIds` - For choice-based questions\n - `correctAnswers` - For text-based questions\n\n2. **Legacy Fields:**\n - `correctAnswer`, `answer`, `correct_answer`\n\n3. **Embedded in Options:**\n - `options[].isCorrect = true`\n - `choices[].isCorrect = true`\n\n4. **Validation Objects:**\n - `validation.correctAnswer`\n - `settings.correctAnswer`\n\nThis multi-path detection ensures compatibility with forms created by different builders and versions.\n\n### Score Calculation\n\n```\ncorrectCount = number of correct answers\ntotalQuestions = number of questions with defined correct answers\nscorePercentage = (correctCount / totalQuestions) × 100\n```\n\n**Example:**\n- Total Questions: 10\n- Correct Answers: 7\n- Score: 70%\n\n---\n\n## Feedback Generation\n\n### AI Prompt\n\nThe system uses a carefully crafted prompt:\n\n```\nYou are an encouraging and supportive K-12 teacher providing \npersonalized feedback to students after completing an exam.\n\nYour feedback should:\n1. Be warm, encouraging, and age-appropriate\n2. Acknowledge effort and completion\n3. Identify 2-3 specific strengths based on their answers\n4. Suggest 1-2 areas for improvement (gently and constructively)\n5. Provide specific study tips related to the exam content\n6. End with an encouraging message\n\nKeep the tone positive and motivating.\nLength: 250-350 words.\n```\n\n### Context Provided\n\n```\nExam: Math Quiz\n\nACTUAL SCORE:\n- Correct Answers: 7 out of 10\n- Incorrect Answers: 3\n- Points Earned: 70 out of 100\n- Score Percentage: 70%\n\nStatistics:\n- Total Questions: 10\n- Questions Answered: 10\n- Total Possible Points: 100\n- Time Taken: 5 minutes\n\nQuestions and Answers (with correctness marked):\nQuestion 1: What is 2+2?\nType: MULTIPLE_CHOICE\nPoints: 10\nStudent's Answer: choice_1\nCorrect Answer: choice_1\n✅ CORRECT\n\nQuestion 2: What is the capital of France?\nType: MULTIPLE_CHOICE\nPoints: 10\nStudent's Answer: choice_2\nCorrect Answer: choice_3\n❌ INCORRECT\n\n...\n```\n\n### Example Output\n\n```\nGreat job on completing the Math Quiz, John! 🎉\n\nI'm impressed by your effort and dedication. You answered all \n10 questions and showed a strong understanding of the material.\n\nYour Strengths:\n✅ Excellent work on arithmetic problems! Your calculations \n were accurate and well-thought-out.\n✅ Strong grasp of geography concepts, especially with \n European capitals.\n✅ Great time management - you completed the quiz efficiently \n without rushing.\n\nAreas for Growth:\n💡 Review fraction operations, particularly when working with \n mixed numbers.\n💡 Practice identifying prime numbers - a quick review of \n factors will help!\n\nStudy Tips:\n- Create flashcards for key math formulas\n- Try online practice quizzes for geography\n- Review your notes on number theory\n\nKeep up the fantastic work! Your dedication to learning shows, \nand I'm confident you'll continue to improve. Remember, every \nchallenge is an opportunity to grow stronger! 🌟\n\nWell done!\n```\n\n---\n\n## Email Template\n\n### Structure\n\n```html\n<!DOCTYPE html>\n<html>\n<head>\n <style>\n /* Modern, responsive CSS */\n /* Mobile-optimized */\n /* Professional appearance */\n </style>\n</head>\n<body>\n <div class=\"container\">\n <!-- Header -->\n <h1>🎓 Exam Completed!</h1>\n <p>Hi John Doe,</p>\n \n <!-- Stats Cards -->\n <div class=\"stats\">\n <div class=\"stat-card\">\n <strong>Questions:</strong> 10/10\n </div>\n <div class=\"stat-card\">\n <strong>Time:</strong> 5 minutes\n </div>\n <div class=\"stat-card\">\n <strong>Completion:</strong> 100%\n </div>\n </div>\n \n <!-- AI Feedback -->\n <div class=\"feedback-section\">\n <h2>📝 Personalized Feedback</h2>\n <div class=\"feedback-content\">\n [AI-generated feedback here]\n </div>\n </div>\n \n <!-- Footer -->\n <div class=\"footer\">\n <p>This is an automated email from Gabay Assessment Platform.</p>\n <p>© 2025 Gabay. All rights reserved.</p>\n </div>\n </div>\n</body>\n</html>\n```\n\n### Features\n\n- ✅ Mobile-responsive\n- ✅ Beautiful typography\n- ✅ Clear hierarchy\n- ✅ Professional branding\n- ✅ Multi-tenant customization\n\n---\n\n## Performance & Cost\n\n### Processing Metrics\n\n| Metric | Value |\n|--------|-------|\n| Queue → Worker | <50ms |\n| Fetch Data | ~100ms |\n| AI Generation | 3-5s |\n| Email Send | 1-2s |\n| **Total Time** | **5-8s** |\n\n### Throughput\n\n- **Rate Limit**: 10 feedbacks/minute\n- **Concurrency**: 3 simultaneous\n- **Daily Capacity**: ~14,400/day\n\n### Cost Analysis\n\n**Per Feedback (gpt-4o-mini):**\n- Input tokens: ~800 (~$0.0001)\n- Output tokens: ~400 (~$0.0001)\n- **Total: ~$0.0002**\n\n**Monthly Estimates:**\n- 1,000 submissions: $0.20\n- 10,000 submissions: $2.00\n- 100,000 submissions: $20.00\n\n💰 **Extremely cost-effective!**\n\n---\n\n## Multi-Tenant Support\n\n### Tenant Isolation\n\nThe system maintains complete data isolation:\n\n```\nStudent submits (Tenant A)\n ↓\nJob includes tenantId + tenantToken\n ↓\nWorker uses Tenant A's Prisma client\n ↓\nFetches from tenant_a.gabay_forms\n ↓\nEmail sent with Tenant A branding\n```\n\n**Guarantees:**\n- ✅ Correct database schema\n- ✅ Proper data isolation\n- ✅ Tenant-specific emails\n- ✅ No cross-tenant leakage\n\n---\n\n## Configuration Options\n\n### Disable AI Feedback\n\n```typescript\n// worker-manager.service.ts\nconst workerManager = WorkerManager.getInstance({\n enableFormResponseWorker: false\n});\n```\n\n### Adjust Concurrency\n\n```typescript\n// form-response-worker.service.ts\nconcurrency: 5 // Process 5 jobs at once\n```\n\n### Change Rate Limit\n\n```typescript\n// form-response-queue.service.ts\nlimiter: {\n max: 20, // 20 jobs\n duration: 60000 // per minute\n}\n```\n\n### Switch AI Provider\n\n```bash\n# Use DeepSeek instead\nOPENAI_FEEDBACK_MODEL=deepseek-chat\n```\n\n---\n\n## Monitoring\n\n### Log Output\n\n```bash\n📧 Form Response - Active: 2, Waiting: 5, Completed: 1,234, Failed: 0\n🔄 [FormResponse] Processing response abc-123\n📧 [FormResponse] Student email: student@school.com\n🏢 [FormResponse] Tenant: school_alpha\n📚 [FormResponse] Analyzing 10 questions\n\n[SCORING] Summary: {\n totalQuestions: 10,\n correctCount: 7,\n incorrectCount: 3,\n pointsEarned: 70,\n totalPoints: 100,\n scorePercentage: 70,\n hasCorrectAnswers: 10\n}\n\n[FormResponse] Using openai provider with model: gpt-4o-mini\n[FormResponse] AI feedback generated (1,958 chars)\n📤 [FormResponse] Sending email to student@school.com...\n✅ [FormResponse] Feedback email sent\n✅ [FormResponse] Response processed successfully\n```\n\n**Scoring Warnings** (only shown if issues detected):\n```bash\n[SCORING] 2/10 questions missing correct answers. Sample: {\n id: 'q_1',\n type: 'MULTIPLE_CHOICE',\n hasCorrectAnswerIds: false,\n fields: ['id', 'type', 'title', 'choices']\n}\n\n[SCORING] Question \"What is X?\" (MULTIPLE_CHOICE) missing correct answer. \nAvailable fields: ['id', 'type', 'title', 'choices', 'required']\n```\n\n### Queue Stats\n\n```bash\nredis-cli LLEN bull:form-response-processing:wait\nredis-cli LLEN bull:form-response-processing:active\nredis-cli LLEN bull:form-response-processing:completed\n```\n\n---\n\n## Troubleshooting\n\n### No Email Received\n\n**Possible Causes:**\n1. Missing `respondentEmail` in form metadata\n2. Invalid email address\n3. Brevo API key not configured\n4. Worker not running\n\n**Solutions:**\n```bash\n# Check worker status\nnpm run start:workers\n\n# Verify email in metadata\n# Form must collect respondentEmail field\n\n# Check Brevo API key\necho $BREVO_API_KEY\n\n# View logs\ntail -f logs/worker.log\n```\n\n### Generic Feedback\n\n**Cause:** AI API key invalid or quota exceeded\n\n**Solution:**\n```bash\n# Verify OpenAI API key\ncurl https://api.openai.com/v1/models \\\n -H \"Authorization: Bearer $OPENAI_API_KEY\"\n\n# Check quota\n# https://platform.openai.com/usage\n```\n\n### Score Shows 0% Despite Correct Answers\n\n**Cause:** Correct answers not properly configured in form questions\n\n**Solution:**\n1. **Check logs** for scoring details:\n ```bash\n [SCORING] Summary: {\n correctCount: 0,\n hasCorrectAnswers: 0 # ← Should match totalQuestions\n }\n ```\n\n2. **Verify question structure** - ensure questions have:\n - Multiple Choice: `correctAnswerIds` field (array of choice IDs)\n - Text Questions: `correctAnswers` field (array of strings)\n\n3. **Example correct format:**\n ```json\n {\n \"type\": \"MULTIPLE_CHOICE\",\n \"correctAnswerIds\": [\"choice_1\"],\n \"choices\": [...]\n }\n ```\n\n4. **Check warning logs:**\n ```bash\n [SCORING] Question \"...\" missing correct answer\n ```\n This shows which questions lack correct answer definitions.\n\n### Worker Not Processing\n\n**Cause:** Redis connection issue\n\n**Solution:**\n```bash\n# Check Redis\nredis-cli ping\n\n# Restart Redis (if needed)\nredis-server\n\n# Restart workers\nnpm run start:workers\n```\n\n---\n\n## Security & Privacy\n\n### Data Protection\n\n- ✅ Email addresses validated\n- ✅ Student data not logged externally\n- ✅ Minimal context sent to AI\n- ✅ Tenant isolation maintained\n- ✅ GDPR/FERPA compliant\n\n### AI Safety\n\n- No PII sent to AI (only answers)\n- No student names in prompts (anonymized)\n- Feedback reviewed for appropriateness\n- Family-friendly content only\n\n---\n\n## Future Enhancements\n\n**Planned Features:**\n- [ ] Customizable feedback templates\n- [ ] Teacher review before sending\n- [ ] Multilingual support\n- [ ] Voice feedback (audio)\n- [ ] Video feedback\n- [ ] Parent CC option\n- [ ] Feedback history tracking\n- [ ] A/B testing for prompts\n\n---\n\n## Related Documentation\n\n- [Worker System](./worker-system.md)\n- [Multi-Tenant Support](../../MULTI_TENANT_WORKER_SUPPORT.md)\n- [Form Response Worker](../../FORM_RESPONSE_WORKER_SYSTEM.md)\n- [API Cost Estimate](../../AI_FEEDBACK_COST_ESTIMATE.md)\n"}}}} |