1 line
18 KiB
JSON
1 line
18 KiB
JSON
{"repo": "brian-munene/emailattachmentclassifier", "files": {"IMPROVEMENTS.md": {"71d1dec47cde34df529aa2aede0e1aa00cd23629": {"t": 1785746502.9534338, "n": 8397, "h": "d34fc654a0f9ef59", "c": "# Project Structure & Maintainability Improvements\n\nThis document outlines recommended improvements to enhance the project's structure, maintainability, and developer experience.\n\n## 1. Project Structure Improvements\n\n### 1.1 Move `prompt.py` into Package\n**Current Issue:** `prompt.py` is at the root level but is imported by package code.\n\n**Recommendation:** Move to `email_attachment_classifier/prompts/` directory.\n\n```\nemail_attachment_classifier/\n prompts/\n __init__.py\n email.py # email_analysis_prompt\n pdf.py # pdf_analysis_prompt\n excel.py # excel_analysis_prompt\n word.py # word_analysis_prompt\n default.py # default_file_analysis_prompt\n```\n\n**Benefits:**\n- Better organization\n- Clearer module boundaries\n- Easier to test and maintain\n\n### 1.2 Organize Entry Points\n**Current Issue:** `main.py` and `server.py` are at root level.\n\n**Recommendation:** Create `scripts/` or `cli/` directory:\n```\nscripts/\n cli.py # Move main.py here\n server.py # Move server.py here (or keep at root for uvicorn)\n```\n\n**Alternative:** Keep entry points at root but create proper CLI structure:\n```\nemail_attachment_classifier/\n cli/\n __init__.py\n main.py # CLI logic\n server.py # Server logic (or keep at root)\n```\n\n### 1.3 Create Proper Test Structure\n**Current Issue:** Empty `tests/` directory.\n\n**Recommendation:**\n```\ntests/\n __init__.py\n conftest.py # Pytest fixtures\n unit/\n test_gmail.py\n test_drive.py\n test_decryption.py\n test_workflow.py\n integration/\n test_api.py\n test_cli.py\n fixtures/\n sample_emails.json\n sample_attachments/\n```\n\n## 2. Code Organization Improvements\n\n### 2.1 Separate API Routes\n**Current Issue:** All routes in `server.py`.\n\n**Recommendation:** Create `email_attachment_classifier/api/`:\n```\nemail_attachment_classifier/\n api/\n __init__.py\n routes/\n __init__.py\n classify.py\n health.py\n dependencies.py # FastAPI dependencies\n middleware.py # Custom middleware\n exceptions.py # API-specific exception handlers\n```\n\n### 2.2 Extract Business Logic from Server\n**Current Issue:** Business logic mixed with API handlers.\n\n**Recommendation:** Create service layer:\n```\nemail_attachment_classifier/\n services/\n email_service.py # Email classification orchestration\n password_service.py # Password handling logic\n```\n\n### 2.3 Improve Error Handling\n**Current Issue:** Generic exception handling in routes.\n\n**Recommendation:**\n- Create custom exception handlers in FastAPI\n- Use proper HTTP status codes\n- Add structured error responses\n\n### 2.4 Configuration Management\n**Current Issue:** Basic config, no validation.\n\n**Recommendation:**\n- Add Pydantic Settings for better validation\n- Create `.env.example` file\n- Add config validation on startup\n\n## 3. Development Tooling\n\n### 3.1 Add Development Dependencies\n**Recommendation:** Update `pyproject.toml`:\n```toml\n[project.optional-dependencies]\ndev = [\n \"pytest>=7.4.0\",\n \"pytest-asyncio>=0.21.0\",\n \"pytest-cov>=4.1.0\",\n \"black>=23.0.0\",\n \"ruff>=0.1.0\",\n \"mypy>=1.5.0\",\n \"types-requests\",\n]\n```\n\n### 3.2 Add Pre-commit Hooks\n**Recommendation:** Create `.pre-commit-config.yaml`:\n```yaml\nrepos:\n - repo: https://github.com/pre-commit/pre-commit-hooks\n rev: v4.5.0\n hooks:\n - id: trailing-whitespace\n - id: end-of-file-fixer\n - id: check-yaml\n - id: check-added-large-files\n - repo: https://github.com/psf/black\n rev: 23.12.0\n hooks:\n - id: black\n - repo: https://github.com/astral-sh/ruff-pre-commit\n rev: v0.1.6\n hooks:\n - id: ruff\n```\n\n### 3.3 Add Type Checking\n**Recommendation:** Create `pyproject.toml` section:\n```toml\n[tool.mypy]\npython_version = \"3.12\"\nwarn_return_any = true\nwarn_unused_configs = true\ndisallow_untyped_defs = false\n```\n\n### 3.4 Add Linting Configuration\n**Recommendation:** Create `ruff.toml` or add to `pyproject.toml`:\n```toml\n[tool.ruff]\nline-length = 100\ntarget-version = \"py312\"\n\n[tool.ruff.lint]\nselect = [\"E\", \"F\", \"I\", \"N\", \"W\", \"UP\"]\nignore = [\"E501\"] # Line too long (handled by black)\n```\n\n## 4. Testing Infrastructure\n\n### 4.1 Add Pytest Configuration\n**Recommendation:** Create `pytest.ini` or add to `pyproject.toml`:\n```toml\n[tool.pytest.ini_options]\ntestpaths = [\"tests\"]\npython_files = [\"test_*.py\"]\npython_classes = [\"Test*\"]\npython_functions = [\"test_*\"]\naddopts = [\n \"--strict-markers\",\n \"--cov=email_attachment_classifier\",\n \"--cov-report=term-missing\",\n \"--cov-report=html\",\n]\nmarkers = [\n \"unit: Unit tests\",\n \"integration: Integration tests\",\n \"slow: Slow running tests\",\n]\n```\n\n### 4.2 Add Test Fixtures\n**Recommendation:** Create `tests/conftest.py` with:\n- Mock Gmail service\n- Mock Drive service\n- Sample email data\n- LLM response mocks\n\n## 5. Documentation Improvements\n\n### 5.1 Add API Documentation\n**Recommendation:**\n- Use FastAPI's automatic OpenAPI docs\n- Add detailed docstrings to all endpoints\n- Create `docs/api.md` with examples\n\n### 5.2 Improve Code Documentation\n**Recommendation:**\n- Add docstrings to all public functions/classes\n- Use Google or NumPy style consistently\n- Add type hints everywhere\n\n### 5.3 Create Architecture Documentation\n**Recommendation:** Create `docs/ARCHITECTURE.md`:\n- System architecture diagram\n- Data flow diagrams\n- Component interactions\n- Design decisions\n\n## 6. Configuration & Environment\n\n### 6.1 Create `.env.example`\n**Recommendation:**\n```env\n# DeepSeek API\nDEEPSEEK_API_KEY=your_api_key_here\nDEEPSEEK_MODEL=deepseek-reasoner\n\n# Google OAuth\nGOOGLE_CREDENTIALS_PATH=credentials.json\n\n# Attachments\nATTACHMENTS_DIR=attachments\n\n# Server\nPORT=8000\nHOST=0.0.0.0\n```\n\n### 6.2 Add Configuration Validation\n**Recommendation:** Use Pydantic Settings:\n```python\nfrom pydantic_settings import BaseSettings\n\nclass Settings(BaseSettings):\n deepseek_api_key: str\n deepseek_model: str = \"deepseek-reasoner\"\n # ... with validation\n```\n\n## 7. Dependency Management\n\n### 7.1 Pin Dependency Versions\n**Recommendation:** Use exact versions or version ranges:\n```toml\ndependencies = [\n \"fastapi==0.104.1\", # or use ~= for compatible versions\n # ...\n]\n```\n\n### 7.2 Separate Production/Development Dependencies\n**Recommendation:** Use optional dependencies groups.\n\n## 8. Logging Improvements\n\n### 8.1 Structured Logging\n**Recommendation:**\n- Use `structlog` for structured logging\n- Add request IDs for tracing\n- Configure log levels via environment\n\n### 8.2 Logging Configuration\n**Recommendation:** Create `email_attachment_classifier/logging.py`:\n```python\nimport logging\nimport sys\nfrom logging.config import dictConfig\n\ndef setup_logging(level: str = \"INFO\"):\n dictConfig({\n \"version\": 1,\n \"formatters\": {...},\n \"handlers\": {...},\n \"loggers\": {...},\n })\n```\n\n## 9. Security Improvements\n\n### 9.1 Environment Variable Security\n**Recommendation:**\n- Never commit `.env` files\n- Use secrets management in production\n- Validate all environment variables\n\n### 9.2 API Security\n**Recommendation:**\n- Add authentication/authorization\n- Rate limiting\n- Input validation\n- CORS configuration for production\n\n## 10. CI/CD Pipeline\n\n### 10.1 GitHub Actions\n**Recommendation:** Create `.github/workflows/ci.yml`:\n```yaml\nname: CI\non: [push, pull_request]\njobs:\n test:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v3\n - uses: actions/setup-python@v4\n - run: uv sync\n - run: pytest\n - run: ruff check\n - run: mypy email_attachment_classifier\n```\n\n## Priority Recommendations\n\n### High Priority (Do First)\n1. ✅ Move `prompt.py` into package structure\n2. ✅ Create `.env.example` file\n3. ✅ Add development dependencies\n4. ✅ Set up basic test structure\n5. ✅ Add type hints and improve error handling\n\n### Medium Priority\n1. Separate API routes\n2. Add pre-commit hooks\n3. Improve logging configuration\n4. Add API documentation\n5. Create architecture docs\n\n### Low Priority (Nice to Have)\n1. CI/CD pipeline\n2. Advanced testing (integration tests)\n3. Performance monitoring\n4. Advanced security features\n\n## Implementation Order\n\n1. **Week 1:** Structure improvements (move files, organize)\n2. **Week 2:** Development tooling (linting, type checking, tests)\n3. **Week 3:** Code quality (error handling, logging, docs)\n4. **Week 4:** Advanced features (CI/CD, monitoring)\n\n"}, "10fac5c0e9407bc02ac3339d93de6b5a44a41f0f": {"t": 1785765017.4225256, "n": 8397, "h": "d34fc654a0f9ef59", "c": "# Project Structure & Maintainability Improvements\n\nThis document outlines recommended improvements to enhance the project's structure, maintainability, and developer experience.\n\n## 1. Project Structure Improvements\n\n### 1.1 Move `prompt.py` into Package\n**Current Issue:** `prompt.py` is at the root level but is imported by package code.\n\n**Recommendation:** Move to `email_attachment_classifier/prompts/` directory.\n\n```\nemail_attachment_classifier/\n prompts/\n __init__.py\n email.py # email_analysis_prompt\n pdf.py # pdf_analysis_prompt\n excel.py # excel_analysis_prompt\n word.py # word_analysis_prompt\n default.py # default_file_analysis_prompt\n```\n\n**Benefits:**\n- Better organization\n- Clearer module boundaries\n- Easier to test and maintain\n\n### 1.2 Organize Entry Points\n**Current Issue:** `main.py` and `server.py` are at root level.\n\n**Recommendation:** Create `scripts/` or `cli/` directory:\n```\nscripts/\n cli.py # Move main.py here\n server.py # Move server.py here (or keep at root for uvicorn)\n```\n\n**Alternative:** Keep entry points at root but create proper CLI structure:\n```\nemail_attachment_classifier/\n cli/\n __init__.py\n main.py # CLI logic\n server.py # Server logic (or keep at root)\n```\n\n### 1.3 Create Proper Test Structure\n**Current Issue:** Empty `tests/` directory.\n\n**Recommendation:**\n```\ntests/\n __init__.py\n conftest.py # Pytest fixtures\n unit/\n test_gmail.py\n test_drive.py\n test_decryption.py\n test_workflow.py\n integration/\n test_api.py\n test_cli.py\n fixtures/\n sample_emails.json\n sample_attachments/\n```\n\n## 2. Code Organization Improvements\n\n### 2.1 Separate API Routes\n**Current Issue:** All routes in `server.py`.\n\n**Recommendation:** Create `email_attachment_classifier/api/`:\n```\nemail_attachment_classifier/\n api/\n __init__.py\n routes/\n __init__.py\n classify.py\n health.py\n dependencies.py # FastAPI dependencies\n middleware.py # Custom middleware\n exceptions.py # API-specific exception handlers\n```\n\n### 2.2 Extract Business Logic from Server\n**Current Issue:** Business logic mixed with API handlers.\n\n**Recommendation:** Create service layer:\n```\nemail_attachment_classifier/\n services/\n email_service.py # Email classification orchestration\n password_service.py # Password handling logic\n```\n\n### 2.3 Improve Error Handling\n**Current Issue:** Generic exception handling in routes.\n\n**Recommendation:**\n- Create custom exception handlers in FastAPI\n- Use proper HTTP status codes\n- Add structured error responses\n\n### 2.4 Configuration Management\n**Current Issue:** Basic config, no validation.\n\n**Recommendation:**\n- Add Pydantic Settings for better validation\n- Create `.env.example` file\n- Add config validation on startup\n\n## 3. Development Tooling\n\n### 3.1 Add Development Dependencies\n**Recommendation:** Update `pyproject.toml`:\n```toml\n[project.optional-dependencies]\ndev = [\n \"pytest>=7.4.0\",\n \"pytest-asyncio>=0.21.0\",\n \"pytest-cov>=4.1.0\",\n \"black>=23.0.0\",\n \"ruff>=0.1.0\",\n \"mypy>=1.5.0\",\n \"types-requests\",\n]\n```\n\n### 3.2 Add Pre-commit Hooks\n**Recommendation:** Create `.pre-commit-config.yaml`:\n```yaml\nrepos:\n - repo: https://github.com/pre-commit/pre-commit-hooks\n rev: v4.5.0\n hooks:\n - id: trailing-whitespace\n - id: end-of-file-fixer\n - id: check-yaml\n - id: check-added-large-files\n - repo: https://github.com/psf/black\n rev: 23.12.0\n hooks:\n - id: black\n - repo: https://github.com/astral-sh/ruff-pre-commit\n rev: v0.1.6\n hooks:\n - id: ruff\n```\n\n### 3.3 Add Type Checking\n**Recommendation:** Create `pyproject.toml` section:\n```toml\n[tool.mypy]\npython_version = \"3.12\"\nwarn_return_any = true\nwarn_unused_configs = true\ndisallow_untyped_defs = false\n```\n\n### 3.4 Add Linting Configuration\n**Recommendation:** Create `ruff.toml` or add to `pyproject.toml`:\n```toml\n[tool.ruff]\nline-length = 100\ntarget-version = \"py312\"\n\n[tool.ruff.lint]\nselect = [\"E\", \"F\", \"I\", \"N\", \"W\", \"UP\"]\nignore = [\"E501\"] # Line too long (handled by black)\n```\n\n## 4. Testing Infrastructure\n\n### 4.1 Add Pytest Configuration\n**Recommendation:** Create `pytest.ini` or add to `pyproject.toml`:\n```toml\n[tool.pytest.ini_options]\ntestpaths = [\"tests\"]\npython_files = [\"test_*.py\"]\npython_classes = [\"Test*\"]\npython_functions = [\"test_*\"]\naddopts = [\n \"--strict-markers\",\n \"--cov=email_attachment_classifier\",\n \"--cov-report=term-missing\",\n \"--cov-report=html\",\n]\nmarkers = [\n \"unit: Unit tests\",\n \"integration: Integration tests\",\n \"slow: Slow running tests\",\n]\n```\n\n### 4.2 Add Test Fixtures\n**Recommendation:** Create `tests/conftest.py` with:\n- Mock Gmail service\n- Mock Drive service\n- Sample email data\n- LLM response mocks\n\n## 5. Documentation Improvements\n\n### 5.1 Add API Documentation\n**Recommendation:**\n- Use FastAPI's automatic OpenAPI docs\n- Add detailed docstrings to all endpoints\n- Create `docs/api.md` with examples\n\n### 5.2 Improve Code Documentation\n**Recommendation:**\n- Add docstrings to all public functions/classes\n- Use Google or NumPy style consistently\n- Add type hints everywhere\n\n### 5.3 Create Architecture Documentation\n**Recommendation:** Create `docs/ARCHITECTURE.md`:\n- System architecture diagram\n- Data flow diagrams\n- Component interactions\n- Design decisions\n\n## 6. Configuration & Environment\n\n### 6.1 Create `.env.example`\n**Recommendation:**\n```env\n# DeepSeek API\nDEEPSEEK_API_KEY=your_api_key_here\nDEEPSEEK_MODEL=deepseek-reasoner\n\n# Google OAuth\nGOOGLE_CREDENTIALS_PATH=credentials.json\n\n# Attachments\nATTACHMENTS_DIR=attachments\n\n# Server\nPORT=8000\nHOST=0.0.0.0\n```\n\n### 6.2 Add Configuration Validation\n**Recommendation:** Use Pydantic Settings:\n```python\nfrom pydantic_settings import BaseSettings\n\nclass Settings(BaseSettings):\n deepseek_api_key: str\n deepseek_model: str = \"deepseek-reasoner\"\n # ... with validation\n```\n\n## 7. Dependency Management\n\n### 7.1 Pin Dependency Versions\n**Recommendation:** Use exact versions or version ranges:\n```toml\ndependencies = [\n \"fastapi==0.104.1\", # or use ~= for compatible versions\n # ...\n]\n```\n\n### 7.2 Separate Production/Development Dependencies\n**Recommendation:** Use optional dependencies groups.\n\n## 8. Logging Improvements\n\n### 8.1 Structured Logging\n**Recommendation:**\n- Use `structlog` for structured logging\n- Add request IDs for tracing\n- Configure log levels via environment\n\n### 8.2 Logging Configuration\n**Recommendation:** Create `email_attachment_classifier/logging.py`:\n```python\nimport logging\nimport sys\nfrom logging.config import dictConfig\n\ndef setup_logging(level: str = \"INFO\"):\n dictConfig({\n \"version\": 1,\n \"formatters\": {...},\n \"handlers\": {...},\n \"loggers\": {...},\n })\n```\n\n## 9. Security Improvements\n\n### 9.1 Environment Variable Security\n**Recommendation:**\n- Never commit `.env` files\n- Use secrets management in production\n- Validate all environment variables\n\n### 9.2 API Security\n**Recommendation:**\n- Add authentication/authorization\n- Rate limiting\n- Input validation\n- CORS configuration for production\n\n## 10. CI/CD Pipeline\n\n### 10.1 GitHub Actions\n**Recommendation:** Create `.github/workflows/ci.yml`:\n```yaml\nname: CI\non: [push, pull_request]\njobs:\n test:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v3\n - uses: actions/setup-python@v4\n - run: uv sync\n - run: pytest\n - run: ruff check\n - run: mypy email_attachment_classifier\n```\n\n## Priority Recommendations\n\n### High Priority (Do First)\n1. ✅ Move `prompt.py` into package structure\n2. ✅ Create `.env.example` file\n3. ✅ Add development dependencies\n4. ✅ Set up basic test structure\n5. ✅ Add type hints and improve error handling\n\n### Medium Priority\n1. Separate API routes\n2. Add pre-commit hooks\n3. Improve logging configuration\n4. Add API documentation\n5. Create architecture docs\n\n### Low Priority (Nice to Have)\n1. CI/CD pipeline\n2. Advanced testing (integration tests)\n3. Performance monitoring\n4. Advanced security features\n\n## Implementation Order\n\n1. **Week 1:** Structure improvements (move files, organize)\n2. **Week 2:** Development tooling (linting, type checking, tests)\n3. **Week 3:** Code quality (error handling, logging, docs)\n4. **Week 4:** Advanced features (CI/CD, monitoring)\n\n"}}}} |