{"repo": "bughouse-wizzard/test-nmc-mvp", "files": {"README.md": {"9545a7441f8261a95493a9540a454b40d69af6c9": {"t": 1785746656.0646763, "n": 6643, "h": "cc307c30363f5dbc", "c": "# NMCK Calculation System\n\nA system for calculating the initial maximum contract price (NMCK) by analyzing historical procurement contracts using AI and web scraping.\n\n## Features\n\n- **AI-Powered Contract Analysis**: Uses DeepSeek AI to extract technical specifications from contracts\n- **Web Scraping**: Automatically fetches contract data from zakupki.gov.ru\n- **Specification Comparison**: Compares target specifications with actual contract specifications\n- **NMCK Calculation**: Calculates the initial maximum contract price based on historical data\n- **Debug Logging**: Comprehensive logging of AI responses and parser HTML for debugging\n\n## Prerequisites\n\n- Docker and Docker Compose\n- DeepSeek API key (free tier available)\n\n## Quick Start\n\n### 1. Set up environment variables\n\nCopy the example environment file and configure your settings:\n\n```bash\ncp .env.example .env\n```\n\nEdit the `.env` file and set your DeepSeek API key:\n\n```env\n# DeepSeek AI API Configuration\nDEEPSEEK_API_KEY=your-deepseek-api-key-here\nDEEPSEEK_BASE_URL=https://api.deepseek.com\nDEEPSEEK_MODEL=deepseek-chat\n```\n\n### 2. Get a DeepSeek API Key\n\n1. Go to [DeepSeek Platform](https://platform.deepseek.com/)\n2. Sign up for a free account\n3. Navigate to the API section\n4. Generate a new API key\n5. Copy the key and paste it in your `.env` file\n\n### 3. Run the application\n\nStart the full stack with Docker Compose:\n\n```bash\ndocker-compose up -d\n```\n\nThis will start:\n- **Backend API** (port 8000)\n- **PostgreSQL database** (port 5432)\n- **Redis** (port 6379)\n- **Nginx** (port 80)\n- **Celery worker** for background tasks\n- **Celery beat** for scheduled tasks\n\n### 4. Access the application\n\n- **Frontend**: http://localhost\n- **Backend API**: http://localhost:8000\n- **API Documentation**: http://localhost:8000/docs\n\n## Project Structure\n\n```\n├── backend/              # FastAPI backend application\n│   ├── app/             # Main application code\n│   │   ├── api/         # API endpoints\n│   │   ├── core/        # Core configuration and utilities\n│   │   ├── models/      # Database models\n│   │   ├── schemas/     # Pydantic schemas\n│   │   └── worker/      # Celery worker tasks\n│   ├── services/        # Business logic services\n│   │   ├── ai/          # AI client for DeepSeek API\n│   │   └── parser/      # Web parsers for zakupki.gov.ru\n│   └── tests/           # Test suite\n├── frontend/            # React frontend application\n├── nginx/               # Nginx configuration\n├── alembic/             # Database migrations\n└── docker-compose.yml   # Docker Compose configuration\n```\n\n## Configuration\n\n### Environment Variables\n\nKey environment variables to configure:\n\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `DEEPSEEK_API_KEY` | **Required**: Your DeepSeek API key | - |\n| `DEBUG` | Enable debug mode | `false` |\n| `DATABASE_URL` | PostgreSQL connection URL | `postgresql://postgres:postgres@db:5432/nmck_db` |\n| `REDIS_URL` | Redis connection URL | `redis://redis:6379/0` |\n| `ZAKUPKI_BASE_URL` | Base URL for procurement portal | `https://zakupki.gov.ru` |\n| `CRAWL_DELAY` | Delay between requests (respects robots.txt) | `60` |\n| `WORKER_CONCURRENCY` | Number of concurrent worker processes | `2` |\n\n### Database Migrations\n\nThe application uses Alembic for database migrations. To apply migrations:\n\n```bash\n# Run inside the backend container\ndocker-compose exec backend alembic upgrade head\n```\n\nTo create a new migration after model changes:\n\n```bash\ndocker-compose exec backend alembic revision --autogenerate -m \"Description of changes\"\n```\n\n## Debugging and Logging\n\n### AI Response Debugging\n\nThe system logs all AI request and response data for debugging:\n\n1. **Raw AI responses** are stored in the `debug_data` field of `ContractResult` records\n2. **Parser HTML** is stored in both `SearchRequest.debug_data` and `ContractResult.raw_data_json.debug_data`\n3. **Debug data includes**:\n   - AI request parameters and response metadata\n   - Raw HTML from zakupki.gov.ru (first 5000 characters)\n   - Processing timestamps\n   - Error information\n\n### Accessing Debug Data\n\nDebug data can be accessed through:\n- Database queries on the `debug_data` JSON field\n- API responses (when debug mode is enabled)\n- Application logs\n\n### Robots.txt Compliance\n\nThe system respects zakupki.gov.ru's robots.txt requirements:\n- **Crawl delay**: 60 seconds between requests (configured via `CRAWL_DELAY`)\n- **Respects allowed/disallowed paths**\n- **Implements exponential backoff** for rate limiting\n\n## API Usage\n\n### Search for Contracts\n\n```bash\ncurl -X POST \"http://localhost:8000/api/v1/search\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"object_name\": \"Laptop\",\n    \"ktru_code\": \"32.30.11.110\",\n    \"customer_region\": \"СЗФО\",\n    \"date_from\": \"2024-01-01\",\n    \"date_to\": \"2024-12-31\"\n  }'\n```\n\n### Check Search Status\n\n```bash\ncurl \"http://localhost:8000/api/v1/search/{search_id}/status\"\n```\n\n### Get Search Results\n\n```bash\ncurl \"http://localhost:8000/api/v1/search/{search_id}/results\"\n```\n\n## Development\n\n### Running Tests\n\n```bash\n# Run tests inside the backend container\ndocker-compose exec backend pytest\n```\n\n### Code Quality\n\n```bash\n# Format code with black\ndocker-compose exec backend black .\n\n# Check code style with flake8\ndocker-compose exec backend flake8 .\n\n# Type checking with mypy\ndocker-compose exec backend mypy .\n```\n\n### Local Development Without Docker\n\n1. Install Python dependencies:\n   ```bash\n   cd backend\n   pip install -r requirements.txt\n   ```\n\n2. Set up environment variables:\n   ```bash\n   export DEEPSEEK_API_KEY=your-key-here\n   export DATABASE_URL=postgresql://postgres:postgres@localhost:5432/nmck_db\n   ```\n\n3. Run the application:\n   ```bash\n   uvicorn app.main:app --reload --host 0.0.0.0 --port 8000\n   ```\n\n## Troubleshooting\n\n### Common Issues\n\n1. **AI API errors**: Verify your `DEEPSEEK_API_KEY` is valid and has sufficient credits\n2. **Database connection errors**: Ensure PostgreSQL is running and accessible\n3. **Parser errors**: Check network connectivity to zakupki.gov.ru\n4. **Rate limiting**: The system implements retry logic with exponential backoff\n\n### Logs\n\nView application logs:\n\n```bash\n# Backend logs\ndocker-compose logs backend\n\n# Worker logs\ndocker-compose logs worker\n\n# Database logs\ndocker-compose logs db\n```\n\n### Database Reset\n\nTo reset the database:\n\n```bash\n# Stop containers\ndocker-compose down\n\n# Remove volumes\ndocker-compose down -v\n\n# Restart\ndocker-compose up -d\n```\n\n## License\n\n[Add your license information here]\n\n## Support\n\nFor issues and feature requests, please use the issue tracker or contact the development team."}, "ebc2c99fc8f147e7526c265151df4b35e83d1056": {"t": 1785766638.4093218, "n": 6643, "h": "cc307c30363f5dbc", "c": "# NMCK Calculation System\n\nA system for calculating the initial maximum contract price (NMCK) by analyzing historical procurement contracts using AI and web scraping.\n\n## Features\n\n- **AI-Powered Contract Analysis**: Uses DeepSeek AI to extract technical specifications from contracts\n- **Web Scraping**: Automatically fetches contract data from zakupki.gov.ru\n- **Specification Comparison**: Compares target specifications with actual contract specifications\n- **NMCK Calculation**: Calculates the initial maximum contract price based on historical data\n- **Debug Logging**: Comprehensive logging of AI responses and parser HTML for debugging\n\n## Prerequisites\n\n- Docker and Docker Compose\n- DeepSeek API key (free tier available)\n\n## Quick Start\n\n### 1. Set up environment variables\n\nCopy the example environment file and configure your settings:\n\n```bash\ncp .env.example .env\n```\n\nEdit the `.env` file and set your DeepSeek API key:\n\n```env\n# DeepSeek AI API Configuration\nDEEPSEEK_API_KEY=your-deepseek-api-key-here\nDEEPSEEK_BASE_URL=https://api.deepseek.com\nDEEPSEEK_MODEL=deepseek-chat\n```\n\n### 2. Get a DeepSeek API Key\n\n1. Go to [DeepSeek Platform](https://platform.deepseek.com/)\n2. Sign up for a free account\n3. Navigate to the API section\n4. Generate a new API key\n5. Copy the key and paste it in your `.env` file\n\n### 3. Run the application\n\nStart the full stack with Docker Compose:\n\n```bash\ndocker-compose up -d\n```\n\nThis will start:\n- **Backend API** (port 8000)\n- **PostgreSQL database** (port 5432)\n- **Redis** (port 6379)\n- **Nginx** (port 80)\n- **Celery worker** for background tasks\n- **Celery beat** for scheduled tasks\n\n### 4. Access the application\n\n- **Frontend**: http://localhost\n- **Backend API**: http://localhost:8000\n- **API Documentation**: http://localhost:8000/docs\n\n## Project Structure\n\n```\n├── backend/              # FastAPI backend application\n│   ├── app/             # Main application code\n│   │   ├── api/         # API endpoints\n│   │   ├── core/        # Core configuration and utilities\n│   │   ├── models/      # Database models\n│   │   ├── schemas/     # Pydantic schemas\n│   │   └── worker/      # Celery worker tasks\n│   ├── services/        # Business logic services\n│   │   ├── ai/          # AI client for DeepSeek API\n│   │   └── parser/      # Web parsers for zakupki.gov.ru\n│   └── tests/           # Test suite\n├── frontend/            # React frontend application\n├── nginx/               # Nginx configuration\n├── alembic/             # Database migrations\n└── docker-compose.yml   # Docker Compose configuration\n```\n\n## Configuration\n\n### Environment Variables\n\nKey environment variables to configure:\n\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `DEEPSEEK_API_KEY` | **Required**: Your DeepSeek API key | - |\n| `DEBUG` | Enable debug mode | `false` |\n| `DATABASE_URL` | PostgreSQL connection URL | `postgresql://postgres:postgres@db:5432/nmck_db` |\n| `REDIS_URL` | Redis connection URL | `redis://redis:6379/0` |\n| `ZAKUPKI_BASE_URL` | Base URL for procurement portal | `https://zakupki.gov.ru` |\n| `CRAWL_DELAY` | Delay between requests (respects robots.txt) | `60` |\n| `WORKER_CONCURRENCY` | Number of concurrent worker processes | `2` |\n\n### Database Migrations\n\nThe application uses Alembic for database migrations. To apply migrations:\n\n```bash\n# Run inside the backend container\ndocker-compose exec backend alembic upgrade head\n```\n\nTo create a new migration after model changes:\n\n```bash\ndocker-compose exec backend alembic revision --autogenerate -m \"Description of changes\"\n```\n\n## Debugging and Logging\n\n### AI Response Debugging\n\nThe system logs all AI request and response data for debugging:\n\n1. **Raw AI responses** are stored in the `debug_data` field of `ContractResult` records\n2. **Parser HTML** is stored in both `SearchRequest.debug_data` and `ContractResult.raw_data_json.debug_data`\n3. **Debug data includes**:\n   - AI request parameters and response metadata\n   - Raw HTML from zakupki.gov.ru (first 5000 characters)\n   - Processing timestamps\n   - Error information\n\n### Accessing Debug Data\n\nDebug data can be accessed through:\n- Database queries on the `debug_data` JSON field\n- API responses (when debug mode is enabled)\n- Application logs\n\n### Robots.txt Compliance\n\nThe system respects zakupki.gov.ru's robots.txt requirements:\n- **Crawl delay**: 60 seconds between requests (configured via `CRAWL_DELAY`)\n- **Respects allowed/disallowed paths**\n- **Implements exponential backoff** for rate limiting\n\n## API Usage\n\n### Search for Contracts\n\n```bash\ncurl -X POST \"http://localhost:8000/api/v1/search\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"object_name\": \"Laptop\",\n    \"ktru_code\": \"32.30.11.110\",\n    \"customer_region\": \"СЗФО\",\n    \"date_from\": \"2024-01-01\",\n    \"date_to\": \"2024-12-31\"\n  }'\n```\n\n### Check Search Status\n\n```bash\ncurl \"http://localhost:8000/api/v1/search/{search_id}/status\"\n```\n\n### Get Search Results\n\n```bash\ncurl \"http://localhost:8000/api/v1/search/{search_id}/results\"\n```\n\n## Development\n\n### Running Tests\n\n```bash\n# Run tests inside the backend container\ndocker-compose exec backend pytest\n```\n\n### Code Quality\n\n```bash\n# Format code with black\ndocker-compose exec backend black .\n\n# Check code style with flake8\ndocker-compose exec backend flake8 .\n\n# Type checking with mypy\ndocker-compose exec backend mypy .\n```\n\n### Local Development Without Docker\n\n1. Install Python dependencies:\n   ```bash\n   cd backend\n   pip install -r requirements.txt\n   ```\n\n2. Set up environment variables:\n   ```bash\n   export DEEPSEEK_API_KEY=your-key-here\n   export DATABASE_URL=postgresql://postgres:postgres@localhost:5432/nmck_db\n   ```\n\n3. Run the application:\n   ```bash\n   uvicorn app.main:app --reload --host 0.0.0.0 --port 8000\n   ```\n\n## Troubleshooting\n\n### Common Issues\n\n1. **AI API errors**: Verify your `DEEPSEEK_API_KEY` is valid and has sufficient credits\n2. **Database connection errors**: Ensure PostgreSQL is running and accessible\n3. **Parser errors**: Check network connectivity to zakupki.gov.ru\n4. **Rate limiting**: The system implements retry logic with exponential backoff\n\n### Logs\n\nView application logs:\n\n```bash\n# Backend logs\ndocker-compose logs backend\n\n# Worker logs\ndocker-compose logs worker\n\n# Database logs\ndocker-compose logs db\n```\n\n### Database Reset\n\nTo reset the database:\n\n```bash\n# Stop containers\ndocker-compose down\n\n# Remove volumes\ndocker-compose down -v\n\n# Restart\ndocker-compose up -d\n```\n\n## License\n\n[Add your license information here]\n\n## Support\n\nFor issues and feature requests, please use the issue tracker or contact the development team."}}, "config.yaml": {"9545a7441f8261a95493a9540a454b40d69af6c9": {"t": 1786068425.9361262, "n": 4628, "h": "4b09b0e182e12d40", "c": "# DeepSeek AI Configuration\n# This file contains configuration for DeepSeek AI client\n\ndeepseek:\n  # API Configuration\n  api_key: ${DEEPSEEK_API_KEY}  # Will be loaded from environment variable\n  base_url: \"https://api.deepseek.com\"\n  model: \"deepseek-chat\"\n  \n  # Model Parameters\n  temperature: 0.1  # Low temperature for more deterministic outputs\n  max_tokens: 4000  # Maximum tokens in response\n  top_p: 0.95\n  frequency_penalty: 0.0\n  presence_penalty: 0.0\n  \n  # Request Settings\n  timeout: 30  # seconds\n  max_retries: 3\n  retry_delay: 2  # seconds between retries\n  \n  # Response Format\n  response_format:\n    type: \"json_object\"  # Ensure JSON responses\n\n# Prompt Templates Configuration\nprompts:\n  extract_specs_from_tz:\n    system_prompt: |\n      You are a technical specification extraction expert. Your task is to analyze raw text or file content and extract structured technical specifications in JSON format.\n      \n      Extract the following information:\n      1. Product/service name\n      2. Technical parameters with their values and units\n      3. Quantity requirements\n      4. Quality standards or certifications\n      5. Delivery requirements\n      6. Any special conditions or notes\n      \n      Return the result as a valid JSON object with the structure:\n      {\n        \"items\": [\n          {\n            \"name\": \"string\",\n            \"parameters\": [\n              {\n                \"name\": \"string\",\n                \"value\": \"string|number\",\n                \"unit\": \"string\"\n              }\n            ],\n            \"quantity\": \"string|number\",\n            \"quality_standards\": [\"string\"],\n            \"delivery_requirements\": \"string\",\n            \"notes\": \"string\"\n          }\n        ]\n      }\n  \n  extract_specs_from_contract_doc:\n    system_prompt: |\n      You are a contract document analysis expert. Your task is to analyze parsed PDF/DOCX text from contract documents and extract structured technical specifications in JSON format.\n      \n      Extract the following information:\n      1. Contract item/product name\n      2. Technical specifications and requirements\n      3. Quantity and unit of measurement\n      4. Quality requirements and standards\n      5. Delivery schedule and terms\n      6. Price information (if available)\n      7. Contract-specific conditions\n      \n      Return the result as a valid JSON object with the structure:\n      {\n        \"contract_items\": [\n          {\n            \"item_name\": \"string\",\n            \"description\": \"string\",\n            \"technical_specs\": [\n              {\n                \"parameter\": \"string\",\n                \"requirement\": \"string\",\n                \"unit\": \"string\"\n              }\n            ],\n            \"quantity\": \"string|number\",\n            \"unit_of_measurement\": \"string\",\n            \"quality_requirements\": [\"string\"],\n            \"delivery_schedule\": \"string\",\n            \"price\": \"string|number\",\n            \"currency\": \"string\",\n            \"contract_conditions\": [\"string\"]\n          }\n        ]\n      }\n  \n  compare_specs_and_score:\n    system_prompt: |\n      You are a technical specification comparison expert. Your task is to compare target specifications with actual specifications and calculate a match score from 0-100.\n      \n      Compare the following:\n      1. Technical parameter matches\n      2. Quantity requirements\n      3. Quality standards compliance\n      4. Delivery requirements alignment\n      \n      Determine the match type:\n      - Identical: All requirements match exactly\n      - Homogeneous: Requirements are functionally equivalent or within acceptable tolerance\n      - Partial: Some requirements match but significant differences exist\n      - Mismatch: Major differences or incompatible requirements\n      \n      Return the result as a valid JSON object with the structure:\n      {\n        \"score\": 0-100,\n        \"match_type\": \"Identical|Homogeneous|Partial|Mismatch\",\n        \"details\": {\n          \"parameter_matches\": [\n            {\n              \"parameter\": \"string\",\n              \"target_value\": \"string|number\",\n              \"actual_value\": \"string|number\",\n              \"match\": true|false,\n              \"notes\": \"string\"\n            }\n          ],\n          \"quantity_match\": true|false,\n          \"quality_standards_match\": true|false,\n          \"delivery_match\": true|false\n        },\n        \"summary\": \"string\"\n      }\n\n# Logging Configuration\nlogging:\n  level: \"INFO\"\n  format: \"%(asctime)s - %(name)s - %(levelname)s - %(message)s\"\n  \n# Validation Settings\nvalidation:\n  require_json: true\n  max_response_size: 10000  # characters\n  timeout_validation: true"}}}}