1 line
33 KiB
JSON
1 line
33 KiB
JSON
{"repo": "backearth1/comfyui-minimax-video", "files": {"README.md": {"91878e7b149c4efa97a076f9cba8ed492f099822": {"t": 1785674030.5073493, "n": 7129, "h": "b76188d65e9b5fc2", "c": "# ComfyUI MiniMax Video Extension\r\n\r\n[English](#english) | [中文](#中文)\r\n\r\n# English\r\n\r\n## Overview\r\nA ComfyUI extension that integrates MiniMax AI's image-to-video, text-to-video, image-to-prompt generation capabilities, and image generation capabilities, allowing users to easily convert static images into dynamic videos with optimized prompts and generate high-quality images.\r\n\r\n\r\n\r\n## Features\r\n- Image to video conversion\r\n- Text to video conversion\r\n- Image to prompt generation\r\n- AI image generation\r\n- Support for multiple video generation models\r\n- Real-time video preview\r\n- Custom prompt optimization\r\n- Flexible API configuration\r\n\r\n## Installation\r\n\r\n1. Clone this repository to your ComfyUI custom_nodes directory:\r\n```bash\r\ncd custom_nodes\r\ngit clone https://github.com/backearth1/ComfyUI-MiniMax-Video.git\r\n```\r\n\r\n2. Install required dependencies:\r\n```bash\r\npip install -r requirements.txt\r\n```\r\n\r\n## Node Parameters\r\n\r\n### MiniMax API Client Node\r\n| Parameter | Description | Type | Default |\r\n|-----------|-------------|------|---------|\r\n| api_key | MiniMax API Key | STRING | - |\r\n| api_url | API Base URL | [\"https://api.minimax.chat/v1\"] | \"https://api.minimax.chat/v1\" |\r\n\r\nhttps://api.minimax.chat/v1 for users in China；\r\nhttps://api.minimaxi.chat/v1 for users in other countries\r\n\r\n### Image to Prompt Node\r\n| Parameter | Description | Type | Default |\r\n|-----------|-------------|------|---------|\r\n| image | Input Image | IMAGE | - |\r\n| api_key | MiniMax API Key | API_KEY | - |\r\n| api_url | API Base URL | API_URL | - |\r\n| model | LLM Model | [\"MiniMax-Text-01\", \"abab6.5s-chat\"] | \"MiniMax-Text-01\" |\r\n| VLM_prompt | Custom VLM Prompt | STRING | Default prompt template |\r\n| temperature | Model Temperature | FLOAT | 0.1 |\r\n\r\n### Image to Video Node\r\n| Parameter | Description | Type | Default |\r\n|-----------|-------------|------|---------|\r\n| client | MiniMax API Client | MINIMAX_CLIENT | - |\r\n| image | Input Image | IMAGE | - |\r\n| prompt | Generation Prompt | STRING | \"\" |\r\n| model | Video Model | [T2V-01-Director,I2V-01-Director,S2V-01,I2V-01-live,I2V-01,T2V-01] | \"I2V-01\" |\r\n| prompt_optimizer | Enable Prompt Optimization | BOOLEAN | true |\r\n\r\n### MiniMax Image Generator Node\r\n| Parameter | Description | Type | Default |\r\n|-----------|-------------|------|---------|\r\n| api_key | MiniMax API Key | API_KEY | - |\r\n| api_url | API Base URL | API_URL | - |\r\n| prompt | Generation Prompt | STRING | \"\" |\r\n| model | Image Model | [\"image-xy01\", \"image-01\"] | \"image-xy01\" |\r\n| aspect_ratio | Image Aspect Ratio | [\"1:1\", \"16:9\", \"4:3\", \"3:2\", \"2:3\", \"3:4\", \"9:16\", \"21:9\"] | \"16:9\" |\r\n| n | Number of Images to Generate | INT | 1 |\r\n| prompt_optimizer | Enable Prompt Optimization | BOOLEAN | true |\r\n\r\n## Example Workflows\r\n\r\n### Basic Image to Video\r\n- Load an image\r\n- Connect to MiniMax API Client\r\n- Use Image to Video node to generate video\r\n- Preview the result\r\n\r\n### Image to Video with Prompt Optimization\r\n\r\n\r\n### AI fit clothes workflow\r\n\r\n\r\n### Subject reference\r\n\r\n\r\n### Image Generation\r\n\r\n- Connect to MiniMax API Client\r\n- Use MiniMax Image Generator node with a descriptive prompt\r\n- Connect the output to an image preview node or use in other workflows\r\n\r\n## Tips\r\n1. Ensure you have a valid MiniMax API key\r\n2. Video generation may take some time\r\n3. Generated videos are saved in ComfyUI's output directory\r\n4. Video files are named as \"time+trace_id\"\r\n5. Use clear frontal images for best results\r\n6. If you don't add any image, the node will support text to video\r\n7. If you want to add watermark, please check \"Add Watermark\" option in the node parameters\r\n8. If you open concurrent requests, please pay attention to the account configured with enough RPM\r\n9. The Image to Prompt node can help generate better video descriptions\r\n10. For image generation, detailed prompts yield better results\r\n\r\n## Troubleshooting\r\n\r\n**Q: Why did video generation fail?**\r\nA: Check:\r\n- API key validity\r\n- Network connection\r\n- Input image requirements\r\n- Prompt appropriateness\r\n\r\n**Q: How to get the best results?**\r\nA: Recommendations:\r\n- Use clear frontal images\r\n- Use Image to Prompt node for better descriptions\r\n- Write clear and specific prompts\r\n- Choose appropriate models\r\n\r\n---\r\n\r\n# 中文\r\n\r\n## 概述\r\nComfyUI MiniMax Video 扩展集成了 MiniMax AI 的图像转视频、文本转视频、图像转提示词生成功能和图像生成功能，让用户能够轻松地将静态图像转换为动态视频，并生成高质量图像。\r\n\r\n\r\n\r\n## 功能特点\r\n- 图片转视频\r\n- 文本转视频\r\n- 图像转提示词\r\n- AI 图像生成\r\n- 支持多种视频生成模型\r\n- 实时视频预览\r\n- 自定义提示词优化\r\n- 灵活的 API 配置\r\n\r\n## 安装方法\r\n\r\n1. 克隆仓库到 ComfyUI 的 custom_nodes 目录：\r\n```bash\r\ncd custom_nodes\r\ngit clone https://github.com/backearth1/ComfyUI-MiniMax-Video.git\r\n```\r\n\r\n2. 安装依赖：\r\n```bash\r\npip install -r requirements.txt\r\n```\r\n\r\n## 节点参数说明\r\n\r\n### MiniMax API 客户端节点\r\n| 参数 | 说明 | 类型 | 默认值 |\r\n|------|------|------|--------|\r\n| api_key | MiniMax API 密钥 | STRING | - |\r\n| api_url | API 基础地址 | [\"https://api.minimax.chat/v1\"] | \"https://api.minimax.chat/v1\" |\r\n\r\n中国用户 https://api.minimax.chat/v1 ；\r\n其他地区用户 https://api.minimaxi.chat/v1 \r\n\r\n### 图像转提示词节点\r\n| 参数 | 说明 | 类型 | 默认值 |\r\n|------|------|------|--------|\r\n| image | 输入图片 | IMAGE | - |\r\n| api_key | MiniMax API 密钥 | API_KEY | - |\r\n| api_url | API 基础地址 | API_URL | - |\r\n| model | 语言模型 | [\"MiniMax-Text-01\", \"abab6.5s-chat\"] | \"MiniMax-Text-01\" |\r\n| VLM_prompt | 自定义 VLM 提示词 | STRING | 默认提示词模板 |\r\n| temperature | 模型温度 | FLOAT | 0.1 |\r\n\r\n### 图像转视频节点\r\n| 参数 | 说明 | 类型 | 默认值 |\r\n|------|------|------|--------|\r\n| client | MiniMax API 客户端 | MINIMAX_CLIENT | - |\r\n| image | 输入图片 | IMAGE | - |\r\n| prompt | 生成提示词 | STRING | \"\" |\r\n| model | 视频模型 | [T2V-01-Director,I2V-01-Director,S2V-01,I2V-01-live,I2V-01,T2V-01] | \"I2V-01\" |\r\n| prompt_optimizer | 启用提示词优化 | BOOLEAN | true |\r\n| watermark | 添加水印 | BOOLEAN | false |\r\n\r\n### MiniMax 图像生成节点\r\n| 参数 | 说明 | 类型 | 默认值 |\r\n|------|------|------|--------|\r\n| api_key | MiniMax API 密钥 | API_KEY | - |\r\n| api_url | API 基础地址 | API_URL | - |\r\n| prompt | 生成提示词 | STRING | \"\" |\r\n| model | 图像模型 | [\"image-xy01\", \"image-01\"] | \"image-xy01\" |\r\n| aspect_ratio | 图像宽高比 | [\"1:1\", \"16:9\", \"4:3\", \"3:2\", \"2:3\", \"3:4\", \"9:16\", \"21:9\"] | \"16:9\" |\r\n| n | 生成图像数量 | INT | 1 |\r\n| prompt_optimizer | 启用提示词优化 | BOOLEAN | true |\r\n\r\n## 示例工作流\r\n\r\n### 基础图像转视频\r\n- 加载图像\r\n- 连接 MiniMax API 客户端\r\n- 使用图像转视频节点生成视频\r\n- 预览结果\r\n\r\n### 带提示词优化的图像转视频\r\n\r\n\r\n### AI试衣服工作流\r\n\r\n\r\n### 人脸主题参考\r\n\r\n\r\n### 图像生成\r\n\r\n- 连接 MiniMax API 客户端\r\n- 使用 MiniMax 图像生成节点并输入描述性提示词\r\n- 将输出连接到图像预览节点或用于其他工作流\r\n\r\n## 使用提示\r\n1. 确保拥有有效的 MiniMax API 密钥\r\n2. 视频生成可能需要一定时间\r\n3. 生成的视频会保存在 ComfyUI 的输出目录中\r\n4. 视频文件会以\"时间+trace_id\"命名\r\n5. 建议使用清晰的正面图片以获得最佳效果\r\n6. 如果未添加任何图片，节点将支持文本转视频\r\n7. 如果需要添加水印，请在节点参数中勾选\"添加水印\"选项\r\n8. 如果开启并发请求，请注意账号配置了足够的RPM\r\n9. 图像转提示词节点可以帮助生成更好的视频描述\r\n10. 对于图像生成，详细的提示词会产生更好的结果\r\n\r\n## 常见问题\r\n\r\n**Q: 为什么视频生成失败？**\r\nA: 请检查：\r\n- API 密钥是否有效\r\n- 网络连接是否正常\r\n- 输入图片是否符合要求\r\n- 提示词是否合适\r\n\r\n**Q: 如何获得最佳效果？**\r\nA: 建议：\r\n- 使用清晰的正面图片\r\n- 使用图像转提示词节点获取更好的描述\r\n- 编写清晰具体的提示词\r\n- 选择合适的模型\r\n\r\n## License\r\nMIT License\r\n\r\n## 贡献\r\n欢迎提交 Issues 和 Pull Requests！"}}, "nodes.py": {"91878e7b149c4efa97a076f9cba8ed492f099822": {"t": 1785674031.5022995, "n": 21034, "h": "6fd6b2586831e832", "c": "import torch\r\nimport numpy as np\r\nimport requests\r\nimport json\r\nimport base64\r\nimport time\r\nfrom PIL import Image\r\nimport io\r\nimport os\r\nimport urllib3\r\n\r\nclass MiniMaxPreviewVideo:\r\n def __init__(self):\r\n print(\"初始化 MiniMaxPreviewVideo 节点\")\r\n self.video_urls = []\r\n \r\n @classmethod\r\n def INPUT_TYPES(cls):\r\n return {\r\n \"required\": {\r\n \"video_url\": (\"STRING\", {\"forceInput\": True, \"multiline\": True}),\r\n }\r\n }\r\n \r\n OUTPUT_NODE = True\r\n FUNCTION = \"run\"\r\n CATEGORY = \"minimax\"\r\n RETURN_TYPES = ()\r\n \r\n def run(self, video_url):\r\n print(\"\\n=== MiniMaxPreviewVideo 节点执行 ===\")\r\n # 处理输入的视频URL\r\n if isinstance(video_url, (list, tuple)):\r\n urls = list(video_url)\r\n else:\r\n # 如果是字符串，尝试解析是否为JSON数组\r\n try:\r\n import json\r\n urls = json.loads(video_url) if isinstance(video_url, str) else [video_url]\r\n except:\r\n urls = [video_url]\r\n \r\n print(f\"接收到 {len(urls)} 个视频URL:\")\r\n for i, url in enumerate(urls):\r\n print(f\"视频 #{i+1}: {url}\")\r\n \r\n # 更新视频URL列表\r\n self.video_urls = urls\r\n \r\n # 返回UI更新信息\r\n return {\"ui\": {\"video_url\": urls}}\r\n\r\nclass MiniMaxAIAPIClient:\r\n def __init__(self):\r\n self.api_url = \"https://api.minimax.chat/v1\"\r\n \r\n @classmethod\r\n def INPUT_TYPES(s):\r\n return {\"required\": {\r\n \"api_key\": (\"STRING\", {\r\n \"default\": \"\", \r\n \"multiline\": True,\r\n \"display\": \"API Key\"\r\n }),\r\n \"api_url\": ([\"https://api.minimax.chat/v1\", \"https://api.minimaxi.chat/v1\"], {\r\n \"default\": \"https://api.minimax.chat/v1\"\r\n }),\r\n }}\r\n \r\n RETURN_TYPES = (\"API_KEY\", \"API_URL\",)\r\n RETURN_NAMES = (\"api_key\", \"api_url\",)\r\n FUNCTION = \"setup_client\"\r\n CATEGORY = \"minimax\"\r\n \r\n def setup_client(self, api_key, api_url):\r\n if not api_key.strip():\r\n raise ValueError(\"API key 不能为空\")\r\n return (api_key, api_url,)\r\n\r\nclass MiniMaxImage2Video:\r\n def __init__(self):\r\n self.output_dir = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(__file__)))), 'output')\r\n if not os.path.exists(self.output_dir):\r\n os.makedirs(self.output_dir)\r\n \r\n @classmethod\r\n def INPUT_TYPES(s):\r\n return {\r\n \"required\": {\r\n \"api_key\": (\"API_KEY\",),\r\n \"api_url\": (\"API_URL\",),\r\n \"model\": ([\"I2V-01\",\"T2V-01\",\"I2V-01-Director\",\"T2V-01-Director\", \"I2V-01-live\", \"S2V-01\"],), # 添加 S2V-01 模型\r\n \"prompt\": (\"STRING\", {\r\n \"default\": \"\", \r\n \"multiline\": True,\r\n \"display\": \"Prompt\"\r\n }),\r\n \"prompt_optimizer\": (\"BOOLEAN\", {\"default\": True}),\r\n \"watermark\": ([\"yes\", \"no\"], {\"default\": \"yes\"}),\r\n \"concurrent_tasks\": (\"INT\", {\r\n \"default\": 1,\r\n \"min\": 1,\r\n \"max\": 5,\r\n \"step\": 1,\r\n \"display\": \"并发生成数量\"\r\n }),\r\n },\r\n \"optional\": {\r\n \"first_frame_image\": (\"IMAGE\",), # 用于 I2V-01 , I2V-01-live和I2V-01-Director\r\n \"subject_reference_image\": (\"IMAGE\",), # 新增：用于 S2V-01 的主体参考图\r\n }\r\n }\r\n \r\n RETURN_TYPES = (\"STRING\",)\r\n RETURN_NAMES = (\"video_url\",)\r\n OUTPUT_NODE = True\r\n FUNCTION = \"generate_video\"\r\n CATEGORY = \"minimax\"\r\n \r\n @classmethod\r\n def IS_CHANGED(s):\r\n return float(\"NaN\") # 确保节点每次都会执行\r\n \r\n def encode_image(self, image_tensor):\r\n print(\"输入图像形状:\", image_tensor.shape)\r\n print(\"输入图像类型:\", image_tensor.dtype)\r\n \r\n # 将 tensor 转换为 PIL Image\r\n if isinstance(image_tensor, torch.Tensor):\r\n if len(image_tensor.shape) == 4:\r\n image_tensor = image_tensor[0] # 移除批次维度\r\n \r\n # 确保数值在 0-1 之间\r\n if image_tensor.max() > 1.0:\r\n image_tensor = image_tensor / 255.0\r\n \r\n # 转换为 numpy 数组\r\n image_np = (image_tensor.cpu().numpy() * 255).astype(np.uint8)\r\n \r\n # 已经是 (H, W, C) 格式，不需要转置\r\n \r\n # 创建 PIL 图像\r\n image = Image.fromarray(image_np, 'RGB')\r\n print(f\"处理后的图像尺寸: {image.size}\")\r\n else:\r\n raise ValueError(\"输入必须是 torch.Tensor 类型\")\r\n \r\n # 转换为 base64\r\n buffered = io.BytesIO()\r\n image.save(buffered, format=\"PNG\", quality=95) # 使用高质量设置\r\n return base64.b64encode(buffered.getvalue()).decode('utf-8')\r\n\r\n def invoke_video_generation(self, api_key, api_url, prompt, model, base64_image=None, base64_subject_image=None):\r\n print(\"-----------------提交视频生成任务-----------------\")\r\n url = f\"{api_url}/video_generation\"\r\n \r\n # 构建基本 payload\r\n payload = {\r\n \"prompt\": prompt,\r\n \"model\": model,\r\n \"prompt_optimizer\": self.prompt_optimizer,\r\n }\r\n \r\n # 根据watermark选项添加水印参数\r\n if self.watermark == \"yes\":\r\n payload[\"watermark\"] = \"hailuo\"\r\n \r\n # 根据不同模型添加不同的图像参数\r\n if model == \"S2V-01\":\r\n if base64_subject_image is not None:\r\n payload[\"subject_reference\"] = [{\r\n \"type\": \"character\",\r\n \"image\": [f\"data:image/png;base64,{base64_subject_image}\"]\r\n }]\r\n else:\r\n if base64_image is not None:\r\n payload[\"first_frame_image\"] = f\"data:image/png;base64,{base64_image}\"\r\n \r\n headers = {\r\n 'authorization': f'Bearer {api_key}',\r\n 'content-type': 'application/json',\r\n }\r\n \r\n # 添加 SSL 验证选项\r\n session = requests.Session()\r\n session.verify = False # 禁用 SSL 验证\r\n \r\n # 忽略 SSL 警告\r\n import urllib3\r\n urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)\r\n \r\n try:\r\n response = session.post(url, headers=headers, data=json.dumps(payload))\r\n response_json = response.json()\r\n print(\"API Response:\", response.text)\r\n \r\n # 保存 trace_id 为类属性，以便后续使用\r\n self.trace_id = response.headers.get('Trace-ID', '')\r\n print(\"Trace-ID:\", self.trace_id)\r\n \r\n if not response_json.get('task_id'):\r\n error_msg = response_json.get('base_resp', {}).get('status_msg', '未知错误')\r\n raise Exception(f\"API 调用失败: {error_msg}\")\r\n \r\n task_id = response_json['task_id']\r\n print(\"视频生成任务提交成功，任务ID：\" + task_id)\r\n return task_id\r\n except requests.exceptions.RequestException as e:\r\n print(f\"API 请求失败: {str(e)}\")\r\n raise Exception(f\"API 请求失败: {str(e)}\")\r\n\r\n def query_video_generation(self, api_key, api_url, task_id):\r\n url = f\"{api_url}/query/video_generation?task_id={task_id}\"\r\n headers = {\r\n 'authorization': f'Bearer {api_key}'\r\n }\r\n \r\n session = requests.Session()\r\n session.verify = False\r\n \r\n # 设置重试策略\r\n retry_strategy = urllib3.Retry(\r\n total=5, # 总重试次数\r\n backoff_factor=1, # 重试间隔\r\n status_forcelist=[500, 502, 503, 504] # 需要重试的HTTP状态码\r\n )\r\n adapter = requests.adapters.HTTPAdapter(max_retries=retry_strategy)\r\n session.mount(\"https://\", adapter)\r\n \r\n try:\r\n # 设置超时时间\r\n response = session.get(url, headers=headers, timeout=(5, 30)) # (连接超时, 读取超时)\r\n result = response.json()\r\n status = result.get('status')\r\n \r\n if status == 'Queueing':\r\n print(\"...队列中...\")\r\n return \"\", status\r\n elif status == 'Processing':\r\n print(\"...生成中...\")\r\n return \"\", status\r\n elif status == 'Success':\r\n return result.get('file_id'), status\r\n else:\r\n error_msg = result.get('base_resp', {}).get('status_msg', 'Unknown')\r\n print(f\"查询状态失败: {error_msg}\")\r\n return \"\", \"Failed\"\r\n except requests.exceptions.Timeout:\r\n print(\"查询超时，将在下一轮重试\")\r\n return \"\", \"Processing\" # 返回 Processing 状态以继续轮询\r\n except requests.exceptions.RequestException as e:\r\n print(f\"请求异常: {str(e)}\")\r\n return \"\", \"Processing\" # 返回 Processing 状态以继续轮询\r\n\r\n def fetch_video_result(self, api_key, api_url, file_id, output_path):\r\n print(\"---------------视频生成成功，下载中---------------\")\r\n url = f\"{api_url}/files/retrieve?file_id={file_id}\"\r\n headers = {\r\n 'authorization': f'Bearer {api_key}',\r\n }\r\n \r\n session = requests.Session()\r\n session.verify = False\r\n \r\n try:\r\n response = session.get(url, headers=headers, timeout=(5, 30))\r\n print(\"获取下载链接响应:\", response.text)\r\n\r\n download_url = response.json()['file']['download_url']\r\n print(\"视频下载链接：\" + download_url)\r\n \r\n # 确保使用完整路径\r\n full_output_path = os.path.join(self.output_dir, output_path)\r\n \r\n # 下载视频\r\n video_response = session.get(download_url, timeout=(5, 60))\r\n if video_response.status_code == 200:\r\n with open(full_output_path, 'wb') as f:\r\n f.write(video_response.content)\r\n print(f\"视频已保存到：{full_output_path}\")\r\n return full_output_path\r\n else:\r\n print(f\"下载失败，状态码：{video_response.status_code}\")\r\n print(f\"错误信息：{video_response.text}\")\r\n raise Exception(\"视频下载失败\")\r\n except Exception as e:\r\n print(f\"下载失败: {str(e)}\")\r\n raise\r\n\r\n def generate_video(self, api_key, api_url, model, prompt, prompt_optimizer, watermark=\"yes\", concurrent_tasks=1, first_frame_image=None, subject_reference_image=None):\r\n try:\r\n print(f\"\\n=== 开始视频生成任务 ===\")\r\n print(f\"并发任务数: {concurrent_tasks}\")\r\n self.prompt_optimizer = prompt_optimizer\r\n self.watermark = watermark\r\n video_urls = []\r\n tasks_info = []\r\n \r\n # 处理图像编码\r\n base64_image = None\r\n base64_subject_image = None\r\n \r\n if model == \"S2V-01\":\r\n if subject_reference_image is None:\r\n raise ValueError(\"使用 S2V-01 模型时必须提供主体参考图片\")\r\n print(\"检测到 S2V-01 模型，编码主体参考图片...\")\r\n base64_subject_image = self.encode_image(subject_reference_image)\r\n else:\r\n if first_frame_image is not None:\r\n print(\"检测到输入图像，进行编码...\")\r\n base64_image = self.encode_image(first_frame_image)\r\n \r\n current_time = time.strftime(\"%m%d%H%M\")\r\n print(f\"\\n--- 开始发起并发请求 ---\")\r\n \r\n for i in range(concurrent_tasks):\r\n if i > 0:\r\n print(f\"等待5秒后发起下一个请求...\")\r\n time.sleep(5)\r\n \r\n try:\r\n print(f\"\\n开始发起任务 #{i+1}/{concurrent_tasks}\")\r\n task_id = self.invoke_video_generation(api_key, api_url, prompt, model, base64_image, base64_subject_image)\r\n trace_id = getattr(self, 'trace_id', 'unknown')\r\n \r\n tasks_info.append({\r\n 'index': i + 1,\r\n 'task_id': task_id,\r\n 'trace_id': trace_id,\r\n 'status': 'Queueing',\r\n 'output_path': f\"MM{current_time}_{i+1}_{trace_id}.mp4\",\r\n 'start_time': time.time()\r\n })\r\n \r\n print(f\"任务 #{i+1} 请求成功:\")\r\n print(f\"- Task ID: {task_id}\")\r\n print(f\"- Trace ID: {trace_id}\")\r\n print(f\"- Output Path: {tasks_info[-1]['output_path']}\")\r\n \r\n except Exception as e:\r\n print(f\"任务 #{i+1} 发起失败: {str(e)}\")\r\n raise\r\n\r\n print(f\"\\n--- 开始监控任务状态 ---\")\r\n while tasks_info:\r\n for task in tasks_info[:]:\r\n try:\r\n file_id, status = self.query_video_generation(api_key, api_url, task['task_id'])\r\n task['status'] = status\r\n \r\n elapsed_time = int(time.time() - task['start_time'])\r\n print(f\"\\n任务 #{task['index']} 状态更新:\")\r\n print(f\"- Status: {status}\")\r\n print(f\"- 已用时: {elapsed_time}秒\")\r\n \r\n if status == \"Success\" and file_id:\r\n print(f\"\\n任务 #{task['index']} 生成成功!\")\r\n video_url = self.get_video_url(api_key, api_url, file_id)\r\n video_path = self.fetch_video_result(api_key, api_url, file_id, task['output_path'])\r\n video_urls.append(video_url)\r\n print(f\"- Video URL: {video_url}\")\r\n print(f\"- Saved to: {video_path}\")\r\n \r\n print(f\"\\n正在更新预览界面...\")\r\n print(f\"当前视频URLs: {video_urls}\")\r\n result = self.update_preview(video_urls)\r\n print(f\"预览更新结果: {result}\")\r\n \r\n tasks_info.remove(task)\r\n elif status in [\"Fail\", \"Unknown\"]:\r\n print(f\"\\n任务 #{task['index']} 失败!\")\r\n tasks_info.remove(task)\r\n raise Exception(f\"视频生成失败 (Task #{task['index']})\")\r\n \r\n except Exception as e:\r\n print(f\"\\n任务 #{task['index']} 处理异常: {str(e)}\")\r\n tasks_info.remove(task)\r\n raise\r\n \r\n if tasks_info:\r\n print(\"\\n等待20秒后继续查询...\")\r\n time.sleep(20)\r\n \r\n print(f\"\\n=== 所有 {concurrent_tasks} 个视频生成完成 ===\")\r\n print(f\"最终视频URLs: {video_urls}\")\r\n return {\r\n \"ui\": {\"video_url\": video_urls},\r\n \"result\": (video_urls,)\r\n }\r\n \r\n except Exception as e:\r\n print(f\"\\n=== 发生错误 ===\")\r\n print(f\"错误详情: {str(e)}\")\r\n raise e\r\n\r\n def update_preview(self, video_urls):\r\n \"\"\"实时更新预览\"\"\"\r\n return {\"ui\": {\"video_url\": video_urls}}\r\n\r\n def get_video_url(self, api_key, api_url, file_id):\r\n \"\"\"获取视频下载链接\"\"\"\r\n url = f\"{api_url}/files/retrieve?file_id={file_id}\"\r\n headers = {\r\n 'Authorization': f'Bearer {api_key}',\r\n 'Content-Type': 'application/json'\r\n }\r\n response = requests.get(url, headers=headers)\r\n result = response.json()\r\n return result['file']['download_url']\r\n\r\nclass ImageToPrompt:\r\n def __init__(self):\r\n self.last_result = None\r\n self.last_inputs = None\r\n print(\"初始化 ImageToPrompt 节点\")\r\n\r\n @classmethod\r\n def INPUT_TYPES(cls):\r\n return {\r\n \"required\": {\r\n \"api_key\": (\"API_KEY\",),\r\n \"api_url\": (\"API_URL\",),\r\n \"model\": ([\"MiniMax-Text-01\", \"abab6.5s-chat\"],),\r\n \"image\": (\"IMAGE\",),\r\n \"LLM_prompt\": (\"STRING\", {\r\n \"default\": \"You will receive an image (which will be the first frame of the video) and user input (which describes what the user wants to generate based on this image). \\n\"\r\n \"Your task is to combine their information to construct a video description that will be used to generate a video.\\n\\n\"\r\n \"- Do NOT add any content that is not in the image or user input!!!\\n\"\r\n \"- Do NOT add any camera movement that is not in the image or user input!!!\\n\"\r\n \"- If the user input does not mention any movement for the main subject, add subtle motions based on the current actions of the subject in the image to avoid stillness in the video.\\n\"\r\n \"- The user input may be empty, meaning the user did not provide any information. Please follow the above rules to generate a video description.\\n\"\r\n \"- Please return the description in English and in 150 words.\\n\\n\"\r\n \"User Input:\\n\",\r\n \"multiline\": True,\r\n \"display\": \"LLM Prompt\"\r\n }),\r\n }\r\n }\r\n\r\n RETURN_TYPES = (\"STRING\",)\r\n RETURN_NAMES = (\"prompt\",)\r\n FUNCTION = \"generate_prompt\"\r\n CATEGORY = \"minimax\"\r\n\r\n def generate_prompt(self, api_key, api_url, model, image, LLM_prompt):\r\n print(\"\\n=== ImageToPrompt 节点执行 ===\")\r\n \r\n # 检查输入是否与上次相同\r\n current_inputs = (\r\n api_key, \r\n api_url, \r\n model, \r\n hash(image.cpu().numpy().tobytes()), # 使用 numpy 数组的哈希值\r\n LLM_prompt\r\n )\r\n \r\n if self.last_inputs == current_inputs and self.last_result is not None:\r\n print(\"使用缓存的结果\")\r\n return self.last_result\r\n\r\n # 确保 API URL 包含完整的端点路径\r\n if not api_url.endswith('/chat/completions'):\r\n api_url = f\"{api_url.rstrip('/')}/chat/completions\"\r\n \r\n # 将 Tensor 转换为 PIL Image\r\n if isinstance(image, torch.Tensor):\r\n if len(image.shape) == 4:\r\n image = image[0] # 移除批次维度\r\n image = image.permute(1, 2, 0) # 将通道维度移到最后\r\n image = (image * 255).clamp(0, 255).byte().cpu().numpy() # 转换为 numpy 数组\r\n if image.shape[2] == 1: # 如果是单通道图像\r\n image = image.squeeze(axis=2) # 移除单通道维度\r\n mode = \"L\" # 灰度图像模式\r\n else:\r\n mode = \"RGB\" # 彩色图像模式\r\n image = Image.fromarray(image, mode)\r\n\r\n # 将图像编码为 base64\r\n buffered = io.BytesIO()\r\n image.save(buffered, format=\"PNG\")\r\n base64_image = base64.b64encode(buffered.getvalue()).decode('utf-8')\r\n\r\n headers = {\r\n \"Content-Type\": \"application/json\",\r\n \"Authorization\": f\"Bearer {api_key}\"\r\n }\r\n\r\n payload = {\r\n \"model\": model,\r\n \"messages\": [\r\n {\r\n \"role\": \"user\",\r\n \"content\": [\r\n {\r\n \"type\": \"text\",\r\n \"text\": LLM_prompt\r\n },\r\n {\r\n \"type\": \"image_url\",\r\n \"image_url\": {\r\n \"url\": f\"data:image/png;base64,{base64_image}\"\r\n }\r\n }\r\n ]\r\n }\r\n ],\r\n \"max_tokens\": 300\r\n }\r\n\r\n try:\r\n print(f\"请求的 API URL: {api_url}\")\r\n response = requests.post(api_url, headers=headers, json=payload)\r\n print(f\"API 响应状态码: {response.status_code}\")\r\n print(f\"API 响应内容: {response.text}\")\r\n\r\n # 检查响应状态码\r\n if response.status_code != 200:\r\n raise Exception(f\"API 请求失败，状态码: {response.status_code}\")\r\n\r\n response_json = response.json()\r\n prompt = response_json.get('choices', [{}])[0].get('message', {}).get('content', '')\r\n print(f\"生成的文本提示: {prompt}\")\r\n \r\n # 缓存结果\r\n self.last_inputs = current_inputs\r\n self.last_result = (prompt,)\r\n return self.last_result\r\n except requests.exceptions.JSONDecodeError as e:\r\n print(f\"JSON 解析错误: {str(e)}\")\r\n raise\r\n except Exception as e:\r\n print(f\"API 请求失败: {str(e)}\")\r\n raise\r\n\r\n# 导出节点映射\r\nNODE_CLASS_MAPPINGS = {\r\n \"MiniMaxPreviewVideo\": MiniMaxPreviewVideo,\r\n \"MiniMaxAIAPIClient\": MiniMaxAIAPIClient,\r\n \"MiniMaxImage2Video\": MiniMaxImage2Video,\r\n \"ImageToPrompt\": ImageToPrompt,\r\n}\r\n\r\n# 导出节点显示名称映射\r\nNODE_DISPLAY_NAME_MAPPINGS = {\r\n \"MiniMaxPreviewVideo\": \"MiniMax Preview Video\",\r\n \"MiniMaxAIAPIClient\": \"MiniMax API Client\",\r\n \"MiniMaxImage2Video\": \"MiniMax Image to Video\",\r\n \"ImageToPrompt\": \"Image to Prompt\",\r\n}"}}}} |