ComfyUI Node: HunyuanVideo Custom Prompt Template

HunyuanVideo Custom Prompt Template:

The HyVideoCustomPromptTemplate node is designed to facilitate the creation of custom prompt templates specifically for video encoding tasks within the HunyuanVideoWrapper framework. This node allows you to define and customize the structure of prompts used in video processing, enabling a more tailored and precise description of video content. By providing a flexible template system, it enhances the ability to generate detailed and contextually relevant prompts that can guide AI models in understanding and processing video data. The node's primary function is to process input parameters to produce a structured prompt template, which can be used to describe various aspects of a video, such as its main content, objects, actions, and environmental details. This customization capability is particularly beneficial for AI artists who wish to have more control over how video content is interpreted and described by AI models.

HunyuanVideo Custom Prompt Template Input Parameters:

custom_prompt_template

The custom_prompt_template parameter is a string input that allows you to define a custom template for video prompts. This template serves as a blueprint for how video content should be described, including placeholders for dynamic content insertion. The default value is derived from the PROMPT_TEMPLATE['dit-llm-encode-video']['template'], which provides a comprehensive structure for video description. This parameter supports multiline input, enabling complex and detailed prompt structures. By customizing this template, you can influence the focus and detail level of the video description, tailoring it to specific needs or artistic goals.

crop_start

The crop_start parameter is an integer that specifies the starting point for cropping the system prompt. This parameter is used to adjust the portion of the prompt that is considered during processing, allowing you to exclude certain parts of the prompt if necessary. The default value is set according to PROMPT_TEMPLATE['dit-llm-encode-video']["crop_start"]. Adjusting this parameter can impact the final prompt by removing unwanted sections, thus refining the focus of the video description. It is particularly useful when you want to streamline the prompt to highlight specific aspects of the video content.

HunyuanVideo Custom Prompt Template Output Parameters:

hyvid_prompt_template

The hyvid_prompt_template output is a dictionary that contains the processed prompt template. This dictionary includes the customized template and the crop start value, encapsulating the final structure that will be used for video description. The output is crucial as it represents the tailored prompt that guides the AI model in interpreting and processing video data. By providing a structured and detailed prompt, this output ensures that the video content is described accurately and comprehensively, aligning with the user's artistic vision and requirements.

HunyuanVideo Custom Prompt Template Usage Tips:

• Customize the custom_prompt_template to include specific placeholders and structures that align with your artistic goals, ensuring that the video description captures the desired details and nuances.
• Use the crop_start parameter to refine the prompt by excluding unnecessary sections, which can help focus the AI model's attention on the most relevant aspects of the video content.

HunyuanVideo Custom Prompt Template Common Errors and Solutions:

Invalid prompt_template: {prompt_template_dict}

• Explanation: This error occurs when the specified prompt_template is not recognized or is incorrectly formatted.
• Solution: Ensure that the prompt_template is one of the valid options such as "custom", "video", "image", "I2V_video", or "I2V_image". Check for any typos or incorrect values.

prompt_template must be a dictionary with a key 'template', got {prompt_template_dict}

• Explanation: This error indicates that the prompt_template is not structured as a dictionary with the required 'template' key.
• Solution: Verify that the prompt_template is correctly formatted as a dictionary and includes the 'template' key. Ensure that the value associated with 'template' is a valid string.

prompt_template['template'] must contain a placeholder {} for the input text, got {prompt_template_dict['template']}

• Explanation: This error arises when the template string does not include the necessary placeholder {} for dynamic content insertion.
• Solution: Modify the custom_prompt_template to include the {} placeholder, which is essential for inserting input text dynamically into the prompt.