ComfyUI Node: ✂️ AI Clipping

✂️ AI Clipping:

The AiClipping node is designed to transform long-form videos into a series of ranked, viral-ready short clips. It leverages the capabilities of muapi.ai's /ai-clipping endpoint to handle the entire process, ensuring that ComfyUI users can focus on creative tasks without delving into technical complexities. This node automates the extraction of highlights from videos by employing advanced techniques such as Whisper transcription, emotional peak detection, and face-tracked vertical auto-cropping. The result is a selection of clips that are optimized for virality, each accompanied by a score indicating its potential impact. This node is particularly beneficial for content creators looking to maximize engagement by repurposing longer videos into concise, impactful segments that are ready for social media sharing.

✂️ AI Clipping Input Parameters:

video_url

This parameter requires the URL of the source video, which should be in MP4 or MOV format. It serves as the primary source from which the node will extract highlights. If you prefer to upload a local file instead, you can use the video_path parameter. The video URL is crucial as it directly influences the content and quality of the clips generated by the node.

video_path

An optional parameter that allows you to specify the path to a local video file. This is useful if your video is not hosted online or if you prefer to work with local files. The node will upload this file to the server for processing. Ensure that the file is accessible and in a supported format to avoid errors.

num_highlights

This parameter determines the number of highlight clips you wish to extract from the source video. It directly impacts the breadth of content you receive, allowing you to specify how many top moments you want to capture. The more highlights you request, the broader the selection of clips you will have to choose from.

aspect_ratio

This parameter specifies the desired aspect ratio for the output clips. Options include 9:16, 1:1, and 4:5, catering to different social media platforms and viewing preferences. Choosing the right aspect ratio is important for ensuring that your clips are visually appealing and optimized for the platform where they will be shared.

return_coordinates_only

An optional boolean parameter that, when set to true, instructs the node to return only the coordinates of the highlights without downloading the actual video clips. This can be useful for users who want to process the clips further or integrate them into other workflows without immediately needing the video files.

✂️ AI Clipping Output Parameters:

clip_url

This output provides the URL of the first clip generated by the node. It is a direct link to the hosted MP4 file, allowing you to easily access and share the clip. This URL is essential for viewing and distributing the content created by the node.

first_frame

The first frame of the first clip is returned as an image. This is particularly useful for previewing the clip or using the frame as a thumbnail. It provides a quick visual reference to the content of the clip without needing to play the entire video.

highlights_json

This output contains a JSON-formatted string detailing all the clips generated by the node. It includes information such as the title, hook sentence, start and end times, and scores for each clip. This comprehensive data allows you to analyze and select clips based on various criteria.

rid

The request ID associated with the processing task. This ID is useful for tracking and managing the request, especially if you need to reference it for troubleshooting or further processing.

len(clips)

This output indicates the total number of clips generated by the node. It provides a quick overview of the breadth of content available for review and selection.

✂️ AI Clipping Usage Tips:

• Ensure that your video URL or path is correct and accessible to avoid processing errors.
• Choose an aspect ratio that aligns with your target platform to optimize the visual appeal of your clips.
• Use the num_highlights parameter to control the number of clips generated, balancing between quantity and quality.
• Consider setting return_coordinates_only to true if you plan to process the clips further in another application.

✂️ AI Clipping Common Errors and Solutions:

"Provide either video_url or video_path."

• Explanation: This error occurs when neither a video URL nor a video path is provided, leaving the node without a source video to process.
• Solution: Ensure that you provide either a valid URL or a path to a local video file before running the node.

"highlights_json is not valid JSON"

• Explanation: This error indicates that the highlights_json input is not properly formatted as JSON, which is necessary for processing.
• Solution: Verify that the JSON string is correctly formatted and free of syntax errors before inputting it into the node.

"no frame"

• Explanation: This error suggests that the node was unable to extract the first frame from the video, possibly due to an issue with the video file or URL.
• Solution: Check the video file or URL for accessibility and format issues, and ensure that the video is not corrupted.