Save Quantized Model (DynamicVRAM Safe):
The INT8ModelSave node is designed to facilitate the saving of quantized models, specifically those utilizing Toolkit INT8 or native ConvRot INT4 layers, in a manner that is safe for DynamicVRAM environments. This node is part of the ComfyUI Quantization Toolkit, which aims to optimize model storage and execution by reducing the precision of model weights, thereby decreasing memory usage and potentially increasing inference speed. The primary function of this node is to ensure that models are saved with the necessary metadata and configurations to maintain their quantized state, while also providing a mechanism to include additional information such as prompts or extra metadata. This is particularly beneficial for AI artists and developers who need to manage large models efficiently without compromising on performance or accuracy.
Save Quantized Model (DynamicVRAM Safe) Input Parameters:
model
The model parameter is a required input that specifies the model to be saved. It must be a model that has been quantized using the Toolkit INT8 or native ConvRot INT4 methods. This parameter is crucial as it determines the model that will undergo the saving process, ensuring that it retains its quantized properties.
filename_prefix
The filename_prefix parameter is a required string input that sets the prefix for the saved model's filename. By default, it is set to "quantized_models/Quantized_Model". This prefix helps in organizing and identifying saved models, especially when dealing with multiple versions or iterations. It impacts the naming convention of the output file, making it easier to manage and retrieve specific models.
prompt
The prompt parameter is a hidden input that allows you to include a textual prompt associated with the model. This can be useful for documentation or for providing context about the model's intended use or the conditions under which it was trained or quantized.
extra_pnginfo
The extra_pnginfo parameter is another hidden input that enables the inclusion of additional metadata in the form of key-value pairs. This can be used to store supplementary information that might be relevant for future reference or for other users who might utilize the saved model.
Save Quantized Model (DynamicVRAM Safe) Output Parameters:
(No output parameters)
The INT8ModelSave node does not produce any direct output parameters. Its primary function is to save the model to a specified location with the appropriate configurations and metadata, rather than generating an output that can be used in subsequent nodes.
Save Quantized Model (DynamicVRAM Safe) Usage Tips:
- Ensure that the model has been properly quantized using the Toolkit INT8 or native ConvRot INT4 methods before attempting to save it with this node. This will prevent errors and ensure that the model is saved correctly.
- Utilize the
filename_prefixparameter to organize your saved models effectively. This can help in maintaining a clear structure, especially when dealing with multiple models or versions.
Save Quantized Model (DynamicVRAM Safe) Common Errors and Solutions:
Save Quantized Model requires a MODEL containing Toolkit INT8 or native ConvRot INT4 weights.
- Explanation: This error occurs when the model being saved does not contain the required quantized weights.
- Solution: Ensure that the model has been quantized using the appropriate methods before attempting to save it.
Save Quantized Model cannot serialize Dynamic LoRAs because they are runtime-only.
- Explanation: This error indicates that the model contains Dynamic LoRAs, which cannot be serialized.
- Solution: Use Stochastic mode or bake stock LoRAs before quantization to avoid this issue.