Model Metadata Extractor (C2C):
The ModelMetadataExtractorMEC is a specialized node designed to read and extract metadata from model files without the need to load the model weights into memory. This capability is particularly beneficial for efficiently handling large model files, as it allows you to gather essential information without the overhead of full model loading. The node supports various file formats, including .safetensors, .ckpt, .pt, and .pth. For .safetensors, it parses the header to extract metadata in a constant-time manner without requiring PyTorch imports. For other formats, it utilizes zipfile to handle modern PyTorch save formats and falls back to pickletools.dis for legacy pickle formats, ensuring safe enumeration of top-level opcodes without executing any code. The node outputs JSON metadata, identifies the model's training type if recognized, and generates a SHA256 fingerprint based on the file's head and tail, which is useful for cache key generation. Importantly, the node is designed with security in mind, avoiding the import of unknown modules and the unpickling of untrusted data.
Model Metadata Extractor (C2C) Input Parameters:
file_path
The file_path parameter specifies the location of the model file from which metadata is to be extracted. It is crucial for the node's operation as it determines the source file to be analyzed. The parameter must be a valid path to a model file, and the file must exist at the specified location. There are no default values for this parameter, and it must be provided by the user.
compute_fingerprint
The compute_fingerprint parameter is a boolean flag that indicates whether a SHA256 fingerprint of the model file should be computed. This fingerprint is derived from the first and last 1 MB of the file, providing a unique identifier that can be used for caching purposes. The default value for this parameter is True, meaning the fingerprint will be computed unless explicitly set to False.
Model Metadata Extractor (C2C) Output Parameters:
metadata
The metadata output parameter provides a JSON object containing the extracted metadata from the model file. This includes information such as the file format, size, detected model kind, and any additional metadata found within the file. This output is crucial for understanding the characteristics and properties of the model without loading it into memory.
fingerprint
The fingerprint output parameter is a SHA256 hash string that serves as a unique identifier for the model file. It is particularly useful for caching and ensuring the integrity of the model file. The fingerprint is computed from the head and tail of the file, providing a balance between uniqueness and computational efficiency.
Model Metadata Extractor (C2C) Usage Tips:
- Ensure that the
file_pathprovided is correct and points to a valid model file to avoid errors during metadata extraction. - Utilize the
compute_fingerprintparameter to generate unique identifiers for model files, which can be helpful for caching and version control.
Model Metadata Extractor (C2C) Common Errors and Solutions:
Model file not found: <file_path>
- Explanation: This error occurs when the specified file path does not point to an existing file.
- Solution: Verify that the file path is correct and that the file exists at the specified location.
Metadata extraction failed: <error_message>
- Explanation: This error indicates that an issue occurred during the metadata extraction process, possibly due to an unsupported file format or a corrupted file.
- Solution: Check the file format and ensure it is supported. If the file is corrupted, try obtaining a new copy of the model file.
