Configuration and model blueprint
The config.json file records the model architecture details that the library needs before any weights are loaded. It lists the number of hidden layers, vocabulary size, attention heads, hidden dimension per head, activation function, and other hyperparameters. This file also stores metadata such as the Transformers version used when the checkpoint was saved and the original source of the weights.
Without config.json the library cannot construct the correct layer stack, so the file is required even when weights are stored separately. Some models add a generation_config.json file that holds sampling parameters, special start and stop tokens, and temperature settings used during text generation.
Weight files and serialization formats
The actual learned parameters live in weight files, most commonly model.safetensors or a set of sharded safetensors files. Each tensor holds the numeric values for one layer or component such as embeddings or attention projections. For models larger than roughly 10 GB the weights are split across multiple files named model-00001-of-00004.safetensors and so on, with an accompanying index file that maps every parameter name to its shard.
Safetensors is now the default because it supports lazy loading, parallel reads, and a restricted header that limits certain attack vectors. Older repositories may still contain pytorch_model.bin files that rely on Python pickle; these should be converted or avoided when possible. GGUF files combine configuration, tokenizer data, and quantized weights into a single file that loads directly into GGML-based runtimes.
- Sharded checkpoints keep individual file sizes under 50 GB by default.
- An index file records the mapping from parameter names to shard locations.
- Disk offloading moves unused layers to storage when GPU and CPU memory are exhausted.
Tokenizer files and vocabulary
Three or four files handle text preprocessing and detokenization. tokenizer_config.json defines special tokens, chat templates, and the maximum sequence length the model accepts. tokenizer.json stores the learned vocabulary and merge rules for byte-pair or word-piece tokenization. special_tokens_map.json maps token names such as to their numeric IDs.
These files ensure that input text is converted to the exact token sequence the model saw during training. Changing or omitting any tokenizer file usually produces garbled output or causes the model to reject valid prompts.
Storage layout and caching behavior
The Hugging Face Hub stores each downloaded repository in a flat folder under the local cache. Actual file contents reside in a content-addressed blobs directory while snapshots contain symlinks that point to the correct blobs. This layout lets multiple revisions share the same weight data and reduces disk usage when several models reuse common components such as a text encoder.
Users who need to run models offline can download the required files once with snapshot_download or hf_hub_download and then point the library at the local directory. Large models benefit from setting max_shard_size during export so that no single file exceeds available storage or transfer limits.
- Blobs are named by their content hash, allowing deduplication across revisions.
- Symlinks in the snapshots directory keep the layout readable for humans.
- Offline environments must fetch config and tokenizer files before attempting to load weights.
Practical tradeoffs when choosing a format
Full-precision safetensors files give the highest numerical fidelity but require the most memory and disk space. Quantized GGUF files reduce size dramatically and run on consumer GPUs or even CPUs, yet they introduce small accuracy losses that vary by quantization level. Sharded safetensors files load faster on multi-GPU systems because each shard can be read in parallel, while a single-file layout simplifies transfer to air-gapped machines.
When memory is constrained, Big Model Inference features can dispatch layers across GPU, CPU, and disk, but disk offloading slows inference because every forward pass reads from storage. Users should test both safetensors and GGUF versions of the same model on their target hardware before committing to a production workflow.