Parameters

• pretrained_model_name_or_path (str or os.PathLike) — This can be either:
  • a string, the model id of a pretrained feature_extractor hosted inside a model repo on huggingface.co. Valid model ids can be located at the root-level, like bert-base-uncased, or namespaced under a user or organization name, like dbmdz/bert-base-german-cased.
  • a path to a directory containing a feature extractor file saved using the save_pretrained() method, e.g., ./my_model_directory/.
  • a path or url to a saved feature extractor JSON file, e.g., ./my_model_directory/preprocessor_config.json.
• cache_dir (str or os.PathLike, optional) — Path to a directory in which a downloaded pretrained model feature extractor should be cached if the standard cache should not be used.
• force_download (bool, optional, defaults to False) — Whether or not to force to (re-)download the feature extractor files and override the cached versions if they exist.
• resume_download (bool, optional, defaults to False) — Whether or not to delete incompletely received file. Attempts to resume the download if such a file exists.
• proxies (Dict[str, str], optional) — A dictionary of proxy servers to use by protocol or endpoint, e.g., {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request.
• use_auth_token (str or bool, optional) — The token to use as HTTP bearer authorization for remote files. If True, will use the token generated when running transformers-cli login (stored in ~/.huggingface).
• revision(str, optional, defaults to "main") — The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so revision can be any identifier allowed by git.
• return_unused_kwargs (bool, optional, defaults to False) — If False, then this function returns just the final feature extractor object. If True, then this functions returns a Tuple(feature_extractor, unused_kwargs) where unused_kwargs is a dictionary consisting of the key/value pairs whose keys are not feature extractor attributes: i.e., the part of kwargs which has not been used to update feature_extractor and is otherwise ignored.
• kwargs (Dict[str, Any], optional) — The values in kwargs of any keys which are feature extractor attributes will be used to override the loaded values. Behavior concerning key/value pairs whose keys are not feature extractor attributes is controlled by the return_unused_kwargs keyword parameter.

Parameters

• save_directory (str or os.PathLike) — Directory where the feature extractor JSON file will be saved (will be created if it does not exist).
• push_to_hub (bool, optional, defaults to False) — Whether or not to push your feature extractor to the Hugging Face model hub after saving it.

  Using push_to_hub=True will synchronize the repository you are pushing to with save_directory, which requires save_directory to be a local clone of the repo you are pushing to if it’s an existing folder. Pass along temp_dir=True to use a temporary directory instead.

  kwargs — Additional key word arguments passed along to the push_to_hub() method.

Parameters

• processed_features (BatchFeature, list of BatchFeature, Dict[str, List[float]], Dict[str, List[List[float]] or List[Dict[str, List[float]]]) — Processed inputs. Can represent one input (BatchFeature or Dict[str, List[float]]) or a batch of input values / vectors (list of BatchFeature, Dict[str, List[List[float]]] or List[Dict[str, List[float]]]) so you can use this method during preprocessing as well as in a PyTorch Dataloader collate function.

  Instead of List[float] you can have tensors (numpy arrays, PyTorch tensors or TensorFlow tensors), see the note above for the return type.

• padding (bool, str or PaddingStrategy, optional, defaults to True) — Select a strategy to pad the returned sequences (according to the model’s padding side and padding index) among:
  • True or 'longest': Pad to the longest sequence in the batch (or no padding if only a single sequence if provided).
  • 'max_length': Pad to a maximum length specified with the argument max_length or to the maximum acceptable input length for the model if that argument is not provided.
  • False or 'do_not_pad' (default): No padding (i.e., can output a batch with sequences of different lengths).
• max_length (int, optional) — Maximum length of the returned list and optionally padding length (see above).
• truncation (bool) — Activates truncation to cut input sequences longer than max_length to max_length.
• pad_to_multiple_of (int, optional) — If set will pad the sequence to a multiple of the provided value.

  This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability

  = 7.5 (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128.

• return_attention_mask (bool, optional) — Whether to return the attention mask. If left to the default, will return the attention mask according to the specific feature_extractor’s default.

  What are attention masks?

• return_tensors (str or TensorType, optional) — If set, will return tensors instead of list of python integers. Acceptable values are:
  • 'tf': Return TensorFlow tf.constant objects.
  • 'pt': Return PyTorch torch.Tensor objects.
  • 'np': Return Numpy np.ndarray objects.

Parameters

• tensor_type (str or TensorType, optional) — The type of tensors to use. If str, should be one of the values of the enum TensorType. If None, no modification is done.

Parameters

• device (str or torch.device) — The device to put the tensors on.

The same instance after modification.

Parameters

• image (PIL.Image.Image or np.ndarray or torch.Tensor) — The image to resize.
• size (int or Tuple[int, int]) — The size to which crop the image.

Parameters

• image (PIL.Image.Image or np.ndarray or torch.Tensor) — The image to normalize.
• mean (List[float] or np.ndarray or torch.Tensor) — The mean (per channel) to use for normalization.
• std (List[float] or np.ndarray or torch.Tensor) — The standard deviation (per channel) to use for normalization.

Parameters

• image (PIL.Image.Image or np.ndarray or torch.Tensor) — The image to resize.
• size (int or Tuple[int, int]) — The size to use for resizing the image. If size is a sequence like (h, w), output size will be matched to this.

  If size is an int and default_to_square is True, then image will be resized to (size, size). If size is an int and default_to_square is False, then smaller edge of the image will be matched to this number. i.e, if height > width, then image will be rescaled to (size * height / width, size).

• resample (int, optional, defaults to PIL.Image.BILINEAR) — The filter to user for resampling.
• default_to_square (bool, optional, defaults to True) — How to convert size when it is a single int. If set to True, the size will be converted to a square (size,size). If set to False, will replicate torchvision.transforms.Resize with support for resizing only the smallest edge and providing an optional max_size.
• max_size (int, optional, defaults to None) — The maximum allowed for the longer edge of the resized image: if the longer edge of the image is greater than max_size after being resized according to size, then the image is resized again so that the longer edge is equal to max_size. As a result, size might be overruled, i.e the smaller edge may be shorter than size. Only used if default_to_square is False.

Parameters

• image (PIL.Image.Image or np.ndarray or torch.Tensor) — The image to convert to a NumPy array.
• rescale (bool, optional) — Whether or not to apply the scaling factor (to make pixel values floats between 0. and 1.). Will default to True if the image is a PIL Image or an array/tensor of integers, False otherwise.
• channel_first (bool, optional, defaults to True) — Whether or not to permute the dimensions of the image to put the channel dimension first.

Parameters

• image (PIL.Image.Image or numpy.ndarray or torch.Tensor) — The image to convert to the PIL Image format.
• rescale (bool, optional) — Whether or not to apply the scaling factor (to make pixel values integers between 0 and 255). Will default to True if the image type is a floating type, False otherwise.