update from climategan space

README.md

@@ -11,11 +11,10 @@ emoji: 🌎
 colorFrom: blue
 colorTo: green
 sdk: gradio
-sdk_version: 4.6
+sdk_version: 3.6
 app_file: app.py
 inference: true
-# datasets:
-# -
+pinned: true
 ---
 
 # ClimateGAN: Raising Awareness about Climate Change by Generating Images of Floods
@@ -38,7 +37,33 @@ If you use this code, data or pre-trained weights, please cite our ICLR 2022 pap
 }
 ```
 
-## Using pre-trained weights
+## Using pre-trained weights from this Huggingface Space and Stable Diffusion In-painting
+
+[**Huggingface ClimateGAN Space**](https://huggingface.co/spaces/vict0rsch/climateGAN)
+
+1. Download code and model
+  ```
+  git clone https://huggingface.co/spaces/vict0rsch/climateGAN hf-spaces-climategan
+  cd hf-spaces-climategan
+  git lfs install
+  git lfs pull
+  ```
+2. Install requirements
+  ```
+  pip install requirements.txt
+  ```
+3. **Enable Stable Diffusion Inpainting** by visiting the model's card: https://huggingface.co/runwayml/stable-diffusion-inpainting **and** running `$ huggingface-cli login`
+4. Run `$ python climategan_wrapper.py help` for usage instructions on how to infer on a folder's images.
+5. Run `$ python app.py` to see the Gradio app.
+   1. To use Google Street View you'll need an API key and set the `GMAPS_API_KEY` environment variable.
+   2. To use Stable Diffusion if you can't run `$ huggingface-cli login` (on a Huggingface Space for instance) set the `HF_AUTH_TOKEN` env variable to a [Huggingface authorization token](https://huggingface.co/settings/tokens)
+   3. To change the UI without model overhead, set the `CG_DEV_MODE` environment variable to `true`.
+
+For a more fine-grained control on ClimateGAN's inferences, refer to `apply_events.py` (does not support Stable Diffusion painter)
+
+**Note:** you don't have control on the prompt by design because I disabled the safety checker. Fork this space/repo and do it yourself if you really need to change the prompt. At least [open a discussion](https://huggingface.co/spaces/vict0rsch/climateGAN/discussions).
+
+## Using pre-trained weights from source
 
 In the paper, we present ClimateGAN as a solution to produce images of floods. It can actually do **more**:
 

app.py

@@ -2,69 +2,302 @@
 # thank you @NimaBoscarino
 
 import os
-import gradio as gr
+from textwrap import dedent
+from urllib import parse
+from requests import get
+
 import googlemaps
+import gradio as gr
+import numpy as np
+from gradio.components import (
+    HTML,
+    Button,
+    Column,
+    Dropdown,
+    Image,
+    Markdown,
+    Radio,
+    Row,
+    Textbox,
+)
 from skimage import io
-from urllib import parse
-from inferences import ClimateGAN
+from datetime import datetime
+
+from climategan_wrapper import ClimateGAN
+
+TEXTS = [
+    dedent(
+        """
+        <p>
+            Climate change does not impact everyone equally.
+            This Space shows the effects of the climate emergency,
+            "one address at a time".
+            Visit the original experience at
+            <a href="https://thisclimatedoesnotexist.com/">
+                ThisClimateDoesNotExist.com
+            </a>
+        </p>
+        <br>
+        <p>
+            Enter an address or upload a Street View image, and ClimateGAN
+            will generate images showing how the location could be impacted
+            by flooding, wildfires, or smog if it happened there.
+        </p>
+        <br>
+        <p>
+            This is <strong>NOT</strong> an exercise in climate prediction,
+            rather an exercise of empathy, to put yourself in others' shoes,
+            as if Climate Change came crushing on your doorstep.
+        </p>
+        <br>
+        <p>
+            After you have selected an image and started the inference you
+            will see all the outputs of ClimateGAN, including intermediate
+            outputs such as the flood mask, the segmentation map and the
+            depth maps used to produce the 3 events.
+        </p>
+        <br>
+        <p>
+            This Space makes use of recent Stable Diffusion in-painting
+            pipelines to replace ClimateGAN's original Painter. If you
+            select 'Both' painters, you will see a comparison
+        </p>
+        <br>
+        <p style='text-align: center'>
+            Visit
+            <a href='https://thisclimatedoesnotexist.com/'>
+                ThisClimateDoesNotExist.com</a>
+            &nbsp;for more information
+            &nbsp;&nbsp;|&nbsp;&nbsp;
+            Original
+            <a href='https://github.com/cc-ai/climategan'>
+                ClimateGAN GitHub Repo
+            </a>
+            &nbsp;&nbsp;|&nbsp;&nbsp;
+            Read the original
+            <a
+                href='https://openreview.net/forum?id=EZNOb_uNpJk'
+                target='_blank'>
+            ICLR 2021 ClimateGAN paper
+            </a>
+        </p>
+        """
+    ),
+    dedent(
+        """
+        ## How to use this Space
+
+        1. Enter an address or upload a Street View image
+        2. Select the type of Painter you'd like to use for the flood renderings
+        3. Click on the "See for yourself!" button
+        4. Wait for the inference to complete, typically around 30 seconds
+            (plus queue time)
+        5. Enjoy the results!
+
+            1. The prompt for Stable Diffusion is `An HD picture of a street with
+            dirty water after a heavy flood`
+            2. Pay attention to potential "inventions" by Stable Diffusion's in-painting
+            3. The "restricted to masked area" SD output is the result of:
+            `y = mask * flooded + (1-mask) * input`
+
+        """
+    ),
+]
+CSS = dedent(
+    """
+    a {
+        color: #0088ff;
+        text-decoration: underline;
+    }
+    strong {
+        color: #c34318;
+        font-weight: bolder;
+    }
+    #how-to-use-md li {
+        margin: 0.1em;
+    }
+    #how-to-use-md li p {
+        margin: 0.1em;
+    }
+    """
+)
+
 
+def toggle(radio):
+    if "address" in radio.lower():
+        return [
+            gr.update(visible=True),
+            gr.update(visible=False),
+            gr.update(visible=True),
+        ]
+    else:
+        return [
+            gr.update(visible=False),
+            gr.update(visible=True),
+            gr.update(visible=True),
+        ]
 
-def predict(api_key):
+
+def predict(cg: ClimateGAN, api_key):
     def _predict(*args):
-        print("args: ", args)
-        image = place = None
-        if len(args) == 1:
-            image = args[0]
+        print(f"Starting inference ({str(datetime.now())})")
+        image = place = painter = radio = None
+        if api_key:
+            radio, image, place, painter = args
         else:
-            assert len(args) == 2, "Unknown number of inputs {}".format(len(args))
-            image, place = args
+            image, painter = args
 
-        if api_key and place:
+        if api_key and place and "address" in radio.lower():
             geocode_result = gmaps.geocode(place)
 
             address = geocode_result[0]["formatted_address"]
             static_map_url = f"https://maps.googleapis.com/maps/api/streetview?size=640x640&location={parse.quote(address)}&source=outdoor&key={api_key}"
             img_np = io.imread(static_map_url)
+            print("Using GSV image")
         else:
+            print("Using user image")
             img_np = image
-        flood, wildfire, smog = model.inference(img_np)
-        return img_np, flood, wildfire, smog
+
+        painters = {
+            "ClimateGAN Painter": "climategan",
+            "Stable Diffusion Painter": "stable_diffusion",
+            "Both": "both",
+        }
+        print("Using painter", painters[painter])
+        output_dict = cg.infer_single(
+            img_np,
+            painters[painter],
+            concats=[
+                "input",
+                "masked_input",
+                "climategan_flood",
+                "stable_copy_flood",
+            ],
+            as_pil_image=True,
+        )
+
+        input_image = output_dict["input"]
+        masked_input = output_dict["masked_input"]
+        wildfire = output_dict["wildfire"]
+        smog = output_dict["smog"]
+        depth = np.repeat(output_dict["depth"], 3, axis=-1)
+        segmentation = output_dict["segmentation"]
+
+        climategan_flood = output_dict.get(
+            "climategan_flood",
+            np.ones(input_image.shape, dtype=np.uint8) * 255,
+        )
+        stable_flood = output_dict.get(
+            "stable_flood",
+            np.ones(input_image.shape, dtype=np.uint8) * 255,
+        )
+        stable_copy_flood = output_dict.get(
+            "stable_copy_flood",
+            np.ones(input_image.shape, dtype=np.uint8) * 255,
+        )
+        concat = output_dict.get(
+            "concat",
+            np.ones(input_image.shape, dtype=np.uint8) * 255,
+        )
+
+        return (
+            input_image,
+            masked_input,
+            segmentation,
+            depth,
+            climategan_flood,
+            stable_flood,
+            stable_copy_flood,
+            concat,
+            wildfire,
+            smog,
+        )
 
     return _predict
 
 
 if __name__ == "__main__":
 
+    ip = get("https://api.ipify.org").content.decode("utf8")
+    print("My public IP address is: {}".format(ip))
+
     api_key = os.environ.get("GMAPS_API_KEY")
     gmaps = None
     if api_key is not None:
         gmaps = googlemaps.Client(key=api_key)
 
-    model = ClimateGAN(model_path="config/model/masker")
-
-    inputs = inputs = [gr.inputs.Image(label="Input Image")]
-    if api_key:
-        inputs += [gr.inputs.Textbox(label="Address or place name")]
-
-    gr.Interface(
-        predict(api_key),
-        inputs=[
-            gr.inputs.Textbox(label="Address or place name"),
-            gr.inputs.Image(label="Input Image"),
-        ],
-        outputs=[
-            gr.outputs.Image(type="numpy", label="Original image"),
-            gr.outputs.Image(type="numpy", label="Flooding"),
-            gr.outputs.Image(type="numpy", label="Wildfire"),
-            gr.outputs.Image(type="numpy", label="Smog"),
-        ],
-        title="ClimateGAN: Visualize Climate Change",
-        description='Climate change does not impact everyone equally. This Space shows the effects of the climate emergency, "one address at a time". Visit the original experience at <a href="https://thisclimatedoesnotexist.com/">ThisClimateDoesNotExist.com</a>.<br>Enter an address or place name, and ClimateGAN will generate images showing how the location could be impacted by flooding, wildfires, or smog.',  # noqa: E501
-        article="<p style='text-align: center'>This project is an unofficial clone of <a href='https://thisclimatedoesnotexist.com/'>ThisClimateDoesNotExist</a> | <a href='https://github.com/cc-ai/climategan'>ClimateGAN GitHub Repo</a></p>",  # noqa: E501
-        # examples=[
-        #     "Vancouver Art Gallery",
-        #     "Chicago Bean",
-        #     "Duomo Siracusa",
-        # ],
-        css=".footer{display:none !important}",
-    ).launch()
+    cg = ClimateGAN(
+        model_path="config/model/masker",
+        dev_mode=os.environ.get("CG_DEV_MODE", "").lower() == "true",
+    )
+    cg._setup_stable_diffusion()
+
+    radio = address = None
+    pred_ins = []
+    pred_outs = []
+
+    with gr.Blocks(css=CSS) as app:
+        with Row():
+            with Column():
+                Markdown("# ClimateGAN: Visualize Climate Change")
+                HTML(TEXTS[0])
+            with Column():
+                Markdown(TEXTS[1], elem_id="how-to-use-md")
+        with Row():
+            HTML("<hr><br><h2 style='font-size: 1.5rem;'>Choose Inputs</h2>")
+        with Row():
+            with Column():
+                if api_key:
+                    radio = Radio(["From Address", "From Image"], label="Input Type")
+                    pred_ins += [radio]
+                im_inp = Image(label="Input Image", visible=not api_key)
+                pred_ins += [im_inp]
+                if api_key:
+                    address = Textbox(label="Address or place name", visible=False)
+                    pred_ins += [address]
+            with Column():
+                pred_ins += [
+                    Dropdown(
+                        choices=[
+                            "ClimateGAN Painter",
+                            "Stable Diffusion Painter",
+                            "Both",
+                        ],
+                        label="Choose Flood Painter",
+                        value="Both",
+                    )
+                ]
+                btn = Button(
+                    "See for yourself!",
+                    label="Run",
+                    variant="primary",
+                    visible=not api_key,
+                )
+        with Row():
+            Markdown("## Outputs")
+        with Row():
+            pred_outs += [Image(type="numpy", label="Original image")]
+            pred_outs += [Image(type="numpy", label="Masked input image")]
+            pred_outs += [Image(type="numpy", label="Segmentation map")]
+            pred_outs += [Image(type="numpy", label="Depth map")]
+        with Row():
+            pred_outs += [Image(type="numpy", label="ClimateGAN-Flooded image")]
+            pred_outs += [Image(type="numpy", label="Stable Diffusion-Flooded image")]
+            pred_outs += [
+                Image(
+                    type="numpy",
+                    label="Stable Diffusion-Flooded image (restricted to masked area)",
+                )
+            ]
+        with Row():
+            pred_outs += [Image(type="numpy", label="Comparison of flood images")]
+        with Row():
+            pred_outs += [Image(type="numpy", label="Wildfire")]
+            pred_outs += [Image(type="numpy", label="Smog")]
+            Image(type="numpy", label="Empty on purpose", interactive=False)
+        btn.click(predict(cg, api_key), inputs=pred_ins, outputs=pred_outs)
+
+        if api_key:
+            radio.change(toggle, inputs=[radio], outputs=[address, im_inp, btn])
+
+    app.launch(show_api=False)

climategan/trainer.py

@@ -22,7 +22,8 @@ from torch import autograd, sigmoid, softmax
 from torch.cuda.amp import GradScaler, autocast
 from tqdm import tqdm
 
-from climategan.data import get_all_loaders
+from climategan.data import get_all_loaders, decode_segmap_merged_labels
+
 from climategan.discriminator import OmniDiscriminator, create_discriminator
 from climategan.eval_metrics import accuracy, mIOU
 from climategan.fid import compute_val_fid
@@ -41,6 +42,7 @@ from climategan.tutils import (
     print_num_parameters,
     shuffle_batch_tuple,
     srgb2lrgb,
+    tensor_to_uint8_numpy_image,
     vgg_preprocess,
     zero_grad,
 )
@@ -223,18 +225,21 @@ class Trainer:
         bin_value=-1,
         half=False,
         xla=False,
-        cloudy=False,
+        cloudy=True,
         auto_resize_640=False,
         ignore_event=set(),
-        return_masks=False,
+        return_intermediates=False,
     ):
         """
-        Create a dictionnary of events from a numpy or tensor,
+        Create a dictionary of events from a numpy or tensor,
         single or batch image data.
 
-        stores is a dictionnary of times for the Timer class.
+        stores is a dictionary of times for the Timer class.
 
         bin_value is used to binarize (or not) flood masks
+
+        all values in the output dictionary have 4 dimensions:
+            BxHxWxC if numpy else BxCxHxW
         """
         assert self.is_setup
         assert len(x.shape) in {3, 4}, f"Unknown Data shape {x.shape}"
@@ -308,28 +313,39 @@ class Trainer:
         if xla:
             xm.mark_step()
 
+        output_data = {}
+
         if numpy:
             with Timer(store=stores.get("numpy", [])):
-                # normalize to 0-1
-                flood = normalize(flood).cpu()
-                smog = normalize(smog).cpu()
-                wildfire = normalize(wildfire).cpu()
-
-                # convert to numpy
-                flood = flood.permute(0, 2, 3, 1).numpy()
-                smog = smog.permute(0, 2, 3, 1).numpy()
-                wildfire = wildfire.permute(0, 2, 3, 1).numpy()
-
-                # convert to 0-255 uint8
-                flood = (flood * 255).astype(np.uint8)
-                smog = (smog * 255).astype(np.uint8)
-                wildfire = (wildfire * 255).astype(np.uint8)
-
-        output_data = {"flood": flood, "wildfire": wildfire, "smog": smog}
-        if return_masks:
-            output_data["mask"] = (
-                ((mask > bin_value) * 255).cpu().numpy().astype(np.uint8)
-            )
+                if "flood" not in ignore_event:
+                    # normalize to 0-1
+                    flood = tensor_to_uint8_numpy_image(flood)
+                    # convert to 0-255 uint8
+                    output_data["flood"] = flood
+                if "wildfire" not in ignore_event:
+                    wildfire = tensor_to_uint8_numpy_image(wildfire)
+                    output_data["wildfire"] = wildfire
+                if "smog" not in ignore_event:
+                    smog = tensor_to_uint8_numpy_image(smog)
+                    output_data["smog"] = smog
+
+        if return_intermediates:
+            if numpy:
+                output_data["mask"] = (
+                    ((mask > bin_value) * 255).cpu().numpy().astype(np.uint8)
+                )
+                output_data["depth"] = tensor_to_uint8_numpy_image(depth)
+                output_data["segmentation"] = (
+                    decode_segmap_merged_labels(segmentation, "r", False)
+                    .cpu()
+                    .permute(0, 2, 3, 1)
+                    .numpy()
+                    .astype(np.uint8)
+                )
+            else:
+                output_data["mask"] = mask
+                output_data["depth"] = depth
+                output_data["segmentation"] = segmentation
 
         return output_data
 

climategan/tutils.py

@@ -564,14 +564,29 @@ def lrgb2srgb(ims):
     return outs[0]
 
 
-def normalize(t, mini=0, maxi=1):
+def normalize(t, mini=0.0, maxi=1.0):
+    """
+    Normalizes a tensor to [0, 1].
+    If the tensor has more than 3 dimensions, the first one
+    is assumed to be the batch dimension and the tensor is
+    normalized per batch element, not across the batches.
+
+    Args:
+        t (torch.Tensor): Tensor to normalize
+        mini (float, optional): Min allowed value. Defaults to 0.
+        maxi (float, optional): Max allowed value. Defaults to 1.
+
+    Returns:
+        torch.Tensor: The normalized tensor
+    """
     if len(t.shape) == 3:
         return mini + (maxi - mini) * (t - t.min()) / (t.max() - t.min())
 
     batch_size = t.shape[0]
-    min_t = t.reshape(batch_size, -1).min(1)[0].reshape(batch_size, 1, 1, 1)
+    extra_dims = [1] * (t.ndim - 1)
+    min_t = t.reshape(batch_size, -1).min(1)[0].reshape(batch_size, *extra_dims)
     t = t - min_t
-    max_t = t.reshape(batch_size, -1).max(1)[0].reshape(batch_size, 1, 1, 1)
+    max_t = t.reshape(batch_size, -1).max(1)[0].reshape(batch_size, *extra_dims)
     t = t / max_t
     return mini + (maxi - mini) * t
 
@@ -644,7 +659,7 @@ def write_architecture(trainer):
             f.write(output)
 
 
-def rand_perlin_2d(shape, res, fade=lambda t: 6 * t ** 5 - 15 * t ** 4 + 10 * t ** 3):
+def rand_perlin_2d(shape, res, fade=lambda t: 6 * t**5 - 15 * t**4 + 10 * t**3):
     delta = (res[0] / shape[0], res[1] / shape[1])
     d = (shape[0] // res[0], shape[1] // res[1])
 
@@ -719,3 +734,30 @@ def tensor_ims_to_np_uint8s(ims):
         nps.append(n.astype(np.uint8))
 
     return nps[0] if len(nps) == 1 else nps
+
+
+def tensor_to_uint8_numpy_image(tensor):
+    """
+    Turns a BxCxHxW tensor into a numpy image:
+    * normalize
+    * to [0, 255]
+    * detach
+    * channels last
+    * to uin8
+    * to cpu
+    * to numpy
+
+    Args:
+        tensor (torch.Tensor): Tensor to transform
+
+    Returns:
+        np.array: BxHxWxC np.uint8 array in [0, 255]
+    """
+    return (
+        normalize(tensor, 0, 255)  # [0, 255]
+        .detach()  # detach from graph if needed
+        .permute(0, 2, 3, 1)  # BxHxWxC
+        .to(torch.uint8)  # uint8
+        .cpu()  # cpu
+        .numpy()  # numpy array
+    )

climategan/utils.py

@@ -917,14 +917,14 @@ def all_texts_to_array(texts, width=640, height=40):
 
 
 class Timer:
-    def __init__(self, name="", store=None, precision=3, ignore=False, cuda=True):
+    def __init__(self, name="", store=None, precision=3, ignore=False, cuda=None):
         self.name = name
         self.store = store
         self.precision = precision
         self.ignore = ignore
-        self.cuda = cuda
+        self.cuda = cuda if cuda is not None else torch.cuda.is_available()
 
-        if cuda:
+        if self.cuda:
             self._start_event = torch.cuda.Event(enable_timing=True)
             self._end_event = torch.cuda.Event(enable_timing=True)
 

climategan_wrapper.py

@@ -0,0 +1,624 @@
+# based on https://huggingface.co/spaces/NimaBoscarino/climategan/blob/main/inferences.py # noqa: E501
+# thank you @NimaBoscarino
+
+import os
+import re
+from pathlib import Path
+from uuid import uuid4
+from minydra import resolved_args
+import numpy as np
+import torch
+from diffusers import StableDiffusionInpaintPipeline
+from PIL import Image
+from skimage.color import rgba2rgb
+from skimage.transform import resize
+
+from climategan.trainer import Trainer
+
+
+CUDA = torch.cuda.is_available()
+
+
+def concat_events(output_dict, events, i=None, axis=1):
+    """
+    Concatenates the `i`th data in `output_dict` according to the keys listed
+    in `events` on dimension `axis`.
+
+    Args:
+        output_dict (dict[Union[list[np.array], np.array]]): A dictionary mapping
+            events to their corresponding data :
+            {k: [HxWxC]} (for i != None) or {k: BxHxWxC}.
+        events (list[str]): output_dict's keys to concatenate.
+        axis (int, optional): Concatenation axis. Defaults to 1.
+    """
+    cs = [e for e in events if e in output_dict]
+    if i is not None:
+        return uint8(np.concatenate([output_dict[c][i] for c in cs], axis=axis))
+    return uint8(np.concatenate([output_dict[c] for c in cs], axis=axis))
+
+
+def clear(folder):
+    """
+    Deletes all the images without the inference separator "---" in their name.
+
+    Args:
+        folder (Union[str, Path]): The folder to clear.
+    """
+    for i in list(Path(folder).iterdir()):
+        if i.is_file() and "---" in i.stem:
+            i.unlink()
+
+
+def uint8(array, rescale=False):
+    """
+    convert an array to np.uint8 (does not rescale or anything else than changing dtype)
+    Args:
+        array (np.array): array to modify
+    Returns:
+        np.array(np.uint8): converted array
+    """
+    if rescale:
+        if array.min() < 0:
+            if array.min() >= -1 and array.max() <= 1:
+                array = (array + 1) / 2
+            else:
+                raise ValueError(
+                    f"Data range mismatch for image: ({array.min()}, {array.max()})"
+                )
+        if array.max() <= 1:
+            array = array * 255
+    return array.astype(np.uint8)
+
+
+def resize_and_crop(img, to=640):
+    """
+    Resizes an image so that it keeps the aspect ratio and the smallest dimensions
+    is `to`, then crops this resized image in its center so that the output is `to x to`
+    without aspect ratio distortion
+    Args:
+        img (np.array): np.uint8 255 image
+    Returns:
+        np.array: [0, 1] np.float32 image
+    """
+    # resize keeping aspect ratio: smallest dim is 640
+    h, w = img.shape[:2]
+    if h < w:
+        size = (to, int(to * w / h))
+    else:
+        size = (int(to * h / w), to)
+
+    r_img = resize(img, size, preserve_range=True, anti_aliasing=True)
+    r_img = uint8(r_img)
+
+    # crop in the center
+    H, W = r_img.shape[:2]
+
+    top = (H - to) // 2
+    left = (W - to) // 2
+
+    rc_img = r_img[top : top + to, left : left + to, :]
+
+    return rc_img / 255.0
+
+
+def to_m1_p1(img):
+    """
+    rescales a [0, 1] image to [-1, +1]
+    Args:
+        img (np.array): float32 numpy array of an image in [0, 1]
+        i (int): Index of the image being rescaled
+    Raises:
+        ValueError: If the image is not in [0, 1]
+    Returns:
+        np.array(np.float32): array in [-1, +1]
+    """
+    if img.min() >= 0 and img.max() <= 1:
+        return (img.astype(np.float32) - 0.5) * 2
+    raise ValueError(f"Data range mismatch for image: ({img.min()}, {img.max()})")
+
+
+# No need to do any timing in this, since it's just for the HF Space
+class ClimateGAN:
+    def __init__(self, model_path, dev_mode=False) -> None:
+        """
+        A wrapper for the ClimateGAN model that you can use to generate
+        events from images or folders containing images.
+
+        Args:
+            model_path (Union[str, Path]): Where to load the Masker from
+        """
+        torch.set_grad_enabled(False)
+        self.target_size = 640
+        self._stable_diffusion_is_setup = False
+        self.dev_mode = dev_mode
+        if self.dev_mode:
+            return
+        self.trainer = Trainer.resume_from_path(
+            model_path,
+            setup=True,
+            inference=True,
+            new_exp=None,
+        )
+        if CUDA:
+            self.trainer.G.half()
+
+    def _setup_stable_diffusion(self):
+        """
+        Sets up the stable diffusion pipeline for in-painting.
+        Make sure you have accepted the license on the model's card
+        https://huggingface.co/CompVis/stable-diffusion-v1-4
+        """
+        if self.dev_mode:
+            return
+
+        try:
+            self.sdip_pipeline = StableDiffusionInpaintPipeline.from_pretrained(
+                "runwayml/stable-diffusion-inpainting",
+                revision="fp16" if CUDA else "main",
+                torch_dtype=torch.float16 if CUDA else torch.float32,
+                safety_checker=None,
+                use_auth_token=os.environ.get("HF_AUTH_TOKEN"),
+            ).to(self.trainer.device)
+            self._stable_diffusion_is_setup = True
+        except Exception as e:
+            print(
+                "\nCould not load stable diffusion model. "
+                + "Please make sure you have accepted the license on the model's"
+                + " card https://huggingface.co/CompVis/stable-diffusion-v1-4\n"
+            )
+            raise e
+
+    def _preprocess_image(self, img):
+        """
+        Turns a HxWxC uint8 numpy array into a 640x640x3 float32 numpy array
+        in [-1, 1].
+
+        Args:
+            img (np.array): Image to resize crop and rescale
+
+        Returns:
+            np.array: Resized, cropped and rescaled image
+        """
+        # rgba to rgb
+        data = img if img.shape[-1] == 3 else uint8(rgba2rgb(img) * 255)
+
+        # to args.target_size
+        data = resize_and_crop(data, self.target_size)
+
+        # resize() produces [0, 1] images, rescale to [-1, 1]
+        data = to_m1_p1(data)
+        return data
+
+    # Does all three inferences at the moment.
+    def infer_single(
+        self,
+        orig_image,
+        painter="both",
+        prompt="An HD picture of a street with dirty water after a heavy flood",
+        concats=[
+            "input",
+            "masked_input",
+            "climategan_flood",
+            "stable_flood",
+            "stable_copy_flood",
+        ],
+        as_pil_image=False,
+    ):
+        """
+        Infers the image with the ClimateGAN model.
+        Importantly (and unlike self.infer_preprocessed_batch), the image is
+        pre-processed by self._preprocess_image before going through the networks.
+
+        Output dict contains the following keys:
+        - "input": The input image
+        - "mask": The mask used to generate the flood (from ClimateGAN's Masker)
+        - "masked_input": The input image with the mask applied
+        - "climategan_flood": The flooded image generated by ClimateGAN's Painter
+            on the masked input (only if "painter" is "climategan" or "both").
+        - "stable_flood": The flooded image in-painted by the stable diffusion model
+            from the mask and the input image (only if "painter" is "stable_diffusion"
+            or "both").
+        - "stable_copy_flood": The flooded image in-painted by the stable diffusion
+            model with its original context pasted back in:
+            y = m * flooded + (1-m) * input
+            (only if "painter" is "stable_diffusion" or "both").
+
+        Args:
+            orig_image (Union[str, np.array]): image to infer on. Can be a path to
+                an image which will be read.
+            painter (str, optional): Which painter to use: "climategan",
+                "stable_diffusion" or "both". Defaults to "both".
+            prompt (str, optional): The prompt used to guide the diffusion. Defaults
+                to "An HD picture of a street with dirty water after a heavy flood".
+            concats (list, optional): List of keys in `output` to concatenate together
+                in a new `{original_stem}_concat` image written. Defaults to:
+                ["input", "masked_input", "climategan_flood", "stable_flood",
+                "stable_copy_flood"].
+
+        Returns:
+            dict: a dictionary containing the output images {k: HxWxC}. C is omitted
+                for masks (HxW).
+        """
+        if self.dev_mode:
+            return {
+                "input": orig_image,
+                "mask": np.random.randint(0, 255, (640, 640)),
+                "masked_input": np.random.randint(0, 255, (640, 640, 3)),
+                "climategan_flood": np.random.randint(0, 255, (640, 640, 3)),
+                "stable_flood": np.random.randint(0, 255, (640, 640, 3)),
+                "stable_copy_flood": np.random.randint(0, 255, (640, 640, 3)),
+                "concat": np.random.randint(0, 255, (640, 640 * 5, 3)),
+                "smog": np.random.randint(0, 255, (640, 640, 3)),
+                "wildfire": np.random.randint(0, 255, (640, 640, 3)),
+                "depth": np.random.randint(0, 255, (640, 640, 1)),
+                "segmentation": np.random.randint(0, 255, (640, 640, 3)),
+            }
+            return
+
+        image_array = (
+            np.array(Image.open(orig_image))
+            if isinstance(orig_image, str)
+            else orig_image
+        )
+
+        pil_image = None
+        if as_pil_image:
+            pil_image = Image.fromarray(image_array)
+        print("Preprocessing image")
+        image = self._preprocess_image(image_array)
+        output_dict = self.infer_preprocessed_batch(
+            images=image[None, ...],
+            painter=painter,
+            prompt=prompt,
+            concats=concats,
+            pil_image=pil_image,
+        )
+        print("Inference done")
+        return {k: v[0] for k, v in output_dict.items()}
+
+    def infer_preprocessed_batch(
+        self,
+        images,
+        painter="both",
+        prompt="An HD picture of a street with dirty water after a heavy flood",
+        concats=[
+            "input",
+            "masked_input",
+            "climategan_flood",
+            "stable_flood",
+            "stable_copy_flood",
+        ],
+        pil_image=None,
+    ):
+        """
+        Infers ClimateGAN predictions on a batch of preprocessed images.
+        It assumes that each image in the batch has been preprocessed with
+        self._preprocess_image().
+
+        Output dict contains the following keys:
+        - "input": The input image
+        - "mask": The mask used to generate the flood (from ClimateGAN's Masker)
+        - "masked_input": The input image with the mask applied
+        - "climategan_flood": The flooded image generated by ClimateGAN's Painter
+            on the masked input (only if "painter" is "climategan" or "both").
+        - "stable_flood": The flooded image in-painted by the stable diffusion model
+            from the mask and the input image (only if "painter" is "stable_diffusion"
+            or "both").
+        - "stable_copy_flood": The flooded image in-painted by the stable diffusion
+            model with its original context pasted back in:
+            y = m * flooded + (1-m) * input
+            (only if "painter" is "stable_diffusion" or "both").
+
+        Args:
+            images (np.array): A batch of input images BxHxWx3
+            painter (str, optional): Which painter to use: "climategan",
+                "stable_diffusion" or "both". Defaults to "both".
+            prompt (str, optional): The prompt used to guide the diffusion. Defaults
+                to "An HD picture of a street with dirty water after a heavy flood".
+            concats (list, optional): List of keys in `output` to concatenate together
+                in a new `{original_stem}_concat` image written. Defaults to:
+                ["input", "masked_input", "climategan_flood", "stable_flood",
+                "stable_copy_flood"].
+            pil_image (PIL.Image, optional): The original PIL image. If provided,
+                will be used for a single inference (batch_size=1)
+
+        Returns:
+            dict: a dictionary containing the output images
+        """
+        assert painter in [
+            "both",
+            "stable_diffusion",
+            "climategan",
+        ], f"Unknown painter: {painter}"
+
+        ignore_event = set()
+        if painter == "stable_diffusion":
+            ignore_event.add("flood")
+
+        if pil_image is not None:
+            print("Warning: `pil_image` has been provided, it will override `images`")
+            images = self._preprocess_image(np.array(pil_image))[None, ...]
+            pil_image = Image.fromarray(((images[0] + 1) / 2 * 255).astype(np.uint8))
+
+        # Retrieve numpy events as a dict {event: array[BxHxWxC]}
+        print("Inferring ClimateGAN events")
+        outputs = self.trainer.infer_all(
+            images,
+            numpy=True,
+            bin_value=0.5,
+            half=CUDA,
+            ignore_event=ignore_event,
+            return_intermediates=True,
+        )
+
+        outputs["input"] = uint8(images, True)
+        # from Bx1xHxW to BxHxWx1
+        outputs["masked_input"] = outputs["input"] * (
+            outputs["mask"].squeeze(1)[..., None] == 0
+        )
+
+        if painter in {"both", "climategan"}:
+            outputs["climategan_flood"] = outputs.pop("flood")
+        else:
+            del outputs["flood"]
+
+        if painter != "climategan":
+            if not self._stable_diffusion_is_setup:
+                print("Setting up stable diffusion in-painting pipeline")
+                self._setup_stable_diffusion()
+
+            mask = outputs["mask"].squeeze(1)
+            input_images = (
+                torch.tensor(images).permute(0, 3, 1, 2).to(self.trainer.device)
+                if pil_image is None
+                else pil_image
+            )
+            input_mask = (
+                torch.tensor(mask[:, None, ...] > 0).to(self.trainer.device)
+                if pil_image is None
+                else Image.fromarray(mask[0])
+            )
+            print("Inferring stable diffusion in-painting for 50 steps")
+            floods = self.sdip_pipeline(
+                prompt=[prompt] * images.shape[0],
+                image=input_images,
+                mask_image=input_mask,
+                height=640,
+                width=640,
+                num_inference_steps=50,
+            )
+            print("Stable diffusion in-painting done")
+
+            bin_mask = mask[..., None] > 0
+            flood = np.stack([np.array(i) for i in floods.images])
+            copy_flood = flood * bin_mask + uint8(images, True) * (1 - bin_mask)
+            outputs["stable_flood"] = flood
+            outputs["stable_copy_flood"] = copy_flood
+
+        if concats:
+            print("Concatenating flood images")
+            outputs["concat"] = concat_events(outputs, concats, axis=2)
+
+        return {k: v.squeeze(1) if v.shape[1] == 1 else v for k, v in outputs.items()}
+
+    def infer_folder(
+        self,
+        folder_path,
+        painter="both",
+        prompt="An HD picture of a street with dirty water after a heavy flood",
+        batch_size=4,
+        concats=[
+            "input",
+            "masked_input",
+            "climategan_flood",
+            "stable_flood",
+            "stable_copy_flood",
+        ],
+        write=True,
+        overwrite=False,
+    ):
+        """
+        Infers the images in a folder with the ClimateGAN model, batching images for
+        inference according to the batch_size.
+
+        Images must end in .jpg, .jpeg or .png (not case-sensitive).
+        Images must not contain the separator ("---") in their name.
+
+        Images will be written to disk in the same folder as the input images, with
+        a name that depends on its data, potentially the prompt and a random
+        identifier in case multiple inferences are run in the folder.
+
+        Output dict contains the following keys:
+        - "input": The input image
+        - "mask": The mask used to generate the flood (from ClimateGAN's Masker)
+        - "masked_input": The input image with the mask applied
+        - "climategan_flood": The flooded image generated by ClimateGAN's Painter
+            on the masked input (only if "painter" is "climategan" or "both").
+        - "stable_flood": The flooded image in-painted by the stable diffusion model
+            from the mask and the input image (only if "painter" is "stable_diffusion"
+            or "both").
+        - "stable_copy_flood": The flooded image in-painted by the stable diffusion
+            model with its original context pasted back in:
+            y = m * flooded + (1-m) * input
+            (only if "painter" is "stable_diffusion" or "both").
+
+        Args:
+            folder_path (Union[str, Path]): Where to read images from.
+            painter (str, optional): Which painter to use: "climategan",
+                "stable_diffusion" or "both". Defaults to "both".
+            prompt (str, optional): The prompt used to guide the diffusion. Defaults
+                to "An HD picture of a street with dirty water after a heavy flood".
+            batch_size (int, optional): Size of inference batches. Defaults to 4.
+            concats (list, optional): List of keys in `output` to concatenate together
+                in a new `{original_stem}_concat` image written. Defaults to:
+                ["input", "masked_input", "climategan_flood", "stable_flood",
+                "stable_copy_flood"].
+            write (bool, optional): Whether or not to write the outputs to the input
+                folder.Defaults to True.
+            overwrite (Union[bool, str], optional): Whether to overwrite the images or
+                not. If a string is provided, it will be included in the name.
+                Defaults to False.
+
+        Returns:
+            dict: a dictionary containing the output images
+        """
+        folder_path = Path(folder_path).expanduser().resolve()
+        assert folder_path.exists(), f"Folder {str(folder_path)} does not exist"
+        assert folder_path.is_dir(), f"{str(folder_path)} is not a directory"
+        im_paths = [
+            p
+            for p in folder_path.iterdir()
+            if p.suffix.lower() in [".jpg", ".png", ".jpeg"] and "---" not in p.name
+        ]
+        assert im_paths, f"No images found in {str(folder_path)}"
+        ims = [self._preprocess_image(np.array(Image.open(p))) for p in im_paths]
+        batches = [
+            np.stack(ims[i : i + batch_size]) for i in range(0, len(ims), batch_size)
+        ]
+        inferences = [
+            self.infer_preprocessed_batch(b, painter, prompt, concats) for b in batches
+        ]
+
+        outputs = {
+            k: [i for e in inferences for i in e[k]] for k in inferences[0].keys()
+        }
+
+        if write:
+            self.write(outputs, im_paths, painter, overwrite, prompt)
+
+        return outputs
+
+    def write(
+        self,
+        outputs,
+        im_paths,
+        painter="both",
+        overwrite=False,
+        prompt="",
+    ):
+        """
+        Writes the outputs of the inference to disk, in the input folder.
+
+        Images will be named like:
+        f"{original_stem}---{overwrite_prefix}_{painter_type}_{output_type}.{suffix}"
+        `painter_type` is either "climategan" or f"stable_diffusion_{prompt}"
+
+        Args:
+            outputs (_type_): The inference procedure's output dict.
+            im_paths (list[Path]): The list of input images paths.
+            painter (str, optional): Which painter was used. Defaults to "both".
+            overwrite (bool, optional): Whether to overwrite the images or not.
+                If a string is provided, it will be included in the name.
+                If False, a random identifier will be added to the name.
+                Defaults to False.
+            prompt (str, optional): The prompt used to guide the diffusion. Defaults
+                to "".
+        """
+        prompt = re.sub("[^0-9a-zA-Z]+", "", prompt).lower()
+        overwrite_prefix = ""
+        if not overwrite:
+            overwrite_prefix = str(uuid4())[:8]
+            print("Writing events with prefix", overwrite_prefix)
+        else:
+            if isinstance(overwrite, str):
+                overwrite_prefix = overwrite
+                print("Writing events with prefix", overwrite_prefix)
+
+        # for each image, for each event/data type
+        for i, im_path in enumerate(im_paths):
+            for event, ims in outputs.items():
+                painter_prefix = ""
+                if painter == "climategan" and event == "flood":
+                    painter_prefix = "climategan"
+                elif (
+                    painter in {"stable_diffusion", "both"} and event == "stable_flood"
+                ):
+                    painter_prefix = f"_stable_{prompt}"
+                elif painter == "both" and event == "climategan_flood":
+                    painter_prefix = ""
+
+                im = ims[i]
+                im = Image.fromarray(uint8(im))
+                imstem = f"{im_path.stem}---{overwrite_prefix}{painter_prefix}_{event}"
+                im.save(im_path.parent / (imstem + im_path.suffix))
+
+
+if __name__ == "__main__":
+    print("Run `$ python climategan_wrapper.py help` for usage instructions\n")
+
+    # parse arguments
+    args = resolved_args(
+        defaults={
+            "input_folder": None,
+            "output_folder": None,
+            "painter": "both",
+            "help": False,
+        }
+    )
+
+    # print help
+    if args.help:
+        print(
+            "Usage: python inference.py input_folder=/path/to/folder\n"
+            + "By default inferences will be stored in the input folder.\n"
+            + "Add `output_folder=/path/to/folder` for a different output folder.\n"
+            + "By default, both ClimateGAN and Stable Diffusion will be used."
+            + "Change this by adding `painter=climategan` or"
+            + " `painter=stable_diffusion`.\n"
+            + "Make sure you have agreed to the terms of use for the models."
+            + "In particular, visit SD's model card to agree to the terms of use:"
+            + " https://huggingface.co/runwayml/stable-diffusion-inpainting"
+        )
+    # print args
+    args.pretty_print()
+
+    # load models
+    cg = ClimateGAN("models/climategan")
+
+    # check painter type
+    assert args.painter in {"climategan", "stable_diffusion", "both",}, (
+        f"Unknown painter {args.painter}. "
+        + "Allowed values are 'climategan', 'stable_diffusion' and 'both'."
+    )
+
+    # load SD pipeline if need be
+    if args.painter != "climate_gan":
+        cg._setup_stable_diffusion()
+
+    # resolve input folder path
+    in_path = Path(args.input_folder).expanduser().resolve()
+    assert in_path.exists(), f"Folder {str(in_path)} does not exist"
+
+    # output is input if not specified
+    if args.output_folder is None:
+        out_path = in_path
+
+    # find images in input folder
+    im_paths = [
+        p
+        for p in in_path.iterdir()
+        if p.suffix.lower() in [".jpg", ".png", ".jpeg"] and "---" not in p.name
+    ]
+    assert im_paths, f"No images found in {str(im_paths)}"
+
+    print(f"\nFound {len(im_paths)} images in {str(in_path)}\n")
+
+    # infer and write
+    for i, im_path in enumerate(im_paths):
+        print(">>> Processing", f"{i}/{len(im_paths)}", im_path.name)
+        outs = cg.infer_single(
+            np.array(Image.open(im_path)),
+            args.painter,
+            as_pil_image=True,
+            concats=[
+                "input",
+                "masked_input",
+                "climategan_flood",
+                "stable_copy_flood",
+            ],
+        )
+        for k, v in outs.items():
+            name = f"{im_path.stem}---{k}{im_path.suffix}"
+            im = Image.fromarray(uint8(v))
+            im.save(out_path / name)
+        print(">>> Done", f"{i}/{len(im_paths)}", im_path.name, end="\n\n")

inferences.py

@@ -1,108 +0,0 @@
-# based on https://huggingface.co/spaces/NimaBoscarino/climategan/blob/main/inferences.py # noqa: E501
-# thank you @NimaBoscarino
-
-import torch
-from skimage.color import rgba2rgb
-from skimage.transform import resize
-import numpy as np
-
-from climategan.trainer import Trainer
-
-
-def uint8(array):
-    """
-    convert an array to np.uint8 (does not rescale or anything else than changing dtype)
-    Args:
-        array (np.array): array to modify
-    Returns:
-        np.array(np.uint8): converted array
-    """
-    return array.astype(np.uint8)
-
-
-def resize_and_crop(img, to=640):
-    """
-    Resizes an image so that it keeps the aspect ratio and the smallest dimensions
-    is `to`, then crops this resized image in its center so that the output is `to x to`
-    without aspect ratio distortion
-    Args:
-        img (np.array): np.uint8 255 image
-    Returns:
-        np.array: [0, 1] np.float32 image
-    """
-    # resize keeping aspect ratio: smallest dim is 640
-    h, w = img.shape[:2]
-    if h < w:
-        size = (to, int(to * w / h))
-    else:
-        size = (int(to * h / w), to)
-
-    r_img = resize(img, size, preserve_range=True, anti_aliasing=True)
-    r_img = uint8(r_img)
-
-    # crop in the center
-    H, W = r_img.shape[:2]
-
-    top = (H - to) // 2
-    left = (W - to) // 2
-
-    rc_img = r_img[top : top + to, left : left + to, :]
-
-    return rc_img / 255.0
-
-
-def to_m1_p1(img):
-    """
-    rescales a [0, 1] image to [-1, +1]
-    Args:
-        img (np.array): float32 numpy array of an image in [0, 1]
-        i (int): Index of the image being rescaled
-    Raises:
-        ValueError: If the image is not in [0, 1]
-    Returns:
-        np.array(np.float32): array in [-1, +1]
-    """
-    if img.min() >= 0 and img.max() <= 1:
-        return (img.astype(np.float32) - 0.5) * 2
-    raise ValueError(f"Data range mismatch for image: ({img.min()}, {img.max()})")
-
-
-# No need to do any timing in this, since it's just for the HF Space
-class ClimateGAN:
-    def __init__(self, model_path) -> None:
-        torch.set_grad_enabled(False)
-        self.target_size = 640
-        self.trainer = Trainer.resume_from_path(
-            model_path,
-            setup=True,
-            inference=True,
-            new_exp=None,
-        )
-
-    # Does all three inferences at the moment.
-    def inference(self, orig_image):
-        image = self._preprocess_image(orig_image)
-
-        # Retrieve numpy events as a dict {event: array[BxHxWxC]}
-        outputs = self.trainer.infer_all(
-            image,
-            numpy=True,
-            bin_value=0.5,
-        )
-
-        return (
-            outputs["flood"].squeeze(),
-            outputs["wildfire"].squeeze(),
-            outputs["smog"].squeeze(),
-        )
-
-    def _preprocess_image(self, img):
-        # rgba to rgb
-        data = img if img.shape[-1] == 3 else uint8(rgba2rgb(img) * 255)
-
-        # to args.target_size
-        data = resize_and_crop(data, self.target_size)
-
-        # resize() produces [0, 1] images, rescale to [-1, 1]
-        data = to_m1_p1(data)
-        return data

requirements.txt

@@ -0,0 +1,169 @@
+addict==2.4.0
+aiohttp==3.8.3
+aiosignal==1.2.0
+anyio==3.6.2
+appnope==0.1.3
+APScheduler==3.7.0
+asttokens==2.0.8
+async-timeout==4.0.2
+attrs==21.2.0
+backcall==0.2.0
+bcrypt==4.0.1
+black==22.10.0
+blis==0.7.9
+Brotli==1.0.9
+catalogue==2.0.8
+certifi==2021.5.30
+cffi==1.15.1
+charset-normalizer==2.0.4
+click==8.0.1
+codecarbon==1.2.0
+comet-ml==3.15.3
+commonmark==0.9.1
+confection==0.0.3
+configobj==5.0.6
+contourpy==1.0.5
+cryptography==38.0.1
+cycler==0.10.0
+cymem==2.0.7
+dash==2.0.0
+dash-bootstrap-components==0.13.0
+dash-core-components==2.0.0
+dash-html-components==2.0.0
+dash-table==5.0.0
+dataclasses==0.6
+decorator==5.0.9
+diffusers==0.6.0
+dulwich==0.20.25
+everett==2.0.1
+executing==1.1.1
+fastapi==0.85.1
+ffmpy==0.3.0
+filelock==3.0.12
+fire==0.4.0
+flake8==5.0.4
+Flask==2.0.1
+Flask-Compress==1.10.1
+fonttools==4.38.0
+frozenlist==1.3.1
+fsspec==2022.10.0
+future==0.18.2
+gdown==3.13.0
+googlemaps==4.6.0
+gradio==3.6
+h11==0.12.0
+httpcore==0.15.0
+httpx==0.23.0
+huggingface-hub==0.10.1
+hydra-core==0.11.3
+idna==3.2
+imageio==2.9.0
+importlib-metadata==5.0.0
+ipython==7.27.0
+itsdangerous==2.0.1
+jedi==0.18.0
+Jinja2==3.0.1
+joblib==1.0.1
+jsonschema==3.2.0
+kiwisolver==1.3.2
+kornia==0.5.10
+langcodes==3.3.0
+linkify-it-py==1.0.3
+markdown-it-py==2.1.0
+MarkupSafe==2.0.1
+matplotlib==3.4.3
+matplotlib-inline==0.1.2
+mccabe==0.7.0
+mdit-py-plugins==0.3.1
+mdurl==0.1.2
+minydra==0.1.6
+multidict==6.0.2
+murmurhash==1.0.9
+mypy-extensions==0.4.3
+networkx==2.6.2
+numpy==1.21.2
+nvidia-ml-py3==7.352.0
+omegaconf==1.4.1
+opencv-python==4.5.3.56
+orjson==3.8.0
+packaging==21.0
+pandas==1.3.2
+paramiko==2.11.0
+parso==0.8.2
+pathspec==0.10.1
+pathy==0.6.2
+pexpect==4.8.0
+pickleshare==0.7.5
+Pillow==8.3.2
+platformdirs==2.5.2
+plotly==5.3.1
+preshed==3.0.8
+prompt-toolkit==3.0.20
+ptyprocess==0.7.0
+pure-eval==0.2.2
+py-cpuinfo==8.0.0
+pycodestyle==2.9.1
+pycparser==2.21
+pycryptodome==3.15.0
+pydantic==1.10.2
+pydub==0.25.1
+pyflakes==2.5.0
+Pygments==2.10.0
+PyNaCl==1.5.0
+pynvml==11.0.0
+pyparsing==2.4.7
+pyrsistent==0.18.0
+PySocks==1.7.1
+python-dateutil==2.8.2
+python-multipart==0.0.5
+pytorch-ranger==0.1.1
+pytz==2021.1
+PyWavelets==1.1.1
+PyYAML==5.4.1
+regex==2022.9.13
+requests==2.26.0
+requests-toolbelt==0.9.1
+rfc3986==1.5.0
+rich==12.6.0
+scikit-image==0.18.3
+scikit-learn==0.24.2
+scipy==1.7.1
+seaborn==0.11.2
+semantic-version==2.8.5
+six==1.16.0
+smart-open==5.2.1
+sniffio==1.3.0
+spacy==3.4.2
+spacy-legacy==3.0.10
+spacy-loggers==1.0.3
+srsly==2.4.5
+stack-data==0.5.1
+starlette==0.20.4
+tenacity==8.0.1
+termcolor==1.1.0
+thinc==8.1.5
+threadpoolctl==2.2.0
+tifffile==2021.8.30
+tokenizers==0.13.1
+tomli==2.0.1
+torch==1.7.1
+torch-optimizer==0.1.0
+torchvision==0.8.2
+tqdm==4.62.2
+traitlets==5.1.0
+transformers==4.23.1
+typer==0.4.2
+typing_extensions==4.4.0
+tzlocal==2.1
+uc-micro-py==1.0.1
+urllib3==1.26.6
+uvicorn==0.19.0
+wasabi==0.10.1
+wcwidth==0.2.5
+websocket-client==1.2.1
+websockets==10.3
+Werkzeug==2.0.1
+wrapt==1.12.1
+wurlitzer==3.0.2
+yarl==1.8.1
+zipp==3.10.0