Did someone have a problem with JupyterLab when start training model?

#20
by artyomboyko - opened

Hi. I use JupiterLab notebook based on this article.

JupiterLab is started via the command line: jupyter-lab --NotebookApp.iopub_data_rate_limit=1e10

When I run the model training process, I get the following error:
image.png

Versions of the libraries I have installed (just in case):

artyom@MSK-PC-01:~/Whisper_Train$ pip list
Package                   Version
------------------------- -------------
absl-py                   1.4.0
accelerate                0.21.0
aiofiles                  23.1.0
aiohttp                   3.8.4
aiosignal                 1.3.1
altair                    5.0.1
annotated-types           0.5.0
anyio                     3.7.1
appdirs                   1.4.4
argon2-cffi               21.3.0
argon2-cffi-bindings      21.2.0
arrow                     1.2.3
asttokens                 2.2.1
async-lru                 2.0.3
async-timeout             4.0.2
attrs                     23.1.0
audioread                 3.0.0
Babel                     2.12.1
backcall                  0.2.0
beautifulsoup4            4.12.2
bleach                    6.0.0
blinker                   1.4
cachetools                5.3.1
certifi                   2023.5.7
cffi                      1.15.1
chardet                   5.1.0
charset-normalizer        3.2.0
click                     8.1.4
cloudpickle               2.2.1
cmake                     3.26.4
comm                      0.1.3
command-not-found         0.3
contourpy                 1.1.0
cryptography              3.4.8
cuda-python               12.2.0
cudf-cu12                 23.6.1
cuml-cu12                 23.6.0
cupy-cuda12x              12.1.0
cycler                    0.11.0
Cython                    0.29.36
dask                      2023.3.2
dask-cuda                 23.6.0
dask-cudf-cu12            23.6.0
datasets                  2.13.1
dbus-python               1.2.18
debugpy                   1.6.7
decorator                 5.1.1
defusedxml                0.7.1
dill                      0.3.6
distributed               2023.3.2.1
distro                    1.7.0
distro-info               1.1build1
et-xmlfile                1.1.0
evaluate                  0.4.0
exceptiongroup            1.1.2
executing                 1.2.0
Faker                     18.13.0
faker-marketdata          0.4
fastapi                   0.100.0
fastjsonschema            2.17.1
fastrlock                 0.8.1
ffmpeg                    1.4
ffmpy                     0.3.0
filelock                  3.12.2
fonttools                 4.40.0
fqdn                      1.5.1
frozenlist                1.3.3
fsspec                    2023.6.0
git-lfs                   1.6
google-auth               2.22.0
google-auth-oauthlib      1.0.0
gradio                    3.36.1
gradio_client             0.2.7
grpcio                    1.56.2
h11                       0.14.0
httpcore                  0.17.3
httplib2                  0.20.2
httpx                     0.24.1
huggingface-hub           0.16.4
idna                      3.4
importlib-metadata        6.8.0
inquirerpy                0.3.4
ipykernel                 6.24.0
ipython                   8.14.0
ipywidgets                8.0.7
isoduration               20.11.0
jedi                      0.18.2
jeepney                   0.7.1
Jinja2                    3.1.2
jiwer                     3.0.2
joblib                    1.3.1
json5                     0.9.14
jsonpointer               2.4
jsonschema                4.18.0
jsonschema-specifications 2023.6.1
jupyter_client            8.3.0
jupyter_core              5.3.1
jupyter-events            0.6.3
jupyter-lsp               2.2.0
jupyter_server            2.7.0
jupyter_server_terminals  0.4.4
jupyterlab                4.0.2
jupyterlab-pygments       0.2.2
jupyterlab_server         2.23.0
jupyterlab-widgets        3.0.8
keyring                   23.5.0
kiwisolver                1.4.4
launchpadlib              1.10.16
lazr.restfulclient        0.14.4
lazr.uri                  1.0.6
lazy_loader               0.3
librosa                   0.10.0.post2
linkify-it-py             2.0.2
lit                       16.0.6
llvmlite                  0.40.1
locket                    1.0.0
Markdown                  3.4.3
markdown-it-py            2.2.0
MarkupSafe                2.1.3
matplotlib                3.7.2
matplotlib-inline         0.1.6
mdit-py-plugins           0.3.3
mdurl                     0.1.2
mistune                   3.0.1
more-itertools            8.10.0
mpmath                    1.3.0
msgpack                   1.0.5
multidict                 6.0.4
multiprocess              0.70.14
nbclient                  0.8.0
nbconvert                 7.6.0
nbformat                  5.9.0
nest-asyncio              1.5.6
netifaces                 0.11.0
networkx                  3.1
notebook_shim             0.2.3
numba                     0.57.1
numpy                     1.24.4
nvidia-cublas-cu11        11.10.3.66
nvidia-cuda-cupti-cu11    11.7.101
nvidia-cuda-nvrtc-cu11    11.7.99
nvidia-cuda-runtime-cu11  11.7.99
nvidia-cudnn-cu11         8.5.0.96
nvidia-cufft-cu11         10.9.0.58
nvidia-curand-cu11        10.2.10.91
nvidia-cusolver-cu11      11.4.0.1
nvidia-cusparse-cu11      11.7.4.91
nvidia-nccl-cu11          2.14.3
nvidia-nvtx-cu11          11.7.91
nvtx                      0.2.5
oauthlib                  3.2.0
openpyxl                  3.1.2
orjson                    3.9.2
overrides                 7.3.1
packaging                 23.1
pandas                    1.5.3
pandocfilters             1.5.0
parso                     0.8.3
partd                     1.4.0
pexpect                   4.8.0
pfzy                      0.3.4
pickleshare               0.7.5
Pillow                    10.0.0
pip                       22.0.2
platformdirs              3.8.1
plotly                    5.15.0
pooch                     1.6.0
prometheus-client         0.17.0
prompt-toolkit            3.0.39
protobuf                  4.21.12
psutil                    5.9.5
ptyprocess                0.7.0
pure-eval                 0.2.2
pyarrow                   11.0.0
pyasn1                    0.5.0
pyasn1-modules            0.3.0
pycountry                 22.3.5
pycparser                 2.21
pydantic                  2.0.2
pydantic_core             2.1.2
pydub                     0.25.1
Pygments                  2.15.1
PyGObject                 3.42.1
PyJWT                     2.3.0
pylibraft-cu12            23.6.2
pynvml                    11.4.1
pyparsing                 2.4.7
python-apt                2.4.0+ubuntu1
python-dateutil           2.8.2
python-json-logger        2.0.7
python-multipart          0.0.6
pytz                      2023.3
PyYAML                    5.4.1
pyzmq                     25.1.0
raft-dask-cu12            23.6.2
rapidfuzz                 2.13.7
referencing               0.29.1
regex                     2023.6.3
requests                  2.31.0
requests-oauthlib         1.3.1
responses                 0.18.0
rfc3339-validator         0.1.4
rfc3986-validator         0.1.1
rmm-cu12                  23.6.0
rpds-py                   0.8.10
rsa                       4.9
safetensors               0.3.1
scikit-learn              1.3.0
scipy                     1.11.1
SecretStorage             3.3.1
semantic-version          2.10.0
Send2Trash                1.8.2
setuptools                59.6.0
six                       1.16.0
sniffio                   1.3.0
sortedcontainers          2.4.0
soundfile                 0.12.1
soupsieve                 2.4.1
soxr                      0.3.5
stack-data                0.6.2
starlette                 0.27.0
sympy                     1.12
systemd-python            234
tblib                     2.0.0
tenacity                  8.2.2
tensorboard               2.13.0
tensorboard-data-server   0.7.1
terminado                 0.17.1
threadpoolctl             3.1.0
tinycss2                  1.2.1
tokenizers                0.13.3
tomli                     2.0.1
toolz                     0.12.0
torch                     2.0.1
torchaudio                2.0.2
torchvision               0.15.2
tornado                   6.3.2
tqdm                      4.65.0
traitlets                 5.9.0
transformers              4.30.2
transliterate             1.10.2
treelite                  3.2.0
treelite-runtime          3.2.0
triton                    2.0.0
typing_extensions         4.7.1
tzdata                    2023.3
ubuntu-advantage-tools    8001
uc-micro-py               1.0.2
ucx-py-cu12               0.32.0
ufw                       0.36.1
unattended-upgrades       0.1
uri-template              1.3.0
urllib3                   1.26.16
uvicorn                   0.22.0
wadllib                   1.3.6
wcwidth                   0.2.6
webcolors                 1.13
webencodings              0.5.1
websocket-client          1.6.1
websockets                11.0.3
Werkzeug                  2.3.6
wheel                     0.37.1
widgetsnbextension        4.0.8
xxhash                    3.2.0
yarl                      1.9.2
zict                      3.0.0
zipp                      1.0.0

@sanchit-gandhi Hi Sanchit, мaybe you can offer some advice on what the cause might be?

Sign up or log in to comment