Analyzing tool for inference

#13
by cllatMTK - opened

Hi,

I've come across this incredibly handy tool, and I'm looking to delve deeper into understanding memory consumption and inference speed. I developed this tool and would greatly appreciate any feedback or collaboration.
My hope is that we can together develop a comprehensive set of analytical tools that aid in forecasting these things.
You can check it out here: https://huggingface.co/spaces/cllatMTK/TransformerAnalyzer.
Currently, I've only tackled the inference side of things, but I'm keen to expand towards predicting training times as well.

Sign up or log in to comment