README.md

---
license: llama3
---

This is the code and data repo of paper [HDFlow: Enhancing LLM Complex Problem-Solving with Hybrid Thinking and Dynamic Workflows](https://arxiv.org/abs/2409.17433). If you think this repo is useful, please cite us!

```bibtex
@article{yao2024hdflow,
  title={HDFlow: Enhancing LLM Complex Problem-Solving with Hybrid Thinking and Dynamic Workflows},
  author={Yao, Wenlin and Mi, Haitao and Yu, Dong},
  journal={arXiv preprint arXiv:2409.17433},
  year={2024}
}
```

Download our [model](https://huggingface.co/wenlinyao/HDFlow-Llama-3-8B-Instruct/tree/main) into the dir /xxxx/checkpoints/wenlinyao/HDFlow-Llama-3-8B-Instruct/. Next, use vllm to host the model service. Here provides an example to host two models at the same time (using different ports).

```sh
CUDA_VISIBLE_DEVICES=0 python -m vllm.entrypoints.openai.api_server --model /xxxx/checkpoints/wenlinyao/HDFlow-Llama-3-8B-Instruct/ --tensor-parallel-size 1 --max-num-batched-tokens 8192 --dtype bfloat16 --port 4231
```

Refer to https://github.com/wenlinyao/HDFlow for more details.