Check out the preprint of our work “Mélange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneity”. Here are the Code and Blog.