Our workshop will start at 13:00 and is located at UCD O'Brien Centre for Science.. First we present the current research on sustainability and large language models in software engineering. Then we will listen to the presentation of
| 13:00 | Opening |
| The organisers will welcome participants and introduce the goals and format of the workshop. |
| 13:10 | Introduction to topic |
| Pepijn de Reus | |
| A short presentation on the intersection of LLMs, software development, and sustainability. |
| 13:30 | Paper: Benchmarking Energy Efficiency of Large Language Models Using vLLM |
| Kalle Pronk | |
| View the Paper The prevalence of Large Language Models (LLMs) is having an growing impact on the climate due to the substantial energy required for their deployment and use. To create awareness for developers who are implementing LLMs in their products, there is a strong need to collect more information about the energy efficiency of LLMs. While existing research has evaluated the energy efficiency of various models, these benchmarks often fall short of representing realistic production scenarios. In this paper, we introduce the LLM Efficiency Benchmark, designed to simulate real-world usage conditions. Our benchmark utilizes vLLM, a high-throughput, productionready LLM serving backend that optimizes model performance and efficiency. We examine how factors such as model size, architecture, and concurrent request volume affect inference energy efficiency. Our findings demonstrate that it is possible to create energy efficiency benchmarks that better reflect practical deployment conditions, providing valuable insights for developers aiming to build more sustainable AI systems. |
| 14:00 | Coffee break |
| 14:15 | Compiling challenges |
| Everyone | |
| Compiling challenges -- everyone in the audience introduces themselves and briefly mentions one key challenge they face related to (sustainability of) using large language models in software engineering. Hot topics -- a selection of 3-4 common interest hot topics is made, and a table is allocated to each topic. Rapid problem solving -- during 2-3 rounds, participants visit a table and propose a solution (or a way forward) for the respective hot topic. |
| 15:45 | Closure of the workshop |
| A short wrap-up session to reflect on the day and outline plans for future collaboration, including the post-workshop position paper. |