SafeSieve is an economical, simple, and robust multi-agent communication framework, which can seamlessly integrate into mainstream multi-agent systems and prunes redundant or even malicious communication messages.
We provide the code of our paper. The algorithm implementation code is in SafeSieve folder, and the experimental code is in experiments folder.
conda create -n safesieve python=3.10
conda activate safesieve
pip install -r requirements.txt# Update in SafeSieve/llm/gpt_chat.py
API_URL = "" # the BASE_URL of OpenAI LLM backend
API_KEY = "" # for OpenAI LLM backendDownload MMLU, HumanEval and SVAMP datasets from MMLU, HumanEval and SVAMP. And put them in different folders.
python experiments/run_mmlu.py --use_llm_similarity --optimized_spatial The above code verifies the experimental results of the mmlu dataset.
If you want to try on heterogeneous setting, please add --heterogeneous_llms.
python experiments/run_mmlu.py --use_llm_similarity --optimized_spatial --heterogeneous_llmsWe also provide experimental code for other datasets and topologies.You can refer to experiments/run_humaneval.py and experiments/run_svamp.py.
This code refers to GPTSwarm and AgentPrune.
