A single eval run on swebench verified mini can cost $1300+ for agents based on SOTA LLMs. If you are compute-poor, it becomes impossible to maintain an independent swebench verified mini leaderboard under these conditions. But, what if there is a technique to help reduce the
It's a classic move. You hire consultant so they can be held accountable of the decision to layoff. It's a political move, and since they are not internal, they can be more ruthless.
This is probably the best DSPy course on the internet. And I'm frankly not biased at all.
Learn to program, not prompt language models and improve your quality of life in the process 😂
This is exactly what the DeepSeek paper shows. I expect a lot of gains from RL in the future. The recipe will become pretraining=>RL=>synthetic data from RL modified model=>finetuning base pretrained model based on synthetic data=>fineturing=>final RL
For the most advanced use cases of DSPy, I highly recommend this module. It covers everything from building knowledge graphs to implementing RAG, as well as integrating DSPy with FastAPI to develop user-facing LLM-based applications.
My contribution has been accepted and synthetic data generation has been added to DSPy. 🔥
You can either generate data by defining a pydantic model or by feeding some initial examples. And it is blazingly fast !
Here is how to use it:
I am going even further. Now, with a bit of meta-programming, I can dynamically create signatures on the fly and generate synthetic datasets based on pydantic models. Crazy nice to see it in action 🔥
I propose to add the synthetic_data_generation function in the utils of DSPy.
