Julien Valentin

Comparing models.. Finding that a small 1B #llm hallucinates less than a 4B for semantic to structural matching. Small llms for targeted tasks for the win. The new Meta llama3.2 1B is better than expected and cheap on token cost.

3

Why "On-Device" is the most ethical AI strategy. We talk a lot about the size of models. Parameters. Trillions of tokens. The race for AGI. But in my academic life at EPFL and my work in Speech, I’ve always been obsessed with the opposite end of the spectrum: Accessibility. When we published our research in Nature Communications on AI for social projects, one conclusion was clear: If AI requires a massive data center and a high-speed fiber connection to run, it excludes half the world. Cloud models are undeniably impressive and they are pushing the boundaries of what is possible. But they shouldn't be the only way we deploy intelligence. I'm seeing a resurgence of interest in on-device solutions, not to replace the cloud, but to handle the sensitive, immediate, and personal tasks where the edge shines: - Privacy: Processing data right where it’s created. This is crucial for multimodal data (voice, video) that is highly identifiable and shouldn't leave the user's hand. - Latency: Real-time processing that feels like a conversation, not a transaction. - Energy Efficiency: Running a small, optimized model locally is often far more sustainable than round-tripping every query to a power-hungry data center. - Reliability: True accessibility means working offline. AI should be helpful even when the signal drops or the fiber hasn't been laid yet. Better locally-run alternatives make these powerful tools accessible to everyone, even for problems and data that are highly sensitive. I’m planning to dive deeper into this myself by experimenting with some of the latest open-weights models specifically for local deployment. If you’ve been testing small language models (SLMs) or edge-optimized architectures recently, I’d love to hear about your experience. What frameworks or models are you finding most promising for mobile/edge/small server hardware right now? #AIforGood #OnDeviceML #NLP #EthicalAI #TechForGood #EdgeAI #SustainableTech #OpenWeights

7

6 Comments

My co-founder put together an excellent summary of the latest scientific findings around agents.md files in agentic coding. If you're building or thinking about AI agents, this is well worth your time. Give it a read 🐿️

10

1 Comment

🚀 The authors from Roche and EPFL claim their Minerva system can handle 96-reaction batches while existing tools like EDBO max out around 16 reactions leveraging Bayesian Optimization. They claim it is largely due to the difference in acquisition function choices. 🧠 The paper suggests EDBO's algorithm crashes when you try to scale up because it uses an acquisition function, q-EHVI, that has exponential computational complexity. Minerva switches to different algorithms (q-NEHVI, q-NParEgo) that supposedly scale much better. But I'm curious about the real-world impact? 🧪 I understand EDBO has its limitations but has a switch to these acquisition functions solved these problems? I would think there is also a need to scale the compute power of the server running these jobs. 🧠 The literature also claims that one should use DFT descriptors versus one-hot encoding to represent categorical variables. Now in theory this would build dense vectors typically better for all ML models versus OHE's sparsity issues but one should consider the trade-offs. The time and power to compute certain levels of theory for physics-based descriptors can be quite long and the uplift might be negligible versus risking a debatably less accurate objective function due to OHE. Abigail Doyle Would love to get the community's thoughts! Relevant link in the comments Chemetrian #Chemistry #MachineLearning #AI #DrugDiscovery #ProcessOptimziation #SmallMoleculeDiscovery

6

1 Comment

FunctionGemma, a new 270M parameter language model from Google, designed specifically for function calling and tool execution — not general chat.   Why is FunctionGemma special? 🔹 Extremely lightweight FunctionGemma can run in ~550MB RAM, making it possible to deploy on laptops, edge devices, and even phones — all without large GPUs. 🔹 Purpose-built for function calling Unlike general LLMs, this model is trained for text-only tool calling, making it ideal for automation, agents, and API-driven workflows. 🔹 Easy local deployment The model is released in GGUF format, so it works seamlessly with llama.cpp and Ollama using quantized versions (Q4/Q5/Q8). 🔹 Fine-tuning ready from day one Thanks to Unsloth’s day-zero support, you can: Run FunctionGemma on CPU or GPU Fine-tune it locally using Colab notebooks Choose full fine-tuning or LoRA   What can you fine-tune it for? FunctionGemma is meant to be customized for specific tasks, such as: Multi-turn function-calling agents Mobile actions (calendar events, reminders, flashlight, timers) Task-specific automation and workflows Lightweight, offline AI assistants This model clearly shows a shift toward small, focused, and efficient AI — where models are trained to do one thing well and can be adapted quickly to real-world use cases.   Model (GGUF): https://lnkd.in/gxVuBhZA Docs & fine-tuning guides: https://lnkd.in/gchpAjUF Run and Deploy LLMs on your iOS or AndroidPhone: https://lnkd.in/grk72Bey

5

🚀 New Publication Alert 📄 From Text to Data: Open-Source Large Language Models in Extracting Cancer-Related Medical Attributes from German Pathology Reports I’m excited to share our latest study, now published open access: 👉 https://lnkd.in/eYye9pnu In this work, we explored how open-source large language models (LLMs) can extract structured oncological information from unstructured German pathology reports — fully locally and in compliance with strict data privacy regulations. 🧠 We evaluated models like Llama 3.3 70B and Mistral Small 24B, combining retrieval-augmented generation (RAG) with tailored prompt engineering. Our results show that even smaller models can reach high accuracy (F1 > 0.90) with the right strategy — making real-world clinical use both feasible and resource-efficient. 🏥 The methodology is about to be implemented in our local clinical cancer registry at the #UKE to support automated tumor documentation and improve data quality. 🛠️ The full code and prompt templates are available open source: 👉 https://lnkd.in/e6qppChq This is a step toward practical, privacy-aware AI integration in oncology — built on reproducible tools and focused clinical needs. #MedicalAI #RAG #CancerRegistry #UCCHamburg #LLMs #OncologyInformatics #ClinicalNLP #DigitalHealth #OpenSourceAI #HealthData #Pathology #AIinHealthcare

13

3 Comments

Yann LeCun on architectures that could lead to AGI LLMs can take us only so far. "If you are interested in human-level AI, don’t work on LLM Abandon generative models in favor joint-embedding architectures Abandon probabilistic model in favor of energy-based models Abandon contrastive methods in favor of regularized methods Abandon Reinforcement Learning in favor of model-predictive control Use RL only when planning doesn’t yield the predicted outcome, to adjust the world model or the critic."

1

4 Comments

🚀 Reranking that's smarter, not just stronger Our paper "E2Rank: Efficient and Effective Layer-wise Reranking" was presented at ECIR 2025 — the result of a collaboration between Sapienza and Pinecone with Cesare Campagnano, Jack Pertschuk, and myself. 🔍 What is E2Rank? It's a cross-encoder-based reranker designed to adapt its computational budget layer by layer — starting light and only getting deeper for the most promising candidates. That means: 🔄 Fewer FLOPs, smarter compute reuse 🎯 SOTA effectiveness across TREC and BEIR ⚡️ Lower latency, even with large models like DeBERTa v3 📊 Instead of one-size-fits-all reranking, E2Rank introduces a multi-step layerwise strategy: progressively refine the candidate set using deeper transformer layers at each step — reusing hidden states, saving compute, and keeping latency low. 🏅 Bonus: it's on the Pareto frontier for efficiency vs. effectiveness. #InformationRetrieval #NeuralSearch #ML #RAG #Retrieval #ECIR2025 #Pinecone

28

1 Comment

OpenAI's paper on Why Language Models Hallucinate is pretty interesting. Paper here: https://lnkd.in/eSb6itBJ The TLDR is that during fine tuning the reward function, similar to multiple choice exams values any guess, right or wrong, higher than "I don't know". Maybe hallucinations are not intrinsic to the architecture of LLM's. This seems obvious in retrospect and I imagine it will result in better models for a lot of use cases.

1

𝗦𝗜𝗡𝗤 🇨🇭 𝗔𝗽𝗲𝗿𝘁𝘂𝘀: Open Source AI, Made in Switzerland. We’re proud to announce a 𝗻𝗲𝘄 𝗾𝘂𝗮𝗻𝘁𝗶𝘇𝗲𝗱 𝘃𝗲𝗿𝘀𝗶𝗼𝗻 𝗼𝗳 𝘁𝗵𝗲 𝗔𝗽𝗲𝗿𝘁𝘂𝘀-𝟴𝗕 model using our novel 𝗦𝗜𝗡𝗤 method! This release combines Apertus, an open-source large language model born from EPFL, Eidgenössische Technische Hochschule Zürich, and the Swiss National Supercomputing Centre (CSCS), with 𝗦𝗜𝗡𝗤, our Zurich-born quantization method released under the Apache 2.0 license. Thanks to the integration with fast kernels, you can now experience 𝗵𝗶𝗴𝗵-𝘀𝗽𝗲𝗲𝗱 𝗶𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 using our 𝗵𝗶𝗴𝗵-𝗾𝘂𝗮𝗹𝗶𝘁𝘆 𝟰-𝗯𝗶𝘁 𝗺𝗼𝗱𝗲𝗹𝘀. I am personally excited to continue contributing to the Swiss open-source AI ecosystem and this is just the beginning. New quantized versions, including for the 70B model, are coming soon! 👉 New SINQ-quantized Apertus model: https://lnkd.in/d5hvFVDC 👉 SINQ GitHub Repo: https://lnkd.in/dctt5jK2 ⭐ Don’t forget to star our repo and join the journey toward fast, open, and collaborative AI made in Switzerland.

104

3 Comments

Are you adding AGENTS.md or CLAUDE.md to all your repositories because "everyone says so"? You might want to hold that thought. 🛑 A new paper from researchers at ETH Zurich and LogicStar.ai just dropped the first rigorous evaluation of repository-level context files, and the results are... counter-intuitive. While industry leaders are pushing these files to help AI agents navigate code, this study reveals that "more context" doesn't always mean "better performance". Using a novel benchmark called AGENTBENCH, they tested top-tier agents like Claude Code, Codex, and Qwen Code across real-world GitHub issues. The TLDR: Surprisingly, LLM-generated context files actually reduced task success rates in most settings while spiking inference costs by over 20%. While these files successfully prompt agents to follow specific instructions and test more thoroughly, they often make the core task harder by introducing unnecessary requirements and redundant information. The data suggests that for now, we should skip the automated /init commands and stick to minimal, human-written requirements. Is your "context engineering" actually hurting your agent's productivity? Read the full paper to see the detailed breakdown of the AGENTBENCH results and why our current recommendations might be leading agents astray. 🔗 https://lnkd.in/eDf-XWnY #AI #SoftwareEngineering #LLMs #CodingAgents #ContextEngineering #Research #Github

5

+1000 points for the Swiss!! "Switzerland launched Apertus, a national LLM developed by EPFL, ETH Zurich, and the Swiss National Supercomputing Centre as an alternative to ChatGPT, Meta’s Llama models, and DeepSeek. The model comes in two sizes — 8 billion and 70 billion parameters — and was trained on 15 trillion tokens across more than 1,000 languages. 40 percent of Apertus’s training data is non-English, including underrepresented languages like Swiss German and Romansh. Unlike commercial models, Apertus’s (from the Latin for “open”) architecture, model weights, training data, and development recipes are all openly accessible and fully documented, ensuring compliance with Swiss data protection laws and EU AI Act transparency requirements. The models are freely available under a permissive open source license for educational, research, and commercial applications." https://lnkd.in/e74P8pvZ

15

2 Comments

Introducing Generative Adversarial Distillation (GAD): a novel GAN-style formulation and framework that facilitates both on-policy and black-box distillation of large language models (LLMs). GAD is the first technique to enable block-box on-policy distillation from proprietary teachers where internal logits or parameters are inaccessible, or distillation between teacher and student LLMs with incompatible vocabularies. GAD expands our prior work on white-box on-policy distillation (i.e., MiniLLM), pioneering block-box on-policy distillation for LLM training. Specifically, GAD frames the student LLM as a generator and trains a discriminator to distinguish its responses from the teacher LLM’s, creating a minimax game. The discriminator acts as an on-policy reward model that co-evolves with the student, providing stable, adaptive feedback. Experimental results show that GAD consistently surpasses the commonly used sequence-level knowledge distillation.  In particular, Qwen2.5-14B-Instruct (student) trained with GAD becomes comparable to its teacher, GPT-5-Chat, on the LMSYS-Chat automatic evaluation. The results establish GAD as a promising and effective paradigm for black-box LLM distillation. Our team has been conducting fundamental research in knowledge distillation with wide adoptions across the industry. -         MiniLM: We introduced multi-head attention distillation, establishing the most effective distillation method for BERT-style models. The open-source MiniLM models (e.g., 6x384) have become the most widely utilized small encoder models on the Hugging Face. -         MiniLLM: Our proposed Reverse KLD is recognized as one of the most effective, de facto on-policy distillation approaches for modern LLM training, which has been widely used by Thinking Machines, Gemma, and many other teams and models. -         BitDistill: We proposed BitNet Distillation to finetune off-the-shelf full-precision LLMs (e.g., Qwen) into 1.58-bit precision (ternary weights {-1, 0, 1}), achieving performance parity with the full-precision counterparts on specific downstream tasks. -         GAD: The development of Generative Adversarial Distillation (GAD) now allows for black-box on-policy distillation, overcoming two major prior limitations: (1) Distillation from proprietary teachers where internal logits or parameters are inaccessible; (2) Distillation between teacher and student LLMs with incompatible vocabularies. https://lnkd.in/gMaP2c7w

125

2 Comments

What if training powerful AI models didn’t require centralizing data, compute, or control? 👇 That’s the core question behind the SPRIND - Bundesagentur für Sprunginnovationen Composite Learning Challenge, and we’re excited to share that CambridgeFlower (team University of Cambridge & Flower Labs) has been selected to move into Stage 2! Today’s AI training pipelines assume, centralized data, homogeneous hardware, and always-on connectivity. Composite Learning challenges this by enabling decentralized, federated training across heterogeneous and real-world systems; from edge devices to data centers, across organizations. Over the next 12 months, with SPRIND - Bundesagentur für Sprunginnovationen’s support, we’ll be focusing on Federated & decentralized learning at scale, and robust training under heterogeneity and intermittency. 📣 Huge congratulations to the other Stage 2 teams exalsius (Alexander Acker, Soeren Becker), SymphonyLearn (PanocularAI, Arya Mazaheri), AETHER (Tilmann Bartsch, SEMRON), and Planetary Compute Platform (Phil Rohr, DeltaWave). And a big thank you to SPRIND - Bundesagentur für Sprunginnovationen and the jury -- Johannes Otterbach, Katharine Jarmul, Wolfgang Stille, Hans Ramsl, Gerhard Wunder, Andreas Unseld and Tian Li -- for supporting ambitious, systems-level AI research. 👉 If you’re building or thinking about federated, decentralized, or edge AI systems; follow along or reach out.

45

Great discussion in our lab today, thanks to Alhusain Abdalla , on Recursive Language Models (RLMs) and how they challenge our assumptions about long-context reasoning. A common belief in the LLM community is that long-horizon tasks are primarily a context window problem: if models could simply “see more,” they would reason better. In practice, we increasingly observe the opposite—context rot—where performance degrades as information density grows, even in frontier models. 🧠 RLMs propose a different framing Rather than forcing models to ingest ever-larger contexts, RLMs treat context as something to interact with programmatically. Conceptually, this represents an evolution: • RAG is efficient but inherently lossy—if retrieval misses the right evidence, the model fails. • Agentic loops improve flexibility, but remain linear and constrained by short-term memory. • Recursive Language Models take a more radical step: the model writes code, recursively decomposes the data, spawns sub-calls, and aggregates results—without ever needing to load the full corpus into its own context window. 🔑 The key insight is that RLMs decouple reasoning capability from context length. The root model never sees the entire dataset—only structured outputs from recursive sub-tasks—enabling scalable reasoning over effectively unbounded data while keeping inference focused and efficient. We’re currently experimenting with their code on several clinical trial–related use cases, and we’re excited to share more results soon 🔥 This feels like an important step toward reasoning systems that scale not by reading more, but by thinking more structurally.

101

4 Comments

 Beyond Transformers: What’s Next for LLMs in the Deep Learning Era? In just a few years, large language models (LLMs) have gone from research curiosities to powering everyday tools — from coding assistants and marketing copywriters to legal advisors and medical copilots. Under the hood, deep learning has driven this revolution — particularly the Transformer architecture. But what if we’ve only scratched the surface? The Transformer Era: Brilliant but Limited? Transformers brought us the superpowers of: Parallel training over large datasets, Attention mechanisms that let models weigh context efficiently, And scaling laws showing that “bigger is better” — at least until recently. Yet, we’re also seeing diminishing returns: GPT-4’s leap over GPT-3 is real, but not revolutionary. Training and inference costs are skyrocketing. And many LLMs still hallucinate, lack reasoning, and struggle with real-time constraints. It’s time to ask: What comes after Transformers? Neurosymbolic LLMs: Logic Meets Language A new wave is brewing — hybrid architectures that blend deep learning with symbolic reasoning. Imagine LLMs that: Not only generate fluent text, but also verify facts logically, Can explain their decisions, not just output tokens And act like reasoning agents rather than black boxes. Think OpenCog, DeepMind’s Gato, or Meta’s Toolformer as early signals. Memory-Augmented LLMs: From Static to Stateful The next generation of LLMs may be stateful and evolving, not just stateless text generators. New research shows promise in: External memory modules (like RETRO, RAG, or MemGPT), Long-context reasoning across multiple sessions, And lifelong learning — updating models without retraining from scratch. This makes LLMs more like digital companions than just chatbots. LLMs as Cognitive Engines, Not Just Models Here’s a new perspective: LLMs are not just NLP models anymore. They’re cognitive frameworks — bridging perception, memory, reasoning, and action. What if we start designing LLMs as: Multimodal agents (text, image, speech, video), World simulators (for gaming, robotics, strategy), Or even scientific assistants that explore hypotheses and automate literature review? Call to Builders: Don’t Just Use LLMs — Reinvent Them As engineers, researchers, and entrepreneurs, we’re entering a rare moment: A new “post-transformer” paradigm is forming. Ask yourself: What if we mix symbolic AI with LLMs? What if models verify instead of just generate? What if context is not just tokens, but structured memories? Final Thought LLMs have brought us far, but the next leap won’t come from just adding more parameters. It’ll come from rethinking the architecture, memory, reasoning, and embodiment. Let’s not wait for someone else to define what’s next. Let’s build it. #DeepLearning #LLMs #AIResearch #SymbolicAl #NeuroSymbolic #CognitiveAl #MLEngineering #FutureOfAl #LinkedInAl

6

After years of building and refining #MyFoodRepo 📱 for our own nutritional studies at EPFL's Digital Epidemiology Lab, we're making it available to researchers everywhere. If you've ever run a study with dietary data, you know the pain: food frequency questionnaires that rely on faulty memory, manual food diaries that cause dropout, and research-grade apps that work in a pilot but frustrate participants at scale. Commercial apps might seem tempting, but they're often privacy-invasive, and you never really know if you'll get the data you need, or under what conditions. We built something different. Participants snap a photo, scan a barcode, or type what they ate in plain language. Cutting-edge AI, i.e. the same vision and language models powering today's leading AI applications, identifies foods, estimates portions, and maps to nutritional databases in real time. No endless dropdowns. No clunky search interfaces. And you get full access to all the raw, timestamped data. Privacy is built into the architecture, not bolted on as an afterthought. We collect zero personal information from participants, no names, no emails, no phone numbers, no location data. We don't know who your participants are, because we neither want to nor need to. Over a dozen research groups across multiple countries already use MyFoodRepo, from clinical trials at university hospitals to large-scale digital cohorts with over 1,000 participants. The platform has generated hundreds of thousands of annotated meal entries and contributed to peer-reviewed publications. It works because we made it easy enough that people actually stick with it. Our own studies depend on it. Now you can use it too. You get your own isolated cohort, real-time data monitoring, automatic nutritional mapping to official databases or your own custom food lists, and full data export via API or CSV. For academic research, it's typically free through our collaboration model. If you're planning a clinical trial, digital cohort, or any study involving dietary data, this might be what you've been looking for. 👉 https://lnkd.in/eraiPNf3

136

9 Comments

New paper from Yann LeCun. It's a great read as always and it comes out at the right time when I think thousands [of software engineers] are really getting into their coding agents and discovering glimpses of their static nature (forget to save a session?). I think part of the art of writing a high-impact paper is generating a reaction. I can't help but feel like the threat of a biologically inspired engineered optimization algorithm is more for the dramatic story telling. https://lnkd.in/eZAuqWkN

5

AI built for society, not for profit. 💡 ETH Zurich and EPFL are about to make a bold move. They are launching a language model without chasing profit. Built for the public, researchers, and education. It’s designed with transparency, data privacy, and long-term public interest in mind. We’re used to seeing Big Tech race ahead with closed systems. Now we have something in Europe which is more open. Some interesting facts: It’s hosted on Swiss infrastructure for data sovereignty. 🇨🇭 It’s trained on publicly available, licensed data, not user content. ETH wants to invite universities, authorities, and civil society to co-develop and explore it together. They want to understand these models instead of just using them. I love that. More of this mindset, please. What do you think? Do we have a chance to lead in ethical, public-focused AI? -J Content: Jonas and AI

5