The Field Robot Event is more than just a competition—it’s a crucible where academic theory meets the reality of autonomous systems in the field. This year, our 9-member team from Politecnico di Milano took on the challenge of adapting, programming (and soon repairing) our agricultural robot to master four challenging tasks, competing against 14 other teams from across Europe.

The Challenge

Under the guidance of supervisors Simone Mentasti, Mirko Usuelli, and Matteo Matteucci, we embarked on this project as part of our coursework. Most of the team did it within the envelope of the current robots course, while other did it as a dedicated semester project. The Field Robot Event, an annual academic competition organized by participating European universities, presented us with four mandatory tasks:

1. Line navigation in a field - Basic autonomous navigation following crop rows
2. Line navigation + object detection - Navigation with simultaneous object recognition
3. Navigating around trees and mapping fruits - Complex obstacle avoidance with fruit detection and mapping
4. Navigating in a field of mushrooms - Detecting and mapping UV-glowing mushrooms

My responsibility centered on the third task: building a vision and mapping pipeline capable of detecting and mapping five different fruit types (apples, bananas, grapes, lemons, and oranges) in 3D space.

The Technical Deep Dive

Vision System Architecture

The foundation of my approach was YOLO11 nano, which we fine-tuned for fruit detection. This required extensive data collection and labeling—a process that taught me the unglamorous but crucial reality of machine learning applications, in most cases only data help. We paired this with a ZED2i stereo camera, leveraging its depth capabilities to estimate 3D fruit positions.

The theoretical pipeline seemed straightforward:

1. Detect fruits in the camera frame using YOLO11 nano
2. Use depth information to calculate 3D positions in camera coordinates
3. Transform coordinates to global map frame using tf2 and SLAM
4. Map individual fruit instances for competition scoring and return a mapping csv file.

Reality, as usual, had other plans.

Navigation Nightmares

Our initial plan involved integrating Nav2, ROS2’s navigation stack, with SLAM for robust autonomous navigation. What seemed like a standard implementation quickly became a nightmare of compatibility issues. Implementing custom odometry, refactoring control systems, and ensuring everything played nicely with Nav2 proved more challenging than anticipated.

But Nav2 was not the only problem we faced with navigation in task three and four. Because our Lidars where mounted quit low and the competition field was a nice field of gras we picked up to much noise from the gras which rendered SLAM more or less useless. Our solution, one night before rebuilding the Lidar setup and the tf-transformations for navigation (sounds again straightforward, but the robotics engineers among those who read this will understand the struggles which could / and have caused this). Unfortunately this did not fix the problems we had with Nav2 so…. after another sleepless night debugging on the competition venue itself, we made the difficult decision to abandon Nav2 entirely. Instead, we implemented a simplified Model Predictive Control (MPC) approach: sample control parameters, simulate collision scenarios, and select the safest trajectory. It wasn’t elegant, but it worked and got the robot moving—a recurring theme in this project.

While seeing the robot just moving randomly on the field and not crashing into obstacles I felt relieved we pulled off what seemed a couple of hours earlier impossible. It was not an effective policy of individually circling trees to get nice detections but rather bouncing from corner to corner, from tree to tree. At this point I have to thank my teammates Adriano and Lorenzo who improvised the moving nodes on the morning such that we could start right after lunch.

The Tracking Dilemma

Task 3 required mapping individual fruit instances, not just detections. This meant we needed either tracking or clustering algorithms to maintain fruit identity across multiple observations.

I initially chose visual tracking, implementing algorithms like DeepSORT, ByteSORT, and StrongSORT. These tools work excellently in traditional scenarios where the camera is stationary and objects move. However, our scenario was inverted—the entire scene moved as our robot navigated, while fruits remained stationary. This fundamental difference broke the assumptions these algorithms were built on.

With just days before the competition, I pivoted to clustering approaches based on global coordinates. Using Gaussian Mixture Models (GMM) and DBSCAN with hyperparameter optimization through random search to minimize Bayesian Information Criterion (BIC), we achieved acceptable fruit instance mapping. Not perfect, but functional—another lesson in pragmatic engineering. On the competition reality said ‘Hi’ again-this meant something in the pipeline broke again which resulted in collapsed 3D coordinates and not correct mapping.

Lessons from Outside Research

Up to this point in my career I focused on research projects, carefully designing software for usability, reproducibility and innovation. But after seeing this robotics project on the initial list (without knowing what it is going to be like) and the thought of what to do after my master degree I wanted to try something which is more related to engineering and maybe real-world project work. In the end I took a couple of key-lessons learned with me.

Beyond the Comfort Zone

This project put me far outside my research comfort zone. Instead of careful experimentation and methodical development, we operated in permanent hackathon mode. The mantra became: “If it works, move on to the next problem.” Perfectionism was a luxury we couldn’t afford. Reflecting this made me realize that I should stick for the moment maybe closer to research and development and step away from engineering. Although I think engineering has some pleasing moments in it (especially when something works out).

Team Dynamics and Skill Gaps

Working with a team spanning vastly different skill levels and backgrounds proved both challenging and educational. Some members were beginners heavily reliant on AI assistants like ChatGPT, while others brought technical knowledge and genuine curiosity to learn. Bridging these gaps while maintaining project momentum required constant adaptation of communication and task delegation strategies. Additionally it became clear that clear communication, knowledge unification (instead of fragmentation) and a project lead with a clear plan in mind, leading skills and technical oversight of the project is crucial. Especially if you work with a newly put together team and on a thigh time-budget where you do not have time to get back on, not even off track.

The Robustness Reality Check

My biggest technical takeaway lessons is similar to previous projects: simplicity trumps sophistication in real-world applications. Complex solutions might perform better in controlled lab environments, but robustness is infinitely more valuable in the field. A simple system that works consistently (but not as good) beats an elegant one that fails under different conditions. This is definitely something I will lookout for in future projects and design methods more closely to the designated schedule (simple to complex approaches).

During this project we where interacting with ROS2 almost all the time, therefore I’m thankful for the opportunity to to learn from experience working with a real robotic system. Topics like node design, node interactions, modular development in the context of ROS2 where tackled by tutorials and guides but as almost always developing your project raises questions which those tutorials cannot really answer but hands on experience can.

The Competition Experience

Despite the stress, sleep deprivation, and occasional (more towards frequent ;P ) frustration, the competition itself was remarkable. Seeing teams from across Europe converge in Milan, each bringing unique solutions to the same fundamental challenges, like completely different robot design or approaches in Software, created an atmosphere of collaborative competition, where people also where helping out across their team border, that I found genuinely inspiring and motivating.

As a recurrent theme in this Blog-post I can only repeat: there’s something magical about watching machines act autonomously—robots navigating complex environments, making decisions, and accomplishing tasks without human intervention. These moments made all the debugging sessions and architectural rewrites worthwhile.

Mixed Feelings and Future Reflections

I’m leaving this experience with decidedly mixed feelings about robotics as a field. On one hand, the fundamental fascination remains strong. The technical challenges are engaging, the community is welcoming and intellectually inspiring, and the potential applications are genuinely exciting.

On the other hand, I’ve gained a healthy appreciation for the frustrations inherent in robotics: hardware that breaks at inconvenient moments (or almost all the time), mysterious driver issues, embedded systems throwing unprecedented errors, and the general unpredictability of physical systems.

While reflecting this project I’ve also confirmed my preference for building persistent, well-architected software over hackathon-style rapid prototyping. While the latter has its place and can be exhilarating, my natural inclination leans toward creating robust, maintainable systems that outlast their initial use case.

Conclusion

The Field Robot Event was simultaneously one of the most challenging and rewarding experiences of my academic career. It forced me to confront the gap between theoretical knowledge and practical implementation, taught me valuable lessons about team dynamics and project management under pressure, and provided invaluable insights into the realities of robotics development.

Would I do it again? Absolutely. Would I approach it differently? Definitely. But that’s perhaps the most valuable lesson of all—every project, successful or not, teaches us something essential about both technology and ourselves.

For anyone considering similar competitions or robotics projects, my advice is simple: embrace the chaos, prioritize robustness over elegance, and remember that the most valuable learning often happens when things don’t go according to plan.

Special thanks to our supervisors Simone Mentasti, Mirko Usuelli, and professor Matteo Matteucci for their guidance, and to my teammates: Angela Remolina​, Alessio Spineto​, Erdal Yalçın​, Alisa Pesotskaia​, Luis Caballero​, Lorenzo Arcangeli​, Filippo Di Fiore​ (and Paolo Ginefra, who participated in the last years edition of FRE and helped out especially during difficult time) for making this challenging journey memorable despite the occasional 3 AM debugging session.

I have written this blog post jointly with Claude.ai.

Exploring the linguistic patterns hidden in the royal game through machine learning

Introduction

What if chess moves could be treated as words in a sentence? What stories might emerge from the strategic dance between kings and pawns? These were the questions that drove our team of four through an exciting Natural Language Processing project at Politecnico di Milano, under the guidance of Professor Mark Carman.

Our journey began with a simple yet profound hypothesis: chess games, recorded in standard UCI notation, might exhibit linguistic properties similar to natural languages. Armed with a dataset of chess games played by Stockfish against itself, we embarked on an investigation that would take us from Shakespeare’s English to word embeddings, from sequence prediction to building our own chess-playing bot.

The Dataset: Stockfish vs. Stockfish

Our foundation was a rich collection of chess games in UCI (Universal Chess Interface) format, where the world’s strongest chess engine battled against itself. Each game came with valuable metadata including game outcomes and termination reasons. This self-play approach eliminated human bias and inconsistency, giving us a pure distillation of optimal chess strategy translated into move sequences.

The UCI format provided us with a clean, tokenizable representation of chess moves – each move essentially becoming a “word” in our chess “language.” Moves like “e2e4” (pawn to e4) became the vocabulary of our linguistic analysis.

Research Questions: From Language to Strategy

Our investigation centered around several compelling research questions:

1. Chess as Natural Language

Can chess games be meaningfully described as a natural language? To answer this, we conducted comparative analyses between chess move sequences and English text, specifically examining works of Shakespeare. We measured statistical properties like token frequency distributions, sequence lengths, and pattern repetition to quantify the linguistic nature of chess.

2. Information Extraction from Chess Notation

How much strategic information can we extract from move sequences compared to the complete game state? We developed models to predict various chess-specific features directly from notation:

• Piece captures
• Piece movements and positioning
• Check situations
• Game outcomes

Looking for instance at game 20 int the test dataset we get inference for piece prediction and outcome prediction:

We also looked into the game embeddings (last hidden state) and see if the hidden state also learns features which could be also used to predict other labels like the reason for termination (checkmate, stalemate, insufficient material, …)

3. Move Embeddings and Clustering

We applied Word2Vec to chess moves, creating dense vector representations that capture strategic relationships. Clustering these embeddings revealed fascinating patterns – moves that often appear in similar contexts (like opening sequences or endgame techniques) naturally grouped together, suggesting that our “chess vocabulary” had learned meaningful strategic concepts.

The Chess Bot: Next-Token Prediction Meets Strategy

Perhaps our most entertaining application was a chess bot built on next-token prediction using a small-scale LSTM. This bot approached chess not as a traditional search problem, but as a language modeling task – predicting the most likely next “word” (move) given the conversation (game) so far.

To make the bot’s decision-making transparent, we implemented several interpretability features:

• Move Certainty: How confident the model is about its chosen move
• Move Entropy: The uncertainty across all possible moves, indicating position complexity
• Win Probability: Real-time assessment of winning chances from the current position

Watching this bot play was like observing someone who had learned chess by reading 70.000 games without ever seeing a board – sometimes brilliant, sometimes baffling, but always following patterns it had learned from the data. By playing with the bot we found the bot is lacking robustness in a sense that if you play not optimal (and move the move sequence out of distribution) then the bot is also not playing optimal because the bot has not seen a move sequence like this before and does not know what to predict in those cases.

My Deep Dive: Sequence Prediction Architectures

My primary contribution focused on training recurrent neural networks for multiple prediction tasks simultaneously. The goal was ambitious: build models that could predict next moves, piece movements, captures, checks, and even game outcomes from incomplete games.

The Architecture Journey

I began with what I thought would be a sophisticated approach, experimenting with various architectures:

• LSTM: The reliable workhorse of sequence modeling
• GRU: A more efficient alternative to LSTM
• MiniGRU: Inspired by the “Were RNNs All We Needed?” paper, though frustratingly unstable during training
• Transformer: The state-of-the-art attention-based architecture

The Surprising Lesson: Simplicity Wins

Here’s where the project taught me a humbling lesson about machine learning. Despite my initial inclination toward complexity, the simplest LSTM architecture consistently outperformed more sophisticated models. Scaling up and adding complexity didn’t just fail to improve performance – it actively hurt accuracy.

This counterintuitive result challenged my assumptions about model capacity and task complexity. In retrospect, chess notation might be structured enough that simple recurrent architectures can capture the essential patterns without requiring the representational power of more complex models. The lesson was clear: sometimes the most elegant solution is also the simplest.

The training difficulties with more complex architectures were particularly puzzling. Sequence models that should have benefited from additional capacity became harder to optimize and less accurate. This experience reinforced the importance of starting simple and scaling complexity only when justified by performance gains.

Team Dynamics: A Rare University Experience

Working on this project was refreshingly different from typical university group work. All four team members were genuinely committed to producing quality results and meeting deadlines. The collaborative energy was infectious – everyone brought unique perspectives and skills that enriched the final outcome.

Despite our collective enthusiasm, the project’s scope was perhaps overly ambitious for our time constraints. We touched on many fascinating aspects – from linguistic analysis to deep learning architectures to interpretable AI – but the tight schedule prevented us from diving as deep as we would have liked into any single area.

What We Wished We Could Explore Further

Given unlimited time, several directions beckoned:

Generalization Capabilities: How well do fine-tuned language models trained on chess notation generalize to different playing styles or time controls? Could a model trained on rapid games understand classical chess?

Dedicated Small-Scale Predictors: Rather than trying to solve everything with one model, what could we achieve with specialized architectures designed for specific chess tasks?

Transfer Learning: Could chess-trained language models transfer knowledge to other strategic games or even back to natural language tasks?

Temporal Dynamics: How do model predictions change as games progress from opening through middlegame to endgame?

Account for Uncertainty: For game prediction of next token, game outcome, … how could we effectively make use of Bayesian-Neural-Networks to account for uncertainty for a time series prediction?

Conclusions and Reflections

This project demonstrated that chess notation does indeed exhibit linguistic properties that can be leveraged by NLP techniques. The success of our Word2Vec embeddings and sequence prediction models suggests that chess moves follow distributional patterns similar to words in natural language.

The interpretability features we built into our chess bot point toward a future where AI decision-making in strategic domains becomes more transparent and explainable. Understanding not just what move an AI chooses, but how certain it is and what alternatives it considered, could transform how we interact with AI systems.

Most importantly, this project reminded me that in machine learning, as in chess, sometimes the most straightforward approach is the strongest. The elegance of a simple, well-executed strategy often trumps the complexity of an over-engineered solution.

This project was completed as part of the Natural Language Processing course at Politecnico di Milano under Professor Mark Carman. Special thanks to my three teammates who made this exploration both productive and enjoyable.

Technical Details

For those interested in replicating or extending this work:

• Dataset: Stockfish self-play games in UCI format found on hugging-face
• Primary Architecture: Single-layer LSTM with embedding layers
• Evaluation Metrics: Prediction accuracy for various chess-specific tasks
• Implementation: Python with PyTorch for deep learning components

The code and detailed results are available GitHub. The notebooks with waaaaaaay more (but not less interesting) material and findings can be found here

Acknowledgements

At the end of this report I would like to thank my teammates, Martina Missana, Paolo Ginefra and Ferdinando Onori, for the great experience during this project. Despite the huge workload we all experienced in this short amount of time we managed to deliver, at least from my point of view, a nice exploratorive study on this dataset.

Please don’t be mad at me when I only covered a veeeery small portion of the work we have done in this blog post.

I have written this blog post jointly with Claude.ai and in the retrospective as I setup my webpage in mid-early 2025.

During my time as a student assistant at the Institute of Medical Bioinformatics and System Biology (IBSM) at the University Clinic Freiburg, I had the opportunity to work on a project with real-world impact in the field of cancer research. Our team was tasked with developing a comprehensive database solution for the Molecular Tumor Board, which would store and organize crucial data for personalized cancer treatments.

The Challenge

Cancer treatment is increasingly becoming personalized based on the molecular characteristics of each patient’s tumor. Molecular Tumor Boards bring together research from medicine, datascience, biology and also sometimes physics to discuss complex cases and determine the best treatment approaches based on genetic and other molecular data.

Our challenge was to create a database system that could:

• Store complex molecular and clinical data
• Enable efficient querying across multiple parameters for each patient
• Maintain strict medical data privacy standards
• Provide an intuitive interface for medical professionals

From MongoDB to SQL: A Development Journey

Our initial approach was to build the system using MongoDB, a NoSQL database that offers flexibility for handling diverse and complex data structures similar to the document structure you find in medical reports. We spent several months developing a prototype with this technology, but as the project evolved, we encountered some limitations:

1. The need for complex relational queries became increasingly apparent.
2. Data consistency requirements were more stringent than initially expected.
3. The data was also able to be captured in a rigid structure by making some cutoff to flexibility.
4. The team had more expertise in SQL-based systems.

After careful consideration and evaluation, we made the strategic decision to pivot to a SQL-based approach using MariaDB. This decision wasn’t taken lightly, as it meant rebuilding significant portions of our work. However, it proved to be the right choice for the project’s long-term success.

Building the Web Interface

With the database foundation solidified, we took off for developing a web interface that would make the system accessible to the medical team. We chose Flask as our web framework, paired with SQLAlchemy as an ORM (Object-Relational Mapping) layer, as Django seemed a bit to much of an overkill at this time.

The interface development focused on:

• Creating intuitive forms for data entry.
• Designing comprehensive but understandable API interfaces to create automatic reports as convenient as possible.
• Enabling efficient search and filtering capabilities.
• Writing tested software to pinpoint errors in future modifications as best as possible.
• Easy rollback capabilities to remove patients data at their will to be conform with data-privacy regulations.
• Inherit portability with Docker container orchestration.
• Deployment of state of the art security frameworks for API user management.

Impact and Outcomes

By the end of my involvement with the project, we had successfully:

• Deployed a functional database system which was able to manage complex.
• Established protocols for data backup and security.
• Ready to deploy database and REST-API for research purposes.
• Built a system architecture that could scale with growing data needs.

Wet-Lab Stay

As a very nice add-on professor Börries granted me a 6 weeks to the groups wet labs to see the pipeline of data I was handling in the database from the very beginning. Meaning I was able to see steps from collecting of tissue samples from the clinic, preparing them and sending them to the DKFZ. For me this was very important to understand some (off course not everything ;)) of the biology behind and what are the core characteristics of the data to take care of.

Reflections

This was my first big programming project and I absolutely loved it. It taught me valuable lessons about database design decisions and the importance of understanding user requirements thoroughly before committing to a technical approach. The transition from MongoDB to SQL highlighted how different database paradigms serve different needs, and how choosing the right tool for the specific context is crucial. The experience of building both NoSQL and SQL implementations of the same system gave me a much deeper understanding of database technologies than any classroom exercise could have provided. Technical perspectives aside it also thought me how valuable and effective a well working team actually is, as I enjoyed my time working and talking with the lab. The database we built was build as an academic research project if handling highly individual molecular data was even feasible but by the end it has the potential to be a tool that could help physicians make better treatment decisions for cancer patients.

By the end of the project I left the project and sadly also the group because I have seen how much potential data-driven application and methods actually have and wanted to learn more about them on a more theoretical level. Still at I fell in love with the interaction of computer-science and medicine / biology. Especially the wet-lab stay was an extremely nice and eye-opening experience which I would do again any time. the

I have written this blog post jointly with Claude.ai and in the retrospective as I setup my webpage in mid-early 2025.