Well, now you can do just that! I’m offering MPS Office Hours, a regular group call where you can demonstrate your problem and get hints from myself or other present experts.

And if you don’t have a problem yet but just want to watch and learn from others, you can, too!

Best of all, it’s free!

I wasn’t expecting to participate in the Office Hours today. My plan was to just watch and learn. I’m used to the try-it-and-see and RTFM before asking for help. Still, I’m glad that I did participate, got good feedback and pointers. It’s a valuable resource for the community.

– Nuba Princigalli

Interested?

Here are the important details:

• The call takes place every Monday and Thursday from 14:30 until 15:00 CE(S)T (GMT+1 or +2, dependent on daylight savings).
• We meet on Google Meet. To avoid zoombombing, the link is not shared publicly but will be posted to the #office-hours channel in the offical MPS Slack.
• Discussion related to MPS Office Hours also takes place in the #office-hours channel.
• The office hours are not recorded currently but this may change in the future.
• Attendance is free and the meeting is public. If you want a private consultation, contact me.

To help reduce the risk of your team getting stuck on a difficult issue, I offer MPS Advisory Service where I answer your technical questions related to MPS with a response time of one business day.

Examples of topics that are covered

• How to implement a particular editing feature
• Understanding an error message and suggestions on how to best fix the underlying cause
• How to automate the build process
• How to best integrate MPS with a particular system or environment

What is not covered

• Developing production code – but I might create a small example to illustrate a point or a technique, if necessary.
• Maintaining or debugging existing code – but I could give you tips to help you do it.

How it works

The service is offered to you as a single person (a team lead, an architect, a CTO). You ask questions by email, I reply to my best ability within one business day, but usually faster. If necessary and convenient, we may jump on a videocall to understand the problem better, but the service is mostly asynchronous.

Terms

• The service costs 3 000 EUR/month (30 days), paid in full in advance.
• In case of vacations/holidays the duration of the service is extended.
• To ensure a high quality of service I only take 2 clients in a given month.

Interested? Contact me at sergej@specificlanguages.com.

Here is an example Groovy script that returns the names of all Java classes in a project:

def conceptName = 'jetbrains.mps.baseLanguage.ClassConcept'

return project.read {
  def concept = project.concept(conceptName)
  def names = []
  mops.search.eachInstanceOf(concept, project.scope) { node ->
    names << node.properties['name']
  }
  return names
}

Note project.read method for executing code in a read action, project.concept for looking up concepts by name, as well as mops.search.eachInstanceOf for executing an instance search.

And here is how you execute it, say, against MPS-extensions:

mops --project-root=code --mps-home='.../mps/2025.1.3' --java-home='.../jbr' code run .../mops/examples/concept-names.groovy

Since it is tedious to figure out the right --mps-home and --java-home parameters, I have also implemented a mops wrapper subcommand which can guess their values for common Gradle builds and write a wrapper script for launching mops in the context of the project.

I think the Groovy scripting interface is going to become more important in the future. For agents, Groovy is a good fit because they are already quite adept at writing it, and editing models from Groovy is likely to be easier than alternatives. For humans, it could provide an easier-to-run alternative to MPS Console scripts and a convenient way to export data. Groovy also makes it easy to add language-specific functionality and helps keep the script readable (as illustrated by the direct access to the name property above).

I am still iterating on both the functionality and the best way to expose it. I have added Java parsing (under mops.parsing.java) and build script reloading (mops.editing.build.reloadModulesFromDisk). I want it to be easy to add further extensions, such as parsing other languages (KernelF, or even structure and editor languages), or executing an intention.

These changes were motivated by trying to get mops to a state where it could autonomously reproduce an issue reported against MPS-extensions and write an automated test for it. I am of course discovering bugs along the way and fixing them. (Did you know that headless MPS will save your models only when explicitly asked, and will not save the project’s list of modules even if explicitly asked? Well, I know now.) I am reaching the point where the tools are in place and my focus will have to shift to helping agents use them, which means I am going to turn to skill writing at some point in the near future.

I must admit, I am not entirely happy with the code quality of mops because I am letting agents (Codex/Astra these days) write most of it. It does have tests, but some are of questionable utility. The code is not modularized the way I would like, and despite having prompted most of the code into existence, I am not that familiar with it. I am however deliberately trading off code quality for exploration at the moment (also known as taking on technical and cognitive debt). After we reach AGI the state where mops can write editor tests on its own, I plan to pause and clean things up.

I will be talking about mops at LangDev (confirmed) and at the MPS Community Meetup (pending acceptance). Let me know if you are going to attend one of these events and want to meet for a chat.

As I wrote previously, I was unable to get the agent to look for duplicates without tooling. However, I must admit I didn’t try too hard either. I probably could have altered the prompt or disabled all the tools but the problem is, I wouldn’t be able to trust the reply too much. What if the model overlooks a clear duplicate? What if it hallucinates one that is not there? Well, this last problem can be solved by asking the model for a proof and checking it by a deterministic script. But as soon as deterministic scripts enter the picture, why not do more work in such a script and only use the AI for the parts that cannot be automated?

Thinking about the problem some more, I decided to ask the model to write a “canonicalizing” script: I give it a list of German/English names such as ‘DSGVO’, ‘Datenschutz-Grundverordnung’, or ‘GDPR’, and out comes “general data protection regulation”. This turns out to be a very simple deterministic script, all the difficulty lies in the data: you have to have a German-English dictionary and a glossary of abbreviations.

So the hard problem is now reduced to providing this dictionary. And this is a task that should nowadays be solvable even by a rather simple language model: “Given the task of searching for duplicate names, this list of English/German names, and this current dictionary/glossary, what entries are missing in the dictionary?”

Overall, the reason for using a deterministic script rather than an LLM seems to be analogous to the reason to use a machine instead of a human: machines are faster and more reliable than humans, but need some initial investment and ongoing maintenance by humans. Deterministic scripts are faster and more reliable than LLMs but need some initial investment and maintenance (by humans, but perhaps even LLMs would be enough?)

Anyway, the next step would be to set this up as a simple workflow/check, including regular dictionary updates and some machinery to mark duplicates as acceptable.

How do we detect these duplicates? It sounds like a fairly simple problem, just collect them and check for similar names. It is, however, made complicated by the fact that one team may choose to model the components in English while another models them in German. Or one team uses an abbreviation and the other spells out the full name. Language translation is something that large language models have always been good for, so it feels obvious that AI could be part of the solution. But what should be its place exactly?

I decided to start simple: let’s export all components to a text file, feed the file to the LLM and ask it if it sees any duplicates. In fact, even simpler: let’s just export the component names, paste them in, and ask the LLM to canonicalize them by translating from German to English and expanding abbreviations where possible.

Sounds trivial? Well, not so fast. One does not simply paste a 3,000-line file into a Copilot prompt. Copilot just stows it away in some session-specific directory and tells the agent where it is. This is basically the same as @-mentioning it in the prompt. The result is that the LLM only sees the file name and instead of trying to produce an answer from context, it reads a few hundred lines to understand what it’s up against, then rolls up its sleeves and starts writing Python scripts. Just like a real, lazy programmer: it’s far more interesting to spend a day writing a script that takes a minute to run, than producing an answer in five minutes. (Of course, being a lazy programmer myself, I kind of empathize and agree with the approach. Especially since my goal is to automate this and run it repeatedly, so I welcome deterministic scripts.)

Interestingly, in the scripts it writes, it will include a small German-English dictionary, based on the actual words it encounters in the input. So this gave me another idea: structure the workflow so that we do as much mechanical work as possible outside of the LLM, and only give the LLM a simple, focused translation task. This kind of goes against The Bitter Lesson (which says that general-purpose methods will outperform special-purpose human-engineered approaches given enough compute), but I guess that lesson applies only as a macro trend, not on a micro scale.

All this was just dealing with the names. Who knows, maybe in the end just using some structural similarity score will fare better. We’ll see next time.

We covered the functionality, architecture, and future plans for mops, as well as why mops was created and how it is different to other approaches.

If you have any comments or questions, just reply to this email.

In the demo I will show how mops works, how it can be used by agents and humans, and where I would like to take it. There will also be time to answer your questions.

The demo will take place in the usual MPS Office Hours Google Meet on Thursday, July 16th, at 14:30 CEST. The meeting will be recorded.

MPS pre-commit hooks are Python scripts that do not launch MPS but inspect files directly. This makes them complementary to MPS model checks. Hooks are much faster but they cannot check domain logic. Rather, they focus on checking the basic shape of the project for consistency.

For example, hooks can check for:

• zero-length XML files (caused by MPS misresolving deleted/modified conflicts),
• modules present in the repository but missing from the MPS project,
• modules present in the MPS project but missing on disk,
• modules present in the repository but not mentioned in any MPS build script,
• models present on disk but not mentioned in any model root (e.g. leftover generator models),
• modules whose file/directory name does not match their name in MPS,
• path variables used in modules.xml or libraries.xml,
• and so on.

I am currently setting up the mbeddr platform build to use them (see PR mbeddr/mbeddr.core#3466). On this repository they only take about a second or so to run, and detect many consistency issues that have accumulated over the years.

If you are interested in trying them out, install prek and add a .pre-commit-config.yaml to your repository with this contents:

repos:
  - repo: https://github.com/specificlanguages/mps-pre-commit-hooks
    rev: v0.1.0
    hooks:
      - id: mps-check-orphan-modules
      - id: mps-check-unbuilt-modules
      - id: mps-check-missing-modules
      - id: mps-check-orphan-models
      - id: mps-check-orphan-mpsr-files
      - id: mps-check-zero-sized-xmls
      - id: mps-check-module-naming
      - id: mps-check-path-variables

Then run prek install in the repository to have the hooks run on the files changed in a commit. If you want to run hooks on all files, run prek run -a.

This has left me wondering how I can properly evaluate whether a tool is beneficial or harmful for certain tasks. So far, I have been evaluating the performance of my tooling manually. I asked the developers of other MPS AI tooling how they were approaching the measurements of their tools, and the answer has been the same: we run it and observe what it does. While this is certainly a possible and in some cases good enough approach, I felt the need for a more reproducible, automated, and overall rigorous approach.

This is why I have spent the past several weeks developing a harness for benchmarking MPS tooling. The tool lets you start from a known good state (reproducibility), set up MPS in a way that helps the tools run unattended (automation), and record the results: the final state on disk, the session transcript, and the metadata such as the versions of MPS and the coding agent used, the token count and the estimated cost (rigor).

Running a first benchmark with the harness showed that, on a simple prompt to write a Java method manipulating some nodes in the project, the agent did better with tooling than without, and more consistently so, but at almost double the cost.

However, the results can definitely provoke many objections:

• The task should have been more complex.
• The prompt should have been less ambiguous.
• The tooling has evolved, a newer version should have been used.
• The setup was wrong and was missing parameter foo.
• A different model should have been used.
• A different agent should have been used.
• The results should be scored differently – in my view, for example, the baseline was almost as good as mps-mcp on this particular task, even though the scores suggest otherwise.

Instead, I invite you to check out the harness and try it yourself. The harness is available on GitHub under specificlanguages/mps-ai-benchmarks. Check it out, set it up and let it run.

I have developed it with the help of Fable 5 (during the few days it was available) and Opus 4.8, in Claude Code. It is a set of Python and Bash scripts, currently only supporting macOS and Claude Code. However, support for other operating systems, agents, and tools is probably a single prompt away.

So far this is 99 % vibe-coded, I have let agents write all of the code and documentation and decide upon the architecture. If it catches on, I will invest some time in refactoring it.

If you have questions or want help with running the benchmarks, I’m happy to help by email, on Slack, or in office hours.

I have also had several contacts admit to me that they are considering moving away from MPS towards more mainstream languages to make use of the newly available AI capabilities. Instead of generating code from MPS, they would generate code by talking to AI.

I was left to wonder whether I should consider a different specialization. Instead of continuing to work with MPS and around its numerous quirks, should we all switch to writing Kotlin and TypeScript and go find ourselves new jobs as full-stack developers?

After some time and experience, I think I can now give the traditional answer to the title question: it depends.

I believe the recent advances in LLMs and coding agents make the weakest case for MPS even weaker: using it mainly as an expensive boilerplate generator when the domain structure is shallow. If the model is little more than a description of database entities for a CRUD application, MPS was probably not the right tool in the first place. Recent AI advances only make that clearer.

However, most of the value of MPS has never been in just generating code. MPS is most valuable when the hard part is not producing text but maintaining a model of a complex domain over time. Its utility comes from things like:

• structure: explicitly defined concepts with typed properties, children, and references;
• constraints, typesystem and checking rules;
• rich projectional editing with tables, diagrams, go-to-reference navigations, including in diffs;
• custom IDE support with actions;
• scripting;
• migrations;
• generators.

AI makes boilerplate code generation less special, but it does not replace the maintenance value of explicit domain structure, executable checks, or migrations.

With MPS, you encode the domain knowledge in the language definitions and models, not in the generated code. The models and language definitions are what you maintain. With AI-generated code the knowledge moves to the generated code itself, potentially into Markdown documents, or in the worst case into the developers’ heads. The AI-written code, along with the documentation, now has to be understood, reviewed, changed, and deleted. This means that your agent had better help you reduce maintenance costs, not just generate more code faster, as James Shore argues in his article, You Need AI That Reduces Maintenance Costs.

Thus, we can say that AI narrows the case for MPS mostly by removing weak arguments for it. MPS remains relevant for complex domains where the value is in domain modeling, controlled maintenance and evolution.