In Mopidy 4.0 we’ve:
These changes make the codebase less scary and more fun to work in, which matters a lot for a volunteer-maintained project. You can read more in the full changelog.
All 12 extensions in the mopidy GitHub organization have been updated to require Mopidy 4.0 and have gotten new major releases.
This includes mopidy-spotify, which
finally has a non-pre-release version using gst-plugin-spotify and the Spotify
Web API 🎉
We can’t test all of them fully (we don’t have anyone with a Pandora subscription, for instance), so please report any issues you run into.
There’s not a lot here for the average end user, and that’s fine. This release is about laying a better foundation, and hopefully it’ll breathe some life back into Mopidy development.
Packages are already on PyPI. Distribution packages will follow when the packagers get around to it, including apt.mopidy.com, which now serves as a backport channel for the latest Mopidy packages from Debian unstable.
There’s a considerable backlog of open issues and PRs across the Mopidy repos. If you’ve filed an issue or opened a PR, it’d be a big help if you could check whether it’s still valid and rebase or update as needed. We now have a clean slate, making it easier to get improvements shipped.
If you maintain a Mopidy extension, now’s the time to:
It’s also worth checking out the updated mopidy-ext-template, which has been heavily modernized. It now uses Copier, with much better support for keeping existing projects in sync with template improvements, not just bootstrapping new ones.
A few distributions packaged alpha versions during the long gap between releases. We appreciate the enthusiasm, and hope those will be swapped out for the release versions quickly, which will go a long way toward getting the broader Mopidy ecosystem back into shape.
All extensions in the mopidy org are starting from a fairly clean slate. If you’ve been thinking about getting involved, whether fixing bugs, reviewing PRs, or helping maintain an extension you use, this is a good moment to jump in. We’d love more hands on deck.
]]>The most significant change in Mopidy 3.0 is adding support for Python 3.7+ and dropping support for Python 2.x. Other release highlights include:
Moving the MPD frontend and Local backend out of Mopidy Core and into their own projects.
The new extension registry.
A configurable default web app for the HTTP frontend.
Simplified logging configuration.
Improvements to Mopidy’s Core API including removal of many long-deprecated APIs.
For a comprehensive list of changes please see the changelog.
Mopidy 3 also brings new versions of all extensions, some of which include new features. Most noticeably, Mopidy-Spotify 4 supports (read-only) playlists again, and also offers additional browse sections.
Yes! The Debian packages for Mopidy and those extensions maintained under the Mopidy GitHub organisation should now be available from apt.mopidy.com for all supported architectures.
Likewise, packages for Arch Linux and everything required to install from source is ready.
This depends on how you installed Mopidy 2.x. See our documentation.
Python 2 will not be maintained after January 1, 2020 and there is no future for Python projects that are not compatible with Python 3. Python 2 and Python 3 are similar but do have significant differences and so there’s considerable burden in also supporting Python 2. If you are limited to Python 2 then stay with Mopidy v2.3.1.
The minimum supported version is Python 3.7 but we also support newer versions
such as Python 3.8. Python 3.7 was primarily chosen as it, at the time of
porting, is the version of Python shipped in Debian stable (Buster). Python
3.7, or newer, is also available by default in Ubuntu 18.10, Arch Linux,
Fedora 29, and in macOS via Homebrew. You can see what the default version of
Python 3 is on your system by running python3 --version.
Ubuntu 18.04 (Bionic Beaver) is nearly 2 years old and Long Term Support releases are enterprise focused with stability in mind; they are not designed for running cutting-edge software, such as Mopidy 3. The next Ubuntu LTS release is 20.04 (Focal Fossa) and scheduled for April 2020. It will ship with Python 3.8 and so Mopidy will work out of the box with that. If you are stuck using Ubuntu 18.04 then you’ll find it uses Python 3.6 by default. Getting Mopidy 3 running is possible, but requires some work. One such way is detailed below.
Alternatively, you can continue using Mopidy v2.3.1 until Ubuntu 20.04 is released - just be sure to use Mopidy’s 2.x documentation and only install old Python 2 versions of any extensions from PyPI.
A Docker based installation of Mopidy 3 on Ubuntu 18.04 should also be possible.
Newer versions of Python are available from Ubuntu’s software repositories and installing Python 3.7 is as easy as running:
sudo apt install -y python3.7
Next, follow the first three steps of the instructions for installing from source. Then install Mopidy under Python 3.7 with:
sudo python3.7 -m pip install --ignore-installed mopidy
The GStreamer side of things is a little trickier. We need versions
of python3-gi and python3-gst-1.0 that are compatible with Python 3.7 but
the versions available in Ubuntu 18.04
(-gi,
-gst-1.0) only support the
system Python version.
First up is pygobject: the Python 3 bindings for gobject-introspection
(python3-gi). This is easy to build from source using pip, once the
required dependencies have been installed:
sudo apt install -y \
libcairo2-dev \
libffi-dev \
libgirepository1.0-dev \
libglib2.0-dev \
python3.7-dev
sudo python3.7 -m pip install --ignore-installed --no-cache pygobject
The GStreamer Python binding overrides (python3-gst-1.0) are not so simple as
they are not available from
PyPI. The
easiest way is to get 3.7 compatible files from an alternate version of
python3-gst-1.0, and copy them into the Python 3.7 installation:
wget http://archive.ubuntu.com/ubuntu/pool/universe/g/gst-python1.0/python3-gst-1.0_1.14.4-1_amd64.deb
ar x python3-gst-1.0_1.14.4-1_amd64.deb
tar -pxvf data.tar.xz
sudo mv usr/lib/python3/dist-packages/gi/overrides/* /usr/local/lib/python3.7/dist-packages/gi/overrides/
If you want to run Mopidy in a
terminal then you are
all done and python3.7 -m mopidy should now work. But if you want to run
Mopidy as a service then
there are a few more steps and things get a little messier.
Download the latest Debian package for Mopidy 3 (as of writing this is v3.0.2 but adjust as required):
wget https://apt.mopidy.com/pool/main/m/mopidy/mopidy_3.0.2-1_all.deb
Find any missing dependencies by attempting to install; this will likely fail:
sudo dpkg -i mopidy_3.0.2-1_all.deb
Install those missing dependencies and then install again; this should succeed:
sudo apt install --fix-broken
sudo dpkg -i mopidy_3.0.2-1_all.deb
Force the Mopidy service to use our Python 3.7 installation:
sudo sed -i 's/python3/python3.7/' /usr/bin/mopidy
Finally, remove packages we no longer need, including those left over from Mopidy 2.x:
sudo apt autoremove \
libcairo2-dev \
libffi-dev \
libgirepository1.0-dev \
libglib2.0-dev
Be sure to install all Mopidy extensions from source using
sudo python3.7 -m pip install mopidy-foo-extension.
Debian Stretch (oldstable) is not supported. Mopidy 3 requires GStreamer >= v1.14.0 but Stretch provides v1.10.x. There are great new features in v1.14 and now we get to finally start using them.
The Mopidy-MPD and Mopidy-Local extensions are no longer bundled with Mopidy and have been moved to independent extension. We hope this will help ease development and encourage contributions. After the move, Mopidy-Local merged with Mopidy-Local-SQLite and Mopidy-Local-Images; the functionality of these three projects is now available under one roof in Mopidy-Local. If you used either of these extensions you will now need to explicitly install them.
To install Mopidy-MPD: sudo python3 -m pip install mopidy-mpd
To install Mopidy-Local: sudo python3 -m pip install mopidy-local.
The configuration for Mopidy-Local has changed and you may need to merge/move your previous settings. Refer to Mopidy-Local’s documentation for the available config values.
Some Mopidy extensions do not support Mopidy 3 yet. You can find information on
which ones do at our extension registry. Any Python
3 compatible extensions installed using your package manager should be updated
when you upgrade Mopidy. But any extensions that were installed from PyPI will
need to be explicitly re-installed for Python 3
e.g. sudo python3 -m pip install Mopidy-dLeyna.
Extensions maintained under the Mopidy GitHub organisation have been ported to Python 3 and support Mopidy 3.0. For those commmunity maintained extensions not yet ported to Python 3, visit the project’s GitHub page for more info and also consider lending the maintainer a helping hand. Please be patient during this transitional period.
If installed from source or PyPI you may want to remove Mopidy 2 from your system, but it’s not strictly necessary. Mopidy on Python 3 won’t in any way be affected by any remnants of Mopidy on Python 2.
Mopidy installed using your system package manager (e.g. apt) will automatically remove the old Python 2 version of Mopidy as part of the upgrade.
When updating the Debian package you will probably get the following:
Configuration file '/etc/mopidy/mopidy.conf'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
What would you like to do about it ? Your options are:
Y or I : install the package maintainer's version
N or O : keep your currently-installed version
D : show the differences between the versions
Z : start a shell to examine the situation
The default action is to keep your current version.
*** mopidy.conf (Y/I/N/O/D/Z) [default=N] ?
If you enter D here it will display the differences. You’ll see the default
service-specific [core] and [logging] configuration has been removed (it
now lives elsewhere). The remaining differences will be anything you added
yourself e.g. account details for extensions, the path to your music files, a
custom audio output etc. You’ll probably still need all of that. The main
exception is any configuration for Mopidy-Local
as some things changed following the move, please check the project’s
documentation for
details.
If you are not sure what to do, select option Y and get the new default
Mopidy 3 service configuration. You will still find your old configuration at
/etc/mopidy/mopidy.conf.dpkg-old, which you can compare against the output of
sudo mopidyctl config to see what configuration you actually still need to
specify in /etc/mopidy/mopidy.conf to obtain the setup you want. Afterwards,
feel free to delete mopidy.conf.dpkg-old.
This is the story of the first decade of Mopidy.
After a brief discussion of how it could work and what we could build upon, Johannes came up with the name “Mopidy.” The name is, maybe quite obviously, a combination of the consonants from “MPD” combined with the vowels from “Spotify.” At the same time, the name is different enough from both of its origins not to be mixed up with them. Even during the first few hours we had some thoughts about maybe adding file playback and support for other backends in the future. Thus we quickly appreciated that the “Mopidy” name would still work, even if Spotify wasn’t always the sole focus of the project.
Within a couple of hours we had a Git repo with some plans written up. We joined the #mopidy IRC channel on Freenode and we had recruited Thomas Adamcik to the project. Over the next few years, he designed many of our most essential components, including the extension system. Today, ten years later, Thomas is still involved with Mopidy and many of its extensions.
After a couple of days, it worked! We had built a primitive MPD server in Python that at least worked with the Sonata MPD client. On the backend side, we used the reverse-engineered “despotify” library to interface with Spotify as it already had some Python bindings available. For all three of us, coming mostly from web development and Django, I believe we already had a feeling of achievement and expanding horizons. If we could pull this off, we could build anything.
The story of Mopidy is a story of thousands of small iterative improvements that, over time, add up to something far greater than the sum of its parts. It was a hack, but a hack with good test coverage from the very start, making changes and iteration safe and joyful.
In March 2010 we released our first alpha release. Over the next decade this would become the first of 74 releases, not counting the numerous releases of the extensions we later extracted from Mopidy or built from the ground up.
Later in 2010, as we added an alternative Spotify backend using the official libspotify library, we started seeing the first traces of our current system architecture. At that time, you switched between the single active backend by manually changing the config file and restarting Mopidy. Support for multiple active backends wouldn’t come until much later, after we had built the muxing “core layer” in-between the backend and frontend layers. However, multiple frontends were supported from the start. The MPD server was the first frontend, and during the first year, we added a second one with the Last.fm scrobbler.
By the end of 2010 we had made GStreamer a requirement and thrown out the
support for direct audio output to OSS and ALSA. In 2011 we built upon the
great power of GStreamer to support multiple outputs; this allowed us to play
audio locally whilst also streaming it to a Shoutcast/Icecast server at the same
time. This support for multiple outputs survived for about a year before we
removed it. Instead, we exposed the GStreamer pipeline configuration directly,
just like we still do today with the audio/output configuration. Streaming to
Icecast is also still possible but is involved enough to require specific
documentation for the setup. However, by exposing the GStreamer pipeline
directly, we didn’t have to guess what kind of installations people would use
Mopidy in and exposed the full power of GStreamer to our end-users.
We added support for Python 2.7, but we definitely didn’t plan to stay with it for eight years. During the winter of 2010/2011, I designed and built the Pykka actor library based on the concurrency patterns we had established in Mopidy. When we started using Pykka in Mopidy in March 2011, it already supported Python 3.
Towards the end of the year, we added support for the Ubuntu Sound Menu and the MPRIS D-Bus specification in our third frontend.
Most of 2012 went by without much happening other than a few maintenance releases. However, in November, we released the almost revolutionary Mopidy 0.9 after finally building out the muxing core layer in-between frontends and backends. Depending on the type of request from a frontend, the core layer would either forward the request to the correct backend or, e.g., in the case of search, fan out the request to all backends and then merge the returned search results before passing the result back to the frontend. We had accomplished one of our original goals from the very first day of development: we had a music server that could play music from both Spotify and local files.
Less than a month later, the wheel turned again and we released Mopidy 0.10 with the HTTP frontend. This exposed the full Core API using JSON-RPC over a WebSocket. With this it was suddenly possible to build clients for Mopidy directly, instead of going through MPD.
In a lazy and ingenious moment, we decided that we had no interest in manually keeping the Core API and the JSON-RPC API in sync for the indefinite future. Thus, we based the JSON-RPC API on introspection of the Core API and included a JSON-RPC endpoint which returned a data structure that described the full API. On top of this, I built the Mopidy.js library and released it together with Mopidy 0.10. Mopidy.js uses the API description data structure to dynamically build a mirror of our Core API in JavaScript, working both on Node.js and in the browser.
Even as the JSON-RPC implementation and the Mopidy.js library became the foundation for several popular Mopidy web clients over the next few years, no bug was ever reported that originated in this library. To this day, I testament this to two things: proper test-driven development and excellent code review by Thomas, making these few weeks in November-December 2012 one of the highlights of my years as an open source maintainer.
Jumping just a few months ahead to Easter 2013, the next revolution was about to happen. In a single long and intense day, Thomas and I hashed out and implemented Mopidy’s extension support. Up to this point, there was just one Mopidy. Today, a search for “mopidy” on PyPI returns 127 results.
The Stream backend was created early in 2013, and later in the year, it learned how to parse several playlist formats to find the streamable URL they contained. This made it easy to build backend extensions for music services that exposed playable URIs, like SoundCloud, Google Music, and thousands of radio stations. The backends only had to find and present the playable streams as a playlist or as a virtual file hierarchy; the heavy-lifting of actually playing the audio could be fully delegated to Mopidy and its Stream backend.
By the end of 2013 we had performed the first round of shrinking Mopidy’s core. The Spotify support, Last.fm scrobbler and MPRIS server were all extracted to new extensions living outside the core project. I believe that pulling extensions out of core has helped reduce the amount one must juggle in one’s head to effectively develop on Mopidy.
In 2013 we eased the on-ramping for new users by automatically creating an initial configuration file on the first run. Mopidy also got support for announcing its servers through Zeroconf so they could be autodetected by mobile apps, like Mopidy-Mobile.
Elsewhere in 2013, Wouter van Wijk built the first iterations of the Pi MusicBox distribution for Raspberry Pi. Pi MusicBox provided a turn-key jukebox setup built on Mopidy. This made Mopidy more approachable for the masses that didn’t know Mopidy, Python, or even Linux; allowing them to create their own hi-fi setups.
The next year, in 2014, Fon launched a Kickstarter campaign to build a “modern cloud jukebox” named Gramofon. It turned out that they based their prototype on Mopidy and Javier Domingo Cansino from their development team started submitting patches and becoming active in Mopidy development.
In the summer of 2014 we had our first real-life development sprint at EuroPython in Berlin. Javier and I were joined by several newcomers that got up and running with Mopidy development and squashed a few bugs.
Mopidy 0.19.5 was released on Mopidy’s fifth anniversary in December 2014. A few months later, we released Mopidy 1.0. The release of Mopidy 1.0 did not mark a breaking change, but rather the decision that we could commit to the current APIs for a while and bring stability to the extension ecosystem.
In the summer of 2015, Thomas joined us as we had our second development sprint at EuroPython in Bilbao, Javier’s hometown.
The year between the sprints of 2014 and 2015, and the 0.19, 1.0, and 1.1 releases, were quite significant when looking back at the project’s history. They were not as notable in features as when we added multi-backend, the HTTP API, and extension support back in 2012/2013. Still, they were significant in that the project garnered lots of interest. Up to twenty different people contributed code to each of these three releases.
More than five years ago, in July 2014, I opened Mopidy’s issue #779 for tracking the port to Python 3. There were three large buckets of work that had to be completed. First, our libspotify Python bindings had no Python 3 support. Second, we needed to upgrade to GStreamer 1. Finally, we had to port Mopidy and all of its extensions.
Over the next year, I rewrote pyspotify from scratch using CFFI. Just a couple of weeks before I shipped the final version, Spotify deprecated the libspotify API. However, since they’ve never provided a replacement API for audio playback, we’re still using pyspotify 2 and libspotify to play music from Spotify more than four years later.
During the autumn of 2015 I ported Mopidy from GStreamer 0.10 to 1.x. GStreamer 0.10 was quickly being deprecated, so this work was necessary solely for Mopidy to continue being packaged in Debian and Ubuntu. It was a nice bonus that the new PyGObject wrapper also supported Python 3. With the release of Mopidy 2.0 in February 2016, the move to GStreamer 1 was complete, and thanks to Thomas, we finally supported gapless playback.
Shortly after the 2.0 release, life caught up with several of the most active contributors. A mix of more kids and more work threw Mopidy into a three-year-long period with lower activity and almost exclusively maintenance releases. The only significant development of Mopidy during this period was the support for persisting playback and tracklist state across restarts, contributed by Jens Lütjen, and released in 2.1 in 2017.
Jumping ahead to 2019. Five years after I wrote the tracking issue for moving Mopidy to Python 3, the world looks quite different. Python 2.7’s announced end-of-life is looming at the end of the year, and Python 3 is the standard for all new projects.
So, finally, in the middle of October, we got started on the third and final step toward Python 3.
Once we had a small part of the test suite running on both Python 2 and 3, it took Nick Steel and myself about three weeks of porting modules one by one until suddenly the full test suite was running. Once Mopidy without extensions ran well on Python 3, we axed all Python 2 support, cleaned up all hacks left over from the porting process, and reformatted the code with Black. All of a sudden Mopidy felt like a modern Python codebase.
The six weeks since then have mostly been spent on extensions.
We’ve built a new Mopidy extension registry. We believe that the new registry will ease the discovery of extensions in general. Short-term, we also hope it will help users navigate the extension ecosystem while it is temporarily split in two between Python 2 and 3.
All extensions in the Mopidy GitHub organization are now running on Python 3, as well as a few popular extensions elsewhere. In total, we have almost 20 extensions compatible with Mopidy 3.0 on the day of the release.
Some extensions have also recently received some extra tender loving care.
The Mopidy-Local extension was pushed out of core early in the Python 3 porting. After becoming an independent extension, Thomas Kemmer’s excellent Mopidy-Local-SQLite and Mopidy-Local-Images were merged into Mopidy-Local. We now have a single comprehensive extension for using Mopidy with pre-indexed local music collections.
Next, Nick did a great job fixing up the Mopidy-Spotify extension. Since Spotify suddently broke the playlist part of libspotify a while ago, Mopidy-Spotify has been without functional playlist support. It now uses the Spotify Web API for everything related to playlists and Nick has continued adding several new features using the Web API. Some are shipped in the release that went out together with Mopidy 3.0, and more are right around the corner. Once complete, Mopidy-Spotify will support everything Mopidy-Spotify-Web provides, once again leaving us with a single comprehensive extension for using Mopidy with Spotify.
Just a few days ago, the Mopidy-MPD frontend was pushed out to an extension too. With this, we’ve come full circle from Mopidy being named after “MPD” and “Spotify.” As of Mopidy 3.0, both Mopidy-MPD and Mopidy-Spotify are independent extensions, and Mopidy core knows nothing of either. I wasn’t entirely sure whether moving the MPD server would bring any benefits. Still, once the move was complete, it was evident that Mopidy-MPD deserves to be a project by itself. The split reduced the amount of code in Mopidy by 25% and cut the test suite run time in half. Hopefully this will also make it easier for newcomers to start contributing to either Mopidy-MPD or Mopidy itself.
Finally, December 22, the day before the Mopidy project’s 10th anniversary, we published Mopidy 3.0.
Together with Mopidy 3.0, we uploaded ten updated extensions to PyPI, with at least six more having compatible pre-releases, and, hopefully, final releases over the next few days. For an up to date overview of what’s ready for Python 3 right now, the extension registry is the place to look.
It’s still early days for Mopidy 3.0. Our Homebrew tap is up to date with the new releases, and parts of Arch Linux are already up to date. Updated Debian packages, both at apt.mopidy.com and in Debian, are still some days away. Nonetheless, Mopidy 3.0 will definitively be a part of Ubuntu 20.04 LTS come spring.
Now, go forth and update your extensions to work with Python 3.7+ and Mopidy 3.0.
In a week, Python 2.7 reaches end-of-life. Mopidy, however, is ready for the next decade.
Thanks to Nick Steel for reviewing this blog post.
]]>Mopidy 3.0 is a backward-incompatible release in a pretty significant way: Mopidy no longer runs on Python 2.
Mopidy 3.0 requires Python 3.7 or newer.
While extensions have been able to continue working without changes throughout the 1.x and 2.x series of Mopidy, this time is different:
All extensions must be updated to work on Python 3.7 and newer.
Some extensions need to replace their use of a few long-deprecated APIs that we’ve removed. See the full changelog for details.
Extension maintainers are also encouraged to update their project’s setup to match our refreshed extension cookiecutter.
In parallel with the development of Mopidy 3.0, we’ve coordinated with a few extension maintainers and upgraded almost 20 of the most popular extensions. These will all be published shortly after the release of Mopidy 3.0.
We’ve also built a new extension registry, where you can quickly track what extensions are ready for Python 3.
In other news, the Mopidy-MPD and Mopidy-Local extensions have grown up and moved out to flourish as independent extension projects. After the move, Mopidy-Local merged with Mopidy-Local-SQLite and Mopidy-Local-Images, which are now both a part of the Mopidy-Local extension.
For further details, please see the changelog.
To read more about the story behind Mopidy 3.0, see our next post, 10 years of Mopidy.
]]>Mopidy-MPRIS is a Mopidy extension that makes Mopidy controllable from other programs on the same machine through D-Bus. This makes it possible to control Mopidy from various widgets in GNOME/KDE/etc, as well as with keyboard media keys.
This release replaces the python-dbus D-Bus bindings with python-pydbus to
modernize the code base and prepare it for the move to Python 3. It also wires
up a lot of events so that various UI elements are immediately updated when the
server state changes.
As part of the release, the documentation has been greatly extended, including a survey of some MPRIS clients and tips on how to run Mopidy-MPRIS on the system bus. Throughout the documentation, I’ve added calls for help wherever something isn’t working perfectly and I haven’t figured it out yet. Even with these rough spots, this is easily the best Mopidy-MPRIS release so far.
For all the details, check out the changelog.
This blog post was originally published at jodal.no.
]]>Four years is quite a long time in the world of JavaScript.
In the beginning of October, I stopped bundling Mopidy.js with the Mopidy Python project and thus had to update the Mopidy.js documentation. While doing so, I tried to run the test suite, written using Buster.JS, which has been abandoned for a few years. To no avail, npm and Yarn were not able to install the library’s development dependencies.
After four years of standing still while the fast-moving world of JavaScript sped on, I seemed forced to spend some time on modernizing the library and it’s development tooling to ensure that it would be maintainable going forward.
I asked a couple of coworkers for advice on what to replace Buster.JS with that could test a library in both Node.js and the browser, and what they would use instead of Browserify today. I was quickly pointed towards Jest for testing and Parcel for zero-configuration builds.
Porting the full test suite–three times the size of the library–to Jest was work, but mostly mechanical work. The only part I needed to read a bit up on was Jest’s mocking capabilities compared to Sinon.JS.
After getting the test suite running again the modernization continued.
JSHint and a list of rules were replaced with ESLint and Airbnb’s style guide.
All code was reformatted with Prettier. This required
almost no
configuration,
once figuring out how to make ESLint and Prettier stay friends. However, the
experience of getting an entirely consistent code style without any effort is
priceless. I can accept most differences from my previous styles in exchange
for not having to do the work myself and not having to enforce that my
contributors and coworkers do the same. Formatters like Prettier,
Black, gofmt, and rustfmt are here to stay
and will probably be an integral part of all new programming languages.
The Mopidy.js code was originally hand-written ES5 using all the hard-learned
tricks of the trade from David Herman’s splendid book Effective
JavaScript. At least it was splendid in 2013. Now it
was
converted
to a modern ES6 class with a single click in VS Code, passing all tests bar one:
you cannot instantiate an ES6 class without the new keyword.
Removing Buster.JS and JSHint reduced the project’s Grunt setup a bit. The move
from Browserify for browser builds and Uglify for minification to Parcel lead
to the removal of Grunt entirely, and the replacement of a monitor height or
two
of Grunt config with the command parcel build src/mopidy.js.
The time had now come to runtime dependencies.
The faye-websocket package was replaced with the
ws package, which made it possible to
replace our own Node.js/browser compatibility
layer
for WebSocket usage with
isomorphic-ws.
Since the last release of Mopidy.js in 2015, ES6’s Promise implementation has become quite universally available. Thus, when could be replaced with Promise, shrinking the minified library from 42 kB to 12 kB.
BANE, an event emitter library that was
part of the now-dead Buster.JS testing tool, was replaced with the
EventEmitter implementation from Node.js’ standard library. Parcel
helpfully and entirely automatically included the EventEmitter implementation
from Node.js as part of the web bundle, without having to add a dependency on
one of the npm packages that has
extracted this lib from the Node.js standard library.
Once everything was modernized, I added two demo applications to the project.
The
web-based
application is served by Parcel’s web server. The configuration needed? Add
parcel examples/web.html as the start script in package.json and I was
done: yarn start now runs the demo web app on http://localhost:1234, with
automatic code reloading on source changes.
For the Node.js console
application,
I quickly remembered that while promises are better than callbacks, they can
still quickly become quite mind-boggling to work with. Enter async/await,
spreading like wildfire from language to language the last few years, and
fully available in Node.js since 2017. You just have to remember to create all
the promises you need, and only then await them. This makes it possible for
the promises to be fulfilled in parallel instead of sequentially, avoiding that
the sequential round trip times to the server add up.
Somewhere around here I read a bit up on TypeScript and tested out porting the library to TypeScript with great help from VS Code. Due to Mopidy.js’ tiny static API and quite large dynamic API, automatically generated based on the API description retrieved from the server, this didn’t seem a worthy path to go down this time around. However, the experience was a good one, and I’ll probably revisit TypeScript in the future on other projects.
Finally, the Mopidy.js docs were moved from the Mopidy docs to the Mopidy.js project and updated accordingly. Being used to writing docs in Sphinx, Markdown’s lack of features like generating a table of contents can feel limiting. Of course, several VS Code extensions can automatically generate a ToC and keep it up to date every time you save your file.
In the late 1990s, I tweaked JavaScript-based web calculators. In the 2000s I surfed with JavaScript disabled and required apps we built to work without it. From 2011 through 2015, I spent a small majority of my time in frontend JavaScript. Since the summer of 2016, I’ve done very little, if any, JavaScript.
It seems that my relationship with and usage of JavaScript varies like the tide, even if on another time scale. After a while away, JavaScript as of late 2018 seems to be in a lot better place than in 2015. The language is nicer. The chore of project setup and maintenance is reduced. Tooling is better, and way better integrated. I’ve felt the JavaScript fatigue before, but from my perspective, it seems like many things are stabilizing in a quite good place.
I’d go as far as claiming that the exercise of modernizing Mopidy.js and its development tooling was, at times, fun and inspiring.
Here’s to four more years of Mopidy.js. Then I might be back with a port to WebAssembly, implemented in Rust, or a language yet to be designed. We’ll see!
This blog post was originally published at jodal.no.
]]>The new version adjusts the LSB facilities the sysvinit script depends on, to make sure that DNS lookups, DBus (used by Mopidy-MPRIS), and Avahi (used for Zeroconf service publishing) are available before Mopidy starts.
Most importantly, maybe, the new package adds a systemd service file. The
sysvinit init script included a custom action called “run”, which is used to
run Mopidy subcommands with the same user and configuration as the system
service use. To replace this with something that can also be used on a system
running systemd, a new executable called mopidyctl has been added. The new
executable does the same: it runs Mopidy with the same user and
configuration as the init system uses to start Mopidy as a system service.
In short, sudo service mopidy run config now becomes sudo mopidyctl config,
both on systems running sysvinit and systems running systemd.
This blog post was originally published at jodal.no.
]]>In Mopidy 0.10, released in the middle of December, we added an HTTP frontend. The HTTP frontend takes Mopidy’s full core API and makes it available from JavaScript in the browser. This means that you now can make your own web clients for Mopidy in JavaScript, and Wouter van Wijk has already started on his client (updated link).
I’d like to write a bit about how we made the HTTP client.
From the start, I was quite clear on having enough work to do on the server-side of Mopidy. Thus, making my own client to test out REST APIs and to make sure the APIs we exposed were usable was mostly out of the question. Also, making a REST API would require us to spend huge amounts of time on figuring out how to best remap our procedural “core API” to REST resources. When the remapping would be complete, if ever, we would be stuck with the maintenance of yet another API on top of the core API, with all the associated mismatches and hacks required to match them up. We got enough of them in the MPD frontend.
Wouter had experimented a bit himself with making an HTTP frontend and suggested that we should go for an RPC model. I was reluctant at first. RPC reminds me of SOAP, the nineties, and other similar ideas that I don’t exactly regard as the state of the art.
After some thought, I figured that with an RPC API, I could probably make the entire API on the server-side dynamically. In other words, I could get it done in a lot less time than a REST API, and it would maintain itself. At least, it would maintain itself to any degree such a thing is possible. If we added a new method to the core API, it would immediately be available through the HTTP RPC API. This would of course also mean that if we changed anything in the existing core API, we’d break any web client that uses that part of the API. After some discussion, we decided that we were OK with this drawback. After all, we intend the core API to become quite stable with time, where time approximately equals the release of v1.0. Also, the initial development work and future maintenance work associated with making an RPC API were within our reach, without distracting us for too long from work on the core code, the MPD and MPRIS frontends, and the local storage and Spotify backends. You see, we already got a healthy list of moving parts to keep oiled and working.
The part that got me excited on an RPC API (of all things to get excited about) was the insight that if I added some introspection support to the API, I could make a JavaScript library to rebuild the entire API in the browser. So, instead of just providing a web service endpoint URL to potential client providers, we could offer them a complete API in the browser. Ready for development. Queue your first music track in a few minutes of development. That’s a good value proposition for aspiring client developers if you ask me.
So, I made a new HTTP frontend. It started a CherryPy web server. Then it plugged ws4py into CherryPy, and we got a working WebSocket. Then I made my custom RPC API, using JSON as the transport format. I sent messages from Chrome’s console, and music started playing. It worked. It wasn’t tested, but I was happy so far.
Then Thomas, my main co-developer, wise as always, pasted the URL to the JSON-RPC 2.0 specification. Humph, I thought, not leaving it much chance. I read through the rather short spec and concluded that it was really close to what I’d reinvented, minus support for calls with both positional and named arguments at the same time.
Cutting it short, I spent the next day or so implementing JSON-RPC 2.0, this time with tests. I plugged it into the HTTP frontend, and it worked.
Now you might say: Why didn’t you use one of the 25 or so existing JSON-RPC implementations on PyPI. Because there are 25 or so. How am I to review them in less time than it takes to implement the perfect one for my needs? Many of the alternatives provide examples of how to execute a Python file, and then magically a web server will be running. That’s not a selling point to me since JSON-RPC got nothing to do with webservers or the message transport for that matter. JSON-RPC is simply a mapping between a JSON format and method calls. It should be implemented by some function/object that accepts JSON, makes the required Python calls, and then returns the return values as JSON again. That’s it. No web server. Maybe some API introspection.
Digression aside, I’m considering extracting and releasing our JSON-RPC adapter. Then it’ll be N+1 competing standards, eh, I mean, implementations.
Next up was the JavaScript library. The main discussion here was actually where
to place the code in our pure-Python repo. Bikeshedding of easily reversible
decisions continues to be the easiest discussions to have. We ended on js/ in
the root of the repo. How imaginative of us.
The first thing in any new JavaScript project is, of course, to set up
Buster.JS for testing. I also tried out the Grunt
build tool for the first time. (My friend Pål recently
wrote an introduction to Grunt
featured at HN and in JavaScript Weekly.) Buster.JS in combination with Grunt
and PhantomJS was a delight. If you simply run
grunt watch and then modify a source file, your code is linted and tested
in a headless browser in second or so. This makes JavaScript development for
the browser feel like server-side development. If this sounds interesting,
check out our JavaScript project
setup (updated link).
TODO
This is a blog post draft that was originally written in January 2013, left unattended for 1689 days, and rediscovered and published unedited in August 2017.
During the five years since Mopidy entered the browser with its HTTP JSON-RPC API and the Mopidy.js JavaScript library, many successful Mopidy web clients have been built on top of this foundation. The APIs themselves have survived the test of time and have required minimal maintenance, just as I hoped when implementing the APIs back in November 2012.
To quickly address the above TODOs from January 2013: Usage examples can be found in the Mopidy docs, and an invitation to develop clients wasn’t needed, as a dozen clients was made without it.
This blog post was originally published at jodal.no.
]]>