The way it works is the following: there’s one “admin” view (/gallery handler) and one “display” view (/viewer handler). Clicking on images in the gallery automatically replaces the displayed image on viewer screens (no page refresh needed). You provide a path to an image folder on server startup. The server will recursively collect all images in that folder hierarchy and present them in the gallery.

This is super-useful when projecting one browser tab to a different screen with say a Chromecast since the gallery tab can then control the projected one. You can also have multiple viewer devices/browsers connected at the same time and they’ll all be updated.

Example of running the server:

./server.py --images_folder=./test_images --port=8080 --host=localhost

Then go to http://localhost:8080/gallery for the admin view and to http://localhost:8080/viewer for the viewer page. Tested with latest Chrome, Firefox and Safari across desktop, Nexus 5/7/9 and iPad.

I need this for running my local Dungeons & Dragons game; as a DM, I want to show the players pictures of maps, characters and other stuff without exposing filenames or thumbnails of pictures that they aren’t supposed to see yet. I was surprised to find that there was no simple and easy way to do this so I wrote Picture Commander.

Picture Commander is free and open-source under the Apache v2 License. Enjoy!

Work on splitting YCM into a client-server model started about a year ago. I’ve written on this topic before; suffice it to say that the new architecture brought extra performance, stability and fixes for long-standing problems.

YCM switched to this new implementation back in October 2013. Ever since then I’ve been tweaking ycmd to remove Vim-specific cruftiness so that it’s more generic. I’ve also added some docs and written the example client which demonstrates how to talk to ycmd.

Why make the server generic?

I’ve learned a lot over the last two years since YCM came out. I’ve integrated several different semantic engines into YCM, from libclang for C/C++/Objective-C to Jedi for Python and OmniSharp for C# (more will be added with time). There’s a pretty simple ycmd-internal API for adding semantic support for other languages. YCM has also moved past code-completion and is now targeting code-comprehension features like GoTo and integrated diagnostic errors.

Of the things I’ve learned, an important lesson is that there’s a lot of common ground between languages when it comes to tools that provide semantic understanding of code. Much infrastructure is built over and over again for no benefit.

For instance, there are many Vim plugins that provide semantic code-completion for a specific language (like jedi-vim for Python, OmniSharp for C#, gocode for Go etc). They all need to have the following, even though some don’t:

1. A semantic engine that can provide a list of available function/class names for a given location in a source code file.
2. A filtering system that can intelligently remove completion strings that aren’t relevant.
3. A ranking system that will (hopefully) put the most useful completion at the top of the completion menu.
4. Process-level separation between the semantic engines and the editor so that engine crashes don’t take down everything. Also makes it much harder to block the editor’s GUI thread.
5. Vim integration that will provide a UI that auto-shows the relevant completions.
6. Vim integration for showing diagnostic messages (errors and warnings) that undoubtedly arise from their semantic engines.
7. Tons of other stuff too annoying to enumerate.

And then we repeat all of it not just across languages but also across editors like Emacs and Sublime Text which have entirely different plugins that implement (or fail to implement) all of the above as well.

This doesn’t scale and is a massive waste of effort.

In the above list, everything but the first point is common infrastructure. This isn’t a theoretical notion, this is how YCM already works. Integrating a new semantic engine into ycmd is borderline trivial; don’t take my word for it, take a look at how the Jedi engine for Python is integrated. It’s ~120 lines of code and provides both semantic code-completion and GoTo. Everything else is provided on top of that and is common to all the engines.

With ycmd now a server independent of Vim, a simple client can be written for any editor; as soon as a new semantic engine is plugged into ycmd, the old clients Just Work™ with it.

This is the point of ycmd: to sit between the clients written for a specific editor and the semantic engines written for a specific language while providing a simple API on both ends. The semantic engines don’t have to worry about anything beyond “which names are available at this line & column location in this file,” all the common plumbing is built on top of that. The clients on the other hands don’t have to worry about anything beyond “this is the line & column location in the buffer the user is editing.”

So what’s ycmd’s value-add?

It all comes from the shared infrastructure. Other than simplifying APIs for the semantic engines and the editor clients, ycmd for instance also has an identifier-based completion engine that’s language-agnostic; it’s triggered when semantic engines aren’t needed (more details in ycmd docs). There’s also smart caching so that the engines aren’t queried too often (they can be slow), a filepath completer, integration with snippet engines like UltiSnips and the aforementioned completion filtering and ranking (which is neither simple nor easy to implement right).

In case anyone is worried about the overhead of the client-server architecture, I’ve debunked that myth quite throughly (with benchmarks, no less). Even better than benchmarks, this has been implemented for YCM and has been battle-tested over the last 10 months. For a server running on localhost, the communication overhead is effectively zilch.

Conclusion

Let’s stop reinventing the wheel.

For semantic engine writers: expose your engine as a server that talks HTTP+JSON if at all possible; don’t worry about anything beyond listing completions at a specific location in the file. Plug the server into ycmd (it’s easy). All ycmd clients instantly work with your engine.

For editor plugin developers: implement a client for ycmd. Most code is already written for you in the example client, you just need to write the editor-specific parts. Hell, since the example code is all licensed under the Apache License v2, you can copy-paste most of it.² Enjoy good code-completion and code-comprehension features.

Here’s an explanation of what happens in the short GIF demo above.

First, realize that no keyboard shortcuts had to be pressed to get the list of completion candidates at any point in the demo. The user just types and the suggestions pop up by themselves. If the user doesn’t find the completion suggestions relevant and/or just wants to type, he can do so; the completion engine will not interfere.

When the user sees a useful completion string being offered, he presses the TAB key to accept it. This inserts the completion string. Repeated presses of the TAB key cycle through the offered completions.

If the offered completions are not relevant enough, the user can continue typing to further filter out unwanted completions.

A critical thing to notice is that the completion filtering is NOT based on the input being a string prefix of the completion (but that works too). The input needs to be a subsequence match of a completion. This is a fancy way of saying that any input characters need to be present in a completion string in the order in which they appear in the input. So abc is a subsequence of xaybgc, but not of xbyxaxxc. After the filter, a complicated sorting system ranks the completion strings so that the most relevant ones rise to the top of the menu (so you usually need to press TAB just once).

All of the above works with any programming language because of the identifier-based completion engine. It collects all of the identifiers in the current file and other files you visit and searches them when you type (identifiers are put into per-filetype groups).

The demo also shows the semantic engine in use. The current semantic engine supports only C-family languages. When the user presses ., -> or :: while typing in insert mode, the semantic engine is triggered (it can also be triggered with a keyboard shortcut; see the docs).

The last thing that you can see in the demo is YCM’s integration with Syntastic (the little red X that shows up in the left gutter) if you are editing a file with semantic engine support. As Clang compiles your file and detects warnings or errors, they will be piped to Syntastic for display. You don’t need to save your file or press any keyboard shortcut to trigger this, it “just happens” in the background.

In essence, YCM obsoletes the following Vim plugins because it has all of their features plus extra:

• clang_complete
• AutoComplPop
• Supertab
• neocomplcache

Why Did You Build This?

Several reasons, but mostly because a long time ago, in a galaxy far, far away I used to write C# code in Visual Studio. Its IntelliSense engine is a wonder to behold, and I missed it in Vim. Don’t get me wrong, Vim is a much better editor than VS (that’s why I switched to it), but Vim’s various built-in code completion systems are an absolute joke. As in, not-even-funny bad. I’m not going to press some complicated keyboard shortcut every single time I’m typing an identifier; no, screw that, the system should trigger itself and (unobtrusively) offer me completions that I can then either use or ignore.

The awesome Vim community valiantly tries to make the best it can out of the built-in “omni-completion”, but these efforts boil down to putting lipstick on a pig.

You could say that AutoComplPop (ACP) already offered us this type of completion auto-triggering as the user types. Fair point, but there are several issues with that plugin. First, it’s dead. Last update was in 2009. Even worse, it’s slow. Last, there’s no fuzzy search. I really, really like fuzzy search. Oh, and there’s no semantic completion with ACP.

The plugin that really made me see the viability of the subsequence match approach was the truly magnificent Command-T plugin. It does subsequence-based matching on file paths for easy file opening. This is probably my favorite Vim plugin, right up there with EasyMotion. I couldn’t (and wouldn’t want to) imagine writing code without them.

We also want our completion system to be smart about ranking whatever completion candidates survive the filtering step. In a perfect world, the candidate you want is exactly at the top of the menu so you need to press TAB only once. The whole point of a code completion system is to save you keystrokes and if you have to cycle through the menu to get to your candidate the point is lost.

Usually, you know exactly what string you want to type, but its long and you don’t want to type 10 characters if 3 plus a TAB will do the same job.

Well, that’s one use case for a code completion system. There’s another. That’s if you don’t actually know what string you want to use. This is the “API exploration” use case. You might have a vague idea of a function you want to call, maybe you even called it a few times before, but you can’t remember the full name.

This is where semantic completion comes into play. It allows you to explore what methods are available on the object you’re handling, what functions are accessible from the scope you’re in etc. A really good semantic completion engine will even help you with the function’s parameters.

Oh and we also want our completion system to be lightning-fast. If we’re waiting for it to return a response… well it’s hardly doing its job of speeding us up then, is it?

So, let’s see where we are now with requirements for a completion system:

1. Auto-triggers as you type.
2. Uses subsequence-based completion filtering.
3. Uses smart heuristics to intelligently rank whatever candidates survive the filtering step.
4. Offers semantic completions.
5. Fast, fast, fast!

clang_complete fails everything except #4 (for C-family languages), which it excels at (it uses libclang, much like YCM). IMO it also fails #5 pretty badly because it’s both slow (lots of Python logic, whereas YCM only uses Python as glue code) and it blocks the GUI thread (something YCM does its best to avoid¹).

ACP as mentioned fails everything except #1. Neocomplcache can be configured to provide #1 and it kinda sorta sometimes does a half-decent job at #4. I’ve used neocomplcache for over eight months and my frustrations with it directly led to the decision to build YCM.

SuperTab I have limited experience with, but from what I understand, it does a half-decent job at #4, much like neocomplcache.

YCM fulfills all of these requirements. #4 is currently only fulfilled for C-family languages, but YCM has an internal Completer API that is designed so that completers for other filetypes can easily be implemented and hooked-up to the rest of the system. A Python semantic completer will probably be built some time in the future,² and others can help out with completers for other languages. In the mean time, there will probably be a “generic” filetype completer that queries Vim’s omni-completion system for completions if a specific completer has not yet been written. YCM would provide the other components on top of the results coming from omni-complete. The “generic” completer hasn’t been built yet but it’s on the roadmap.

Things We Can’t Build in Vim yet but Would Like to

I can’t show a box with help text related to the function right next to the completion menu in Vim. There’s just no API for that. libclang can actually extract doxygen information from function comment blocks, but there’s no way to spawn such a “sub-window” arbitrarily on the screen in Vim.

The best Vim offers is the “preview” window that can show up at the top of the file.³ YCM supports this, but it’s just not the same. Your eyes have to leave the small area of the screen where you’re currently typing and on a 30 inch monitor this is annoying.

So no, we can’t build the full VS experience. I’ll take what I’ve built so far though.

Closing Words

I’ve been building YCM on-and-off for a year now in my spare time; I’d guess about 4 months of work went into it. It’s not a terribly complicated system to build, the main causes of slowdowns were bugs in Vim I encountered while building this. Those were incredibly annoying and put me off the whole thing for months.

But it’s reached a very releasable state. I’ve been using the identifier completer for roughly eight months now and the semantic completer for about two months (with C++ code). In my experience, they’re both rock-solid. I’ve had co-workers bang on YCM and lots of bugs were fixed as a result; hopefully it will be pretty stable for a wider audience as well.

Enjoy the plugin and feel free to report any issues you encounter!

[TL;DR: For the impatient crowd, go read The Four Pillars of Investing right now. For the super-impatient, here’s a grossly inadequate four-word summary of modern personal investment advice: low-cost index funds. Don’t have a clue what I’m talking about? Read on.]

I’ve been fascinated with Financial Mathematics ever since I took a (gruelingly difficult) class in it in graduate school. Since then, I’ve read a few more books on the topic. It remains the only field of Mathematics other than Discreet Math that truly interests me.¹ But these are merely hobbies; I hold no delusions of grandeur in my abilities in these areas.

Knowledge of Financial Mathematics prepares one for investing in much the same way that Computer Science prepares one for Software Engineering. In other words, not at all. There are many, many reasons why investing is an altogether different beast. Those of you who have read more than one Financial Math university textbook, quick, how many references did those books make to taxes and investment costs?²

So FM textbooks are naturally out. There’s a ridiculous number of investment books out there, most of which are less than stellar. Some of the good ones are also out of date, and a lot of them are tailored to institutional investors. I’ve picked out a nice selection of sage advice for the personal investor in the following list; I’ve read all of the listed books and quite a few I’m not listing. None of the books I’m recommending should be controversial and I’m not affiliated with any of them; there are no referral links here and I have no skin in this game.

There’s a very good reason why I’m giving you a list of books to read instead of just summarizing the great advice that’s in them: never take investment advice you don’t understand. You need to understand why portfolio X makes sense, why investment Y doesn’t, why actively managed funds underperform etc. You need depth here, not superficial “I think I get it” understanding. Your financial future depends upon you fully grasping these concepts.

The Basics

These are must-read books. I’m presenting them in a sensible reading order that will slowly teach you more and more, building on what you have previously read. None of them are difficult reads; in fact, all of them are surprisingly well written and entertaining.

• The Only Investment Guide You’ll Ever Need by Andrew Tobias: Yes, I know, the title is terrible (and patently false). To the author’s credit, he apologizes for it in the preface and says it was the publisher’s idea. But no matter; it’s a really good book.

  It covers a very wide area and spends a considerable amount of time on Personal Finance. If you’re young and just out of college (or soon to be), this is where you should start. If you think you have a firm grasp of Personal Finance, then you should definitely start here. This book also describes in detail many different investment vehicles (with pros & cons of each); it also covers planning for retirement, differences between the various retirement accounts (401k/Roth 401k/IRA/Roth IRA), annuities, insurance, tax laws, estate planning, ESA’s, 529s, SEP’s, SIMPLE’s and the kitchen sink, too. Seriously, start here.

  The book is also well written and remarkably funny. You will enjoy reading it.

  Make sure you get the latest edition of the book. It’s been out for nearly thirty years now, but the author updates it every three or four years as the tax and investment landscape changes. The latest editions have little in common with the earlier ones.

• The Four Pillars of Investing by William Bernstein: This is a brilliant book and the one I recommend the most out of this bunch. It does require the reader to be comfortable with math to a minute extent.

  It will teach you how to think about investment returns and it will provide a brief history of investment folly. It will provide mountains of data on how (and why) actively managed funds fail to beat low-cost index funds and why individual stock-picking is a loser’s game. It will teach you how to tailor an investment portfolio that matches your financial situation (and personal risk disposition) and why you should avoid investment banks, brokers and preferably the entire investment industry, lest it eat up considerable portions of your returns.

  If you come away from this articles with one book in mind, let it be this one. It’s the best and the most important one I can think of.

  I cannot recommend this book highly enough. Instead of peddling vague and anecdotal advice based on some random guy’s experience (which is what most investment books do), Bernstein approaches personal investing as a science. He backs up everything he says with sound data and references to modern investment research. All of his advice is practical and actionable. Read this now.

• A Random Walk Down Wall Street by Burton Malkiel: This book will teach you how to “think” about the market. It will cover pretty much all the stock market bubbles throughout history and what caused them. This will help you recognize similar craziness in the future. It’s also chock-full of more proof on why actively managed funds underperform index funds and why stock-picking is something only monkeys with darts do. It will teach you about the Efficient Market Hypothesis and why it still makes sense after all these years (well, the semi-strong variety at least… and it makes sense the vast majority of the time, except when it doesn’t, for reasons that I will not go into right now). This book is absolutely worth your time.

After reading the Guide, the Pillars and the Walk, you can confidently stop. This is my three-book list for investment newcomers. After completing this book bundle you’ll know more about investing than most “professional” money managers. If you implement the advice in these books, your returns will also beat the returns of ~80% of actively managed funds, over the long term (20+ years).³

If you can read only two, go with the Guide and the Pillars. If just one, then the Pillars. But preferably all of them. That’s it, you can stop reading this article if you wish.

Greater Appetite

If you want to know even more about sound investing, here are some other books I’d recommend:

• The Intelligent Asset Allocator by William Bernstein: This was Bernstein’s first book. It’s very good, but Pillars covers everything that Allocator does and much more. This book goes into more detail about asset allocation, but frankly Pillars gives you all of the really meaty parts. Allocator will go into more data, graphs, proof and math on why the things Bernstein recommends in Pillars make sense. A worthwhile read, certainly, but if you’ve read his second tome you’re not going to get that much from it. Read it if you want some extra in-depth data on asset class behavior, otherwise skip it.

  Some people will tell you that understanding this book requires knowledge of advanced mathematics. Nonsense. It’s certainly more math-heavy than Pillars, but if knowledge to understand it passes for “advanced math”, someone please shoot me now.

• Common Sense on Mutual Funds by John C. Bogle: Mountains upon mountains of data on why index funds beat active funds in the long term. Truly, mountains. The biggest complaint is that the book is extremely repetitive; the same message is repeated over and over again. And then some more. And did you know that index funds beat active funds? They do. You didn’t know that? Fine, here’s evidence item #892374984.

  After a time, it tends to grate a bit. But convince you of its message it will.

  Also, Bogle famously shares some disdain for foreign market diversification. He’s an absolute giant of finance and this is his only major flaw. The book is still a worthwhile read.

• The Intelligent Investor by Benjamin Graham: This is Warren Buffett’s mentor. If you want to know more about the stock market and what makes certain companies better stocks than others, read this. But keep in mind that you shouldn’t be buying individual stocks at all. Index funds were invented after Graham died; had they been around when he was, he would have probably been all over them. But this book will broaden your horizons if you read it, so it’s not a bad read, even if you don’t end up using the knowledge in it.⁴

Uncommonly Recommended

Here are some other good books on investing I don’t usually recommend (they make sense, but the alternatives are better and cover the same areas):

• The Investor’s Manifesto by William Bernstein: This book is a dumbed-down version of Pillars. If you’ve read that book, ignore this one. It offers virtually nothing new, merely a rehashing of Bernstein’s old (still perfectly valid) advice without the data backing it up. Read this book if you’re scared of graphs, charts and tables. I recommend it only to people who can’t handle Pillars because they’re frightened of the most basic high-school level math.

• The Bogleheads’ Guide to Investing by Taylor Larimore, Mel Lindauer and Michael LeBoeuf: This is a pretty good all-around book on investing and retirement planning, but Pillars + Guide is a better option. There’s no bad advice here.

Investing and Finance, Broadly

And now that we’ve reached the end, here are some books that may not teach you much about investing, but will teach you plenty about the specific areas of finance that they cover:

• Liar’s Poker by Michael Lewis: If you ever wanted to know why brokers and investment banks are the biggest enemies of their customers, read this book. It chronicles Lewis’s life as a broker for Salomon Brothers in the ‘80s. It’s not pretty (but it is a fun read). Little has changed since then; the finance industry may have become just a shade⁵ more unethical. Keep in mind, the book starts out a bit slow.

• The Big Short by Michael Lewis: Another great book from Lewis, this time about how and why the 2007 recession started. You get to learn just how bat-shit crazy with greed Wall Street can become. If you ever wanted to know “the real deal” behind the Great Recession, read this. It’s a peculiar mix of laugh-out-loud funny and blow-your-brains-out depressing.

• When Genius Failed by Roger Lowenstein: Covers the fall of the Long Term Capital Management hedge fund in 1998. You may have heard of this event, it damn-nigh brought the financial industry to its knees (much like the recent recession in fact did). The fund had derivative positions with a notional value of ~$1.25 trillion, and had to be bailed out by the Fed of New York. Does this perhaps sound familiar? We never learn.

• Buffett: The Making of an American Capitalist by Roger Lowenstein: A biography covering Warren Buffett. Very well written. It’s an entertaining read with sensible investment advice sprinkled throughout.

Parting Advice

If you’re confused about personal investing even after reading the books in the Basics list, a great place to ask questions is the Bogleheads online forum. It’s a collection of some of the nicest and most investment-savvy people you can meet. Their wiki is also filled with great advice.

Happy investing!

[For those who want to skip to the end, the plugin I built is called MatchTagAlways.]

Sure enough, I wasn’t the only one who noticed that Vim was missing this awesome feature. A kind soul even took it upon himself to implement said missing functionality as a Vim plugin that does exactly what it says on the tin, and does it well.

But you know what? I want more than that.

I don’t want Vim to “just” highlight the other tag in the pair when my cursor is inside the tag, I want it to always highlight the tags that enclose the cursor. I’ve seen Sublime Text 2 trying to point out the enclosing tags with a dotted line, but I’ve also seen it fail in the face of unclosed tags common in HTML5 syntax.²

Here’s an animated GIF of what I’m talking about:

Do you see how the enclosing tags are always highlighted, even when you are inside the element’s content? That’s what I wanted.

So I wrote a plugin that does it. The GIF you just saw is MatchTagAlways in action. It’s plug & play, comes with sane defaults and should require no configuration (although there are knobs to tweak if you want to do that).

Implementing this is actually surprisingly tricky to get right if you want to support use-cases like templating languages and the aforementioned HTML5 syntax. There’s a ridiculous number of corner cases to handle, and MTA implements far too much of an HTML parser for my liking.³

Anyway, use it if you edit HTML or XML if this looks like a feature you’d like to have. Note that your copy of Vim needs to come with Python support. More details can be found in the plugin’s docs.