mon.im

Using the Tatacon to USB converter with a PS4

2018-12-13T22:30:00+00:00

This post has been contributed by Blackkat101, with my thanks.
I do not own any of these devices but this information is invaluable to those who have them

As it so happens, the Tatacon to USB converter works with these commercial USB controller adapters:

These adapters can convert standard USB gamepads and keyboards to work with PS4, Switch and the XBox. Please note, native Switch firmware is coming to the Tatacon to USB adapter. If you are patient, it could save you some money.

ChronusMax is the original and has the most scripts out there to use (as it’s normally used for using one controller on another system, or creating macros/hotkeys/rapidfire/more). The TitanOne is a clone of the ChronusMax in literally everything, from the looks to the software (seriously, they just copy pasted everything….). The TitanOne hasn’t been around as long, thus there are not as many scrips in it’s library, so if you have one, you’ll be mainly making your own, or copying and (slight) converting them to work on the TitanOne from the ChronusMax library.

The software for the ChronusMax+ (and transversely, the TitanOne) is suuuper easy to use. Allowing you to code in normal programing language to a super simple idiot proof tile mode that is drag and drop and one with no programming knowledge whatsoever can use it to make their controllers do whatever they want. The TitanTwo is the newest device, having only been out for a year or so. For some reason it uses new scripts, so you need to convert TitanOne scrips if you want to use those. For some reason, they removed the idiot proof tile programming ability too…

NOW onto how it works with the Taiko USB mon made.

For the ChronusMax+ (and TitanOne), you need to plug this device into both a computer and your console. Thus something like a laptop (or a tablet that can run Windows/Mac standard software) next to your system and then the drum (treated as a keyboard) would then be plugged into the PC device.

The TitanTwo (much to my sadness) wins out because (while being a larger device, looking like a large plastic box while the ChronusMax+/TitanOne is a small USB device) it does not need a computer to go through when doing the keyboard. Do note that any other type of controller does not need a computer pass through for controllers. Just that the ChronusMax+/TitanOne need it for Keyboard and Mouse while the TitanTwo does not.

The Titan Two does cost more money though, so keep that in mind. It is also a partial device that has lots of expensive upgrades to get the most out of it.

ChronusMax+ costs normally $60 (when not near a holiday, for some reason when I checked right now, people want $100 for it on Amazon…. ouch…..) TitanOne costs usually $55-60 (so about the same, often times it is a little cheaper than the ChronusMax+ because of it being considered a cheap knockoff. It is the only one I don’t own personally though…)

TitanTwo costs around $90 for the base unit. If you want the Bluetooth adapter, you need to spend another $40, but at least it is an option for wireless play that the previous two don’t have. You will also want to buy an AC adapter if buying the Bluetooth adapter or using cables that are longer than 3-4ft or you will get lag from a lack of power. The AC adapter is a generic one (make sure you get the right specs) and will cost you around $15-25 for that.

I tried multiple other devices with mon’s USB Taiko adapter and it did not work with things like from 8bitdo or the Magic-NS from Mayflash. It does again however work with the previous mentioned 3 devices. I did buy all those and personally tested them all out. I am currently using the TitanTwo (even though I love my ChronusMax+) because I do not need to have my laptop next to the system. It did take a bit to get the controls right and then tweaking the in game settings for timing.

I personally set it up as so: (This is the Switch set up, but it’s pretty much the same on the PS4)

Left Kan = Z (default) = ZR (on switch pro controller)
Left Don = X (default) = Down D-Pad (on switch pro controller)
Right Don = C (default) = A Button (on switch pro controller)
Right Kan = V (default) = ZL (on switch pro controller)

As you can see, I didn’t change the default 4 keyboard buttons that were set as it didn’t really matter what they were. I used the ZR and ZL (those are the triggers) instead of the R and L (those are the bumpers) as the ZR and ZL allow you to scroll through the song lists and difficulty, like hitting up and down on the d-pad.
Right Don set to A for selecting enter (obviously needed).
The left Don, I really wanted set to the B button and it would still register as a Don hit, but it would register in game as hitting the right side of the drum, which felt odd…. So even though I don’t have a way to go back with the drum, I set it to the down arrow on the d-pad (any arrow would have worked), so that it registers right when playing.

I have the in game controls set to register as Pro Controller setting 2.

I have a Switch Pro Controller also plugged into the TitanTwo (need a USB hub if using a ChronusMax+ or TitanOne) for controller authentication. This controller is then set next to the drum. Turn the rumble off so the controller doesn’t vibrate on the table the drum is on. Since you’re not holding the controller, that rumble on a hard surface is loud. This controller can still be used like normal, so this allows for Pausing and a B button to go back (but since I generally never need to do either of those things, I then rarely have to touch that and can go through the song list, pick my difficulty and play with just the drum.

I found that (so far) setting the timing of the on TV settings for the first bar to be -10 and the second bar to be -5 (the third bar left alone). Not near my Switch so cannot check what the names of the exact settings are but that’s what they’re set at.

So now you know that though doing all that you can use your Wii Taiko Drums on the PS4, Switch or any other system.

Pocket Voltex production update

2018-08-10T10:17:00+00:00

Here’s the latest and greatest progress update - if you’re signed up to the mailing list, you should have got this in email form too!

TL;DR: I have to fix some design aspects and produce advertising materials and we’re good to go.

Here’s a picture of a pre-production unit

What’s new?

Hotswappable switches
USB C
30% bigger FX buttons
Straight FX buttons
Stabilisers for FX (no more “hit the edge, it’s mushy”)
Wider knob design (closer to real muscle memory)
Artwork is now a paper cutout - make your own waifu art!
PCB is now designed using KiCad, making the Pocket Voltex even more open!

What’s blocking the next batch

The paper cutout was the wrong size
The paper cutout lets through way too much light and the LEDs are far too obvious
Still waiting on new box designs
No single supplier has enough encoders (for knobs) for a production run. I hope I don’t exhaust the global supply…

What are you doing to fix it?

Have contacted my manufacturer to rectify the cutouts
Have ordered both white and mirrored acrylic to explore more reflective layers
Once these are confirmed it’s a moment’s decision to move to the new material
Boxes should be here in a week

When can I give you money?

Best estimate? 3 weeks. You’ll receive an email if you signed up!

Will I get one?

There’s currently 3x more people signed up for updates than I plan to produce boards for. It’s quite possible you might miss out. What the hell, mon!? Produce more boards! If there are unforseen issues with this batch, I will bankrupt myself. If you happen to miss this batch, please await the next - once ~30% of orders from this batch are delivered and I confirm they work fine, I’ll order a lot more to fulfil demand.

It might be a while, but I don’t have any plans to stop producing these until they stop selling. Thank you everyone for your continued patience while I work out how to mass manufacture my designs ❤.

Unravelling Konami’s Arcade DRM

2017-12-03T00:00:00+00:00

It’s well known that Konami doesn’t have complex protection on its arcade systems. Programmed World was a fan-made network replacement and update service for Konami arcade games with no support outside of Japan. Nothing of its scale and quality had been seen before, and nothing has been seen since, despite other franchises on other systems having similarly impressive userbases. It existed for so many years, pushing day-1 data updates where no other system came close. Surely that means the crypto is simple? Well, it’s certainly not a walk in the park, but I’ve managed to completely reverse the system.

Background: I consider myself a proficient low-level programmer, but my reverse engineering skills leave a lot to be desired. However, I’m persistent! Despite my techniques being inefficient, it took me about 1.5 months to reverse Konami’s crypto, working in my spare time.

I’ll be focusing on two things for this adventure - my process in reversing the encryption, because by sharing my methods I may help others. Secondly, high level details of Konami’s dongle-based encryption, because it’s rare to find proper information on actual, in-the-wild implementations of USB dongle based security.

Oh, and before you get your hopes up - I couldn’t find any obvious flaws. As far as I can see, there is no way to decrypt data without the private keys stored in the dongle.

Let’s dive in!

Obtaining the hardware

First of all, how’d I get my system? For me, it’s not really a “system”… If you hang around Yahoo! Auctions and rhythm game communities, you’ll eventually see people selling game HDDs along with security dongles (a good start is searching YAJ for ドングル, Japanese for dongle). I waited a while and PM’d some people, and managed to score a Reflec Beat HDD+Dongle combo from 2011, and it only cost me $50!

Without the arcade machine attached, it might not boot far enough to decrypt the game. For $50 I was willing to take the risk.

Eventually they arrived:

The dongle looked far too big for what it is, so I took a guess and pried open the outer shell:

Much easier to fit in my hub. An amusing note - the large shell says “Made in Japan”, the actual dongle is “Made in China”.

After making a HDD backup, what’s on it? Zanneth does a far better job explaining it here. Replace LDJ with KBR (Reflec Beat’s game code) and that’s essentially what’s on the drive.

Firing it up

Next, can I boot the game through a VM? Through a surprisingly laborious process of GParted and SATA->USB adapters, I got the game into a VMWare image. It boots!

And gets to a hardware error screen:

Presumably this is my lack of game IO. I couldn’t find anything remotely like this error in the unencrypted files, so I guessed it’s getting far enough to decrypt. A good fallback!

Running natively

However, I loathe Windows XP Embedded, even with remote debugging tools - running a VM seems so wasteful. Time to see if I can get it running on Windows 10!

By investigating the various startup .bats in the system, the magic happens inside bootstrap.exe, inside the game directory’s modules folder. It contains a few dlls, along with 9 (!) encrypted DLLs. Most of them all start with 0x00, then appear entirely random. Two have a different system - kdm.dll and kws.dll both have plaintext ‘test’ along with some structured data, but quickly turn random as well.

Decoding these headers is what we’re here for!

Bootstrap is given some file arguments - trustcerts.p7s and trustfiles.p7s. These are ASN.1 encoded certificates, with some extra encrypted binary data - a list of trusted system files and their MD5 hashes, to prevent tampering. It also gets a configuration xml (encrypted) and a configuration to run.

reflec> bootstrap.exe ..\prop\bootstrap.xml env_drm ..\prop\trustcerts.p7s ..\prop\trustfiles.p7s
M:boot: bootstrap 1.2.5  build at 2010-09-16 10:43:37+9
F:boot: Unable to initialize the ikey service.

Clearly this isn’t booting as far as Windows XP did. Time to fire up IDA, and holy debugging messages:

Throughout this project, my work was simplified by the thorough, plaintext debugging messages sprinkled judiciously through Konami’s code. I still had to reverse all the low-level details, but high level functionality was very easy to discern thanks to these error messages.

So, bypassing the ikey error… Going into the function, it appears to be restarting services related to the dongle, presumably to get a clean slate. I didn’t have dongle drivers yet, so I made the function return success.

M:boot: bootstrap 1.2.5  build at 2010-09-16 10:43:37+9
F:boot: Unable to initialize the pki world.

This error is thrown by a function called before the ikey initialiser, so what’s up? Seems like some sort of security check - I did just tamper with the exe, after all. On a whim, I used a combination of CreateProcess and WriteProcessMemory to patch the code in memory. I put in a pause to give me time to attach a debugger, and off we go~

Applied 1 patch
Press enter to continue...
M:boot: bootstrap 1.2.5  build at 2010-09-16 10:43:37+9
F:boot: Unable to initialize security process.

We got further, looking good! In the end I made 3 more patches - one to enable verbose logging (why this was left in is beyond me), and two to bypass signature checks the game loaded from trustfiles.p7s. They were easy to implement, since the game prints helpful messages when files fail validation.

A small debugging hiccup came in the form of kbd.exe (I think it’s short for Konami Bootstrap Debugger) - this file attaches a debugger to bootstrap.exe, which fails if I’m already debugging it. If kbd quits, so does bootstrap. As an experiment, I made the most basic replacement for it:

#include <stdio.h>
#include <windows.h>

int main() {
    printf("I:adb proxy launched\n");
    while(1)
        Sleep(10000);
    return 0;
}

This actually worked :D So, where do these patches get us?

E:boot: Unable to load bootstrap.dll: 0x00000003
F:boot: Unable to run bootstrap.dll.

A brick wall of missing files

0x03 is the Windows error code for ERROR_PATH_NOT_FOUND, so it’s time to fire up one of my favourite tools - Process Monitor. If you’ve never heard of it, it logs filesystem, registry, thread and some network activity. It captures a stackdump for every event, which is super useful in finding where files are loaded.

With Procmon, I can see bootstrap loading several encrypted DLLs along with (presumably) their system DLL dependencies. However, just before exiting, bootstrap fruitlessly checks my entire PATH for api-ms-win-core-processenvironment-l1-1-0.dll. It turns out this is a part of Microsoft’s new “API Sets” feature introduced in Windows 8. These are “virtual” DLLs, in that they don’t really exist on the filesystem, but LoadLibrary sorts them right out. For their DRM, Konami appears to have overwritten LoadLibrary with their own version, which breaks this system. This makes sense, as calling LoadLibrary on an encrypted DLL doesn’t return an error - it pops up a MessageBox, which is very programmer unfriendly.

This path filter needs patching, but it was beyond me at the time. Bootstrap already loads several encrypted DLLs, such as kbt, ltsme, and libavs-win32. We can ignore the path issues for the moment and get these DLLs decrypted. I took the CloseFile event for kbt.dll (the first DLL loaded), put a breakpoint in the function that calls it, and began my slow reversing process.

First stage decryption

So what do I know? I know the file has just been loaded. I know that all DLLs start with the magic characters “MZ”. So, I fired up Cheat Engine and searched for “MZ”. There were some libraries loaded, so I got 4 results. I then began stepping through the code after the CloseFile call. I then repeated the search. When another MZ appears in the list, I know the function I just stepped over performs decryption. I can then break on this function, restart the program, and repeat the process until I get to the meat of the algorithm.

This isn’t very efficient! To keep track of things, I also named functions as I drilled down. This results in quite a soup of names as I go deeper and deeper.

However, in the end I managed to work out this “stage 1” crypto. First, I found the function creating the first block of decrypted data. Cryptographic algorithms are very obvious in disassembly, because they’re full of bitshifts, XORs and other bitwise ops.

By looking at the constant arrays referenced in this code, I found AES constants. Alright, so we’re using AES! Moving back up the functions, I found the AES key and what appeared to be an IV. I could also see the first decrypted block inside the encrypted file, and it wasn’t at offset 0. Now I knew how long the header was. I made a quick Python script using PyCrypto, and attempted to decrypt kbt.dll. And it worked… almost. The first block was decrypted, but the rest was garbage. Moving back up, the IV was being modified in a very confusing way - it was split into 4 ints, then the last one was being mangled:

After 15 minutes spent trying to decipher this, I simply translated the code into Python and iterated infinitely. Turns out, this is how you perform big-endian addition without actually converting the number to little-endian. An odd way of doing it, but perhaps a compiler optimisation. It turns out they’re actually using AES_CTR, but instead of incrementing the entire counter IV, they’re only using the last 4 bytes. A little strange, but if your encrypted file is 2³² * 16 bytes long, you have other problems to worry about.

The final step was to find how the encryption key was generated. For this, I repeated my “Cheat Engine binary search”(TM) method, waiting until keys popped into memory. This time I found even more RORs and ROLs along with XORs, and again some data tables - googling some constants, this was SHA1! So far, so good.

It turns out the encryption key comes from hashing parts of the file header, but I was missing something critical. The internal state in the SHA1 structure wasn’t matching the standard initialisation values (0x012345678 etc) plus the buffer was half-filled with a great deal of “PADDINGXXPADDING”, which is inserted into binaries by compilers. I could check what I knew so far, by finding a pure-Python SHA1 library and meddling with its internal state. I set it to the state I found, hashed the stuff I did know… and it gave me the right key! So it’s half solved, but where did this state come from?

Again I repeated the Cheat Engine technique, until I found this state in memory. I’ve probably restarted the game a few hundred times by now, it’s really not a fast way of doing things! This brought me to a very long function full of loops and conditionals making lots of calls to the hash function I encountered before. By inspecting the locations and sizes being passed in, I eventually realised the code was parsing the PE header of bootstrap.exe, and hashing individual sections! The “PADDINGXXPADDING” lives at the end of bootstrap.exe, which explains why it was left in the SHA1 buffer..

This is a very clever way to prevent tampering - if bootstrap.exe is modified, it can’t decrypt anything as the keys will be wrong! By reading up about the PE header, and thanks to the lovely pefile library, I finally got myself a standalone decryptor for “stage 1”. This includes kbt, kdu, kmp, libavs-win32 and ltsme, as well as the configuration files bootstrap.xml and avs-config.xml. It also worked on the binary blobs inside trustfiles/trustcerts, but the decrypted contents are an unknown binary format - all I could glean was the names of the trustfiles checked.

Second stage decryption

This left kdm and kws to reverse, and the game files of course. But before we start, time to look at our newly decrypted DLLs. Inspecting them, kbt is what the game refers to as bootstrap.dll, and actually executes the game. kdu probably stands for “Konami Dongle Utils” and communicates with the dongle drivers. kmp seems to be some sort of memory pool. libavs-win32 is a general purpose library containing a ton of filesystem, logging, memory management and utility functions.

ltsme.dll is a little more interesting - it’s structured far differently from all the k__ DLLs, and contains many strings referring to different crypto algorithms, as well as source file references for throwing errors. Googling these source files reveals it’s probably the RSA BSAFE library - certified implementations of many common cryptographic algorithms. This makes sense, as every Konami game I’ve seen has a splash screen proclaiming:

All of the k__ DLLs have their function exports obfuscated, of the pattern <dll_name>_<n> where n is the ordinal of the export (kdu_1, kdu_2 etc). Thanks again to Konami’s gratuitous debugging, a lot of these functions can have their names reversed anyway. libavs thankfully lacks this obfuscation, which makes it easy to see it’s a utility library:

The pesky path filter

Before we can load the remaining DLLs, it’s time to look back at the DLL loader’s conflicts with API sets. Thanks again to Process Monitor, I know the function that’s making the search:
At first, I tried ignoring the conditonals and always returning success. However, the LoadLibrary replacement is recursive, and loads dependencies from the DLL it finds. By stubbing the function, I simply get null pointer dereferences. Naive! By moving only 2 functions up the call stack, I found my saviour, which in hindsight was very obvious.

To avoid infinite recursion from circular dependencies, bootstrap checks to see if the module is loaded using GetModuleHandle. If I just force Windows DLLs to load using the real LoadLibrary, I’ll avoid the API set debacle. Time to test out some new skills and write a DLL injection library! There’s tons of literature on the web to teach this, and it didn’t take me very long to implement. Before running bootstrap, I save the real LoadLibrary before it’s tampered with, and hook GetModuleHandle to include my extra checks.

int WINAPI my_GetModuleHandleW(LPCWSTR lpModuleName) {
	auto ret = orig_GetModuleHandleW(lpModuleName);
	if (ret) {
		return ret;
	}
	printf("Loading module %ls\n", lpModuleName);
	if (wcsstr(lpModuleName, L"api-")) {
		printf("Windows library, preloading\n");
		orig_LoadLibraryW(lpModuleName);
	}
    // return newly loaded module
	return orig_GetModuleHandleW(lpModuleName);
}

Hacky, but it worked! Now where are we up to?

W:boot: Dongle is not connected.
I:boot: Entry loadlibrary (p7e)
I:boot: DRM module is about to be loading.
W:boot: ERROR:don_manager_slot_license
W:boot: unable to load 'kdm.dll'

Ding dong(le)

Time to use the security dongle! First up, our prime suspect for analysis - kbt.dll. It initialises a lot of our newly decrypted libraries. Importantly, it has references to dongle driver DLLs (through kdu), so we install those drivers. The dongle is a Safenet iKey 2032, and can be treated as an encryption black box containing a secret key. The dongle library thankfully follows a well documented standard called PKCS11, which greatly simplifies login, decryption, and session management.

kbt calls kdu functions in so many places, and kdu calls dongle functions in so many places, that manually stepping through and finding calls is going to be a huge pain. I’ve just learnt how to inject DLLs and hook functions, so why not put this to good use? I looked at the imports of kdu, hooked all of the C_ functions that sounded interesting, and fired up bootstrap once again.

Thanks to my hooks, I could now print the return address every time a dongle function is called. I can also print the arguments and return values, which is incredibly useful. Hooking C_Login led me to the algorithm that generates the dongle PIN. By hooking C_Decrypt, I can print out the data that’s going to the dongle - this makes it super easy to cross-reference with what’s in the encrypted files. Hooks are great!

DONGLE: C_FindObjectsInit(session: 0x09850004, count: 3)
DONGLE: C_FindObjectsInit(session: 0x09850004, count: 3)
M:boot: watcher enabled
I:boot: don_manager_start_lib is running.
DONGLE: C_Login(session: 0x0E150006, u_type: 1, pin: 0xBFA7911976079F6B757A32D69BAA4671, pin_len: 16)

I know I could have used breakpoints with IDC/IDAPython to print information, but I really wanted to learn DLL injection and hooking. kbt.dll also switches from bootstrap.exe’s logging to functions provided by libavs-win32.dll. There is a debug flag to enable here too, so it’s easier to just inject.

When attempting dongle login, if you enter the incorrect password 10 times, the dongle will permanently lock you out. Luckily I didn’t find this by accident: the dongle drivers let you know how many attempts remain. One advantage of dynamic analysis meant that I could test my PIN generation algorithm, fail a few times, and run bootstrap.exe again to let the game perform a correct login. This is a useful safety net when failure results in a brick!

The dongle PIN turned out to be reasonably simple - a salt appended to some identifying info from the dongle, hashed with SHA1. To avoid errant null terminators from the hash, each successive \0 becomes \1, \2 etc etc… However, in newer versions of the dongle drivers these PINs aren’t always valid. The spec says they have to be UTF-8 encoded, and a SHA1 hash sure doesn’t guarantee this. I changed my dongle’s PIN to “helloworld” to perform some tests, but attempting to change it back resulted in an invalid length error. I had to boot up the old XP VM to change the PIN back, as it doesn’t care about bad characters.

Ok, time to inspect kdm. Why kdm and not kws? Well…

Great function name obfuscation guys, really effective.

The hex dump of kdm.dll reminded me of the trustcerts file. Could it be ASN.1?

Brilliant! By hooking more C_* functions related to encryption, and cross referencing data with the ASN.1 dump, “stage 2” became 100% obvious. I never had to look at disassembly for this stage, because the dongle calls told the entire story.

Both key and IV were extracted directly from the ASN.1 blob. The key was decrypted by the dongle and its private key (using RSA PKCS), then the pair were used with the dongle’s AES mechanism to decrypt the DLL. Oddly, the dongle’s AES is identical to standard AES, and I was able to perform the decryption on the PC side. Limited by USB 2.0, the dongle ends up slower, so I wonder why Konami chose to perform this decryption on the dongle. I guessed they will move to a more performant encryption for the real game, since those files are significantly larger. Read on to see if I was right :)

Final stage decryption

With kdm and kws decrypted, it’s time for the final frontier! Decryption of the actual game, not just the bootstrapping DLLs.

Konami’s extensive debug string usage was once again a huge help here - before I even started reversing specific functions I knew exactly what they should do. Wouldn’t every reverse engineer wish for something so nice?

Path obfuscation

The hardest part (for me) didn’t turn out to be the decryption, but the path obfuscation. The actual game files aren’t as conveniently named as the dlls in the modules folder.

KBR/contents$ tree
.
├── 0
│   ├── 1
│   │   ├── 1
│   │   │   ├── 5a7ddaa7bcaca08f81911ad290be72f9e14c1
│   │   │   └── 77c12c396141778ba39762dcda195807dc76b
│   │   └── f
│   │       └── be3ff2c203d690006af3a243f5e78f35d965a
│   ├── 2
│   │   ├── 1
│   │   │   └── d44f8e863315302ddb81868fa4338f16dcc0d
│   │   ├── 5
│   │   │   └── 07c708612ae712921e83cb2db48dcf0b709b0
│   │   └── 6
│   │       └── 814976b86d6913c97d615691cdf2db41545dd
│   ├── 4
│   │   ├── 5
│   │   │   └── 26445d335bcb0da2c2d72accdd3b119a46a30
│   │   └── b
│   │       └── db5f902601454b4c4863bab941647f994af7d

This continues for all 600 files in the game. The first file accessed (thanks again to Process Monitor) is eventually traced back to its “real” name: keyfile.dat. Sounds exciting! Through the slow Cheat Engine process, I found the function generating these paths:

I googled this and discovered it was a MAC algorithm. The input was the real filepath, the key/hash/code was more dongle data and a salt. I then wasted many days trying to reverse exactly what this function was doing. I didn’t know anything about MAC, so I couldn’t reliably differentiate RSA’s boilerplate from the real deal. The entire library is dynamic functions and vtables, so static analysis is useless for me.

After much crying and gnashing of teeth I had a good night’s rest and looked back at the main function again. cry_mac.c. One proper search later, some more Python poking, and I realise that mac is short for _H_MAC. Boom - instant success. Lesson: You can step through code all you want, but do try and understand the basics.

Keys to the kingdom

With path obfuscation solved, the very final step is probably the simplest work in this entire saga. The decryption functions are well commented with error messages, easy to follow, and consistent. The keyring was loaded into memory, so offsets into it are simple to understand. The reversing of this algorithm was quite basic compared to the rest, so I won’t go into detail. Instead, for (in my opinion) the most interesting information of all. How are the files encrypted?

The biggest surprise - to decrypt all the files in the entire game, only one call is made to the dongle. I guess at this point, decrypting files (or even multiple file keys) with the dongle would just slow the game down.
So, here’s the process:

The keyfile contains the keys for the game. Every file has a different key.
The “master key” is decrypted using the dongle.
- This master key changes with every single game update (I had some leftover update files on my HDD)
- This is why you always need a dongle!
When a file is loaded, a key for that file is decrypted using the master key
That key is then used to decrypt a second key
THAT key is then used to decrypt the real file.

Phew! Crypto layers on crypto layers on crypto layers! The final question - how does the game know which file belongs to which key? The very first file decrypted, file.xml, contains a list of every file in the game, along with their key offsets:

<?xml version='1.0' encoding='UTF-8'?>
<fileinfo contents_code="KBR-001">
  <dir name="data">
    <dir name="graphics">
      <dir name="cardentry">
        <file name="card.bin">
          <key_idx __type="u32">1</key_idx>
          <orig_size __type="u32">6982428</orig_size>
          <dst_path __type="bin" __size="20">d2b50a2ae7d3392205ead278d70506d85ae9bee0</dst_path>
        </file>
      </dir>

Combining everything into a (now quite large) Python program, I can decrypt my game HDD for the 7 year old ReflecBeat.

> python .\decrypt_all.py 'D:\reflec\game\KBR\contents'
Successful dongle login
Keys loaded
File list loaded
data/graphics/cardentry/card.bin
data/graphics/cardentry/card_2.bin
data/graphics/caution/caution.bin
data/graphics/common/cmn.bin
data/graphics/common/loca_info.bin
data/graphics/common/paseli.bin
data/graphics/font/f0000.bin
data/graphics/gameover/gameover.bin
data/graphics/gameover/info.bin
...
...

What’s next?

With what I’ve learned, I’m confident in purchasing a real arcade cabinet. No arcades in my city have these games, and frequent trips to Japan become expensive ;). New games can’t run offline by default, so maybe I can write my own personal network server and buy a Sound Voltex cab to fawn over. Of course, this would be useless if Konami has changed their system in the last 7 years (very likely!). If I buy a cab and it’s got a new system, I guess you’ll see another blog post :)

I could also start writing emulation tools for ReflecBeat’s IO system and bypass the hardware error to get it playable on a touchscreen laptop. At this point, I don’t have the time. Perhaps a later date!

I hope you’ve enjoyed reading this as much as I enjoyed writing it! It’s been a fun ride.

$70 SDVX Controller

2016-07-22T10:17:00+00:00

In my neverending quest to play more rhythm games, I discovered SDVX/kshootmania. Unfortunately there really is no good way to play these without a controller. An expensive investment, you’re looking at over $200AUD shipped. Not something I want to spend for a game I may not even like!

This is a work in progress. If you find this on Google and it helps you, leave a comment to inspire me to polish it and post pictures when I complete my setup.

Working off this guide, I’ve made several changes…

All prices in AUD, so if you work in USD, prepare for a happy discount :)

Parts list

Leonardo Pro Micro, $6. Search on eBay for a seller. Should look like this:
Sheet of MDF. Cost me $6 at Bunnings. I wouldn’t go under 6.5mm thick otherwise it flexes too much. Also acceptable is plywood and possibly acrylic.
The switches themselves. “9x Beatmania IIDX Video Game DIY Parts LED Illuminated Rectangular Push Button”. $37 from eBay.
Aluminium knobs: $17.50. These are completely overkill but they feel wonderful in my hands. You can probably find much cheaper ones on aliexpress.
Encoders. The ones listed in the pancake guide are wrong! The don’t have threads so can’t easily be attached to the board. I got these instead. $5.20. After using them for a while, these really aren’t amazing encoders. The next step up are 600ppr ones, which are somewhat insane. See the “upgrades” section.

Total: $70-ish (AUD)

I had these parts lying around, but you may need to buy them:

Female header pins (2.54mm)
Male to female jumper cables (listed as “breadboard jumpers” on ebay usually)
150 ohm resistors

Make a box

The arcade templates on Pancake were far too big for my liking - I want this as compact as possible whilst maintaining decent arcade styling. I shortened everything.

The VOL squares are not centered, you will have to mark these with a pencil when you make the cut.

I printed this out on A4, glued the parts together, then used it for my hole drilling.

Here also is the svg file and high res png. I also have a pdf - if you print with Acrobat reader there’s options to tile it onto 2 sheets of A4.

12v to 5v LED conversion

If you can find switches with 5V LEDs already inside, buy them, this will save you some effort. Take out all the LEDs from their holders, desolder the existing resistors, and solder in your 150 ohm resistors. This means we can drive the LEDs directly from the MCU with no extra hardware! Very convenient.

Firmware

Sorry, Pancake man, but your code was bad. I’ve reworked it to run better and be more configurable. Get it here. For best encoder performance, make sure the encoder pins are 0, 1, 2, 3. Any order is fine, just as long as you use those pins. All other pins you’re free to use whatever you want!

Wiring

Common ground across everything, then switch + LED connectors for each button, and A/B for each encoder.

Future upgrades

The switches are insanely heavy. I’ve bought some 50g omrons. Expensive, however! Cost me $37 shipped to Australia. You could probably get big square switches.

The encoders are kinda crap. I found some arcade quality 600PPR ones, 2 cost me $27. Here’s a link, but I highly recommend just searching ebay for “encoder 600 6mm”. I saved $10 by searching and finding the cheapest for my location.

TEVO Tarantula BLTouch Setup

2016-06-19T05:52:00+00:00

The logical next step to improving your 3D printer experience is an automatically levelled bed. I won’t go into specifics, but the BLTouch is awesome - accurate, simple, and works on any surface. No fuss!

This is a setup guide specifically for the Tevo Tarantula 3D printer. It uses an MKS Base board and runs best with Marlin firmware. Let’s get started!

What you’ll need

The BLTouch itself
Servo extension leads (select 60cm male to female)
A way to mount it
- I use the excellent LPA fan duct - there’s versions for stock and E3D hotends.
- If you have the control board mounted on the printer, I advise mirroring the print so it doesn’t collide with the PCB
Marlin firmware
- Go to the “Files” section on the Facebook group, scroll down until you find a Marlin zip. I use this version:
- If you are using RC6, this guide supports that too!
- Copy the Configuration.h from ABL_YES into the Marlin folder

Wiring

IMPORTANT IMPORTANT IMPORTANT
Some TEVO Tarantula boards (MKS Base) have SWAPPED POWER AND GROUND PINS. YOU MUST SWAP THE BLTOUCH CONNECTOR OR YOU WILL FRY YOUR BLTOUCH.

The BLTouch connector has 2 connectors:
The main connector is Signal, Power, Ground (orange, red, brown)
Using a pair of tweezers or a toothpick, carefully bend the plastic keeping the Power and Ground pins in place, pull them out, and swap them.

The new order should be Signal, Ground, Power (orange, brown, red)
Note how they match up to the MKS Board:

One this is done, attach your 60cm servo extension leads to both cables of the BLTouch. Be sure to line up signal with signal. For the 2 pin plug, just plug it into signal and “power” (which we know is ground).

Plug the 3 wire cable into the bare pins on the right - you want the one marked D11. The signal wire will be right next to the label, and brown (actually power since we swapped) on the back.

Plug the 2 wire cable into the Z- plug. The fit won’t be 100% secure but this is fine. Ensure that if you trace the wires, the pin marked “-” goes to the black wire and the pin marked S goes to the white wire.

Sanity check

Make sure the BLTouch, when retracted, is higher than your hotend. You don’t want it crashing before the hotend does!

If this is the case, loosen the grub screws holding the heat break into the heatsink. Drop it a mm or 2 and the BLTouch should now clear.

Firmware

Marlin is best. No question. You’ll want to edit Configuration.h to setup the BLTouch.

Find and modify these variables:

const bool Z_MIN_ENDSTOP_INVERTING = false;

Comment out this line by adding “//” to the start:

//#define FIX_MOUNTED_PROBE

#define NUM_SERVOS 1

If you are on RC6:

#define Z_ENDSTOP_SERVO_NR 0
#define SERVO_ENDSTOP_ANGLES {{10,90}}

If you are on RC7:

#define BLTOUCH

You will need to set your probe offsets. Luckily, the LPA fan duct documentation has a handy table for us. Here are mine:

#define X_PROBE_OFFSET_FROM_EXTRUDER 38
#define Y_PROBE_OFFSET_FROM_EXTRUDER -4

Don’t worry about Z offset yet, we’ll get that perfect later.

You may want to slightly increase the Z_RAISE (RC6) settings. I increased them by 1 or 2mm each. If you are on RC7, these are called Z_PROBE_DEPLOY_HEIGHT and Z_PROBE_TRAVEL_HEIGHT.

RC6 only: For some reason, Marlin doesn’t like pulling the probe pin up after the last probe. This is easily fixed:

#define Z_PROBE_END_SCRIPT "M280 P0 S90"

Personally, I don’t like EEPROM, so I disable it. This is entirely up to you.

//#define EEPROM_SETTINGS

If the code won’t compile and warns you about unreachable probe points, you will need to modify them. For example, I had to change LEFT_PROBE_BED_POSITION from 30 to 38.

Calibration

BRING YOUR Z VERY HIGH FOR TESTING
You must give yourself time to abort a test if you set things up incorrectly.

Connect your printer to your computer and start executing some GCode:

G28 X0 Y0; Setup X and Y
G28 Z0

This will begin endstop testing with your BLTouch. Using your hand, manually stop it far above the bed to ensure it works. If it does, then you can proceed:

G29 ; autolevel sequence

Keep a very close eye on the first autolevel - if your offsets are wrong, there is a chance you will run off the bed into air, and crash into the bed.

If this succeeds, great! Now time to calibrate the Z offset. Using G1 Z{distance} and a piece of paper, move down until you get a decent amount of friction on the paper. Same as manual levelling!

For example, I ran G1 Z0. This far far too high, so I went G1 Z-1. Getting closer, I started going down in increments of 0.1, so G1 Z-1.1 then G1 Z-1.2 etc etc until I found the perfect number.

Once you have this number, enter it as Z_PROBE_OFFSET_FROM_EXTRUDER in the Configuration.h

You’re almost done!

GCode

Finally, you’ll need to modify your GCode to perform an Autolevel sequence (G29) before each print.

Simply find where you have G28 Z0 and put G29 after it.

Evoluent VerticalMouse C Review, teardown, comparison

2016-03-17T11:55:00+00:00

Computer mice haven’t really changed since they were invented. It’s still a block you move about with your hand in a flat position. However, this twisting of the forearm is unnatural, and frequent mouse users will find they develop wrist pain.
The solution is a vertically oriented mouse. It lets the hand rest in a natural state and in my experience is super comfortable. These are readily available for under $30 on eBay, Amazon and Aliexpress, but are often childsize or simply not comfortable enough. After trying about 4 mice I’ve realised who the reigning champion is: Evoluent.

I’ve been a happy owner of their VerticalMouse 4 (VM4) for almost a year now. It’s comfortable but a little awkward, and required some modding to really make it good. I’ve been using it interchangeably with my Razer DeathAdder, but definitely use the DeathAdder more.

Released in February 2016, the Evoluent VerticalMouse C (VMC) is the company’s next iteration of their flagship product. It comes in wired and wireless versions, and is right handed only for now. If it’s anything like the VM4, a left handed version will follow.

I purchased the VMC Right Wired for $150 AUD shipped from B&H to Australia.

But how does it stack up against the previous VM4? Is it good enough for gaming? Read on to find out…

Look

Evoluent has really leapt into this decade with the design of the VMC. The curves are beautiful and complement the ergonomic shape. It practically looks like a spacecraft now, and that’s a good thing!

Compared to the VM4, it’s a huge upgrade.

Compared to my DeathAdder, it’s quite a difference.

The bottom of the mouse is similarly simple, with only the sensor hole, signature, and feet.

You can also find promotional shots of the mouse on Evoluent’s page.

Feel

The VMC has again come into the modern age by replacing the silver plastic with a lovely matte finish. It’s that slightly rougher plastic that you see on mice like the DeathAdder, and it’s very sweat resistant. My VM4 is covered in dried sweat (yuck!) but I don’t see this being a problem with the VMC. The thumb area is the same material, a welcome change from the gloss finish on the previous versions. The silver gloss is now reserved for the side of the mouse you don’t touch, and it looks great.

The cable isn’t braided, but it’s not standard either - it has a rubbery finish, almost feels like RC silicone wire. Its very pliable but it also gets in the way as it catches on my mouse pad. I will probably replace this with a donor braided cable.

Ergonomics

If you’ve never used a vertical mouse before, the VMC would make an excellent introduction. I gave myself fairly serious hand pain from playing far too much Osu and TF2. I eventually realised that sore hands every breakfast was not at all normal.

I didn’t think it was possible, but Evoluent has bested the VM4 in almost every aspect. The pinky rest lip is bigger than before and a joy to rest my hand on. The entire mouse feels more fluid, and it’s much more natural to hold. However, the grip is slightly smaller than the VM4. I have large hands, and if I fully grasp the mouse my fingers now extend past the edge of the face. It’s not uncomfortably small - the mouse is still much nicer than the VM4 - but it could be just a little bit bigger.

Mousing

Buttons

I have to start off with this - the VMC has no scroll wheel click. It is instead replaced by the third face button. The extra face button is certainly comfortable, but why remove the scroll click? I imagine it’s harder to press a scroll wheel and it was removed for ergo purposes, but two buttons is better than one.

Otherwise, the VMC has a fairly standard button configuration. There’s 2 thumb buttons, scroll wheel, DPI changer and the 3 face buttons. The extra face button is quite comfortable to press with my ring and pinky fingers combined, and it doesn’t feel as awkward as it looks. The thumb buttons are easy to click easy to avoid when you don’t want to click them. The DPI switcher button is conveniently placed, and the LEDs to indicate your current sensitivity are subtle and quickly glanceable. The scroll wheel feels like “upper class office” style - that weird combination of mushy (the good kind) and clicky. Personal preference, but I quite like it. My only real gripe is that Mouse1 requires about half the force to actuate than the other mouse buttons, which resulted in heaps of miss-clicks as I was adjusting to the mouse. Lighter switches are ergonomically superior, but it’s confusing why only 1 of the buttons get this treatment.

Lots of buttons are useless without configuration, and Evoluent’s software comes through in spades. Each button can be set to any standard mouse or keyboard button or a pre-recorded macro. Bindings can also be customised on a per-application basis, which is excellent to see. Also included is a break timer to remind you to get off your arse, and “Click lock” and “Auto click” features to reduce strain from clicking things.

Tracking

Initially I used the VMC on my Goliathus ‘Speed’ edition mouspad. It’s a cloth pad with a smooth surface, and I enjoy it with my DeathAdder. With the VMC it’s unusable. Incredibly slippery. For those who enjoy cloth pads, it’s like switching to a super smooth hard pad - I felt I was tracking on ice. Luckily I bought a Goliathus ‘Control’ by accident some time ago. It’s also cloth but quite rough. It made a huge difference to the feel of the mouse, and tracking now feels good. I’m starting to regain muscle memory with this configuration - headshots and airshots are become more common, and I’m not overshooting on-screen buttons like I was on the Speed. The feet are as smooth, if not smoother than the ones on my DeathAdder. Combined with the vertical stance putting more weight on the mouse, this would explain the slipperiness.

I don’t actually know what DPI Evoluent is using for the VMC. There are 2 ways to adjust sensitivity. The first is using the built in Windows sensitivity settings - the Evoluent software doesn’t add any precise control. Additionally, the mouse itself has a DPI button that changes between 4 different sensitivity settings. These are not configurable. This is pretty annoying, because my DeathAdder was set up perfectly. By forcing me to adjust using the Windows DPI settings, I now have to correct the DeathAdder’s sensitivity to compensate, which is a pain. Adding a DPI slider would be a great improvement to the Evoluent software.

Unlike standard mice where you actually lift the entire mouse, the most natural way to “lift off” using the VMC is to tilt the mouse to the side and slide with one end still on the mat. The manual even recommends this method, and it’s definitely the most comfortable. However, the liftoff distance of the sensor is quite high, and not configurable. I found myself not lifting far enough and sliding back. After a few days (and swapping mouse pads) it’s getting better, but it’s another pain point to consider when adapting to the new mouse - even if you already own a vertical mouse.

Lastly, the mouse updates at 125Hz, but this isn’t a deal breaker for me. Worth mentioning for the 1KHz diehards. I’m just a schmuck on the internet, but I think Evoluent would do well to try and capture the gaming market with a dedicated product. There’s a lot of gamers abusing their hands that probably don’t realise such comfort exists.

Teardown

Basic disassembly is quite simple. By removing 2 of the mouse feet and 4 screws, the bottom simply falls out, revealing the PCB and ribbon cables to the button assemblies. Having to remove the feet is annoying as ever, as it’s never quite the same when you replace them after disassembly. Replacements aren’t offered yet, but I reached out to Evoluent and they said they’ll sell them soon.

Sensor

My VM4 was an early model and had an rubbish laser sensor. The VMC (and the VM4 if bought new today) have a huge upgrade - the Avago ADNS-3000. It’s not gaming class, but it’s optical and made by Avago. It uses a Kingsis lens and should be good for up to 2000dpi. I don’t know much about lenses but from what I’ve read the Kingsis seems pretty standard. The date code on the sensor indicates it was manufactured in April 2015.

Processor

The processor is a Cyprus CY7C64215. Nothing particularly special here, it does what it needs to do.

Wrap-up

The Evoluent VerticalMouse C is a welcome upgrade over the VerticalMouse 4, and may well be the best ergonomic mouse on the market today. For office use its purchase should be a no-brainer. For gaming it certainly holds its own, and I would definitely consider a purchase if hand pain is a problem for you. However, limited DPI customisation, 125Hz update rate and missing scroll click mean the VMC won’t be ideal for hardcore or competitive gamers.

Is it worth $100 USD? It’s a tough purchase, but the answer is simple for me - $100 now or permanent hand pain as I grow old. That it happens to be a great mouse is just a bonus.

cthelper - fix “Failed to allocate pseudoterminal”

2016-01-17T06:46:00+00:00

Kitty/PuttyCyg, Windows 10, cthelper, Cygwin64. This blog post is brief, but my mosh stopped working and I felt I should document the solution.

Open your cygwin installer (just download it again if you don’t have it) and update packages - don’t select anything, just go through until it installs. Boom, fixed!

Evoluent Vertical Mouse 4 Sensor Transplant

2015-09-14T14:58:00+00:00

Maybe I just play games badly, but for many months I was getting serious wrist pain using my Deathadder. I tried a couple of “ergo” mice, including a Logitech ball mouse (the M570) and a no name eBay vertical mouse, but they didn’t really help, and were damn hard to play games with. At the suggestion of a physio friend, I finally bought the “real deal”, the Evoluent VM4.

It’s weird to start off, but by 2 weeks I was playing (almost) as well as I did with the DA, and my wrist pain had practically disappeared. There was only 1 problem. The sensor is garbage! It has a weird jitter I can’t really place. Is it angle snapping? 125Hz update rate? Or simply laser sensors being horrible as always? Either way, it could never replace the DA for “real” play, so I set off on an adventure to fix it.

The donor mouse

Thanks to pvh I found a suitable donor, the EVGA TORQ X5. It has enough buttons to match the 8 on the Evoluent, and a great sensor, the Pixart 3988. On top of this, I grabbed one for $20 used off Amazon. Now came the fun part - shoving it in its new case, and connecting all the switches and lights from the vertical mouse.

Quite the difference.

Disassembly

Evoluent Disassembly

I haven’t found anyone online do a complete disassembly of this mouse, so I’m going to be a little verbose. Feel free to skip past this section.

The VM4 comes apart with 4 screws in the bottom:

Simple PCB, and the crappy sensor:

Next you undo all the screws you can see, then fiddle with a large flathead screwdriver until the side pops off. Then you can take the entire shell apart.

Couple more screws and the scrollwheel assembly comes out:

And that’s it!

TORQ X5 Disassembly

This one’s really easy. 4 screws hidden under the skids, and the whole body comes apart. Nothing much else to say!

Mmmm, that lovely Pixart.

First attempts

It was clear that the X5 wouldn’t fit inside the body of the VM without cutting off the front.

First mistake: I figured putting the X5 scrollwheel inside the VM4 would be a brilliant idea. WRONG. The 2 wheel assemblies are completely different, and to get it to fit I ruined the middle click actuation of the mouse. Learn from my mistakes. I then purchased my second VM4, with a horrible exterior but working interior parts to salvage.

The new bashed up donor:

Was this left out in the sun? Weird wear.

So, to the next failure. I have to cut the front off the X5 to make it fit. The buttons live on the front. Thus, I have to wire jumpers on the microcontroller. Individually. With several right next to each other. I wasted hours on this. I even used a hot air gun to take the entire MCU off the board, bend certain pins up, and resolder it. In doing so I busted the mouse3 pin, which meant I had to map two buttons to the same pin. What a disaster!

PCB surgery: soldering_skills++; sanity--.

Not pictured here is the soldered ribbon cable adapter, after I painstakingly mapped all the VM4 pins to their functions. After all this, I plugged it in… And it worked! …Except for the scroll wheel. Turns out, mechanical scroll wheels (X5) don’t put out the same signals as IR scroll wheel (VM4). I practically gave up at this point, and the project was dead for several weeks, until my hand got sore enough to look back into it.

Final Success

After failing to shove VM4 bits into the X5, I had a stroke of genius - why not use the X5 board just as a sensor, the VM4 board just for buttons, and put a USB hub in there? $2 and 2 weeks later, my 4 port eBay hub arrived. I forgot to take a before when it had all 4 ports, but the extra 2 snapped off with no problems:

From there it was a simple matter of desoldering the USB ports and hardwiring all the relevant mice connectors to the board:

With a solution finally in place, I went ahead and cut the front off the X5 PCB, scribed its outline, dremeled a hole for the lens, and dremeled a bunch of plastic off the VM4 case so the PCB would lay flat. Add a bit of hot glue to the lens and you’re set:

Because the sensor fits to the lens quite solidly, no extra glue is required. I had to reposition the PCB once, as it initially collided with the top half of the shell. After the move it’s a tight fit:

Now comes the easy part, putting it back together. I added some kapton tape to the scroll wheel PCB just to be sure there weren’t going to be any shorts. I probably should have covered the original laser aperture, but with no lens it hasn’t given me any problems yet.

Looks messy, works beautifully. Amazingly, the $2 hub was actually USB 2.0! MouseView was showing 1000Hz from the X5, which is just lovely.

The RGB LEDs on the X5 even show through the Evoluent logo, which is a great bonus:

Compared to all the failed attempts, this mod is now wonderfully easy, assuming you’re capable of desoldering USB connectors. However, since there’s no longer a need for any buttons on the donor, I’d recommend getting a smaller mouse to shove inside this one to save hacking away at PCBs.

Again, if you get wrist pain from gaming, I’d highly recommend a vertical mouse. Don’t get a Chinese copy, they’re simply too small to be useful. I got mine for $70AUD used, they seem to go for about $40USD if you’re in the states. See if you can deal with the bad sensor, and if you can’t, upgrade it!

Modifying AVR PROGMEM at runtime

2014-11-06T12:07:00+00:00

As part of a project this year, I needed about 11.5KiB of data which would be maintained on power off, could be modified on the fly, could be read blazingly fast, and didn’t use the Arduino bootloader. Storing stuff in flash using PROGMEM sounds great, but you can’t modify that, right? Wrong!

First up: Don’t do this unless you have a very good reason to. The flash memory on most AVR microcontrollers has an endurance of only 10,000 writes. Make sure that you’re not going to hit this limit in your application, or bad things could happen! The data is also lost on every program flash (unlike EEPROM, which can be preserved), so keep that in mind if you rapidly iterate.

Here’s some example code to make it all work:

#include <avr/io.h>
#include <avr/boot.h>
#include <avr/interrupt.h>
#include <avr/pgmspace.h>

void rewrite_data(void);
BOOTLOADER_SECTION void boot_program_page(uint32_t page, uint8_t *buf);

const uint8_t flash_data[11520] __attribute__((aligned(SPM_PAGESIZE))) PROGMEM = {};

void rewrite_data(void) {
 uint8_t data[SPM_PAGESIZE];
 // Break data into pages
 for(int i = 0; i < 11520; i+= SPM_PAGESIZE) {
  // Load data
  for(int j = 0; j < SPM_PAGESIZE; j++) {
   // For my project this came from serial, don't just use 42!
   data[i] = 42;
  }
  boot_program_page((uint32_t)flash_data + i, data);
 }
}

// avr/boot.h documentation example
BOOTLOADER_SECTION void boot_program_page(uint32_t page, uint8_t *buf)
{
 uint16_t i;
 uint8_t sreg;
 // Disable interrupts.
 sreg = SREG;
 cli();
 eeprom_busy_wait ();
 boot_page_erase (page);
 boot_spm_busy_wait (); // Wait until the memory is erased.
 for (i=0; i<SPM_PAGESIZE; i+=2)
 {
  // Set up little-endian word.
  uint16_t w = *buf++;
  w += (*buf++) << 8;
  boot_page_fill (page + i, w);
 }
 boot_page_write (page); // Store buffer in flash page.
 boot_spm_busy_wait(); // Wait until the memory is written.
 // Reenable RWW-section again. We need this if we want to jump back
 // to the application after bootloading.
 boot_rww_enable ();
 sei();
 // Re-enable interrupts (if they were ever enabled).
 SREG = sreg;
}

To explain some tricky sections:

const uint8_t flash_data[11520] __attribute__((aligned(SPM_PAGESIZE))) PROGMEM = {};

This defines your data - in my case, a 11520 byte array of unsigned 8 bit integers. PROGMEM is the AVR directive to put this variable inside flash memory instead of SRAM. The ___attribute__((aligned(SPM_PAGESIZE)) is so that the data gets put into its own page. AVR flash memory is per page, and there is no way around that. You could read in the previous page data before yours, add your data, then write it back, but this is unnecessary effort. Aligning the data to a page boundary means you can write data directly to that page without worrying about other things in that memory space.

BOOTLOADER_SECTION void boot_program_page(uint32_t page, uint8_t *buf)

This puts this function (only this function) inside the bootloader section of the flash memory. If you try to execute flash memory modifications outside this area, nothing happens.

Now, if you were to compile this, you’d get a linker error. Whilst you’ve said that your special programmer function should exist in the bootloader, it doesn’t actually know where that is. How do we find out where it should go? The datasheet! I was using the Atmega328P, so I’ll use this for an example.

The bootloader section can be several different sizes depending on which fuses you’ve set. For the 328 we go to section 26.8.16, page 291.

In this case, the page programming code happily fits inside 256 words, so we set our BOOTSZ[1:0] bits to 3. We can also see that the bootloader will then start at 0x3F00. So how do we tell the linker this?

In Atmel Studio you can set this up under the project settings:

Using commandline tools, just add --section-start=.bootloader=0x3F00 to your linker flags. Double check your address is actually in bootloader space! Otherwise the code will run happily but not modify anything.

That’s it! Enjoy your newfound “kind of EEPROM but not really” space!