justinmklam | blog

A Beginner’s Guide to Split Keyboards

Sun, 15 Feb 2026 10:59:22 -0800

So you’ve heard of split keyboards and want to buy one, but don’t know where to start? You’ve come to the right place! There are many offerings these days which can be overwhelming, so this guide aims to provide a high level overview of the landscape so you can figure out which path you want to take.

The intent of this guide is not to tell you which specific keyboard you should buy, but rather to equip you with the knowledge to find one that works for yourself.

The content in this post is mainly a consolidation of content that exists in r/ErgoMechKeyboards, but organized in a way that is hopefully helpful for beginners.

Also, it should be noted that regardless of how ergonomic your setup is, it’s important to take frequent breaks from being in front of a computer when possible, and to get up and move around. If you’re suffering from acute muscular pain (e.g. from keyboard use, or other), consider also seeing a physiotherapist or other healthcare practitioner for professional help.

Introduction

Why Use a Split Keyboard?

There’s already many other resources that talk about the benefits of a split keyboard, but the gist is that by separating the left and right halves of the keys, your hands can rest in a more natural position instead of being forced together like on a traditional keyboard. This reduces outward wrist bending (aka ulnar deviation), shoulder tension, etc. The goal isn’t to magically fix posture overnight, but to remove some of the physical constraints imposed by a standard keyboard so your body can settle into something more relaxed during long sessions at the computer.

Exaggerated depiction of how a split keyboard can help promote a more neutral, comfortable typing position. (Source: Candid Technology)

Types of Keyboards

There are three main categories of keyboards, which define the overall positions of the keys.

Row-staggered: Keys are aligned in rows, like a traditional keyboard. This is what you’re already used to.
Column-staggered: Keys are aligned in columns. This better matches finger anatomy and is very common in ergonomic splits.
Ortholinear: Uniform grid, no stagger. Visually pleasing, but less common in ergonomic splits.

Comparison of keyboard layouts.

For people interested in ergonomics, most transition from row staggered to column staggered. Ortholinear was popular for a time and still has a cult following (e.g. with the Planck), but is not as popular for ergonomic keyboards. Majority of the split keyboards shown in this post will be column staggered.

A common question is whether learning to type on a non-row staggered keyboard will impact your ability to type on a traditional keyboard. If you switch between the two somewhat regularly, the answer is no - the brain appears to be great at compartmentalizing muscle memory for different devices! In my first month or so of learning to type on a column stagger keyboard, it took me a few minutes to get used to a regular keyboard again in that session. But now, I can use either a traditional or split without issue or “ramp up” time.

The Big Question: How Much Time Do You Have?

Despite the promised land of ergonomic benefits being a mere credit card purchase away, it is not without its dark side. There is unfortunately an inevitable learning curve, and it can be a steep one.

Re-training years, if not decades, of muscle memory for typing on a traditional keyboard is tough work, and can have its frustrating moments. It’s common to take at least a week or two of lots of typing practice to get accustomed to the new layout. For me, it took about a month of heavy use to be comfortable typing at a productive level for work, and that’s in a profession that involves a significant amount of daily typing (i.e. software development).

If you’re practicing with something like monkeytype, the default mode of typing lowercase words is only half the battle. Getting familiar with a keyboard involves other things, like:

Using numbers, punctuation, and symbols
Selecting/manipulating text (arrow keys) with shift or by word, home/end
Application-specific hot keys (e.g. browser fwd/back, tab cycling, closing tabs/Windows, window management, Excel motions, etc)

The typing progression with my first split keyboard, over the course of 3 months.

If at this point you’re questioning the time investment required for this whole endeavour, fear not! There’s a keyboard for everyone, and not all require this steep learning curve.

Path 1: “I want a split, but don’t have time to learn”

These keyboards keep a row-staggered layout, so you can keep that well-trained muscle memory. You still get the biggest ergonomic win of splitting your hands apart, but without a massive cognitive load of retraining your fingers and brain for typing on a radically new layout.

Some examples of traditional keyboards with a slight split in them:

Logitech ERGO K860, which is similar to the previously popular, but now discontinued Microsoft Sculpt. (Source: Logitech)

Keychron Q10 in what’s called an Alice Layout. (Source: Keychron)

Going one step further, getting a keyboard with independent halves will provide the most flexibility, since this allows you to freely position the halves (e.g. shoulder width apart), or even set them up with tenting.

Kinesis Freestyle 2, with optional tenting accessories. (Source: Kinesis)

UHK 60 and 80 variants, which come with optional thumb modules for mousing. (Source: Ultimate Hacking Keyboard)

These keyboards may not be perfect for “power” users, but they tick enough boxes for the majority of people. If you do want to use more advanced features like layers and combos, you can use software like Karabiner Elements (macOS), keyd (Linux), or Kanata (cross-platform) to add this functionality, similar to how I modified a regular keyboard’s keymap here.

If you’re in this group, you can stop reading here! You can also find other similar keyboards using the (basic) search term “ergonomic keyboard” to see what’s currently available on the market.

Path 2: “I’m willing to learn for better ergonomics”

If you do have the time and motivation, then welcome to the rabbit hole of features, geometries, and customizability!

Most keyboards in this category will have columnar stagger and multiple thumb keys (in place of a single spacebar), which are the main drivers for relearning how to type. The benefit of thumb keys is they let the thumbs handle high-frequency actions so the pinkies aren’t overworked reaching for shift and modifiers. However, keep in mind that it’s possible for thumbs to get overuse injuries as well!

Features

Number of Keys

When shopping for split keyboards, you’ll notice they can have a wide range of number of keys. A standard US keyboard has 104 keys (or 105 keys for those outside the US), but many have half that or even less!

The rationale for having fewer keys is to keep your fingers as close to home row as possible, minimizing the amount of hand movement while typing. The tradeoff to fewer keys is complexity - instead of just reaching over to press a number, multiple keys will need to be pressed to achieve the same result.

Holding the left thumb key activates the second layer, which contains symbols and numbers.

There’s some discussion of whether smaller layouts are actually better for ergonomics, but some find enjoyment in seeing how few keys one actually needs. This is ultimately a personal choice, but if you’re just starting out, it’s always better to have more keys and not use them, than to not have them and want them. You can also always try smaller layouts on a larger keyboard, like I did when experimenting with a 36 key layout.

Here’s a few examples of keyboards showing the range of key counts:

Kinesis Advantage360, a “maximalist” split keyboard with 76 keys. Omits the F-keys at the top, but offers a hefty 6-key thumb cluster on each side. (Source: Kinesis)

Lily58, “full” keyboard with a number row, outer columns, and 4 thumb keys. (Source: Github)

The popular Corne, with 42 keys. Commonly denoted as 6x3+3 (6 columns x 3 rows + 3 thumb keys). Drops the number row. (Source: Keebd)

Chocofi, 36 keys (5x3+3). No number row or outer columns, so keys like shift, tab, quotation are no longer accessible on the main layer via a single key press. (Source: beekeeb)

Sweep, 34 keys (5x3+2). With only 2 thumb keys, this is about as small as your can go while having all the alpha keys on the main layer. (Source: Github)

Smaller keyboards do exist, but these layouts require extensive use of layers and/or combos since the main layer is no longer large enough to contain all alpha keys as well as basic function keys like layer switch, space, backspace, etc. The Hummingbird is a common keyboard/layout in this form factor, where Z/X/Q/J keys are relegated to combos to make space for other keys.

A tiny 30 key keyboard. Questionable utility, but undeniable aesthetics! (Source: Github)

Geometric Considerations

Aside from the number of physical keys, there are also many options of how the keys are laid out. Keyboards can have different amounts of column stagger, as well as splay - which is when the columns are at an angle instead of parallel with each other.

Hillside 52 with splay in the outer 3 columns. (Source: Github)

You can also find keyboards that are sculpted in 3D like the Glove80 or Charybdis, or you can even design your own Dactyl. Some find great comfort in these, however they come at a higher cost due to manufacturing complexities, unless you handwire your own.

Glove80 sculpted keyboard. (Source: Moergo)

Recently, people started designing sculpted keycaps like the KLP Lamé which brings the benefits of sculpted shapes without needing a special keyboard.

KLP Lamé sculpted keycaps on a Ferris Sweep. (Source: Github)

You can also find unibody/monoblock keyboards, which still have a split-like layout but are connected as a single unit. Some prefer the constraints and reproducibility of having the halves always being in the same position, since independent halves can shift around throughout the day. Unibody keyboards are also generally easier to take and move around, like if you’re working on your lap on the couch or on an airplane tray.

Reviung41 unibody with low profile switches and a Pimoroni trackball. (Source: Holy Keebs)

The pursuit of ergonomics goes quite deep, where there’s also a league of non-keyboard-looking keyboards that use innovative methods for input. The Svalboard, for example, uses multiple feather-light paddles for each finger instead of regular switches. This keyboard was inspired by the DataHand, an alternative keyboard designed to be operated without requiring any finger travel. Similarly, the Charachorder uses joystick-like switches for each finger and makes heavy use of combos, although this makes it closer to stenography in philosophy.

In reality, this is near the bottom of the rabbit hole. Most users never go anywhere near this level of ergonomic experimentation, but it’s where the most interesting (and potentially game changing) ideas tend to emerge!

“Banish pain with the most adaptable ergonomic keyboard and mousing instrument ever made.” (Source: Svalboard)

“Less finger travel = quicker reactions, faster typing, & less strain. Leave 1D typewriting in the past and experience a new dimension built for the digital age.” (Source: Charachorder)

Tenting

Many people find it more comfortable to have their keyboard slightly raised in the middle, which allows the wrist to maintain a more neutral pronation. The ideal tenting angle is dependent on both the user and the keyboard, where at a given angle, a wider keyboard with more keys will feel different than a smaller keyboard with fewer keys.

Some keyboards come with integrated tenting, but if not, there are many DIY solutions to add it on after. For small angles, foldable laptop stand feet/risers work great.

Totem with sculpted keycaps and adjustable laptop stand feet. (Source: Github)

For larger angles, adhesive-backed Magsafe rings and mobile phone stands allow for more adjustability. The stands can be pricey, but there are also cheaper, 3D printed options.

Tenting with Magsafe ring and phone stands. (Source: Keebio)

For extreme angles, ball mount clamps (like those for cameras) can be mounted to a desk (or even chairs)

Tenting solution with ball joint clamps. (Source: Ben Vallack, YouTube)

Pointing Devices and Encoders

With custom keyboards these days, there are many options that include an integrated touchpad, trackball, or even trackpoint to minimize (or remove) the need to move your hand to use the mouse.

Some keyboards also have encoders for easy access to things like volume control or scrolling, which can be controlled via firmware.

Toucan 42 and 36, with an integrated Cirque 40mm Glide touchpad. (Source: beekeeb)

Charybdis, with a thumb trackball. (Source: Bastard Keyboards)

cocot46plus with a finger trackball and encoder. (Source: Github)

TPS42 with an integrated trackpoint. (Source: Github)

Custom keyboards also supports mouse keys to emulate mouse movements with your keyboard. It can feel a bit limiting in functionality, but is better than nothing and can be helpful in a pinch.

Wired vs Wireless

Historically, split keyboards were only wired due to both technical and licensing limitations with QMK, the custom firmware that powers many keyboards. A permissive, wireless, Bluetooth-first alternative called ZMK began development in 2020, and now many keyboards are offered in both wired and wireless variants.

One thing to note - battery management has some quirks for fully wireless split keyboards. The central side needs to do double duty since it needs to communicate with both the host computer and the peripheral side, and as a result its battery discharges significantly faster. Additionally, the more features your keyboard has (display, pointing device, etc.), the shorter the battery life will be.

A common setup of a nice!nano and a 110mAh battery yields an estimated ~2 weeks for the central side, and a whopping ~3 months for the peripheral side. (Source: ZMK Power Profiler)

A popular workaround is to use a ZMK dongle, which makes both sides of a wireless keyboard act as peripherals to maximize battery life, although there are still some limitations to this approach.

One example of the many available ZMK dongle screens. (Source: ZMK Dongle Screen, Github)

Some companies like ZSA still prefer wired, and if your keyboard is going to stay on your desk, I would tend to agree.

High vs Low Profile Switches

There are a ton of online resources that already go into the comparison of these, but the short of it is that low profile mechanical switches are relatively new and are becoming more common in keyboard designs. Lower profile means a shorter keyboard height, alleviating the need to use wrist/palm wrests.

At the time of writing, high profile (MX) switches still have many more options and generally feel and sound better, and finding keycaps for low profile switches (especially Choc v2) can be challenging, but availability is expected to improve over time as they become more widespread.

Comparison of MX (high) and low profile switches. (Source: Kinetic Labs)

Keyboards often have hotswap sockets to allow installing different switches without needing to solder. Some boards like the cheapino have the option to directly solder the switches (instead of using sockets) to keep costs low.

You might notice that many keyboards consist of blank keycaps instead of ones with legends. This is common because with the use of layers and keys having multiple functions, legends don’t end up being that useful for indicating what a key will actually do.

For both switches and keycaps, again it comes down to personal preference.

Keymap Customization

A core part of split keyboards is customizing the keymap to suit your own needs. Although keymaps are defined in firmware, many popular keyboards will have pre-compiled firmware with VIA/Vial support (for QMK, or ZMK Studio for ZMK) to support editing keymaps through a GUI.

Using Vial to customize a keyboard layout. (Source: Jawick)

Basic “programmable” features consist of the following:

Mod-Tap: One key when tapped, another key when held. (E.g. on a traditional keyboard, many programmers remap CAPSLOCK to ESC when tapped, CTRL when held).
Combos : Press two keys to output another. (E.g. J+K to ESC, or F+G to a symbol like -)

At a minimum, you’ll need to get used to using mod-taps on the thumb keys for keys like space, backspace, etc. and to activate the different layers.

Beyond the surface lies more advanced features, where the sky’s the limit to what fancy things can be done:

One shot keys: For modifiers only, allows you to use a modifier without needing to hold it down. Tap a modifiers key to activate it, then tap another key send the key code.
Home row mods: Giving home row keys (ASDF + JKL;) double duty to also act as modifier keys (SHIFT, CTRL, ALT, GUI) when held
Callum style mods: Similar to home row mods, but combined with one shot keys to remove the dependency on timing

If you don’t know where to start with creating your own keymap, you can draw inspiration from existing ones on KeymapDB. For 36 key keyboards, the Miryoku keymap is a good place to start.

What About Using a Non-QWERTY Layout?

Some choose to also change to an alternative layout like Dvorak, Colemak or other, since QWERTY is not designed with ergonomics in mind. However, learning a new layout requires even more time, and I personally haven’t gone down this route since the benefits of a column stagger and split keyboard are sufficient enough for me (and I reckon most people as well).

However, if you’re curious to learn more, there are many other layouts aside from the popular Dvorak layout, which each have their own advantages depending on the metrics important to you (e.g. same-finger bigrams (SFBs), lateral stretch bigrams (LSBs), scissors, redirects, rolls, etc.). More information can be found in the https://layouts.wiki or r/KeyboardLayouts.

Comparison table of different metrics. (Source: Pascal Getreuer)

How Do I Choose?

With the overwhelming number of keyboards one can acquire, how does one choose one for themselves?

The first question is to ask yourself how many keys you think you need, which starts with whether you need a number row, arrow keys, home/end, etc. Generally, the more keys you have, the faster the learning curve will be.

Once you narrow down to the number of keys, next is the form factor. Aside from watching review videos on YouTube, there are also some great web tools to help since this will also depend on your own hand geometry.

Personally, I found the amount of column stagger doesn’t make a huge difference (except for maybe the pinky column), whereas the thumb cluster matters a bit more since an overly tucked or stretched thumb is more noticeably uncomfortable.

Searchable collection of ergonomic keyboards. (Source: YAL Tools)

Web app to compare the physical sizes of common keyboards. (Source: SplitKB Compare)

To get a better feel of your potential keyboard(s), you can use some low-fidelity prototypes to test the form factor. Printing it on paper is the easiest way…

Printing a layout on paper to test hand positions. (Source: Pascal Getreuer)

Or you can take it a step further and cut out the template on to cardboard, then install switches. If you want, you could even handwire it to make it a usable, functional keyboard, which may be worth the effort since actually using a keyboard can identify many more things (e.g. discomforts) than just randomly pressing dummy keys.

Cutting out the keyboard template on to an old shoe box.

Switches and keycaps installed on to the cardboard prototype.

To Buy or DIY?

Buy

Gone are the days of needing to spend >$500 on an ergonomic keyboard, and now you can find a keyboard for every budget! Generally, the smaller the keyboard, the cheaper it will be since fewer keys = fewer switches/keycaps + smaller PCB.

In the upper budget range, companies like Kinesis, ZSA, and Dygma make specialized, closed-source keyboards but offer premium devices and support. There are also more boutique options like MoErgo and Bastard Keyboards that offer high quality products. These keyboards will usually have their own custom software to program them, which usually easier to use than the open-source software versions.

Premium keyboard with a premium website. (Source: ZSA Voyager)

In the middle range, you can find many shops like beekeeb, holykeebs, splitkb, or any of the other vendors listed on the r/ErgoMechKeyboards wiki. They typically sell both pre-assembled boards and kits that require soldering, but are offered at a more affordable price point. Many of the keyboards offered at these vendors are based on or variants of open-sourced designs.

Beekeeb (Source: beekeeb)

Although you do get a much nicer keyboard in the above price ranges, cheaper options exist on sites like Amazon and Aliexpress for those on low budgets. The experience or physical product may not be as refined (e.g. 3D printed cases are often used), but it’ll still be functional and at least let you try it out without breaking the bank. This is where I started on my journey with using a Corne keyboard.

When buying cheap keyboards, there is a security risk to them. If possible, you should reflash the firmware to remove any risk of malicious behaviour.

Aliexpress with many affordable options for split keyboards. (Source: Aliexpress)

DIY

Many keyboard designs are open-sourced, which means you can take the design files (usually hosted on Github, packaged as gerber files) and get them manufactured somewhere like PCBWay. Getting a PCB made with them is pretty painless: you upload your gerber files, pick your specs (defaults are usually fine), and the board typically arrives within a week or two, even to North America. The quality is solid, and their instant quote tool makes it easy to see costs upfront before committing.

This is a great option if you find a design that isn’t available in any of the options above. But in terms of cost, by the time you factor in the PCBs, switches, diodes, microcontrollers, fasteners, etc., not to mention the soldering equipment if you don’t already have that, buying something from Amazon or Aliexpress will likely be the cheapest if you’re starting from zero.

However, if you’re okay with being even more hands on, and if you already have soldering equipment, handwiring a keyboard with a 3D printed enclosure will be the cheapest (and most flexible) option. As long as your soldering is solid, there wouldn’t be any functional difference between this and a PCB!

Popular Keyboards

Still have trouble finding which keyboard you should get? Here’s a list of popular* keyboards for each budget (as of Feb 2026):

*The keyboards listed below are from my personal perspective, based on what I see mentioned and recommended around Reddit. YMMV and I could have missed some - there are many other keyboards out there, so please do your own research!

Keyboard	Budget	# Keys	Open Sourced?
Kinesis Advantage2/360	$$$	80 / 76	no
ZSA Moonlander / Voyager	$$$	76 / 52	no
Dygma Defy	$$$	70	no
Glove80 / Go60	$$$	80 / 60	no
Dactyl / Charybdis	$$$	58 / 42 / 36	yes
Keyball	$$$	61 / 44 / 39	yes
Elora / Kyria	$$	62 / 50	no
Sofle / Lily58	$$	58	yes
Totem	$$	38	yes
Silakka54	$	54	yes
Corne	$	36 / 42	yes
Sweep	$	34	yes

Open sourced means the design files are available for building yourself, as well as there being many variants around (at both reputable vendors and places like Aliexpress). For example, if you bought a Corne on Aliexpress and like the layout but want some premium features like wireless, low profile switches, and/or a sturdier case, you can shop around at different vendors to find boards with these features.

A common starting keyboard is the 42 key Corne if you’re okay with fewer keys, or the Silakka54/Lily58/Sofle if you want more keys. From there, some stay with these to be their daily drivers, and others use them as a stepping stone to move on to other keyboards after learning more about their workflow preferences.

Design Your Own

If no existing keyboard sparks joy, there are many resources and tools to design your own. Tools like Ergogen and kle-ng make it easy to create your own layout, which can be exported to Kicad to finalize the PCB. Or if you want one fully customized to your hand shape, you can use Cosmos to create a sculpted one to be handwired.

Ergogen keyboard layout generator. (Source: Ergogen)

Cosmos keyboard configurator. (Source: Cosmos)

This is a topic in itself, but there are many existing tutorials and guides on YouTube and around the internet. For keyboards on Github, you can also poke around the design files and reference or adapt them to your needs, which is the great thing about open source!

Closing Thoughts

If you’ve made it this far, thanks for reading! Hope this was helpful in starting you off in this journey of finding your perfect keyboard.

Happy typing!

Remapping a Standard Keyboard

Sun, 30 Nov 2025 22:18:06 -0800

As much as I enjoy using my ergonomic keyboards, sometimes I still need to use the built in keyboard on my laptop. Transitioning between the two drastically different layouts is no longer an issue, but I find myself having an itch of wondering whether it’s possible to improve the usability of a standard keyboard.

I personally find typing letters and numbers is actually not so bad; despite the row stagger causing my fingers to move around more than they would need to on a columnar stagger layout, it’s not the worst thing to deal with. The thing that bothers me more are the actions around text manipulation like selecting or deleting text by words or entire lines. The keys for these are at the peripheral of the keyboard, which causes me to move both my hands away from the home row whenever I need to do those operations, which is fairly often while typing.

With a bit of layer remapping that I adapted from using small form factor keyboards, I was able to achieve a layout that keeps my hands relatively centered around home row. It’s not a far departure from a standard layout, so the learning curve is low compared to other, more drastic layout changes to bring improved ergonomics to common keyboards.

The Goal

To enable keeping my hands close to home row as often as possible, I need to move a handful of key keys:

Top left: Escape
Top right: Delete/Backspace
Bottom left: Shift, Control, Option/Alt, Command
Bottom right: Left, Right, Up, Down Arrow Keys

Prior Art

Other efforts exist like SpaceFN or Home Row Mods to improve the usability of standard keyboards, but the issue I found is that they all modify frequently used keys. I find these customizations work best on keys that aren’t as commonly used during regular typing. On my split keyboard, I started using bottom row mods (where the modifiers are on ZXCV instead of ASDF) which I find work much better, but unfortunately on a standard layout, the bottom row is shifted quite far right which makes using it with the left hand more awkward.

After a bit of trial and error, I settled on a remapping that works for my workflow. I even went as far as replacing my daily split keyboard with the Apple Magic Keyboard for a few weeks, since my secondary hypothesis was whether it’s possible to make a standard keyboard ergonomic enough to avoid needing to buy and learn how to use a columnar-stagger split keyboard. Unsurprising spoiler alert: You can get close, but there are tradeoffs.

The Layers

Base

Main layer with three modified keys.

For the main base layer, only three infrequently used keys are changed:

Tab -> Tab when pressed, Control when held
Capslock -> Hold for Extend layer
RightAlt -> Hold for Nav layer

Additionally, I added a combo of J+K = Escape, which is especially useful for Vim.

Tab’s default behaviour doesn’t change, but holding it unlocks a new behaviour of acting as Control. Capslock and Right Alt are not used often (at least by me), so I opted to use those as the keys to activate the two new layers, “Extend” and “Nav”, which I’ll explain more in the following sections.

I considered using LeftCommand to toggle the layer as well, which would help with symmetry and also have both thumbs operating the layers, similar to how it works on typical split keyboards. But I use Command often, and Capslock is basically unused. I already had remapped it to be “escape when tapped, control when held” to improve ergonomics when using vim, since Capslock is in a prime location for the pinky to press while the rest of the fingers are free to move along the home row.

Extend: via Capslock

When the extend (previously capslock) key is pressed, it activates the other coloured keys.

With this layer, I can easily manipulate text with my right fingers on the arrow keys, and left fingers on the modifiers. This lets my hands stay on home row, where typically to do similar actions, I’d need to move both hands to the bottom corners to press the modifiers and the arrow keys.

For example, if I want to select a bunch of text, instead of moving my left hand to the bottom left of the keyboard to press Shift + Option, and my right hand to the arrow keys, I can just press S+D with my left hand, and one of H|J|K|L with my right hand for the arrow key.

In this layer, Spacebar also becomes Delete/Backspace, which avoids having to move my right hand to the top corner to delete text, as well as allowing my left hand to use Option or Command to easily delete by word or line.

I also added a few keys for browser navigation - browser forward/back, and next/previous tab.

If you don’t use macOS, you could consider remapping the Left Alt key to activate the left thumb layer instead, which would keep things symmetrical where both thumbs are used to activate layers.

One caveat is that putting Control on the Tab key is a little weird for vim motions. It’s not the most comfortable and takes a while to get used to, but I’m okay sacrificing a bit here to keep the arrow keys in an easily accessible place.

Nav: via Right Alt

By activating this layer with the right thumb, the keys turn to mouse navigation (similar to QMK Mouse Keys) for those times when you just need to nudge your mouse a bit, e.g. to the next input box or adjacent window. Using a mouse or dedicated pointing device is still much more efficient, but it’s nice to have this option for other situations.

I also added cut/copy/paste here. Since I use a mouse with my left hand, this regains my ability to execute these operations with my non-mousing hand.

For moving mouse keys and navigating tabs. Also adding cut/copy/command so I can use it along with left mouse.

Karabiner Config

Making complex modifications directly with Karabiner isn’t the easiest, where it requires manually writing JSON, which can be error prone and finicky. There are other tools to make it easier to create configs, and I recently started using karaml.

The karaml_config.yaml looks like this (non-exhaustive), which then gets translated to Karabiner JSON format:

/base/:
 caps_lock: /extend/
 j + k: escape
 tab: [left_control, tab]

/extend/:
 # Vim motions, (x) means to include any optional mods
 (x) | h: left
 (x) | j: down
 (x) | k: up
 (x) | l: right

 space: backspace
 semicolon: enter

 ...

GokuRakuJoudo is another popular option, but I personally found the syntax to be confusing and too complicated for my simple mind to comprehend.

For those on Linux, keyd is quite simple to configure in a similar way. However, I haven’t quite figured out how to chord the combos in the extend layer (e.g. to execute Shift + Control + Left), which works fine in Karabiner though. So please comment if you know how to fix it!

[ids]
*

[main]
capslock = overload(extend, esc)
j+k = esc
tab = overload(control, tab)
rightalt = overload(nav, backspace)

[extend]
a = leftalt
s = leftshift
d = leftcontrol
f = leftmeta
h = left
j = down
k = up
l = right
space = backspace

[nav]
u = C-S-tab
p = C-tab
i = A-left
o = A-right

Closing Thoughts

All in all, this was a successful experiment in improving the usability of a regular keyboard, and I’ll definitely continue to iterate on this layout as needed. Use a dedicated split keyboard is still (probably) superior, sometimes you need to make do with what you have, and hopefully this helps anyone who’s on that journey.

Some resources referenced in this post:

Karabiner configurations (Github Gist)
Keyboard Layout Editor visualizations:

Happy typing!

Replacing Disqus With Giscus and Github Discussions

Tue, 26 Aug 2025 21:08:26 -0700

As you might have noticed, I’ve put this website through a number of changes this year. Apart from changing the theme (sorry if you hate blue), I also switched my website metrics from Google Analytics to Umami, which is an open source, privacy focused analytics platform.

Along the same vain, it was finally time to move on from Disqus.

Disqus was the go-to commenting platform back in the 2010s, but over time they’ve become focused on profiting through selling user data and advertisements, as well as just being very bloated. The final straw was adding advertisements to their free plan, and despite them not enabling it (yet) for small, non-commercial websites like this one, it’s likely only a matter of time before they go that direction.

Prior Art

There are a number of cloud and self-hosted Disqus alternatives, but I wanted something open source, simple, no-frills, and low maintenance. For this static site hosted on Github Pages, it seemed silly to self-host a comment platform like Isso or Commento just for the <100 comments I have.

Utterances was previously a common choice which uses Github Issues, but I settled on Giscus since it uses the (more logical) Github Discussions component instead. It also seemed to be the go-to choice these days amongst bloggers, as well as being the most actively maintained of these tools.

One caveat to using a Github-based tool is that users need to have a Github account to comment and react to a post. I mostly write about tech, so most of my users likely already have Github accounts, so this isn’t too much of an issue for me. It does add a friction point of not having easy login options like Google or other social media accounts, though I don’t get a ton of comments on my blog anyway, so again, something I can live with.

Setup

Configuring Giscus

Setting up Giscus was quite straightforward, the instructions on their homepage were easy to follow. Once the Giscus app was installed and Discussions enabled on my repo, I copied the generated snippet to my website, replacing where Disqus would go.

<script src="https://giscus.app/client.js"
	data-repo="[ENTER REPO HERE]"
	data-repo-id="[ENTER REPO ID HERE]"
	data-category="[ENTER CATEGORY NAME HERE]"
	data-category-id="[ENTER CATEGORY ID HERE]"
	data-mapping="pathname"
	data-strict="0"
	data-reactions-enabled="1"
	data-emit-metadata="0"
	data-input-position="bottom"
	data-theme="preferred_color_scheme"
	data-lang="en"
	crossorigin="anonymous"
	async>
</script>

Counting Comments

One of the nice things about Disqus was its “built-in” comment counter. This was enabled with a snippet like this:

<!-- The placeholder to be replaced... -->
<span class="disqus-comment-count" data-disqus-url="{{ .Permalink }}#disqus_thread">0 comments</span>

...

<!-- ... From this script -->
<script defer id="dsq-count-scr" src="//[SHORTNAME].disqus.com/count.js" async></script>

Giscus doesn’t have a built in option for that (see this issue, but I was able to achieve it fairly easily with a bit of Javascript. In my Hugo template:

<!-- The placeholder to be replaced... -->
<span id="giscus-comment-count-{{ $context.File.UniqueID }}" class="giscus-comment-count" data-url="{{ $context.RelPermalink }}">
 <a href="{{ $context.RelPermalink }}#comments">0 comments</a>
</span>

<!-- ... From this script -->
{{ $giscusJs := resources.Get "js/giscus-comments.js" }}
{{ if $giscusJs }}
 {{ $giscusJs := $giscusJs | resources.Minify }}
 <script src="{{ $giscusJs.RelPermalink }}" defer></script>
{{ end }}

And in giscus-comments.js:

// Giscus Comment Count Manager
// Uses PostMessage API to get comment counts from Giscus iframes

class GiscusCommentManager {
 constructor() {
 this.setupMessageListener();
 }

 setupMessageListener() {
 window.addEventListener("message", (event) => {
 if (event.origin !== "https://giscus.app") return;
 if (!(typeof event.data === "object" && event.data.giscus)) return;

 const giscusData = event.data.giscus;
 const currentPath = window.location.pathname;

 // Check if this contains discussion data with comment count
 if ("discussion" in giscusData) {
 const discussion = giscusData.discussion;

 if (discussion && typeof discussion.totalCommentCount === "number") {
 // Calculate total count including both comments and replies
 const commentCount = discussion.totalCommentCount || 0;
 const replyCount = discussion.totalReplyCount || 0;
 const totalCount = commentCount + replyCount;

 // Update comment count displays on current page
 this.updateCommentCountDisplays(
 currentPath,
 totalCount
 );

 console.log(
 `Giscus: Page ${currentPath} has ${commentCount} comments and ${replyCount} replies (${totalCount} total)`
 );
 } else if (discussion === null) {
 // Discussion doesn't exist (404), default to 0 comments
 this.updateCommentCountDisplays(currentPath, 0);
 console.log(`Giscus: Page ${currentPath} has no discussion (defaulting to 0 comments)`);
 }
 }

 // Handle error messages (e.g., when discussion doesn't exist)
 if ("error" in giscusData) {
 const error = giscusData.error;
 
 // Default to 0 comments on any error
 this.updateCommentCountDisplays(currentPath, 0);
 }
 });
 }

 updateCommentCountDisplays(url, count) {
 // Update comment count elements for this URL on the current page
 const elements = document.querySelectorAll(`[data-url="${url}"]`);
 elements.forEach((element) => {
 const link = element.querySelector("a") || element;
 if (count === 0) {
 link.innerHTML = "0 comments";
 } else if (count === 1) {
 link.innerHTML = "1 comment";
 } else {
 link.innerHTML = `${count} comments`;
 }
 });
 }

 // Initialize comment count manager
 initialize() {
 // The PostMessage listener will automatically update counts when Giscus loads
 // No need for stored counts - fresh data every time
 }
}

// Initialize the comment manager when DOM is loaded
document.addEventListener("DOMContentLoaded", function () {
 window.giscusCommentManager = new GiscusCommentManager();
 window.giscusCommentManager.initialize();
});

One mildly annoying thing is that this counter can take a few seconds to update since the page needs to wait for the Giscus widget to fully load, but it’s a small (and acceptable) price to pay to avoid having user data being tracked and sold!

The Great Migration

Giscus unfortunately doesn’t have a built in way to migrate comments from Disqus to Github Discussions. There are quite a few blog posts about this topic, but they were in languages like Java, Ruby, .NET, none of which I have set up on my laptop. Some of them also did things like attempting to map the Disqus usernames to Github users, but I didn’t want to bother with that since the hit rate seems low anyway.

Predominantly being a Python developer, I figured this would also be a good opportunity to see how far vibe coding can take me through this problem and use it to create a script tailored to my heart’s desires.

Exporting from Disqus

Fortunately, Disqus allows exporting comments to XML. After requesting an export here, I shortly received an email with the contents.

Converting Disqus to Github Discussions

There were a few tricky parts with the conversion:

Disqus has support for multiple thread levels, Github only has one level
Export only has usernames of the commenters
Github has a GraphQL rate limit of 5000 requests/hr
Unless you create a separate Github account, all imported comments will show up under your Github user

The last point might be a deal breaker for some, but I wanted to keep this migration as simple as possible, so I dealt with it. The imported comments will have a header that shows who and when it would be posted, which is sufficient enough for my needs. Update: I added support for posting via a Github App, which removes this issue!

The full script is in this gist, and it was definitely way more lines than I thought it’d be. But it does have some nice features like:

Having a --dry-run flag to test parsing without hitting the Github API
Idempotency by keeping track of published comments in a state file

The vibe coding did take quite a few back-and-forths to get right, but I’m overall impressed with the output and speed.

Script Usage (click to expand):

$ uv run disqus_to_giscus.py --help

usage: disqus_to_giscus.py [-h] [--repo-owner REPO_OWNER] [--repo-name REPO_NAME] [--category-name CATEGORY_NAME] [--dry-run] [--output OUTPUT] [--state-file STATE_FILE]
 [--app-id APP_ID] [--private-key-path PRIVATE_KEY_PATH] [--installation-id INSTALLATION_ID]
 xml_file

Migrate Disqus comments to GitHub Discussions

positional arguments:
 xml_file Path to Disqus XML export file

options:
 -h, --help show this help message and exit
 --repo-owner REPO_OWNER
 GitHub repository owner/organization name (required for real migration)
 --repo-name REPO_NAME
 GitHub repository name (required for real migration)
 --category-name CATEGORY_NAME
 GitHub Discussion category name (default: Announcements)
 --dry-run Preview migration without posting to GitHub (recommended)
 --output OUTPUT Output file for dry run preview (default: migration_preview.md)
 --state-file STATE_FILE
 State file for tracking migration progress (default: migration_state.json)

GitHub App Authentication:
 Use GitHub App for authentication instead of personal access token

 --app-id APP_ID GitHub App ID (can also be set via GITHUB_APP_ID environment variable)
 --private-key-path PRIVATE_KEY_PATH
 Path to GitHub App private key file (can also be set via GITHUB_APP_PRIVATE_KEY_PATH environment variable)
 --installation-id INSTALLATION_ID
 GitHub App installation ID (can also be set via GITHUB_APP_INSTALLATION_ID environment variable)

Environment Variables:
 Authentication (choose one):
 GITHUB_TOKEN Personal access token for GitHub API

 OR for GitHub App authentication:
 GITHUB_APP_ID GitHub App ID
 GITHUB_APP_PRIVATE_KEY_PATH Path to GitHub App private key file
 GITHUB_APP_INSTALLATION_ID GitHub App installation ID

State Tracking:
 The script maintains a local state file (migration_state.json by default) to track
 which discussions and comments have been successfully created. This makes the script
 idempotent - you can safely re-run it after failures and it will resume where it
 left off without creating duplicates.

Authentication Methods:
 1. Personal Access Token (original method):
 - Set GITHUB_TOKEN environment variable
 - Token needs 'repo' and 'write:discussion' scopes

 2. GitHub App (recommended for organizations):
 - Create a GitHub App with discussions:write permission
 - Install the app on your repository/organization
 - Provide --app-id, --private-key-path, --installation-id
 - Or set corresponding environment variables

Examples:
 # Dry run (recommended first)
 python disqus_to_giscus.py export.xml --dry-run

 # Real migration with personal access token
 export GITHUB_TOKEN="your_token_here"
 python disqus_to_giscus.py export.xml --repo-owner myusername --repo-name myrepo

 # Real migration with GitHub App
 python disqus_to_giscus.py export.xml --repo-owner myusername --repo-name myrepo \
 --app-id 123456 --private-key-path /path/to/private-key.pem --installation-id 789012

 # With custom category name
 python disqus_to_giscus.py export.xml --repo-owner myusername --repo-name myrepo --category-name "General"

 # Custom output file for dry run
 python disqus_to_giscus.py export.xml --dry-run --output custom_preview.md

 # Custom state file location
 python disqus_to_giscus.py export.xml --repo-owner myusername --repo-name myrepo --state-file my_migration.json

Using the --dry-run mode will export the parsed comments into a migration_preview.md file, in case you want to edit any processing/formatting to suit your own needs.

$ uv run disqus_to_giscus.py justinmklam-2025-08-26T15_34_53.793894-all.xml --dry-run

🚀 Starting Disqus to GitHub Discussions migration
📖 Parsing Disqus XML: justinmklam-2025-08-26T15_34_53.793894-all.xml
✅ Parsed 277 threads with comments
🔍 Performing dry run - generating preview to migration_preview.md
📊 Existing state: 4 discussions, 28 comments already created
✅ Dry run complete! Preview saved to migration_preview.md
📊 Summary: 8 new threads ready for migration, 4 already exist
🚫 Skipped 265 threads with no comments

Option 1: Posting from a Personal Account

If you want to avoid having to go through the extra steps of creating a Github App, and are okay with all the imported comments being under your account, then this method is for you. If not, move on to the next section!

For actually running the migration to Github Discussions, be sure to create a new personal access token and set it as GITHUB_TOKEN in your environment.

I would recommend testing it out on a new repository first to make sure everything works and looks as expected. I did this with a new giscus-import-test repo, and after verifying the results, I changed the repo to the final repo.

$ uv run disqus_to_giscus.py \
 justinmklam-2025-08-26T15_34_53.793894-all.xml \
 --repo-owner justinmklam \
 --repo-name personal-blog \
 --category-name "Blog Comments"

Now, all my comments are in this website’s repo under the Discussions tab here!

Code and user comments, all in one repo!

As mentioned before, the comment threads all appear under my Github handle, which is not ideal but it’s an acceptable trade-off for a free, no-hassle commenting platform.

Example of an imported comment thread, but all under your own account.

Option 2: Posting from a Github App

After doing the previous option, I did a bit more reading and found out that creating a Github App to do the posting isn’t actually that much more work, and the end result is much cleaner since the comments would then be posted under a bot account. This makes it much clearer that the comments are imported from another platform, and that it’s not just you having a conversation with yourself.

I should probably have just started with this since I did have to delete all the previously created discussions, but it was pretty quick and the end result is much nicer.

Steps to set up a Github App:

Go to your Github developer settings and create a new app
1. Give it a name
2. Set a homepage url (this can be anything)
3. Disable webhooks
4. Set repository permissions to have read/write access to Discussions
5. Upload a custom logo if desired
6. Generate a private key and download it (*.private-key.pem file)
Install the app to your target repository
Make note of the app’s:
1. App ID -> from the app’s about page
2. Installation ID -> go to your Applications, click “Configure” for your app, then use the numeric id in the URL path

Then run the same script, but with a few different options (and this time, GITHUB_TOKEN isn’t required):

$ uv run disqus_to_giscus.py \
 justinmklam-2025-08-26T15_34_53.793894-all.xml \
 --repo-owner justinmklam \
 --repo-name personal-blog \
 --category-name "Blog Comments" \
 --app-id 12345 \
 --private-key-path giscus-comment-bot.2025-08-27.private-key.pem \
 --installation-id 67890

Example of an imported comment thread using a Github App. Nice!

Closing Thoughts

I was able to get the migration done in an evening, so I’d call that a success! Claude Code definitely did a lot of heavy lifting, but it was a neat experiment in using AI for menial, one-off automations that would normally be more painful and time consuming. And now, all my website code and comments are in one repository, free of advertisements and selling of user data.

Hope this helps anyone else planning to migrate away from Disqus!

Handwiring a 36-Key Split, Sculpted Keyboard

Tue, 05 Aug 2025 14:35:20 -0700

As I continued the descent into my split keyboard addiction, I decided to dive off the deep end and handwire my own ergonomic keyboard.

My Corne was a good intro keyboard, but I eventually determined it had more keys than I actually needed. I also grew intrigued towards keyboards with better ergonomics - more stagger and splay to improve alignment with ring and pinky fingers, as well as keyboards with sculpted, organic surfaces to better follow my hand’s natural shape.

Other keyboards like the Charybdis seemed to be quite popular, but I was searching for something smaller and less bulky looking. I also didn’t want to design my own using Cosmos just yet, since I knew it’d take a few iterations to get right which would be difficult without my own 3D printer.

The Skeletyl spoke to me on an spiritual level, and although it lacked an integrated pointing device like a trackpoint, it checked enough boxes for me to be sold on that form factor.

Since I had never used a keyboard like this before, I was hesitant to put too much money into it. Prebuilt options and kits were going for >$300 CAD, which was more than I was willing to empty my wallet for something that might just be a fad for me. Although I could have tried sourcing the PCBs myself, this keyboard was a perfect candidate to try handwiring due to its complex shape as well as keeping costs low since I had most of the equipment already.

So with my soldering iron and wires in hand, I began my quest.

The completed Skeletyl in its natural habitat.

Close up of the left half. The sparkle pattern came through quite well!

Backside of the keyboard showing the USB-C and TRS ports.

The Build

Bill of Materials

My friend from Ember Prototypes was kind enough to print the case for me, but if you don’t have access to a 3D printer yourself or a friend who does, many local libraries have them and are usually very affordable. Otherwise, places like JLCPCB offer printing for not too much.

Total cost for my build was ~$70 CAD, where $45 of it was from the keycaps and (relatively expensive) switches. Aside from the case, all parts were sourced from Amazon and Aliexpress.

If you’re looking to build one yourself, based on the parts I found it can range from $45-100 CAD. Getting the build to be <$40 would likely be possible if you hunt a bit harder for better deals, especially since switches and keycaps make up for a large portion of the overall cost. Alternatively, 3D printing your own keycaps is an option to further save some money.

For my build, I chose the same TTC Frozen Silent v2 switches that I used on my Corne keyboard, and I bought blank PBT in the XDA profile. I tried Cherry/OEM profile, but I wasn’t a fan of feel or look of the sharp corners.

Part	Quantity	Cost (in CAD)
3D printed case (top + bottom)	2	$0-15
M4 heat-set threaded inserts	12	$2
M4 5mm fasteners	12	$3
1N4148 diodes (through hole)	36	$1
TRS/TRRS connectors	2	$3
RP2040 Zero dev board	2	$12
TRS cable	1	$4
MX switches	36	$5-30
MX-compatible keycaps	36	$15-30

Schematic

The wiring followed a basic row/column matrix, where the diodes were connected along the rows, and the remaining switch legs were wired directly together in columns.

On the MCU, the pins required were:

Switch matrix, 4 rows x 5 columns = 9 GPIO pins
TRS connector (soft serial) = 1 GPIO pin, VCC, GND

RP2040 Pinout. (Source: Waveshare)

In order for both sides to communicate with each other, there are a few different ways that QMK supports. I opted for the simplest of using soft serial so only three wires were required. For the RP2040, the soft serial pin (using the PIO driver) needed to be GP1.

For the matrix columns and rows, I chose the pins on the right side to make it a bit easier to assemble. Any valid pins can be used, as long as they’re mapped correctly in the QMK keyboard.json, where mine looks something like this (full configuration below).

	...
 "matrix_pins": {
 "cols": ["GP4", "GP5", "GP6", "GP7", "GP8"],
 "rows": ["GP9", "GP10", "GP11", "GP12"]
 },
 "split": {
		...
 "soft_serial_pin": "GP1",
 "matrix_pins": {
 "right": {
 "cols": ["GP8", "GP7", "GP6", "GP5", "GP4"],
 "rows": ["GP9", "GP10", "GP11", "GP12"]
 }
 }
 },
	...

Assembling the Hardware

Despite having quite a bit of soldering experience, handwiring still took longer than expected - the first half took about 3-4h, and the second half took maybe 2-3h once I figured out the best way to do certain things. It seems like such a simple task of just connecting wires together, but as with most things, it still takes time and effort. Though it did turn out to be a pleasant change of pace to my typical programming work, since the manual, dexterous, but not cognitively-challenging labour required a different type of patience that ended up being a bit meditative.

The models for the case (without the additional tenting) were downloaded from Printables, and was printed on a Bambu Lab P1S in the Onyx Black PLA Sparkle.

Test fitting the switches and a few keycaps to get a sense of how the sculpted profile feels.

Using a soldering iron to install the heatset inserts.

Coiling the diode leg to make it easier to attach to the switch pin.

Diodes placed, marking the wire locations to splice.

For splicing the wire, this turned out to be easier to do than expected. Joe Scotto’s way is to use bare copper wire and heat shrink the intersections, but I didn’t think it was actually that much faster since it still takes time to cut, place, and heat the heat shrink.

Using basic wire cutters and an x-acto knife to expose the wire.

Rows and columns all connected!

One weird aspect of sculpted keyboards is how the thumb clusters are on a different plane than the rest of the keys. The wiring ended up looking a little funny, but I wanted to hide it reasonably well to prevent them peeking through the exposed sections of the case.

Close up of how I wired the thumb cluster keys.

With the matrix all done, all that was left was to wire each row and column to the MCU. I chose to solder these blue wires at the ends of each row and column to keep things looking tidy. The MCU was oriented upside down so the reset buttons would still be easily accessible when fully assembled.

For the TRS connector, I initially bought ones on breakout boards, but it turned out to be way too bulky so I had to desolder it. Was a bit of a pain, but it worked out in the end.

RP2040-Zero all wired up.

Doing everything again for the other half, and it was complete!

Build complete!

I initially thought to design a nice 3D printed bracket to hold the MCU and TRS connector, but I was impatient and opted for the hacky way of just using hot glue since I was likely only going to be building this once.

Surprisingly, the glue was sturdy enough to hold the components in place even when plugging/unplugging the cables from them.

Securing components with “structural” hot glue.

Compiling the Firmware

QMK

I followed the QMK tutorial and forked the repo, ran the new keyboard command, and then updated the config. With the RP2040, the bootloader and processor needed to just be RP2040 instead of the dev board, which was one of the default options.

It took a while to figure out what the correct configuration was for the layout, but I eventually got it done with a bit of help from ChatGPT.

For the firmware flashing, I just set the split enabled flag for each side and commented out the opposite side. Maybe next time I’ll use EEPROM or setting a pin high to avoid needing to build different firmware for each side.

#pragma once

/* RP2040- and hardware-specific config */
#define RP2040_BOOTLOADER_DOUBLE_TAP_RESET // Activates the double-tap behavior
#define RP2040_BOOTLOADER_DOUBLE_TAP_RESET_TIMEOUT 500U
#define PICO_XOSC_STARTUP_DELAY_MULTIPLIER 64

#define SERIAL_PIO_USE_PIO1

In rules.mk:

SERIAL_DRIVER = vendor

And the keyboard.json:

{
 "manufacturer": "Justin Lam",
 "keyboard_name": "skeletyl",
 "maintainer": "Justin Lam",
 "bootloader": "rp2040",
 "processor": "RP2040",
 "diode_direction": "COL2ROW",
 "features": {
 "bootmagic": false,
 "extrakey": false,
 "mousekey": true,
 "nkro": false
 },
 "matrix_pins": {
 "cols": ["GP4", "GP5", "GP6", "GP7", "GP8"],
 "rows": ["GP9", "GP10", "GP11", "GP12"]
 },
 "split": {
 "enabled": true,
 "main": "right",
 "soft_serial_pin": "GP1",
 "transport": { "protocol": "serial" },
 "matrix_pins": {
 "right": {
 "cols": ["GP8", "GP7", "GP6", "GP5", "GP4"],
 "rows": ["GP9", "GP10", "GP11", "GP12"]
 }
 }
 },
 "url": "...",
 "usb": {
 "device_version": "1.0.0",
 "pid": "0x0254",
 "vid": "0xFEED"
 },
 "layouts": {
 "LAYOUT_split_3x5_3": {
 "layout": [
 {"matrix": [0, 0], "x": 0, "y": 0.25},
 {"matrix": [0, 1], "x": 1, "y": 0.125},
 {"matrix": [0, 2], "x": 2, "y": 0},
 {"matrix": [0, 3], "x": 3, "y": 0.125},
 {"matrix": [0, 4], "x": 4, "y": 0.25},
 {"matrix": [4, 0], "x": 7, "y": 0.25},
 {"matrix": [4, 1], "x": 8, "y": 0.125},
 {"matrix": [4, 2], "x": 9, "y": 0},
 {"matrix": [4, 3], "x": 10, "y": 0.125},
 {"matrix": [4, 4], "x": 11, "y": 0.25},
 {"matrix": [1, 0], "x": 0, "y": 1.25},
 {"matrix": [1, 1], "x": 1, "y": 1.125},
 {"matrix": [1, 2], "x": 2, "y": 1},
 {"matrix": [1, 3], "x": 3, "y": 1.125},
 {"matrix": [1, 4], "x": 4, "y": 1.25},
 {"matrix": [5, 0], "x": 7, "y": 1.25},
 {"matrix": [5, 1], "x": 8, "y": 1.125},
 {"matrix": [5, 2], "x": 9, "y": 1},
 {"matrix": [5, 3], "x": 10, "y": 1.125},
 {"matrix": [5, 4], "x": 11, "y": 1.25},
 {"matrix": [2, 0], "x": 0, "y": 2.25},
 {"matrix": [2, 1], "x": 1, "y": 2.125},
 {"matrix": [2, 2], "x": 2, "y": 2},
 {"matrix": [2, 3], "x": 3, "y": 2.125},
 {"matrix": [2, 4], "x": 4, "y": 2.25},
 {"matrix": [6, 0], "x": 7, "y": 2.25},
 {"matrix": [6, 1], "x": 8, "y": 2.125},
 {"matrix": [6, 2], "x": 9, "y": 2},
 {"matrix": [6, 3], "x": 10, "y": 2.125},
 {"matrix": [6, 4], "x": 11, "y": 2.25},
 {"matrix": [3, 2], "x": 2.5, "y": 3.25},
 {"matrix": [3, 3], "x": 3.5, "y": 3.5},
 {"matrix": [3, 4], "x": 4.5, "y": 3.75},
 {"matrix": [7, 0], "x": 6.5, "y": 3.75},
 {"matrix": [7, 1], "x": 7.5, "y": 3.5},
 {"matrix": [7, 2], "x": 8.5, "y": 3.25}
 ]
 }
 }
}

Vial?

My previous keyboard came with Vial, and it was immensely helpful to be able to play around with different key layouts and tweaks without having to compile and flash for every small change. Since I had my 36 key layout dialled in (or so I thought, more on that later), I thought that having a relatively stable layout in QMK would be sufficient.

And then I actually went through the process of compiling/flashing firmware, and I immediately wanted Vial back!

The main reason was because the process to make changes on a split keyboard are:

Update configuration
Run command to compile firmware
Unplug USB cable from keyboard
Unplug TRS cable from both halves
With one half, turn upside down and press reset on the MCU to enter bootloader mode
Plug USB cable back in
Open file explorer, move firmware file to bootloader
Unplug USB cable to exit bootloader mode
Repeat step 5-8 for other half (may not be necessary depending on type of change)
Plug TRS cable back in to both halves
Plug USB cable back in to keyboard
Test updated configuration
Repeat from step 1 as needed

Compared to the process with vial:

Open vial.rocks in web browser
Update configuration
Test updated configuration
Repeat from step 2 as needed

Way easier and faster right? Especially when fiddling with timing or mouse key configurations, it’s just a much better development cycle since vial enables immediate changes.

Yes, Vial

Anyway, setting up Vial was relatively straightforward. It involved creating a new config from the QMK one (following the porting guide here, and adding an extra config file that describes the visual representation of the keyboard. This config maps what you see in the UI to the keys in QMK.

It was mildly annoying since I initially forked the QMK repo, and Vial requires forking their repo instead, so I just had to copy files into the Vial repo to continue from there. But I guess I should have read all the docs first!

All files listed below are saved under keymaps/vial.

In config.h:

#pragma once

#define VIAL_KEYBOARD_UID {0x11, 0x6B, 0x9E, 0x21, 0xCB, 0x6C, 0xB7, 0x37}

#define VIAL_UNLOCK_COMBO_ROWS { 0, 2 }
#define VIAL_UNLOCK_COMBO_COLS { 0, 4 }

In keymaps/vial/rules.mk:

VIA_ENABLE = yes
VIAL_ENABLE = yes

And vial.json:

{
 "name": "skeletyl",
 "vendorId": "0xFEED",
 "productId": "0x0254",
 "matrix": {
 "rows": 8,
 "cols": 5
 },
 "layouts": {
 "keymap": [
 [ "0,0", "0,1", "0,2", "0,3", "0,4", { "x": 2 }, "4,0", "4,1", "4,2", "4,3", "4,4" ],
 [ "1,0", "1,1", "1,2", "1,3", "1,4", { "x": 2 }, "5,0", "5,1", "5,2", "5,3", "5,4" ],
 [ "2,0", "2,1", "2,2", "2,3", "2,4", { "x": 2 }, "6,0", "6,1", "6,2", "6,3", "6,4" ],
 [ { "x": 2.5 }, "3,2", "3,3", "3,4", { "x": 1 }, "7,0", "7,1", "7,2" ]
 ]
 }
}

After flashing, my keyboard then showed up under vial.rocks!

Vial web user interface.

Required Adjustments

Keyboard Layout

I thought my 36 key layout would be completely fine as is, but it was not. Since the thumb cluster was wider, the farthest thumb no longer felt as comfortable or easy to hit for me since it required my hand to stretch across a large distance, especially for combos like CMD+T. For modifiers, I started to use bottom row mods in addition to the thumb clusters, which ended up working quite well.

Desk Ergonomics

Since the keyboard is taller than normal, I started experiencing wrist and shoulder fatigue after a few hours in the first week of using it. Wrist/palm rests also did not help. After some research, I eventually came across advice that changing the height of your desk and/or chair are needed to accommodate the keyboard’s height.

Since I use a floor desk, my desk is the only thing that has height adjustability, so I lowered it 1/2” and found it was a big improvement to my posture. Maybe one day I’ll recess my keyboard into my desk?

I also found that having the two halves closer and slightly angled inwards to be more comfortable than strictly at shoulder width apart. I wonder if it’s something to do with the height of the keyboard and how my forearms aren’t completely parallel with the ground? Either way, the nice thing about having a split is the ability to easily move them around to find the best position. Or as physiotherapists like to say, “the best position is your next position”!

Final Thoughts

One interesting thing about sculpted keyboards is that if the rubber feet are a bit dusty, the keyboard would have a tendency to shift inwards while typing. This is not an issue with flat keyboards since the direction of force is vertically down, whereas here, the force vectors are also horizontal. This can be mitigated by making sure the rubber feet are clean and have good grip, or adding small weights to the case (e.g. with motorcycle wheel balancing weights).

I also noticed how my hands wanted to rest in home row (e.g. the lowest sloped area of the keyboard). In contrast to a typical flat keyboard, where my right hand would sometimes rest one key over on HJKL (because of vim and vim-like keybindings on many apps I use), it didn’t matter as much since my hands could just shift laterally without significantly changing my wrist angle/position. It required some adjustment to train my right hand to really stay on home row and use my forefinger to span the two columns YHB and UJN instead of just one, which I found particularly difficult since reaching over with my forefinger often led to bumping into the tallest thumb key.

The curvature for the ring and pinky fingers were quite comfortable for me, and being able to press the top outermost keys with my pinkies was a nice change.

The thumb clusters could also be a little lower and shifted slightly inward, since the outer key is hard to reach for my (somewhat) smaller hands. But this is getting into specific hand morphology, so YMMV.

All in all, I was quite pleased with how it turned out, and I’ve been daily driving this keyboard for a few weeks now. Unfortunately, building my first keyboard from scratch has opened the Pandora’s box of the desire to build even more keyboards, so time will tell if my journey ends here or continues on…

Optimizing My Symbol Layer: A Data Driven Approach

Mon, 28 Jul 2025 10:37:03 -0700

When first designing my layout on my Corne keyboard, I was mostly focused on the macro level of what layers keys should go on as well as the ease of common workflows like selecting text or switching workspaces. I put some thought into the placement of each symbol, but after growing quite comfortable with my keyboard and having a nagging feeling that something about the placement of my symbols felt off, it was finally a good opportunity to revisit the micro details of my symbols layer.

Why Go Through the Hassle?

You might be wondering - why not just copy the symbol positions of a standard keyboard along the top row? While this works, it’s not the most effective, or at least not in my experience. With the numbers across home row and the corresponding symbols along the top row, I found it kind of difficult to remember where the second half of the numbers were (e.g. was 7 on my middle or ring finger?). Switching the numbers to be a numpad layout was much easier to remember for me, since it leveraged the same muscle memory as in standard layouts as well as allowing for single-handed entry.

Standard layout of a QWERTY keyboard. (Source: Das Keyboard)

A popular layout for 3x6 keyboards. Numbers and symbols run left to right to mimic a standard QWERTY layout. (Source: Mark Stosberg)

Although I could have used the numeric position as the symbols’ position (e.g. ! at 1, @ at 2, etc.), I would still need to train myself to commit the new positions to memory. But if I’m doing that anyway, I may as well create a personalized, brand new layout for symbols to memorize!

I initially just put them in order spanning the rows on the left half, which worked okay but I figured it would be better to put my most used symbols on the home row for ease of use.

The pic below is from another blog I came across during my research, which showed their grid of relative effort for each key placement. I generally agree with most of it, except I find the bottom pinky position actually quite easy, likely because of my habit of using my pinky to hold down the SHIFT key (in lieu of using CAPSLOCK), and/or because my pinkies are strong from rock climbing.

Relative effort of each key position; lower is better. (Source: Jonas Hietala)

I mainly use Python at work, so these symbol usages will be biased towards that. Every language will have its own set of common symbols (as explained in depth by Pascal Getreuer, so keep that in mind if you decide to design your own layout.

Getting Nerdy With It

Making a Keylogger

To start, I created a simple python script to log all my keystrokes. I left it running in the background for a week or so to gather enough data, since my day to day involves both coding and writing documentation. The script only saves to a persistent file when it’s terminated, but this didn’t seem to cause any data loss issues for me.

import json
from pynput import keyboard
from pathlib import Path

# Path to the JSON file where stats will be stored
STATS_FILE = Path("key_stats.json")

# Load existing stats or initialize a new dictionary
def load_stats():
 if STATS_FILE.exists():
 with open(STATS_FILE, "r") as f:
 return json.load(f)
 return {}

# Save stats to the JSON file
def save_stats(stats):
 with open(STATS_FILE, "w") as f:
 json.dump(stats, f, indent=4)

# Update key stats
def update_stats(key, stats):
 try:
 key_str = (key.char if hasattr(key, 'char') and key.char else str(key)).lower()
 except AttributeError:
 key_str = str(key)
 
 if key_str in stats:
 stats[key_str] += 1
 else:
 stats[key_str] = 1

# Key press handler
def on_press(key):
 global stats
 update_stats(key, stats)
 save_stats(stats)

# Initialize stats
stats = load_stats()

# Start listening to key presses
print("Keylogger started. Press 'Ctrl + C' to stop.")
with keyboard.Listener(on_press=on_press) as listener:
 try:
 listener.join()
 except KeyboardInterrupt:
 print("Keylogger stopped.")
 save_stats(stats)

The output data file looked something like this, which made it easy to visualize afterwards.

{
 "key.cmd": 1259,
 "k": 5877,
 "key.shift": 3153,
 "l": 1953,
 "h": 1803,
 "e": 3930,
 "y": 1119,
	...
}

I manually split up the json file into alpha, symbols, and misc files to graph the keys in logical groups. I considered building this into the script, but decided not to since it didn’t seem worth the effort for this one-time data collection and analysis.

Patterns, Pain Points, and Priorities

Alphanumeric Keys

J and K were the most used, most likely because of vim. Yes, they aren’t the most efficient motions, but they do add up in usage due to many one off line moves (in comparison to larger motions like CTRL+D, CTRL+U, CTRL+[, CTRL+], etc.).

Nothing too surprising here, except maybe how infrequently X, Q, and especially Z were not used by a relatively wide margin.

Alphanumeric keys

Symbol Keys

As mentioned before, I predominantly use Python (and vim) at work, so these symbols will reflect that (e.g. high usage of : for vim commands, low usage of ; since Python doesn’t use semicolons at the end of each line like C/C++ do). I’m actually surprised that square brackets are my least commonly used bracket types since it’s used for array indices and dictionaries, but I guess it’s not as frequent as I thought.

It’s no surprise that the symbols @|$%&^ are the least used, so the placement of those symbols will be fine in less-optimal positions.

Symbol keys

Miscellaneous Keys

Since I use macOS, CMD and ALT are the most used modifiers (e.g. CMD+C to copy, ALT+LeftArrow to navigate text by word). CTRL would likely be lower if it weren’t for my vim keybindings, although on Linux or Windows, I’d imagine it’d take the place of CMD (e.g. since on those platforms, copying is CTRL+C).

Interestingly, the arrow keys are fairly equal in usage. I also wonder if it’s common for SPACE and BACKSPACE to be so close in usage together - maybe I make a ton of typing mistakes?

Miscellaneous keys

Tweaking for Ease of Use and Comfort

With this data, I can now put the symbols in places that make a bit more sense!

I kept ! and @ in the same positions out of habit, and it didn’t really make sense to move those. Home row now contains #'"-=, and I opted to put _ where SHIFT would be since I find it quite comfortable to hit that bottom corner key with my pinky while typing variable names (e.g. some_variable_like_this), which leverages the same “hold shift while typing for all caps” muscle memory. I also decided to add ' and " as separate keys to make those easier to type, especially with " no longer requiring an additional SHIFT key to hit.

$%^& are now on the bottom row since they don’t see as frequent usage.

Keen readers might notice that the parentheses, brackets, and braces are nowhere to be found. They weren’t forgotten, but rather promoted to my main layer as combos! See my main keymap here for details. I did switch the position of [] and () so the latter is on home row instead, based on it being more frequently used from my data above.

Before…

… And after. Note that no changes were made on the right half.

After a few weeks of using this layout, I can say for sure that it’s quite an improvement. Common symbols are easier to type, and as a result, it was also easier to memorize these new symbol placements.

Admittedly, I still find myself using my alpha-layer combo of L+; to type the quotation more often, so time will tell if I decide to replace those two home row positions with something else. Otherwise, I’m quite happy with the result of this exercise.

Hooray for data driven decisions!

Transitioning to a 36-Key Layout

Wed, 16 Jul 2025 09:50:41 -0700

Having used my Corne keyboard for some time now, I kept seeing 34/36 key layouts on r/ErgoMechKeyboards with a growing amount of envy. I mean, the minimalist aesthetics of the Ferris Sweep are just so enticing! Staring down at my whopping 42 keys, I found myself not actually using the outer columns that often and wondered if I too could join the ranks of those using even fewer keys…

So naturally, I removed the extraneous keys, covered the openings with electrical tape, and began the pursuit of trimming my layout with the goal of (arguably futile) minimalism and “efficiency”.

The aesthetic allure of having only 34 keys. (Source: Github)

Why …

… Fewer Keys?

Citing ergonomics would be the knee-jerk reaction, where with this smaller layout, each key is at most 1u key away from a finger. There’s some debate about whether fewer keys is actually better, since you end up trading off more hand movements for more combos (as well as cognitive load), as described in more detail in this blog post. I personally found smaller layouts to be more comfortable to use since the reduced wrist and finger stretching induced noticeably less strain on those body parts, which far outweighed the need for layers, combos, and memorizing a new layout.

Granted, there’s probably a sweet spot of reduced keys that is greater than ~30-40% to gain ergonomic benefits without the need to learn a radically new layout. Take the Silakka54 as an example, which contains a number row and outer columns, so it’s a minimal departure from the layout of a standard keyboard.

Although for me, it’s honestly fun to dive into creating a personalized layout that suits my own computing needs, and the aesthetics of having such a minimal keyboard are hard to beat (through my rose-tinted glasses anyway).

… 36 and Not 34 Keys?

It should be noted that going from 42 to 36 keys was relatively straightforward, since I was able to place all the required modifiers across the thumb clusters while still having layer keys.

However, going down to 34 keys (thus removing two of the thumb keys) makes the reduction much more challenging, since relocating the modifiers to no longer be on dedicated keys requires a bit more creativity. Mod taps become a necessity (e.g. through the use of home or bottom row mods), or something like Callum mods which uses one shot modifiers on a separate layer. This was something I wasn’t quite ready to do yet, so I decided on a “sane” reduction to 36 keys.

Relocating the Outer Keys

The main challenge in reducing a 42-key layout to 36 keys were figuring out where to put the outer column keys, which consisted of TAB, ESC, SHIFT, BACKSPACE, ', and ENTER. For the keys on the inner column, they didn’t see much use anyway since I just mapped them superfluously to VOL+, VOL-, RGUI, and RALT, so losing those wasn’t a big deal.

Previous 42-key layout.

Tab, Enter

I added these keys to the remaining thumb cluster keys that weren’t using tap modifiers (e.g. LGUI and RSFT). No major issues or adjustments here, since it was fairly similar to hitting SPACE with a thumb. ENTER was placed on the opposite side of LGUI so that combo could still be easily used for some desktop and web apps (e.g. to submit a comment in GitHub or Jira).

Backspace

Out of habit, I had this in the top right corner, but I eventually got used to using backspace with my thumb so it became redundant. I thought I would miss the LALT+BSPC combo for easily deleting words (on macOS), but I found that hitting LALT with my left thumb and BSPC with my right thumb was equally as convenient.

Escape

After adding the J+K combo to be ESC, I quickly became accustomed to it since it was just so convenient! Again, this outer position became redundant, and was an artifact of remapping CAPSLOCK to “CTRL when held, ESC when tapped”.

Apostrophe, Quotation Mark

Not having the apostrophe and quotation mark in the normal position was definitely a bit jarring. I initially tried putting it with the rest of the symbols in my symbols layer, but it just didn’t feel natural. Things clicked once I created a combo on L+;, and I’ve been using that since.

Shift

I previously tried home row mods, but couldn’t get used to it without causing accidental misfires. I also tried using shift on one of the thumb keys, but I found that it was difficult to keep shift held down when typing all caps. Yes, I could just use capslock, or better yet capsword, but the latter was not available on my keyboard’s version of vial (since it was purchased as a prebuilt without access to the firmware), and I’m not a fan of toggling capslock on/off while typing.

I wanted to keep the shift on the pinky, so I changed Z to be “SHIFT when held, Z when pressed” (aka SHIFT_T[Z]). This worked decently until I started typing quickly, which then resulted in shift not being registered (e.g. Instead of “La”, it would output “zla”).

This was mildly frustrating, so I started playing around with one shot modifiers. I mapped it to the S+D combo, and it actually flowed quite well into my typing habits. I still occasionally had misfires of an extra shifted letter when typing quickly (e.g. “LA” instead of “La”), but this happened far less often then with SHIFT_T[Z].

However, for holding down shift while typing as well as for combos (e.g. selecting text), I still use SHIFT_T[Z] for that, which works without issue. Optionally, I can hit S+D twice in succession to enable or disable holding the shift key (so effectively capslock). Some may find it burdensome to having too many different options to remember, but I like how they serve different use cases: shifting while typing is a different rhythm and hand position than shifting while navigating, doing motions, etc.

With SHIFT_T[Z], I figured I might as well try adding the rest of the modifiers (ALT, CTRL, GUI) as tap mods, so home row mods but on the bottom row. This actually worked surprisingly well, and removed many of the misfire issues I experienced with home row mods without having to fiddle around with tapping term or enabling chordal hold or flow tap (which again, my version of vial did not support). Ironically, this made me realize that transitioning to 34 keys would not be as out of reach as I initially thought, but I shelved that idea to be revisited another time.

Revised Keymap

The diagrams below show my “final” 36 key layout.

The main changes were on the main layer, although I did rearrange the function keys on the last layer since I could no longer span F1 to F12 along the top row (which didn’t make much sense to do anyways). The left half RGB controls were replaced with the function keys. In retrospect, I could have matched the order of the function keys to mimic the numpad (or even be in the same position as them), but I’d rather have the media keys on the right side (since I use the mouse on my left hand). I rarely use the function keys anyway, so they’re just there in the odd time that I need it.

Layer 1: Letters

Layer 2: Symbols

Layer 3: Navigation

Layer 4: Miscellaneous

I’ve been using this layout with success for a few weeks now, and it’s been quite good. Now to start shopping for an actual 36 key keyboard…

The pursuit of minimalist computing continues…

Six Months with a Corne Keyboard

Tue, 27 May 2025 15:57:51 -0700

It all started when I first saw a tenkeyless keyboard. Compared to a standard 104-key keyboard, having only 87 keys (an entire 17 keys fewer!) seemed like the next best thing to sliced bread. Since I wasn’t a big user of the numpad, it seemed like a no brainer to reduce desk space, bring my mousing hand a few inches closer, and just feel like I was one with r/PCMasterRace. Oh how naive I was.

Little did I know that it was simply the beginning of a long, arduous journey in search of elusive perfection. Perhaps what I was looking for is actually from within? Or, perhaps I just needed to buy another keyboard. I went with the latter, and ended up with a minimalistic-ly cute 46-key keyboard called the Corne.

Backstory

The table below shows most of the keyboards I’ve tried over the years. I never got too deep into the world of mechanical keyboards since my pockets were (fortunately?) not as deep as my heart wanted them to be.

Date	Keyboard	Switches
July 2016	CM Storm TKL	MX Cherry Blue
Aug 2016	Lenovo TrackPoint Keyboard	Membrane
Sep 2017	Poker 61	MX Cherry Black
Nov 2017	Havit Low Profile TKL	Kaihl Blue
Mar 2019	Tada68	Gateron Silent Red
Apr 2021	Apple Magic Keyboard	🦋
Jan 2025	Corne	TTC Frozen V2 Silent

I stuck with the TKL format for a while, but was always intrigued by smaller layouts. I tried the Poker, but I found it cumbersome not having arrow keys easily accessible, since it made things like selecting text by word require some nasty finger gymnastics (e.g. CTRL+SHIFT+[LAYER+LEFT]). A few boards later and after some ridicule from a coworker for having tried more keyboards than he’s used in his lifetime, the Tada68 became my daily driver.

Tada, what a nice compact layout! Is this the end game? (Source: Drop)

I was previously using Linux computers for personal and work, but when I started my new job in 2021, I was given a MacBook and was quickly frustrated with the difference in layouts. I then purchased a Magic Keyboard so the layout would be the same whether I was at my desk or on the go, and eventually came to appreciate the shallow keys since it allowed me to type quite quickly.

However, a part of me kept wondering if something was missing from my life. Obviously it was another keyboard that was needed, but this time I looked towards ergonomic, mechanical keyboards.

Previously, I had tried the Microsoft Sculpt and the Kinesis Freestyle, but they never really clicked for me. I wanted something that spoke to me on a spiritual level. Perusing r/ErgoMechKeyboards was just what I needed, and the simplistic-yet-still-practical layout of the Corne caught my eye. I bit the bullet with (a relatively inexpensive) prebuilt one on Aliexpress for ~$80 CAD, and a few weeks later it arrived at my doorstep, ready to fill my emotional void.

For reference, here are some of the other keyboards I considered, but ultimately settled on the Corne due to a combination of features, aesthetics, cost, and ease of purchase.

Lily58
Sofle
Totem
Ferris Sweep
Let’s Split
Iris
Ergodox Voyager
Ergodox Moonlander
Naya Create <- Way out of budget but it sure looks nice!

Getting Used to the Corne

Learning Curve

It felt like I was in elementary school again, playing intro typing games like home row and Math Blasters in the computer lab, trying to make sense of this piece of hardware that turns thoughts into words on a screen.

I can usually type around 100-110 wpm, but the first few hours with this layout was definitely an adjustment. Started off with a paltry 30 wpm, steadily increased back up to my normal typing speed. With some practice, I was starting to get the hang of the columnar stagger.

After about 1-2 weeks, I was typing steadily enough to use it for work, albeit still nowhere close to my usual flow. At the 3-4 week mark, I became comfortable with the layout and started to get a better understanding of what worked and didn’t work in my layout.

Steadily increasing WPM over the span of 3 months. (Source: MonkeyType)

Ergonomics

Many of the keyboards I saw online were tented, so it seemed like the norm to try it out. Since mine didn’t come with a tenting kit or mechanism, I used a book to achieve the same angling to see how it felt.

Book-based tenting in action.

It was ok. I didn’t find it to be a drastic improvement over being flat. I also didn’t like how it left my wrists floating above the surface. Perhaps I needed to take the ergonomics more seriously and try a little harder…

So I went deeper. Bought some ball mount clamps and went full ergo, or at least tried to. Unfortunately it didn’t really work with my set up. It was too high for my elbows and, although it was comfortable for my wrists, it felt like my arms and shoulders couldn’t really relax. It also made using the mouse even more inconvenient, but that was a secondary problem.

I think a set up like this would do better on a standing desk, or if you have a better, more flexible way of mounting them. I found the clamped position to not be optimal, and it also pushed me farther from my desk and monitor.

Went a little too hard on the ergo-front.

Ultimately, I went back to just a flat configuration. I found that having the sides separated was more important than the angle, at least for me.

My humble floor desk.

Switches

The keyboard came with some Leopold switches, which actually felt decent but were loud enough to warrant my wife granting me more budget to get something quieter.

Name	Actuation Force	Comment
Leopold Grayborg	40gf	Felt ok, but very loud.
Akko Silent Fairy	50gf	Mushy and scratchy. Would not recommend.
TTC Frozen v2 Silent	39gf	Impressed with how silent they are while feeling crisp and smooth.

After trying a handful of different ones, I settled on the TTC Frozen v2 switches. They might be even better if lubed, but they passed both mine and my wife’s sound test, so they were a keeper! A nice bonus is that due to their clear design, the keyboard lighting shines brightly and diffusely around the keycaps.

Switches galore!

Keymap

I tried to keep a handful of things in mind when designing the layout of my keyboard:

Certain keys/combos should still be usable with one hand, e.g:
- Using arrow keys
- Copy/cut/paste
- Cycling through windows/tabs
- Taking screencaps via selected area
Certain mouse + keyboard workflows still need to work, e.g:
- Holding ALT and clicking to open a link in my IDE
- Horizontal scrolling by holding SHIFT with the opposite hand
- Panning certain diagraming tools by holding GUI
Frequently used keys/combos should be easy and comfortable to use, e.g:
- Selecting text by words/home/end
Optimized for writing and software development
Layout should be somewhat mnemonic to ease the learning curve

Through using this keyboard as my daily driver for almost half a year and occasional layout tweaks, I’ve come up with the mapping described below. It’s far from perfect, but enables a pretty smooth workflow for how I use the computer.

Note: This keyboard uses VIAL, which is QMK based but includes a web interface to update the layout on the fly instead of having to flash new firmware every time. Makes it easy to fiddle around and try new configurations without much friction.

Keyboard graphics were generated with Keymap Drawer, which was fed the QMK layout from QMK Configurator. Source files for both tools can be found in this gist.

Main Layer

Main layer consists of letters, modifiers, and some symbols.

The main alpha layer is fairly straightforward, since it follows a standard QWERTY layout.

Things that worked well:

Key combos: They’re a nice way to bring more keys to the main layer without having to switch layers. Open/closing brackets being on opposite sides was kind of neat; it was easy to remember and convenient for programming.
Thumb mapping combos: Key mappings like SPACE|L2 and BSPC|LTCL were nice because they provide double duty on optimally positioned thumb keys.
Thumb layer/modifier combos Having the layer and modifier keys right next to each other (L1 and LGUI, L2 and LCTL) meant that it was possible to use the thumb on both simultaneously to make certain key combos much easier, e.g. LTCL+[L2+H] to be the equivalent of LCTL+LEFT, which switches virtual desktops on macOS.
Breaking symmetry with the right layer key: I mapped the L2 layer key on the leftmost key instead of center (like how it is on the left side), since I found my right index finger to always hover over H instead of the typical home row position of J. This is likely because on the right side, the second last column contains symbols instead of more frequently used letters. I didn’t like having my thumb curled under my index too much, especially since it made using arrow keys a bit awkward.

Things that didn’t work that well:

Optimizing for mouse + keyboard use: Since I mouse on my left hand, doing things like horizontal scrolling (shift + mouse) and panning (GUI + mouse) was initially hard until I duplicated some keys on the right side (RGUI, RALT, RSFT).
Mod tap on letter keys: Tried home row mods for a bit, but didn’t like how the timing was quite finicky and it would sometimes cause delays in keypresses, which was quite frustrating. With the presence of modifier keys on thumb clusters, it didn’t seem like a necessary feature to implement.

Symbols Layer

Symbols and numbers.

I initially set it up so numbers went from left to right along the middle row, then the corresponding symbols above it on the top row. It worked ok, but I found it hard to remember where the higher numbers were (6-0). I switched to numpad on the right side, and found it to be much better. Typing numbers (and things like numeric versions) is quite a bit faster, since it can all be done with the same hand.

There’s duplication of the hyphen which might not be optimal, but I find it handy to be able to use the left one while typing (e.g. hyphenated variable names), and the right one for number-related tasks (e.g. in spreadsheets).

The random LCTL+Q is to lock my laptop with the hotkey GUI+CTRL+Q on macOS. I initially had it be one macro but found it was too easy to accidentally hit, so made it explicit but still be able to do it with one hand.

Things that worked well:

Numpad was easier to remember than having them all in the top row
Hyphen and underscore are typed often, didn’t like them as combos since didn’t feel as precise/fast
Like having # and * above/below each other for vim, since they’re used for cycling forward/back on the currently selected text.

Things that didn’t:

Less frequently used symbols are still hard to remember, e.g. $%^&

Arrow keys, browser, and mouse navigation.

Things that worked well:

Arrow keys in hjkl are also very convenient, but needed layer toggle on the same side (whereas for numpad, it’s opposite).
Arrow key selection is really handy and fast since I can use both hands for the combo - right hand for the arrows (while hand is in home row), and left hand for the modifiers (e.g. SHIFT plus ALT for selecting by word, or CMD for selecting home/end).
Navigation for browser fwd/back and tab next/prev is a pleasure to use.

Things that didn’t:

Don’t really use the mouse wheel keys on the left cluster.
Sometimes use mouse control, but it’s kind of hard to be precise. Also doesn’t easily work with click/drag. But it’s useful for things like refocusing a window/input that the mouse is nearby.

For right handed users, you probably would want to move all the browser nav hotkeys (top row on the right side) to the left, so your mouse can still be used while your hand is on those keys.

Miscellaneous Layer

Layer with everything else.

I used to use this layer mainly for the volume and media keys, but I recently mapped the common ones (volume control, media play/pause) to the extra inner keys on the first two layers, which are more convenient and allow for single hand control.

Other than that, I don’t use this layer very much, aside from the occasional F2 when I want to rename all instances of a variable in my IDE. They’re here just in case I need them. I’ve also seen the function keys mapped to a single side, so I may try that in the future if I end up needing to use these function keys more often.

Using Regular Keyboards?

For the first three months, I exclusively used my Corne keyboard. When I tried using a regular keyboard after that, it definitely felt jarring and took a few minutes to recalibrate my typing. Now, with regularly switching between my Corne and a regular keyboard, I no longer suffer from that initial transition period, and can now type on either at full proficiency. Thank goodness for muscle memory, which fortunately keeps the different layouts well-compartmentalized!

For the record, I did try using my Corne on top of my laptop while I’m sitting on the couch with it, but I found it cumbersome to move it from my desk, as well as it being precariously placed on top of my laptop. Even though there are solutions for keeping the keyboard in place and even disabling the laptop’s internal keyboard, I found it just not as nice or easy to use.

However, I did look into software like Karabiner Elements (for macOS) and keyd (for Linux) to bring some of the niceties over from my Corne. Main things were:

Mapping j+k to ESC
Overloading CAPSLOCK to ESC if tapped, or CTRL if held
Having a navigation layer to turn hjkl into arrow keys

Here’s the keyd config I use for my Linux laptop:

[ids]
*

[main]
capslock = overload(control, esc)
j+k = esc
=+backspace = C-backspace
meta = layer(nav)

[nav]
h = left
j = down
k = up
l = right

Closing Thoughts

Overall, I’ve been quite happy with this keyboard and the functionality that it enables. Having a wireless option would be nice for making it easier to use with a laptop on the go, but the relatively short battery life is a deal breaker for me, since it’ll just become another thing to regularly charge.

A lower profile, choc-style keyboard might be next on my list, but I actually haven’t suffered from any wrist issues that have previously given me issues with MX-height keyboards (without using a wrist rest).

What’s currently on my mind is figuring out a better pointing/mousing workflow. Haven’t decided whether that would look something like a keyboard with an integrated trackball, or if I should dive into a fully keyboard-driven workflow with something like Mouseless.

Or, perhaps this isn’t the end of my keyboard journey, and I should go even deeper and get something like the CharaChorder.

…Or perhaps I’ll just stick with this one for now :)

Hugo: Some Random Tips and Tricks

Fri, 23 May 2025 08:47:00 -0700

My intro to Hugo was back in 2016, a mere 3 years after its inception, when I was in my early days of learning web development. Quite a bit has changed since then, both in what Hugo is capable of and my understanding of how to actually code. I’m far from being a frontend dev, but I’ve gathered a collection of snippets that I’ve stumbled upon along the way in bringing this website up to more modern standards.

Creating New Posts

To create a new post, typically you would do something like this:

hugo new posts/2025/05/my-new-post/index.md

But manually typing date paths is cumbersome, so instead you can use a target like this in your Makefile:

new:
	@if [ -z "$(title)" ]; then echo "Please provide 'title'"; exit 1; fi
	hugo new posts/$$(date +%Y/%m)/$(title)/index.md

Which can then be used like this:

make new title=my-new-post

Markdown Render Hooks

Render hooks can be used to override markdown to HTML conversions to give full flexibility in how components are presented while staying being able to write markdown. As useful as shortcodes are, they typically aren’t compatible with normal markdown editors/viewers, which adds a bit of friction when you need to have two windows open (one for editing, and the other for previewing).

Full docs are here, but the components below are the ones I found that I actually wanted to change.

Headings

Many websites these days have heading anchors that readers can click on, making it easy to reference or share direct sections of a page. Although Hugo doesn’t have this behaviour by default, a simple template can be used to achieve this.

Create the following file:

layouts/_default/_markup/render-heading.html

With content that looks something like this:

<h{{ .Level }} id="{{ .Anchor | safeURL }}" class="anchor">
 <a href="#{{ .Anchor | safeURL }}">{{ .Text | safeHTML }}</a>
</h{{ .Level }}>

And voila, all your headings in your rendered posts are now anchored!

Images

I wanted to customise how images and captions were displayed, as well as being able to use markdown syntax for both images and videos. This template does that, which conditionally renders the two media types differently.

I also wanted to show a build warning when an image can’t be found (e.g. image was moved or renamed without updating the link), so warnf does that.

In this file:

layouts/_default/_markup/render-image.html

Contents look like so:

<!-- This renders markdown image notation into pics or vids with captions -->
{{ $dest := .Destination | safeURL }}

<div class="row img-captioned">
 <!-- Video -->
 {{ if or (hasSuffix $dest ".mp4") (hasSuffix $dest ".webm") }}
 <video class="img-content" autoplay loop muted controls>
 <source src="{{ $dest }}" type="video/mp4" />
 </video>
 {{ else }}
 <!-- Image -->
 {{ $image := .PageInner.Resources.Get $dest }}
 {{ if $image }}
 {{ $image = partial "resize-image.html" (dict "image" $image) }}
 <a href={{ $image.RelPermalink }}><img loading="lazy" class="img-content" src={{ $image.RelPermalink }}></a>
 {{ else }}
 {{ warnf "Image not found for %s: %s" $.PageInner.RelPermalink $dest }}
 {{ end }}
 {{ end }}
 <p class="caption">{{ .Text | safeHTML }}{{ with .Title }} (Source: {{ . | safeHTML }}){{ end }}</p>
</div>

Note: There’s a resize-image.html partial that’s referenced, more on that next!

Pipes

Compiling and Minifying CSS

If you’re using SASS/SCSS in your stylesheets, Hugo can compile them for you with toCSS without having it part of a separate build step. You can also use resources.Concat to concatenate multiple files into a single one, and then optionally use resources.Minify to minify them, which will work on any CSS, JS, JSON, HTML, SVG, or XML file.

An example that combines all of these operations:

{{ $custom := resources.Get "css/custom.scss" | toCSS }}
{{ $main := resources.Get "css/main.scss" | toCSS }}
{{ $styles := slice $main $custom | resources.Concat "/css/styles.css" | resources.Minify }}
<link rel="stylesheet" href="{{ $styles.Permalink }}">

Processing Images

Before page bundles, images were typically stored under your /static directory. But since its introduction in v0.32, it opens up the ability to process images during the build. For example, if you want to resize and convert all your images that are displayed on your website to jpg or webp, you can use the partial below.

Fun fact: Partials can also be used as functions, not just for rendering snippets of HTML.

{{/*
 Usage:
 {{ partial "resize-image.html" (dict "image" $image) }}
 Returns: resized and processed image resource.
*/}}

<!-- Constants -->
{{ $maxSizePx := 1200 }}
{{ $imageType := "jpg" }}

<!-- Input args -->
{{ $image := .image }}

{{ $image_ext := (lower (path.Ext $.image) ) }}
<!-- do nothing for these filetypes -->
{{ if eq $image_ext ".gif" }}
{{ else if eq $image_ext ".png" }}
{{ else if $image }}
 <!-- Resize landscape -->
 {{ if gt $image.Width $maxSizePx }}
 {{ $resizeStrLandscape := printf "%dx %s" $maxSizePx $imageType }}
 {{ $image = $image.Resize $resizeStrLandscape }}
 <!-- Resize portrait -->
 {{ else if gt $image.Height $maxSizePx }}
 {{ $resizeStrPortrait := printf "x%d %s" $maxSizePx $imageType }}
 {{ $image = $image.Resize $resizeStrPortrait}}
 <!-- No resize necessary, just convert -->
 {{ else }}
 {{ $image = $image.Process $imageType}}
 {{ end }}

{{ end }}

{{ return $image }}

Usage of this partial would look something like:

{{ $image := .Resources.Get "posts/myimage.jpg" }}
{{ $image = partial "resize-image.html" (dict "image" $image) }}
<img src="{{ $image.RelPermalink }}"/>

Adaptive Syntax Highlighting

Hugo has built in support for syntax highlighting using Chroma, but the default configuration doesn’t allow for dynamically setting light/dark syntax highlighting. Fortunately there’s an easy workaround with a few configs!

By default, your config.yaml may look something like this, which tells the renderer to use the monokai style everywhere:

markup:
 highlight:
 style: monokai

Instead, we can set noClasses=false to use an external stylesheet, which will allow us to dynamically set the theme depending if the current colour scheme is light or dark. Change the config.yaml to this instead:

markup:
 highlight:
 guessSyntax: true
 noClasses: false

And then generate your desired stylesheets using the Hugo CLI (see here for the full list of supported themes):

# Light mode
hugo gen chromastyles --style=monokailight > static/css/syntax.css
# Dark mode
hugo gen chromastyles --style=monokai > static/css/syntax-dark.css

And then include the following in your header to use these stylesheets depending on the colour scheme:

<link rel="stylesheet" href="/css/syntax.css" media="screen">
<link rel="stylesheet" href="/css/syntax-dark.css" media="screen and (prefers-color-scheme: dark)">

Depending on your other stylesheets, this may be all you need. However, if this still results in some wonky dark mode colours, you can make some manual edits to fix it. The exported chroma stylesheet contains empty classes, which normally would inherit the default text colour from your site, but since we’re defining two stylesheets, it may end up inheriting from the light theme instead.

To resolve this, we’ll need to do a search and replace of { } to { color:inherit } in the second stylesheet, to properly render the text colours based on the colour scheme. So change this:

/* Generated using: hugo gen chromastyles --style=monokai */

/* Background */ .bg { color:#f8f8f2;background-color:#272822; }
/* PreWrapper */ .chroma { color:#f8f8f2;background-color:#272822; }
/* Other */ .chroma .x { }
...

To this:

/* Generated using: hugo gen chromastyles --style=monokai */

/* Background */ .bg { color:#f8f8f2;background-color:#272822; }
/* PreWrapper */ .chroma { color:#f8f8f2;background-color:#272822; }
/* Other */ .chroma .x { color:inherit }
...

Building and Deploying

Since my website is hosted on GitHub Pages, I used to manually build and deploy the assets with a script that looked something like this:

#!/bin/sh
# Usage:
# ./deploy.sh
# ./deploy.sh "Your optional commit message"

# If a command fails then the deploy stops
set -e

printf "\033[0;32mRebuilding website...\033[0m\n"

# Build the project.
hugo # if using a theme, replace with `hugo -t <YOURTHEME>`

# Go To publish folder
cd docs

# Add changes to git.
git add .

# Commit changes.
msg="Rebuild site ($(date))"
if [ -n "$*" ]; then
 msg="${msg}: $*"
fi
git commit -m "$msg"

printf "\033[0;32mPushing updates to GitHub...\033[0m\n"

# Push source and build repos.
git push origin main

However, after learning about CI/CD and that it can be quite easy to set up these days, that manual deployment has now been replaced with a Github Action. It runs on commits to main, which then Github picks up on the gh-pages branch.

name: Github Pages

on:
 push:
 branches:
 - main
 pull_request:

jobs:
 deploy:
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4
 with:
 submodules: true # Fetch Hugo themes (true OR recursive)
 fetch-depth: 0 # Fetch all history for .GitInfo and .Lastmod

 - name: Setup Hugo
 uses: peaceiris/actions-hugo@v3
 with:
 hugo-version: '0.147.1'
 extended: true

 - name: Build
 run: hugo --minify

 - name: Deploy
 uses: peaceiris/actions-gh-pages@v3
 if: github.ref == 'refs/heads/main'
 with:
 github_token: ${{ secrets.GITHUB_TOKEN }}
 publish_dir: ./docs

Bonus: Using Obsidian as a Local CMS

Since Hugo posts are just markdown files, I started using Obsidian to edit the files. Previously, I was writing content in my IDE, but something about it felt a bit off – the virtual atmosphere that promoted writing code just didn’t hit the same for writing creatively.

I wanted a simple WYSIWYG editor to write my posts in, and with Obsidian being more popular than ever these days, I incorporated it into my workflow and have been finding it quite a joy to use! Now, I can write content in a clean interface using markdown, and have the niceties of having the elements rendered similar to how it’d look on my website without needing to have a separate window open to show the preview.

Settings

Be sure to disable “Use Wikilinks” so that markdown format is used for image links, and select “Default location for new attachments” is set to the “Same folder as current file”, which works well if page bundles are used.

Obsidian settings -> Files and links

Note that the filename of the pasted image may not render correctly by Hugo, since it typically doesn’t play well with spaces in image filenames. To fix this, just rename the image in Obsidian, which will automatically update the link in your markdown document.

Plugins

Some days I want to stay within Obsidian and not have to open a separate terminal to commit my changes to Github, and the Obsidian Git Plugin makes that possible. After setting it up, you can do normal git operations straight from Obsidian for a truly seamless workflow.

There’s a plethora of other community of plugins which you can incorporate into your own writing workflow.

Front Matter

Hugo supports defining your posts’ front matter in yaml or toml, but Obsidian only supports rendering yaml. If you have a front matter that looks like this:

---
title: New Post
date: 2025-05-23T14:34:05-07:00
tagline:
image: placeholder.jpeg
tags:
draft: true

layout: single
type: blog
aliases:
---

Lorem ipsum dolor sit amet...

It’ll be rendered in Obsidian like so, which is pretty nifty!

A nicely rendered markdown document, a welcome upgrade to writing in an IDE.

On the Go

Obsidian also has iOS and Android apps, but the git plugin isn’t the most stable, at least not on iOS. As an alternative, you can follow the steps described in this Obsidian forum thread on setting up either iSH (free) or Working Copy (paid).

With iSH, git can be slow at times (e.g. during git status or git checkout) since to make it work, it emulates an x86 architecture on ARM-based iOS devices, so the added overhead takes a toll on performance.

But hey, at least works! Paired with automated deployments, it provides a relatively painless (and free!) way for me to update my website from my phone.

Using Google Cloud Pubsub for Batch Pipelines in Apache Beam

Wed, 23 Nov 2022 12:31:53 -0800

Google Cloud’s Pub/Sub is a useful service that provides an asynchronous and scalable messaging platform that decouples services producing messages from those that receive and process those messages. When combined with Apache Beam (and/or Dataflow, Google’s managed version of it), you can quickly develop powerful batch and streaming pipelines for data-parallel processing.

However, I recently ran into one slight hiccup - although Apache Beam has a built-in IO connector for pubsub, it only supported streaming pipelines (at the time of development). Fortunately, after a bit of searching, someone else on Stack Overflow figured out a workable solution:

The trick is that if you call future.result() inside the process() method, you will block until that single message is successfully published, so instead collect a list of futures and then at the end of the bundle make sure they’re all either published or definitively timed out. Some quick testing with one of our internal pipelines suggested that this approach can publish 1.6M messages in ~200s.

import logging

import apache_beam as beam
from apache_beam.io.gcp.pubsub import PubsubMessage

from google.cloud.pubsub_v1 import PublisherClient
from google.cloud.pubsub_v1.types import (
 BatchSettings,
 LimitExceededBehavior,
 PublishFlowControl,
 PublisherOptions,
)


class PublishClient(PublisherClient):
 """
 You have to override __reduce__ to make PublisherClient pickleable 😡 😤 🤬

 Props to 'Ankur' and 'Benjamin' on SO for figuring this part out; god knows
 I would not have...
 """

 def __reduce__(self):
 return self.__class__, (self.batch_settings, self.publisher_options)


class BatchPubsubWriter(beam.DoFn):
 """
 beam.io.gcp.pubsub does not yet support batch operations, so
 we do this the hard way. it's not as performant as the native
 pubsubio but it does the job.
 """

 def __init__(self, topic: str):
 self.topic = topic
 self.window = beam.window.GlobalWindow()
 self.count = 0

 def setup(self):
 batch_settings = BatchSettings(
 max_bytes=1e6, # 1MB
 # by default it is 10 ms, should be less than timeout used in future.result() to avoid timeout
 max_latency=1,
 )

 publisher_options = PublisherOptions(
 enable_message_ordering=False,
 # better to be slow than to drop messages during a recovery...
 flow_control=PublishFlowControl(limit_exceeded_behavior=LimitExceededBehavior.BLOCK),
 )

 self.publisher = PublishClient(batch_settings, publisher_options)

 def start_bundle(self):
 self.futures = []

 def process(self, element: PubsubMessage, window=beam.DoFn.WindowParam):
 self.window = window
 self.futures.append(
 self.publisher.publish(
 topic=self.topic,
 data=element.data,
 **element.attributes,
 )
 )

 def finish_bundle(self):
 """Iterate over the list of async publish results and block
 until all of them have either succeeded or timed out. Yield
 a WindowedValue of the success/fail counts."""

 results = []
 self.count = self.count + len(self.futures)
 for fut in self.futures:
 try:
 # future.result() blocks until success or timeout;
 # we've set a max_latency of 60s upstairs in BatchSettings,
 # so we should never spend much time waiting here.
 results.append(fut.result(timeout=60))
 except Exception as ex:
 results.append(ex)

 res_count = {"success": 0}
 for res in results:
 if isinstance(res, str):
 res_count["success"] += 1
 else:
 # if it's not a string, it's an exception
 msg = str(res)
 if msg not in res_count:
 res_count[msg] = 1
 else:
 res_count[msg] += 1

 logging.info(f"Pubsub publish results: {res_count}")

 yield beam.utils.windowed_value.WindowedValue(
 value=res_count,
 timestamp=0,
 windows=[self.window],
 )

 def teardown(self):
 logging.info(f"Published {self.count} messages")

Unfortunately, I eventually ran into an issue where the pubsub client was being overloaded when processing large amounts of data under specifying conditions. Explicitly batching the data mitigated the issue, instead of letting the (Dataflow) runner auto-magically determining the bundle sizes based on the input data.

import apache_beam as beam

# This can probably be higher if needed, but works fine as it is
MIN_PUBSUB_BATCH_SIZE = 10000
# Have tested up to 400k, but instability seems to start occurring at these bundle sizes
MAX_PUBSUB_BATCH_SIZE = 200000

...

with beam.Pipeline(options=pipeline_options) as pipeline:
 (
 pipeline

 ...

 | "Batch Messages for Pubsub"
 >> beam.transforms.util.BatchElements(
 min_batch_size=MIN_PUBSUB_BATCH_SIZE,
 max_batch_size=MAX_PUBSUB_BATCH_SIZE,
 )
 | "Publish Batches to PubSub"
 >> beam.ParDo(BatchPubsubWriter("projects/myproject/topics/my-topic"))
 )

Hopefully Apache Beam eventually adds official support for pubsub in batch pipelines, but until then, this seems to suffice for processing millions of rows of data per job.

Tips and Tricks with Terraform's null_resource

Tue, 17 May 2022 20:21:37 -0800

Terraform’s null_resource resource can be useful when there aren’t any existing modules to satisfy your needs (with some caveats). Hashicorp’s documentation for it is a bit lacking, but fortunately there’s more information about the provisioners in their other docs here. After using these resources in a handful of places across our infrastructure deployments, I’ve developed a small collection of tips I picked up over the past few months that I thought I’d share.

Use timestamp() as a trigger when you need the resource to run on every deployment:

resource "null_resource" "always-run" {
 triggers = {
 timestamp = timestamp()
 }

 provisioner "local-exec" {
 command = "echo foobar"
 }
}

You can also supply multiple provisioner blocks, where one of them can be configured with when = destroy to specify the action to take when the resource will be destroyed. Dynamic values need to be accessed using self.triggers.* since Terraform isn’t able to resolve the values at runtime during resource destruction.

resource "null_resource" "include-cleanup" {
 triggers = {
 name = var.name
 region = var.region
 }

 # Executes on resource creation {
 command = join(" ", [
 "echo",
 "create:",
 "${self.triggers.name}",
 "${self.triggers.region}",
 ])
 }

 # Executes on resource destruction
 provisioner "local-exec" {
 when = destroy
 command = join(" ", [
 "echo",
 "destroy:",
 "${self.triggers.name}",
 "${self.triggers.region}",
 ])
 }
}

The local-exec provisioner also allows for an environment field, which can be used to easily pass values into commands or scripts:

resource "null_resource" "environment-values" {
 provisioner "local-exec" {
 command = "./some-script.sh"
 environment = {
 ENVIRONMENT = var.environment
 JSON_DATA = jsonencode({
 "var1" = var.name,
 "var2" = var.region,
 ...
 })
 }
 }
}

Aside from provisioning resources, null_resource can also be used for debugging values (which I stumbled upon from nexxai.dev):

resource "null_resource" "terraform-debug" {
 provisioner "local-exec" {
 command = "echo $VARIABLE1 >> debug.txt; echo $VARIABLE2 >> debug.txt"

 environment = {
 VARIABLE1 = jsonencode(var.your_variable_name)
 VARIABLE2 = jsonencode(local.piece_of_data)
 }
 }
}

Deploying Google Cloud Functions with Terraform

Tue, 08 Mar 2022 11:47:29 -0800

Cloud Functions are an easy, performant, and potentially inexpensive way to build serverless backends. I recently went down the route of setting up continuous deployments for them, and thought I’d share my learnings with it.

The easiest way to deploy a Cloud Function is using the gcloud CLI like so:

gcloud functions deploy <YOUR_FUNCTION_NAME> \
 --region=<YOUR_REGION> \
 --runtime=<YOUR_RUNTIME> \
 --source=<YOUR_SOURCE_LOCATION> \
 --entry-point=<YOUR_CODE_ENTRYPOINT> \
 <TRIGGER_FLAGS>

Things get a bit more complicated if you want to use Terraform for deployments, which has its own set of advantages. The main trick to getting it to work is in line 43 below, where a checksum is appended to the archive’s filename every time it’s uploaded to the storage bucket.

The reason this is necessary is because the google_cloudfunctions_function resource won’t be triggered for a redeployment on subsequent code changes - by having a checksum generated based on the source code, we ensure that Terraform knows to redeploy the Cloud Function whenever the underlying code changes.

 1locals {
 2 project_id = "my-project"
 3 region = "us-west1"
 4 component = "my-component"
 5
 6 cloud_function = {
 7 name = "${local.component}-cf"
 8 description = "Some description for this cloud function"
 9 runtime = "python39"
10 entry_point = "my_entry_point"
11
12 source_dir = "./src"
13 archive_filepath = "/path/to/file"
14 }
15}
16
17# Service account for the Cloud Function
18resource "google_service_account" "cloud_function_sa" {
19 project = local.project_id
20 account_id = local.component
21 display_name = local.component
22}
23
24# Bucket to store the source code archives
25resource "google_storage_bucket" "function_archive" {
26 name = "${local.component}-cloud-function-archive"
27 location = local.region
28 project = local.project_id
29 uniform_bucket_level_access = true
30}
31
32# Archive the source code as a zip file
33data "archive_file" "function_archive" {
34 type = "zip"
35 source_dir = local.cloud_functions.source_dir
36 output_path = "${path.root}/${local.cloud_function.archive_filepath}"
37}
38
39# Upload the source code archive to the bucket. This will happen each time
40# the source code changes.
41resource "google_storage_bucket_object" "archive" {
42 # Append checksum so file changes trigger a cloud function deployment
43 name = format(
44 "%s#%s",
45 local.cloud_function.archive_filepath,
46 data.archive_file.function_archive.output_md5
47 )
48 bucket = google_storage_bucket.function_archive.name
49 source = data.archive_file.function_archive.output_path
50
51 content_disposition = "attachment"
52 content_encoding = "gzip"
53 content_type = "application/zip"
54}
55
56# Cloud Function that uses the uploaded source code archive
57resource "google_cloudfunctions_function" "some_cloud_function" {
58 name = local.cloud_function.name
59 description = "Some description for my cloud function"
60
61 project = local.project_id
62 region = local.region
63 source_archive_bucket = google_storage_bucket_object.archive.bucket
64 source_archive_object = google_storage_bucket_object.archive.name
65 service_account_email = google_service_account.cloud_function_sa.email
66
67 runtime = local.cloud_function.runtime
68 entry_point = local.cloud_function.entry_point
69 trigger_http = true
70}

Precision Sourdough: A Smart Lid for Your Starter

Mon, 22 Feb 2021 16:36:18 +0000

Featured on Hackaday!

A few years ago, I had the idea to track my sourdough starter using computer vision. It was neat to monitor it this way, but it was fairly impractical to do for each feeding since it required setting up a camera, downloading the images, and doing some manual image cropping before running it through my analysis script. The analysis was also only done after the fact, and what I really wanted was something that could tell me when the starter was ready to be used (or fed), or, if I missed the window of opportunity, how long ago it peaked.

Last year, I came across this Reddit thread and Christine Sunu’s sourd.io project, where they both had distance sensors inside the lid to measure the height of the starter. I thought it was genius, and had to make one for myself! However, in addition to the live monitoring, I wanted to log the data for further analysis, so I also decided to make it internet-connected as a way to get the data off the device (since saving to an SD card would add hardware costs, as well as being less “sexy” in today’s world of everything having wifi connectivity).

Interested in making your own? All the design files and code can be found on GitHub!

Three modes of operation: Max rise and time, graph, stats for nerds.

Selecting, viewing, and downloading data for a given feeding session.

Aside: In honesty, this project took longer than I originally expected, and I had spouts of project fatigue where I had zero motivation to work on this anymore. Eventually, a third (or maybe fourth, I lost count) wave of inspiration came to me, and I managed to finish the last remaining bits of this project. I’m glad I did though, because this turned out to be one of the more nifty gadgets I put together. I’m telling you this because people are often talking about side projects and hustles (especially software engineers), and I want to say that it’s ok to focus on your mental well-being and just be a potato sometimes.

The Development

If you’re only interested in the resulting data that came out of this, you can skip to the end where I visualize the growth, temperature, and humidity from a few weeks of worth of feedings. Otherwise, read on to learn about the development of this high specialized, mildly esoteric kitchen gadget!

Hardware

Electronics

With the idea in mind, I bought the components off Digikey and AliExpress and hooked them up on a breadboard. The parts I used for this project:

NodeMCU ESP8266
VL6180X Time of flight distance sensor
DHT22 Temperature and humidity sensor
SSD1306 128x32 OLED display

I wrote some code to test that all components worked correctly, then went on to design a PCB to make it easier to integrate into the form factor of a jar lid.

Breadboard prototype with off-the-shelf modules.

I’ve made PCBs using protoboards before (in my sous vide controller), but it was extremely time consuming. Since I surpassed the learning curve of designing boards in KiCad, now I’d rather wait a few weeks for the boards to show up from overseas, especially since it’s so cheap ($2 for 5 boards, plus $14 shipping).

PCB layout and schematic.

Top of the PCB with the display (left), bottom with the distance and temperature/humidity sensors (right).

At this point, I was eager to try it out, so I cut a hole in a plastic yogurt lid and taped the assembled PCB on.

It ain’t pretty, but it works.

It worked well enough to test out the initial workflow, and I realized that I needed few crucial things to make the measurements actually useful:

The height of the jar
The starting height of the starter
The height the starter has grown

With a bit of algebra and math, these values were now measured by the distance sensor, mounted at the bottom of the lid.

Algebra? More like alge-bread!

Some definitions:

$$ d_1 = distance \\ to \\ jar $$

$$ d_2 = distance \\ to \\ starting \\ height $$

$$ d_3 = distance \\ to \\ current \\ height $$$$ h_1 = starting \\ height $$

$$ h_2 = current \\ height $$

And the governing equations:

$$ h_1 = d_1 - d_2 $$

$$ h_2 = d_2 - d_3 $$

$$ h_{rise percent} = h_2 / h_1 = (d_2 - d_3) / (d_1 - d_2) $$

Resulting in our formula for calculating the rise percentage of the starter!

Enclosure

I designed the enclosure in Fusion 360 and printed it on my Monoprice Mini 3D Printer.

3D printed enclosure, designed in Fusion 360.

I wasn’t the proudest of this design, since I had to resort to using hot glue to attach the PCB to the enclosure, mainly because I forgot to leave enough hole clearance on the PCB . Ideally, the PCB would drop into the top half and assemble from the back (instead of the bottom half, as designed), but unfortunately I didn’t have this forethought. I was designing the PCB to have a minimal footprint to keep costs low, instead of making it easy to integrate with!

Since I didn’t want to solder the modules directly on to the PCB, I used female headers to keep them removable. However, this gave the assembly quite a bit of height, which resulted in the button needing some type of extension. I didn’t want to make another Digikey order specifically for this button, so I decided to be resourceful and glued a machine screw to the button to make up for the missing height.

Yes, that’s a machine screw hot glued on to a switch…

Anyway, it assembled together without issues, and the only mistake you can see from the outside is the screw head (or let’s just say I wanted to go for an intentional steampunk look)!

An active, healthy starter being carefully monitored.

Firmware

I wanted to use a task-based architecture, since having everything in a single state machine can become convoluted and difficult to debug. With multitasking, the processor is only executing a single task at any given time, but it switches between each task rapidly to give the illusion of concurrency. I’ve used FreeRTOS in other projects, but wanted something more lightweight since I didn’t need all the bells and whistles that it provides.

Multitasking vs concurrency (Source: FreeRTOS)

I eventually came across TaskScheduler, a cooperative multitasking framework, which checked all the boxes I needed.

A lightweight implementation of cooperative multitasking by TaskScheduler. (Source: GitHub)

The code was divided up into the following tasks/files:

measurements.cpp - Read the sensors and make the measurements available for other tasks
userinput.cpp - Handle the button presses from user input (short press, long press, double click)
display.cpp - Display information to the user
iot.cpp - Send the measured data to the cloud

The benefit of this architecture is that each file is less than 200 lines of code, and the clear separation of concerns made it easy to develop and debug.

The display shows how much it peaked, and how much time elapsed since it peaked.

With the architecture in place, the firmware itself was fairly straightforward. The only “fancy” thing I needed to do was save the jar height to EEPROM so that it would be saved between sessions, even if it’s powered off. Interestingly, the ESP8266 doesn’t actually have genuine EEPROM memory, so it’s emulated by using a section of flash memory.

Cloud Connectivity

One of the more time consuming parts of this project was actually getting AWS set up on the ESP8266. I encountered many library compatibility issues, which I should have expected since this chip was released 6 years ago (at the time of development)! If I didn’t have an ESP8266 in my box of parts that I wanted to use, I would have used the newer ESP32, which (hopefully) would have presented fewer issues.

If you’re looking to do the same, let me save you some headache! You can check out the template I made on GitHub to get set up.

Once I got it publishing messages over MQTT to AWS, I set up the cloud infrastructure to receive and save the data. The data flow was:

Device sends the data in a message over MQTT
On a message receive event, an AWS Lambda function is triggered to parse the data from the message and passes it to a Kinesis Firehose delivery stream
Kinesis Firehose receives the data and saves it Amazon S3

AWS architecture for basic IoT applications. (Source: DZone)

I initially was going to use Amazon QuickSight to visualize the data, but there were limitations with the refresh rate that were a deal breaker for me. Instead, I bit the bullet and created a custom dashboard using Flask and HTML/CSS/Javascript, which queries data from S3 using Amazon Athena. I was too lazy to figure out how to host the dashboard on AWS for free, so I opted to use Heroku (as I’ve done previously for my recipe converter web app).

With cloud connectivity and dashboard complete, the sourdough monitor was now ready to be used!

The Analysis

The main purpose of the monitor was to help with my timing of when to use the starter. Since the data was being collected anyway, I figured I might as well do some analysis to see if there was anything significant.

Overview

After using the monitor for a few weeks, I had enough data to play with. The two main things I was curious about were:

Whether the peak height changes over time (i.e. through repeated feedings)
If it still grew as much if the starter was kept in the fridge and hadn’t been fed for a few days

Doing a quick kernel density plot to get a feel for the data, we see some clustering but no obvious trends.

Kernel density plot to show the distribution and clustering of data. No clear correlations are present…

The graph below shows the consistency of my feedings:

5 subsequent feedings, starting on Jan 21
3 subsequent feedings on Jan 30, Feb 3, and Feb 17
2 subsequent feedings on Feb 12

Note: The horizontal axis is a bit off since some days had multiple feedings, the graph assumes each feeding is a separate day.

Feeding schedule over the past few weeks.

The graph below shows no correlation between how long it took for the peak rise to occur. It also didn’t seem to matter if the starter was kept in the fridge for a few days, which is great news (and maybe not news to those who know their starter well)!

No statistical significance between the most relevant metrics.

In Detail

Looking at the time series data, we see the progression of the rise height from the first to last feeding. Some observations:

Peak rise height increased from ~100% to ~200% by the 6th subsequent feeding
Slope of rise height was constant, even with the first feeding out of the fridge
Peak rise height is achieved by ~5 hours, and sustains at this height for ~3-5 hours

Note: I used a temperature controlled proofing box, which is why the temperature and humidity are constant throughout the majority of the fermentation. The temperature was set to 24°C for most days, but interestingly the recorded temperature was typically around 30°C. More testing is required to see which sensor is correct…

Rise height, temperature, and humidity over time.

Pair plot of the time series. Not much to show, except that temperature and humidity are correlated (as expected).

The rest of the feeding data can be found in the combined figure below. There are a few interesting things that occurred, but further experimentation would be required to draw any conclusions:

Jan 30: All feedings were relatively similar, even the first one after being in the fridge for 5 days
Feb 3-5: The second feeding had a steeper slope; I think I adjusted the feeding ratio for this one from 1:2:2 to 1:3:3, but I can’t quite remember (or perhaps my wife was the one who fed it this time?). Also interesting that the temperature for the third feeding was not as consistent. Perhaps the proofing box wasn’t turned on?
Feb 12: Second feeding peaked 100% more than the first feeding. I think the feeding ratio was changed. The first feeding’s humidity was also really weird in this one.
Feb 17-18: First feeding took longer to peak than the subsequent two, which contradicts my observations from Jan 30…

Visualization of the rest of the feeding data, for those interested.

So basically, what the data is telling me is that there are many other variables at play, and I’ll need to record a few other metrics in order for the measured data to be actionable, such as:

Feed ratio
Start and end temperature of the starter (using a probe thermometer)
Optional: Water hardness

Other experiments that I’d like to nerd out on:

Compare feed ratios and/or flour blends
Compare growth performance with and without a temperature controlled environment (i.e. proofing box)
Ideal location if you don’t have a proofing box (i.e. Oven with light on? Microwave with warm water?)

Conclusion

So after all this, the takeaway might actually be that timing the starter isn’t all that important, since it stays active at its peak for at least a few hours. And if you maintain a somewhat regular feeding schedule and have a relatively stable environment, then you can probably get a good feel for how long it takes your starter to grow (or you can set up a timelapse with your smartphone camera to see how it’s doing).

This was still a fun way to nerd out with baking, since engineers like metrics, and there’s a lot to measure with sourdough bread. Although this gadget might seem superfluous to some, I enjoyed the precision and confidence it gave me in taking out the guesswork of how my starter is doing. There’s two types of bakers in the world: those who go by feel, and those have a 0.01 g resolution kitchen scale. Guess which one I am 😏

A darn nice crumb, if I do say so myself!

Hope you found this enlightening, and if you have any other ideas on what experiments to try out, leave a comment below! If you like my content, please consider buying me a coffee!

Happy baking!

Building a Backyard Chicken Coop in the City

Wed, 09 Sep 2020 03:41:54 +0000

Back in 2018, we decided to raise chickens in our urban backyard, all in the name of being able to have fresh eggs every morning. Our city had bylaws on having chickens, which included:

A maximum of 4 hens (no roosters), 4 months or older
Ducks, turkeys, or other livestock are not allowed
Eggs, meat, and manure cannot be used for commercial purposes
Backyard slaughtering is not allowed

Since we were fortunate enough to have access to a backyard at the time, we decided to build a coop and start the lifestyle of having easy access to fresh eggs. Was it cost effective? Heck no, but it sure was a fun journey!

The Development

The Coop

After doing some rough sketching, we settled on dimensions of our coop. We put the frame together to get a feel for how big this thing would actually end up being!

Most coops are made out of an outdoor wood like cedar, but that would have blown our budget for something that we may not keep forever. The other thing is that in general, my engineering style is to see how inexpensive and minimal I can build something, rather than overbuild with an unlimited budget.

Anyone can build a bridge that won’t fail, but it takes an engineer to build one that barely stands.

As such, we built out the frame with the thinnest (and cheapest) construction wood at Home Depot (something like 1x2). With just the perimeter built, it was honestly quite wobbly… but once the trusses were added, rigidity was achieved.

Rough frame of the coop (top) and part of the run (bottom)

Building out the frame, adding the angled roof and the nesting box.

The doors were a bit of a pain, since the opening wasn’t perfectly square. But after an adequate amount of measuring and shimming, we had the doors installed, operational, and smooth.

For the paint, we went to the clearance section at Home Depot and found heavily reduced outdoor paint which suited our needs. This strategy isn’t always fruitful, but luckily for us, we found a decent colour as well as not actually needing that much paint.

Sturdy enough for me, sturdy enough for chickens.

Dry fitting the coop and building out the run.

How the chickens will enter and leave the coop.

Fully assembled and ready for our new guests!

At this point, we were finally ready to go get ourselves some chicken(s)!

The Auction

Turns out buying chickens is easier said than done. We eventually found a farm in the next city over that hosts farm animal auctions, which sounded like a great idea at the time. Upon arrival, potential buyers browse the inventory and make notes about what they would like to purchase. What we didn’t realize was that sellers were there to sell in bulk… While we wanted to buy a couple hens of different breeds, sellers were putting chickens up for sale in multiples of five!

Browsing the inventory before the auction starts.

Having never been to an auction before, it was actually a bit stressful. The auctioneer speaks extremely quickly, and you (as the buyer) need to make quick decisions on how much you want to bid, and if someone else also bids, if you’re willing to outbid them (and how many times you want to repeat this cycle). The auctioneer also scolded a few people for waving their paddles by accident, since the slightest of movement is an indicator that the person holding the paddle wants to bid.

Bidding for our hens.

The Chickens

Within the span of 24 hours, our chicken ownership went something like this:

Bid on two nice, small chickens
Find out that they’re sold as a pair because one of them is a rooster
Panic because we can’t bring back a rooster, so put them back up for auction
Sell the chickens at a slight loss, end up buying 4 chickens
Try to confirm with one of the farmers that all of them are hens, is told that they’re too young to tell the gender
Take all the chickens home, find out the next morning at 6 am that one of them is a rooster
Panic again, find a local farm to trade our rooster (plus another hen since our coop was built for max 3, not 4)

Hens in their new home (some of them anyway…).

Turns out catching a chicken is easier said than done, but eventually we got him!

With the chicken fiasco over, we had ourselves a stable flock.

Three happy hens roosting in their new home.

From left to right: Tamago (Bantam hen), Peidan (Black Copper Maran), and Pecorino (Ameraucana).

This was mildly painful at first since they’d nip our flesh when going for the seeds.

On the first few nights, a bit of training was required to tell the chickens to go inside the coop at dusk. To do this, we shined a flashlight from the inside (such that it’s visible from the outside), which eventually attracted their attention and they flew up into the opening.

After a while, they no longer needed any coaxing and like clockwork, they’d return to the coop right right when the sun set for the day.

Hens entering the coop at sunset…

… And exiting at sunrise

That winter, we had snowfall that stuck (which is rare for the “wet” coast of Vancouver), but the chickens were fortunately ok. We had a heat lamp ready but they didn’t seem to be cold, and we kept the water warm to keep it from freezing overnight.

When winter rolled around, we had to keep an eye on their water to make sure it didn’t freeze.

At least they were still somewhat free run!

The Eggs

We had to “let the taps run for a bit” before our egg production hit full tilt, so to speak. The shells from the first few eggs were too soft (due to a lack of calcium?), but after a week or so, we finally started to get some proper eggs.

Soft shell crab is a delicacy, but soft shell eggs are not.

This one almost made it, but the shell was still too thin. More calcium required!

Waking up to this pair of eggs was egg-tremely egg-citing (sorry, couldn’t resist).

Production eventually entered steady state.

Try doing this with store bought eggs!

The Farewell

Two fruitful years later, life events resulted in having to say goodbye to our beloved hens. We initially put an ad up on Facebook Marketplace, but that quickly got taken down since apparently they don’t allow livestock to be sold on their platform. Back to Craigslist we went, and eventually, we found a buyer. When they arrived, they came prepared to take it all away, albeit with equipment that was a little non-standard…

Half of the coop packed up into the buyer’s SUV…

… And the other half in another SUV.

All that was left was a memory of the chickens on our crashpad, thanks to our good friends who designed and gifted it to us for our wedding.

Memories of our original hens, now forever imprinted on our bouldering crashpads.

Nordic nRF52 Development with Visual Studio Code

Thu, 25 Apr 2019 14:14:25 -0700

A few years ago, I created a tutorial on setting up Visual Studio Code for development with the STM32. Since I’ve also been developing on the Nordic nRF52, I thought I’d share another tutorial to show how a project can be set up, flashed, and debugged using Visual Studio Code.

The template project discussed in this post can be found on Github.

Instructions

The Nordic toolchain is cross-platform, but the instructions below are specifically for Linux. However, they can easily be replicated in Windows as long as installation paths and environment variables are set correctly.

General Comments

When using any editor + terminal for nRF52 development, the things to remember are:

GCC path is set in <sdk>/components/toolchain/gcc/Makefile.posix
Makefile is up to date with:
- SDK_ROOT is pointed to where <sdk> is located
- Source and header files for new components
- Board/component configurations in sdk_config.h

With Visual Studio Code:

In .vscode/c_cpp_properties.json, update defines, includePath, and compilerPath as required
In .vscode/launch.json, update executable and armToolchainPath as required
In .vscode/tasks.json, update the current working directory cwd and command as required (ie. changing to make flash -j8 to use 8 cores to build).

Installation

System Tools

sudo apt install build-essential

# Required by java-based CMSIS Configuration Wizard
sudo apt install default-jre

Code Editor

Visual Studio Code
- C/C++ Extension by Microsoft
- Cortex-Debug Extension by marus25

nRF52 Toolchain

Download

nRF52 SDK
nRF52 Command Line Tools
Segger J-Link Software Tools
GNU-RM Embedded Toolchain for ARM
- It’s recommended to install the GCC version that matches the Nordic SDK version. Check the GCC version in <sdk>/components/toolchain/gcc/Makefile.posix and download the appropriate version.
- For nRF5 SDK 15.3.0, the gcc version is gcc-arm-none-eabi-7-2018-q2-update

Optional:

Segger Ozone Debugger

Setup

Run the commands below to extract the archives to the respective paths.

nRF5_SDK to $HOME
nRF Command Line Tools to /opt/ and /usr/local/bin
gcc-arm-none-eabi to /usr/local/bin

# Unpack SDK to home directory
unzip nRF5_SDK_15.3.0_59ac345.zip -d $HOME

# Unpack nRF command line tools and make accessible in terminal
tar -xvf nRF-Command-Line-Tools_9_8_1_Linux-x86_64.tar --directory /opt/
sudo ln -s /opt/nrfjprog/nrfjprog /usr/local/bin/nrfjprog

# Install Segger
sudo apt install ./JLink_Linux_V644f_x86_64.deb

# Unpack gcc toolchain to /usr/local/bin
sudo tar -xjvf gcc-arm-none-eabi-8-2018-q4-major-linux.tar.bz2 --directory /usr/local/bin

If optional tools are downloaded:

# Segger Ozone
sudo apt install ./Ozone_Linux_V262_x86_64.deb

Check nrfjproj --version that it’s been installed correctly.

Nordic SDK Setup

In the nRF52 SDK folder, update the values in components/toolchain/gcc/Makefile.posix:

GNU_INSTALL_ROOT
GNU_VERSION

This is only required if using a different gcc version than specified. It’s recommended to use the same one as the SDK.

Using the Project

Setup

In blinky/ directory, do a global search and replace to update the SDK root to wherever your nRF5_SDK directory is:

From: SDK_ROOT := ../../../../../..
To: SDK_ROOT := $(HOME)/nRF5_SDK_15.3.0_59ac345

If required, update the following in .vscode/c_cpp_properties.json:

Include paths
Defines based on C/assembler flags in the Makefile

Configuring `sdk_config.h`

CMSIS Configuration Wizard is integrated with example makefiles. In order to open sdk_config.h in this tool, type:

make sdk_config

Build and Flash

cd blinky/pca10056/mbr/armgcc

# To just build. Optional `-jN` flag, where N is number of cores to use
make

# To build and flash
make flash

# If a SoftDevice is included in your project
make flash_softdevice

Build tasks can also be added to .vscode/tasks.json. Pressing CTRL+SHIFT+B will execute the default build task (in this case, the default build task is make flash for PCA10056).

Segger RTT Log

To use the RTT Viewer equivalent on GNU/Linux, start by opening a terminal and starting JLinkExe.

Follow the steps below to connect the device (in this case, NRF52840_XXAA). Press ENTER to accept the default value. The only option that needs to be changed (aside from board, if necessary) is the target interface (use SWD instead of JTAG).

J-Link>connect
Device>
Please specify target interface:
 J) JTAG (Default)
 S) SWD
 T) cJTAG
TIF>S
Specify target interface speed [kHz]. <Default>: 4000 kHz
Speed>
Device "NRF52840_XXAA" selected.

Alternatively, you can specify the configurations in the command line:

JLinkExe -device NRF52832_XXAA -if SWD -speed 4000 -autoconnect 1

In another terminal, start JLinkRTTClient. RTT output should now start displaying. Output should be as follows:

###RTT Client: ************************************************************
###RTT Client: * SEGGER Microcontroller GmbH *
###RTT Client: * Solutions for real time microcontroller applications *
###RTT Client: ************************************************************
###RTT Client: * *
###RTT Client: * (c) 2012 - 2016 SEGGER Microcontroller GmbH *
###RTT Client: * *
###RTT Client: * www.segger.com Support: support@segger.com *
###RTT Client: * *
###RTT Client: ************************************************************
###RTT Client: * *
###RTT Client: * SEGGER J-Link RTT Client Compiled Apr 12 2019 17:30:19 *
###RTT Client: * *
###RTT Client: ************************************************************

###RTT Client: -----------------------------------------------
###RTT Client: Connecting to J-Link RTT Server via localhost:19021 Connected.
SEGGER J-Link V6.44f - Real time terminal output
J-Link OB-SAM3U128-V2-NordicSemi compiled Jan 7 2019 14:07:15 V1.0, SN=683903307
Process: JLinkExe
<info> app: SPI example started.
<info> app: Transfer completed.
<info> app: Transfer completed.
<info> app: Transfer completed.
...

Debugging

Visual Studio Code

Open the debug pane (CTRL+SHIFT+D) and select Cortex-Debug.
To create a new configuration, select Add Configuration and choose Cortex-Debug.
If required, update the executable and/or armToolchainPath in .vscode/launch.json.
Hit F5 to start debugging.

Segger O-zone

Using Segger Ozone provides rich insights on memory, assembly instructions, peripheral registers, etc.

Screencap of using Segger Ozone debugger.

New project settings:

Select Create new project
Choose target device
- Select device: nRF52840_xxAA (or other)
- Peripherals: (blank)
Connection settings
- Target interface: SWD
- Target interface speed: 1 MHz
- Host interface: USB
- Serial no: (blank)
Program file
- Select pca10056/mbr/armgcc/_build/nrf52840_xxaa.out

Resources

Nordic Information Center - Official Documentation

Happy developing!

Monitoring the Fermentation of Sourdough Starter with Computer Vision

Sun, 24 Jun 2018 17:50:22 -0700

Featured on MagPi Issue 72, Hackaday, Hacker News, Adafruit, Hackster.io and The Guardian!

Bread, the quintessence of life. People have survived for centuries off this staple consisting only of flour, water, salt, and yeast. Try consuming all these ingredients separately, and you’ll be in for a digestive surprise. However, mix them together and let time do its thing, and the result is the release of profound flavour, texture, and nutrients that were previously locked away.

Despite it being relatively easy to turn dough into something that looks and feels like bread, the challenge is in squeezing every possible ounce of flavour and texture (using only those four ingredients) to achieve the embodiment of a true loaf of bread. The secret to artisinal bread is all in the fermentation. A baker’s skill is in their ability to manage and control the fermentation process, which is usually achieved through countless months and/or years of trial and error.

But what if there was a better way to understand what happens during the fermentation process?

In this blog post, we dive into the world of wild yeast (commonly known as sourdough starter) by tracking its growth through timelapses, automated image analysis, and cool graph animations. Read on to find out more!

Timelapse taken over ~10 hours at 5 minute intervals. Shown: Two sourdough starters with different feeding ratios.

Image analysis for tracking growth of the above timelapse.

The Backstory

Two key components in making artisinal bread are time and fermentation. If you can afford a long, slow rise, you will be rewarded with a texture like no other, and flavours that are both complex and subtle. Most bakeries unfortunately do not have this luxury (since it’s a business after all), so commercial instant dry yeast is used to expedite the fermentation such that it reaches the desired loaf volume at a reasonable schedule. With bread risen with instant dry yeast, it will definitely resemble visual qualities of bread, but the texture and taste will not be comparable to bread that uses wild yeast.

The quest for achieving the perfect loaf is an arduous one. It can take upwards of 12 hours from the first mix to actually being able to bake the loaf, and environmental factors (ie. temperature and humidity) can alter the dough’s behaviour (ie. in how it absorbs water, or how fast/slow the fermentation occurs). With every loaf being slightly different, it can be challenging to identify what processes need adjusting to get one step closer to that perfect loaf.

For context, a typical process for making an artisinal loaf is roughly described below:

Discard 80-90% of the sourdough starter and feed it with flour and water.
Wait a few hours for the starter to ferment and double or triple in size.
At its peak, take the starter and mix it in with the rest of the bread ingredients.
Gently fold the dough to tighten it up (which creates the gluten) a few times for the first 1-2 hours.
Let rest. Flavours are developed during this bulk fermentation period.
Shape the loaf and place it in a proofing basket.
Let rest. The final volume is achieved during this second fermentation.
Bake.
Place on cooling rack when done, and listen to the song of the crackling crust while you wait to cut it open.

Sourdough starter is a living organism, and as such should be treated with kindness, respect, and most importantly, food. A well-fed starter is an active and happy one; if we can get a feel for how the starter behaves on a day-to-day basis during its feeding (ie. when the fermentation occurs), it may help clear up at least one of the variables in bread baking.

One way to achieve this understanding is simply through trial and error. But we can do better. We can use (rather simple) computer vision!

The loaf’s holey texture (ie. the elusive open crumb) is largely determined by activeness of the starter, so this is where we shall put our initial efforts. Specifically, we want to understand:

When the starter reaches its maximum fermentation,
How consistent the starter’s fermentation is, and
What happens when the starter is neglected and how quickly it can come back to life.

To answer these questions, our plan of action is to:

Take a timelapse of starter
Write an image analysis script to locate the current height of the starter in each image
Plot the height over time to get its growth characteristics

Enough with the walls of text. Let’s get on to the fun stuff!

The Development

Setting Up the Timelapse

For ease and convenience, the timelapse was set up on a Raspberry Pi. I initially had the idea to eventually create a real-time analysis of the starter (ie. a Python script updating a locally hosted dashboard), but I figured post processing a timelapse was good enough for now to answer those questions. You can pretty much use any camera to take a timelapse.

Loading the Raspberry Pi Zero (Headless)

Setting up a Raspberry Pi is very easy these days:

Download Raspbian
Flash it to an SD card with something like Etcher

The only hiccup I ran into was setting up without any monitor or ethernet attached, so getting it connected to WiFi and retrieving its IP address was non-trivial. Luckily, other people had run into the same problem over at the Raspberry Pi forums. The solution:

Create an empty file on the SD’s boot partition called ssh to enable it.
Create another file named wpa_supplicant.conf with the following content:

country=US
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1

network={
 ssid="your_real_wifi_ssid"
 scan_ssid=1
 psk="your_real_password"
 key_mgmt=WPA-PSK
}

Connect the camera module and boot it up. You can use something like Angry IP Scanner to find its IP address, and with any luck you should be able to SSH into it from your own computer!

Creating the Scripts

Fortunately, the Pi comes loaded with raspistill, a command line tool to capture images (see here for documentation). All we need to do is to write a simple shell script to execute this command every N seconds to create a timelapse.

Note: The easiest option is to use the built in timelapse mode. However, I decided to make my own script to have more control over the filenames and where the images get saved.

Folder Structure

I set up two scripts, one to take the timelapses, and another to start the timelapse as a background process. The run script is in the home directory so I can execute the command right when I login through the command line.

 home/pi/
 ├── sourdough-monitor/
 │ ├── imgs/
 │ └── timelapse.sh
 └── run_sourdough_monitor.sh

Scripts

In timelapse.sh:

#!/bin/bash
# Calls 'raspistill' every 300s (ie. 5 mins) and writes the images
# to a new date/timestamped folder.

TOTAL_DELAY=300 # in seconds
CAM_DELAY=1 # need to have a nonzero delay for raspistill

# Must be 1.33 ratio
RES_W=1440
RES_H=1080

# Calculate the total delay time per cycle
SLEEP_DELAY=$(($TOTAL_DELAY-$CAM_DELAY))

FOLDER_NAME=imgs
mkdir -p $FOLDER_NAME # create image root folder if not exist

IDX=0 # image index

function cleanup() {
 echo "Exiting."
 exit 0
}

trap cleanup INT

while true; do
 DATE=$(date +%Y-%m-%d_%H-%M-%S)
 FNAME="${DATE}_(${IDX})" # image filename

 # Create folder for current timelapse set
 if [ $IDX -eq 0 ]
 then
 FOLDER_NAME=$FOLDER_NAME/$DATE
 mkdir -p $FOLDER_NAME
 echo "Created folder: ${FOLDER_NAME}"
 fi
 # Take image
 raspistill --nopreview -t $CAM_DELAY -o ./$FOLDER_NAME/$FNAME.jpg -w $RES_W -h $RES_H

 echo "Captured: ${FNAME}"
 IDX=$((IDX+1))
 sleep $SLEEP_DELAY
done

And in run_sourdough_monitor.sh:

#!/bin/bash
# Convenience script to run the monitor from home directory and launch
# in background

cd sourdough-monitor

# Start in background so ssh session can be closed
nohup ./timelapse.sh &> /dev/null &

Nothing fancy here, so let’s move along.

Taking the Timelapses

Sometimes having kludgy setups is the best way forward. Might not be as pretty, but when bread is on the menu then nothing else matters. All we’re here for is the data!

Sometimes random parts and a bit of tape are the best way forward.

The light was originally placed in front (as shown above), but a better method was to put the light behind the jars to maximize contrast and minimize glare.

To start the timelapse, ssh into the Pi and execute the following:

$ ./run_sourdough_monitor.sh

The next morning, you can ssh back in and kill the process with:

$ pkill timelapse

Once you have it set up on the Pi, you can also control it with your phone and something like JuiceSSH to feel super techy.

Starting the timelapse on Android, because why not?

Now we have some data to analyze!

The Analysis

The computer vision part of this project was quite straightforward, thanks to scikit-learn. All we have to do is to:

Apply a binary threshold to the image to get two distinct regions
Find the location of the boundary line

Fortunately, these are very easy to do with this library! The two main required components:

skimage.filters - For applying thresholds
skimage.measure - For measuring region properties

Everything discussed in this blog post is also available on my Github.

Thresholding the Image

There are many threshold algorithms to choose from, and fortunately scikit-learn comes with a handy function to try them all at once.

import matplotlib.pyplot as plt
from skimage.filters import try_all_threshold

img = io.imread('2018-05-31 Levain Timelapse/test.jpg', as_grey=True)
img = img[0:1000, 650:1100] # crop image to zoom in to jar

fig, ax = try_all_threshold(img, figsize=(10, 8), verbose=False)
plt.show()

Running the above code yields the following image:

The try_all_threshold() function is fast and convenient to see which will likely be the best for an image.

Discussing the different thresholding algorithms is beyond the scope of this post, but what we’re looking for is one that is able to separate the boundary between the clear glass jar and the opaque sourdough starter. From the image above, it looks like isodata, otsu, and yen provide the sharpest thresholded boundary.

Given a hot tip from my coworker (thanks Andreas) that Otsu’s method is a good one to pick, we can dig deeper into it. From the docs:

Otsu’s method calculates an “optimal” threshold (marked by a red line in the histogram below) by maximizing the variance between two classes of pixels, which are separated by the threshold. Equivalently, this threshold minimizes the intra-class variance. - SciKit Image Docs

The main takeaway with Otsu’s method is that it works best with a bimodal distribution. For our image, this means the histogram should be represented by two distinct peaks. Since our cropped image looks to have two visually different areas (ie. transparent glass at the top, flour/water mixture at the bottom), let’s confirm that this method will work.

plt.hist(img.ravel(), bins=256, range=(0.0, 1.0), fc='k', ec='k')
plt.title('Histogram of Original Image')
plt.show()

Plotting the histogram yields the following result:

The image is a bimodal histogram, so Otsu’s method should work well. (Please forgive my unlabelled axes.)

Great, the histogram shows just what we need! (Except for the high concentration of saturated pixels, but let’s just ignore that for now…)

Taking a closer look at the thresholded image:

from skimage.filters import threshold_otsu

thresh = threshold_otsu(img, nbins=5)
binary_img = img < thresh

fig, axes = plt.subplots(ncols=2)
ax = axes.ravel()

ax[0].imshow(img, cmap=plt.cm.gray)
ax[0].set_title('Original image')

ax[1].imshow(binary_img, cmap=plt.cm.gray)
ax[1].set_title('Result')

plt.show()

The result of Otsu’s thresholding method.

Those white blobs on the walls of the jar may still be detected as regions of interest, but applying a minimum thresholded area will help with false positives. Luckily, we have moderate control of the lighting and contrast of the object in question. If we’re lucky we won’t have to change much for the timelapses taken at different times of the day!

Quantifying the Image

With our binary image, we can now use skimage.measure to easily get quantified properties of the regions. Full list of properties can be found in the docs.

height = None

fig, ax = plt.subplots()
ax.imshow(binary_img, cmap=plt.cm.gray)

label_img = label(binary_img)
regions = regionprops(label_img)

for props in regions:
 y0, x0 = props.centroid

 minr, minc, maxr, maxc = props.bbox
 bx = (minc, maxc, maxc, minc, minc)
 by = (minr, minr, maxr, maxr, minr)

 area = (maxc-minc)*(maxr-minr)

 if area >= min_area:
 # Plot the bounding box
 ax.plot(bx, by, '-b', linewidth=2.5)
 # Plot the centroid
 ax.plot(x0, y0, 'ro')
 height = minr

plt.show()

The resultant image below shows how multiple areas are detected and classified. However, we can apply some RI(TM) (aka. real intelligence) to only pick the region of the sourdough starter. There are many ways to do this, but the simplest was to simply set a minimum area requirement. As long as it’s high enough, it should ignore all the other regions.

Minimum area is required since other areas will register as a detected region.

Due to the camera perspective and curvature of the jar, the boundary between the two regions is not actually straight. This was solved by taking a much narrower cropped region, which you will see below in the animated timelapses.

This may have been a bit easier with a square container…

Now that we have a working understanding of the components we need, let’s get scripting and collect a bunch of data! You can check out the full analysis on Github.

The Results

The timelapses below show the sourdough starter from different dates. The boundary tracking algorithm is reasonably well at detecting the correct height, but there are occasional outliers that cause the errors. However, the overall trend of the growth is still captured.

May 29, Left Jar

The rise on the left jar had the cleanest growth trace.

May 29, Right Jar

A narrow crop area had to be used to prevent detection of uneven rise levels.

May 31, First Feeding

Sunlight through the kitchen window created glare on the bottom left area on the jar, which affected the binary thresholding.

May 31, Second Feeding

Having the jar farther away reduced the thresholding accuracy, but the overall trace was still acceptable.

June 10, Out of Fridge

A larger jar was needed for this one! The thresholding algorithm was surprisingly still able ot catch the peak to some extent, despite minimal contrast.

June 23, First Feeding

After some neglect, the starter only rose to ~50%...

June 23, Refeeding

But after another feeding the next morning, it had enough activity to spring back ~80% growth!

The Discussion

The animations are cool to watch, but what can we interpret from it? Plotting all the growths (as shown below), we see that they seem more similar than different. The peaks hover around 60-80%, and the rate of growth coming up to the peak are similar.

All the timelapses plotted to compare normalized growth. Horizontal dotted line indicates 50% mark.

Effect of Regular Feeding

Taking only a select number of days, there is a clearer trend to be seen. The graph below shows how the growth magnitude and rate increases over frequent and regularly scheduled feedings.

On May 29th, we began to feed our sourdough starter after many months of sporadic feeding. With the June 10th growth, the ramp up occured later but eventually rises to ~80%. The ideal growth target is to double or triple in size, but hopefully that time will come with even more regular feeding.

We can see that regularly feeding the sourdough starter greatly increases its rate and growth.

Note: The time delay of each subsequent feeding is not a result of regularity, but rather because the starter to food ratio was reduced over time to promote a more active fermentation. We eventually migrated to a 50g starter + 100g water + 100g flour (50 / 50 white and whole wheat). The starter has more potential for growth by beginning with less starter and more food, but as a result it may take a bit longer for the bacteria to vigorously start multiplying.

Bringing the Starter Back to Life

Previous to June 23, the starter was neglected for a week. After the first (overnight) feeding, it showed little signs of growth. However, feeding it again around noon and tracking its progress shows that the growth springs back up to ~80%, which was around the previous maximum from before.

What doesn't kill you makes you stronger (or at least as strong as before). First feeding is shifted because I forgot to start the timelapse right away.

Thus, a bit of neglect seems to be okay since the starter appears to be fairly resilient to starvation! (But if you’re going on vacation, please feed it more than you typically would so it can sustain itself in your absence.)

The Conclusion

So what can we take away from ~40 hours of watching yeast rise¹? From all this, we learned:

The rate of growth appears to be fairly consistent, even with poorly maintained starters
A healthy, regularly fed starter reaches maximum fermentation growth after 5-6 hours.
The peak has a ~1 hour window for when it should be used
Regularly feeding a neglected starter will make it (and you) happy again

Even just taking a timelapse will give you a good feel for how your starter behaves, especially when you’re starting out with baking bread. If you leave it overnight and come back the next morning to what looks like minimal growth, a timelapse can tell you if it peaked 3 hours in without you noticing, or if it actually didn’t grow. Every starter is different, but if you can give it the attention it deserves, its fermentation will continually reward you with loaves like no other.

Happy baking!

From Prototype to Cloud: A Python Recipe Converter

Fri, 06 Apr 2018 20:33:46 -0700

In case the title wasn’t clear, this blog post is about developing a web application using the Python programming language using Jupyter Lab, Flask, and the Heroku platform. If you were looking for an article on python recipes, you can start off with this one on making a poached Burmese python curry.

The Backstory

The problem with online baking recipes is that the majority of them use volumetric units. As any civilized baker would know, Patricia’s 1 cup of flour may very well be different than Patrick’s 1 cup of flour. Maybe Patricia sifted her flour. Maybe Patrick’s organic flour is a finer texture. Maybe both Pats should measure by mass instead of volume to avoid all this confusion in the first place.

With this in mind, I was tired of constantly typing “1 tbsp baking soda in grams” and “3 1/4 cup flour in grams” into Google every time I came across a new recipe. There had to be a better way, and there was. As usual, Python was staring, patiently waiting for me to make eye contact with it.

My main reqiurement for this application was to be cross-platform. I wanted to be able to carry these conversions out on my laptop, someone else’s desktop, or my smartphone. Although React is what all the cool kids talk about these days, I also wanted to develop something fast. Since Python is both excellent at string parsing and easy to work with, it seemed like an appropriate choice.

After a bit of searching, I came across Flask and Heroku which would allow me to use Python for the entire development process, from prototype to deployment. Super cool.

The Outline

This blog post will outline the development of a simple string-parsing application that converts common ingredients measured in cups, tablespoons, and teaspoons to grams.

Prototype the recipe conversions in Jupyter Lab’s interactive environment
Migrate script to Visual Studio Code to a command line interface
Integrate the Python script with Flask’s web framework
Set up and deploy the app through the Heroku cloud platform

Interested in trying it out? It’s live at recipe-converter-app.herokuapp.com!

The Development

Prototyping in Jupyter

Sure you can do all your development in a text editor and command line. There’s nothing wrong with putting print statements everywhere, or even having a Python console open to do scratchpad-type testing. But when a tool as flexible and great as Jupyter is available for everyone to use, why settle for anything else?

The reason for Jupyter’s immense success is it excels in a form of programming called “literate programming”. It emphasizes a prose first approach where exposition with human-friendly text is punctuated with code blocks. It excels at demonstration, research, and teaching objectives - Unidata

One of its greatest selling points is the option to execute code in chunks and see what the output is at each step. For data science, pairing this with the ability to write markdown above or below the code block makes documentation and explanations a joy to work with.

The power of Jupyter's interactive output makes prototyping almost effortless. (Source: Noam Finkelstein)

Jupyter Lab is the successor to Jupyter (previously iPython) Notebook. The tabbed interface is a godsend. (Source: Jupyter)

On To The Prototype

The proposed use case is described below. Nothing complex happening, just a handful of string parsing operations and a conversion table lookup.

Copy ingredient list to clipboard
Run script
Display converted ingredient list

In the run script method (ie. the meat and potatoes of the app):

Store clipboard text to list data structure*
Parse fractions to floats
Unpack abbreviations (ie. tbsp = tablespoon)
Find each original ingredient in conversion table and convert to grams (where possible)
Repeat Step 4 for each provided ingredient

*Turns out it’s easy to get clipboard data from a locally running script, but not as straightforward when the script is being hosted on a browser. I guess the user will just have to paste the ingredients into a textbox instead.

Imperial volume to metric mass (gram) conversions in a csv file, ie. the wrong units to the right ones.

Checking out my first working version, you’ll see that it’s not the most efficient script. However, when proving out a concept, the priority is to get something working (ie. a minimum viable product) and optimize later. Since uncertainty of this working (or being worth it) is quite high, we don’t want to spend too much time making it perfect!

Testing out the prototype in Jupyter.

Comparing that with the current version, we can see that the code structure went from messy and repetitive to organized and modular. Cleaning up code is always mildly therapeutic.

Repetitive version on left, modular version on right.

Now that we’ve finished testing the prototype and are happy with how it works, we can move on to the next step in turning it into a browser-based application.

Migrating to Visual Studio Code

Fortunately, this is painless! In Jupyter, navigate to File > Export Notebook As... > Executable Script, and that’s it. A Python script will download with all the cells in a single executable file.

Exporting from Jupyter Lab to the next phase of development: the command line.

A bit of clean up later and the python script is ready to be imported as a module.

Feel free to check out the cleaned up file on GitHub. That was easy.

Integrating with Flask

Like the Thing Used in Chemistry?

No, this Flask is different! There are many options for web development in Python, with two heavy hitters being Flask and Django. They’re both great frameworks, but where Django is the more fully-featured framework, Flask follows the Unix philosophy of “do one thing and do it well”.

Flask is a lightweight WSGI web application framework. It is designed to make getting started quick and easy, with the ability to scale up to complex applications. It began as a simple wrapper around Werkzeug and Jinja and has become one of the most popular Python web application frameworks.

Flask offers suggestions, but doesn’t enforce any dependencies or project layout. It is up to the developer to choose the tools and libraries they want to use. There are many extensions provided by the community that make adding new functionality easy. - Flask GitHub

For myself (and this app), I chose Flask because of its simplicity and upfront/visible functionality. The function routing, template rendering is called explicitly with functions, with little magic happening “behind the scenes”. As this answer from StackOverflow puts it, “Django can be a little more mysterious for a beginner to figure out how everything fits together”.

Running With Flask

Now the fun really begins. We can start off by firing up a terminal and installing (and updating) the Flask microframework:

> pip install -U Flask

The simplest example of Flask is shown below (taken from Wikipedia). If you copy it below and execute the script, a blank webpage will render and display “Hello World!” in your browser.

from flask import Flask
app = Flask(__name__)

@app.route("/")
def hello():
 return "Hello World!"

if __name__ == "__main__":
 app.run()

The main difference here is the route() decorator. Modern web frameworks use routing techniques to help keep tracak of application URLs. This decorator is used to bind a Python function to a URL. In this case, the URL is /, otherwise known as the index / home / default page. If we were to change it to /hello, then we’d have to navigate to localhost:5000/hello to see “Hello World!”.

Saying hello from localhost:5000.

Now moving on to our recipe converter, we need two parts to make this work:

Flask python script for the backend
HTML template for the frontend

App.py (aka the Back End)

Looking at the code below, there isn’t much difference from the simple “hello world” application from above. One key change is the inclusion of methods = ['GET', 'POST'] in the route() decorator. From the quickstart guide:

GET - The browser tells the server to just get the information stored on that page and send it.

POST - The browser tells the server that it wants to post some new information to that URL and that the server must ensure the data is stored and only stored once. This is how HTML forms usually transmit data to the server.

Since we are going to have a few buttons on our webpage, we need to include these HTTP methods. With multiple buttons, we can do a simple value check to handle them differently.

The second key change is returning render_template() in the routed function call. Since we want to be able to format an HTML page, we can use a template (running on Jinja2) to create one.

We can also pass variables to the template. In this case, textarea is where the user will enter the ingredient list. When the ‘Convert’ button is clicked, the text will be parsed by parse_form_text(). To display the parsed/converted recipe, we can use the flash() method, which Flask provides as a feedback mechanism to the users (see Message Flashing).

from flask import Flask, request, render_template, flash
import recipeConverter as rc

# Stuff to initialize the Flask app
app = Flask(__name__)
app.config.from_object(__name__)
app.config['SECRET_KEY'] = '7d441f27d441f28567d441f2b6176a' # this can be anything

# This decorator tells Flask to use this function as a webpage handler/renderer
@app.route('/', methods=['GET', 'POST'])
def hello():
 # Default text to display in the HTML template
 input_text = "Hello world!"

 # Normal page load calls 'GET'. 'POST' gets called when one of the buttons is pressed
 if request.method == 'POST':
 # Check which button was pressed
 if request.form['submit'] == 'Convert':
 input_text = request.form.get("text")
 parse_form_text(input_text)

 elif request.form['submit'] == 'Clear':
 input_text = ''

 # Render the HTML template. input_text gets fed into the textarea variable in the template
 return render_template('form.html', textarea=input_text)

def parse_form_text(text):
 # This is where the magic happens
 recipe = rc.RecipeConverter()

 # split the text to get each line in a list
 text2 = text.split('\n')
 text_converted = recipe.parse_recipe(text2)

 for line in text_converted:
 flash(line)

if __name__ == '__main__':
 app.run()

Form.html (aka the Front End)

To accompany our Flask script, we need a usable interface. The interesting bits are encapsulated by {}, and everything else is straight HTML and the Bootstrap library.

The {{textarea}} is the input variable from render_template() above. For the message flashing, we can loop through the message list using get_flashed_messages().

<div class="page-header">
 <h1>Recipe Converter</h1>
 <p class="lead">Tired of reading recipes in cups, tablespoons, and teaspoons when you just want everything in grams? Paste your recipe in volumetric units in the textbox below, and hit "Convert"!</p>
</div>

<!-- Input column (left) -->
<div class="col-md-6">
 <h2>Input Recipe</h2>
 <p>
 <form action="" id="textform" method="post">
 <textarea name="text">{{textarea}}</textarea> <!-- Input text box -->
 <br>
 <input type="submit" name="submit" value="Clear"> <!-- "Clear" button -->
 <input type="submit" name="submit" value="Convert"> <!-- "Convert" button -->
 </form>
 </p>
</div>

<!-- Output column (right) -->
<div class="col-md-6">
 <h2>Output Recipe</h2>
 <p>
 <!-- Display each converted line from the flash method (from Flask) -->
 {% with messages = get_flashed_messages() %}
 {% if messages %}
 {% for message in messages %}
 {{ message }}
 <br>
 {% endfor %}
 {% endif %}
 {% endwith %}
 </p>
</div>

And we’re done! To run the Flask app, we can either execute the Python script by:

> python app.py

Or, as the Flask documentation suggests, we can also use the flask run command after setting the script as the FLASK_APP environment variable.

Unix Bash (Linux, Mac, etc.):

$ export FLASK_APP=app.py
$ flask run

Windows Command Prompt:

> set FLASK_APP=app.py
> flask run

Windows PowerShell:

> $env:FLASK_APP = "app.py"
> flask run

Running Flask in Visual Studio Code.

Deploying on Heroku Cloud

First thing’s first: Create download Heroku and create an account by running (after the installation completes):

> heroku login

Heroku should now be initialized on your local system!

Setting Up the Required Files

To deploy an application/dyno on Heroku, we need the following files:

app.py
- Flask Python script
Procfile
- Tells Heroku what commands are run by your application’s dynos
requirements.txt
- Tells Heroku what modules it needs to install
runtime.txt
- Tells Heroku which version of Python to use

The following sections will detail what goes into where.

Procfile

We’ll be using Gunicorn to run our application on Heroku, which is its preferred HTTP server. So in Procfile, add the following line:

web: gunicorn app:app --log-file=-

However, Gunicorn is only available on Unix systems. If we want to test our Heroku app locally, we’ll need to specify another command to run the app. For this, we’ll put the following in another file named Procfile.windows:

web: python app.py --log-file=-

This is functionally equivalent to running the Flask command from before, with the main difference is that this uses Heroku’s local hosting framework.

Requirements.txt

In Requirements.txt, we need to add the dependencies our app has. When we deploy it to Heroku cloud, it will install these modules on the server. Add the following lines:

gunicorn
Flask

Runtime.txt

Finally, we need to specify that our app runs on Python in runtime.txt. You can check your own version by running python -V or python --version.

In Runtime.txt:

python-3.6.4

Testing Heroku Locally

For a minimal application, our file structure should look like the following:

 ├── app.py
 ├── Procfile
 ├── requirements.txt
 └── runtime.txt

With our recipe converter app (and because we’re on Windows), we have the addition of two files shown below:

 ├── app.py
 ├── recipe-converter.py
 ├── Procfile
 ├── Procfile.Windows
 ├── requirements.txt
 └── runtime.txt

If you’re on a Unix system, you only need to run heroku local web to start a local server. However, since gunicorn is a webserver that isn’t available on Windows (but is required for Heroku), we need to have a separate Procfile for testing on Windows. Thus, we need to add -f procfile.windows to ensure Heroku uses the correct file. So to start Heroku locally on Windows, we can run:

> heroku local web -f procfile.windows

With any luck, the app should compile and run just like it did with Flask!

Pushing to Heroku App

Heroku uses git to receive files to host on herokuapp.com. First, we’ll setup the Heroku application with heroku create (where the second argument is the app name, otherwise a randomly generated one will be used).

> heroku create recipe-converter-app

The following commands will create a local git repository and queue all files to be pushed to Heroku’s servers.

> git init
> git add .
> git commit -m 'First commit'

Finally, we can push and deploy the files to Heroku:

> git push heroku master

Heroku output on compile.

Troubleshooting: If it compiles but fails to load at *.herokuapp.com, you can check the logs with the following command:

> heroku logs

Published to the Cloud!

After all that’s been said and done, the recipe converter app is now published on herokuapp.com! It’s a freemium service, so it’s worth checking out for professional development as well.

The final python app published through Heroku App.

Logging in to dashboard.heroku.com provides a nice dashboard for managing all your deployments, activity, and more.

Heroku dashboard to manage your dynos.

Closing Thoughts

To recap our on our journey, we:

Prototyped the recipe conversions in Jupyter Lab’s interactive environment
Migrated script to Visual Studio Code to a command line interface
Integrated the Python script with Flask’s web framework
Deployed the app through the Heroku cloud platform

Doing everything in Python has its caveats, but it still can be an excellent tool for quickly getting results. Although the deployed version is still (arguably) a prototype, the amount of legwork we had to accomplish to achieve this end product was relatively minimal!

As long as we keep in mind that Python is not the solution to everything, we can use it where appropriate and/or when we are aware of its trade-offs.

Thanks for reading, and I hope you’re motivated to either try developing your own app or to stop using volumetric units. If you had to choose one, my vote is on the latter.

Happy cooking!

Synchronous vs Asynchronous Ping Sweep in C# Windows Form

Fri, 09 Feb 2018 17:02:25 -0800

As a mechatronics engineer (in training), sometimes I like to pretend that I also know how to program.

In my most recent adventures to software land at MistyWest, I needed to write an application in C# that involved doing a ping sweep to find devices that were physically connected through ethernet. Since Google and Stack Overflow are my two best friends, I was able to find (what seemed to be) an off-the-net solution quite quickly.

However, despite this being a relatively well known objective with well-known libraries to accomplish it, my journey to developing a solution was not as easy as I originally thought. The following post outlines the things I tried before arriving to a working solution, which hopefully is at least mildly interesting and/or educational. Also if you’re a software engineer reading this, please go easy on my code. Or not.

The Development

First Attempt: Some-ping Simple

A quick Google search showed that pinging addresses is dead easy in C#. Using the System.Net.NetworkInformation namespace, we can easily use the Ping.Send() command to check if a remote address is alive.

using System.Net.NetworkInformation;

int timeout = 10; //in ms

Ping p = new Ping();
PingReply rep = p.Send("192.168.1.1", timeout);

if (rep.Status == IPStatus.Success)
{
 //host is active
}

This is great if we only had a few addresses to ping, because unfortunately this method is unacceptably slow for a user waiting to see if any devices are found. Although Ping.Send has an overload to accept a timeout interval, it appears that setting very low values doesn’t actually change much. From the MSDN docs:

When specifying very small numbers for timeout, the Ping reply can be received even if timeout milliseconds have elapsed.

In practice, it seems that ~500ms is about the fastest threshold that can be set. Unfortunately, scanning 255 IP addresses this way will be excruciatingly slow (for both the developer and end user). And the search continues…

Second Attempt: Jaime LAN-nister, Pingslayer

A few more Googles later and I had what seemed to be the golden solution.

Thanks to Tim Coker, I thought I was mostly done my application already (knowledge is half the battle, right?). His console application worked wonderfully and was written in C#, which was great news since I was developing the application using Windows Forms. Instead of using the synchronous Ping.Send(), it harnessed the asynchronous Ping.SendAsync() along with the CountdownEvent class in System.Threading. However, upon copying the code into Windows Forms, it didn’t seem to work. What gives?

/* Source: https://stackoverflow.com/a/4042887
 * Original author: Tim Coker
 */

using System;
using System.Diagnostics;
using System.Threading;
using System.Net.NetworkInformation;

namespace ConsoleApplication1
{
 class Program
 {
 static CountdownEvent countdown;
 static int upCount = 0;
 static object lockObj = new object();
 const bool resolveNames = true;

 static void Main(string[] args)
 {
 countdown = new CountdownEvent(1);
 Stopwatch sw = new Stopwatch();
 sw.Start();
 string ipBase = "10.22.4.";
 for (int i = 1; i < 255; i++)
 {
 string ip = ipBase + i.ToString();

 Ping p = new Ping();
 p.PingCompleted += new PingCompletedEventHandler(p_PingCompleted);
 countdown.AddCount();
 p.SendAsync(ip, 100, ip);
 }
 countdown.Signal();
 countdown.Wait();
 sw.Stop();
 TimeSpan span = new TimeSpan(sw.ElapsedTicks);
 Console.WriteLine("Took {0} milliseconds. {1} hosts active.", sw.ElapsedMilliseconds, upCount);
 Console.ReadLine();
 }

 static void p_PingCompleted(object sender, PingCompletedEventArgs e)
 {
 string ip = (string)e.UserState;
 if (e.Reply != null && e.Reply.Status == IPStatus.Success)
 {
 Console.WriteLine("{0} is up: ({1} ms)", ip, e.Reply.RoundtripTime);
 lock(lockObj)
 {
 upCount++;
 }
 }
 else if (e.Reply == null)
 {
 Console.WriteLine("Pinging {0} failed. (Null Reply object?)", ip);
 }
 countdown.Signal();
 }
 }
}

Lesson learned: Console applications and Windows Forms are different beasts and deadlocks occur with the above code. According to Hans Passant from another Stack Overflow thread, the additional UI thread is the culprit:

Winforms has a synchronization provider whereas console apps do not. The problem is that the Ping class makes a best effort to raise the PingCompleted event on the same thread that calls SendAsync(). So it tries to raise the event on the main thread, but that can’t work since the main thread is blocked with the countdown.Wait() call. In a console app however, the PingCompleted event will be raised on a ThreadPool thread.

Hm, turns out this problem wasn’t as easy as copying and pasting random code off the internet. Time for a bit of research!

Digging Deeper: A Battle of Asynchronous Pings

For some reason that can only be to confuse inexperienced programmers like myself, there are two different asynchronous ping methods in this class. First on the list:

Ping.SendAsync(): Asynchronously attempts to send an Internet Control Message Protocol (ICMP) echo message to a computer, and receive a corresponding ICMP echo reply message from that computer. - MSDN

And the second:

Ping.SendPingAsync(): Sends an Internet Control Message Protocol (ICMP) echo message to a computer, and receives a corresponding ICMP echo reply message from that computer as an asynchronous operation. - MSDN

Based on these method descriptions, it appears that SendAsync() does not guarantee asynchronous operation. Since we just learned how threading in console applications vs windows forms is dealt with differently, this may be why it didn’t work as expected in the latter case. What we want are guaranteed asynchronous operations, so hopefully SendPingAsync() should perform to its name.

(One can only assume this second method was added after-the-fact when Microsoft realized that developers wanted a truly asynchronous ping…)

The takeaway: Use SendPingAsync() or bust.

Digging Deeper: Background Worker vs Async/Await

So we’ve selected our asynchronous ping method, but that still leaves us hanging over how we’re going to handle it in a background task. Before .NET 4.0 was released, BackgroundWorker was the de-facto standard¹. However:

The core problem that BackgroundWorker originally solved was the need to execute synchronous code on a background thread. If you’re using it for asynchronous or parallel work, you’re not using the right tool in the first place. - Stephen Cleary

After .NET 4.0, we have the following options:

Tasks (Async Methods)
Tasks (Task Parallel Library)
Delegate.BeginInvoke
ThreadPool.QueueUserWorkItem
Threads

Discussing each of these methods is beyond the scope of this post, but you can read more on Cleary’s article on various implementations of asynchronous background tasks. In short, using Task-returning asynchronous methods is the best overall method to use.

Kind of.

For my application, sending each ping to its own task using async/await is logical, as I could then call Task.WhenAll() to wait until all pings have been received back to the main thread.

However, I could still use BackgroundWorker for SFTP file transfer from the remote devices to a local directory (required in my final application, but not included in this ping sweep demo). Doing so would prevent the main UI thread from hanging while files are being transferred. Although async/await may also be used for this, BackgroundWorker seemed to be the more appropriate (and easier) implementation since each file is serially transferred from remote to local device. Additionally, it’s just drag and drop in WinForms!

The takeaway: Use async/await for asynchronous ping sweep, and BackgroundWorker for SFTP file transfers.

Third Attempt: One Ping to Rule Them All, One Ping to Find Them

Finally, we have a working solution! Using async/await with Tasks in System.Threading.Tasks yields promising results. See below for the implementation.

private string BaseIP = "192.168.1.";
private int StartIP = 1;
private int StopIP = 255;
private string ip;

private int timeout = 100;
private int nFound = 0;

static object lockObj = new object();
Stopwatch stopWatch = new Stopwatch();
TimeSpan ts;

public async void RunPingSweep_Async()
{
 nFound = 0;

 var tasks = new List<Task>();

 stopWatch.Start();

 for (int i = StartIP; i <= StopIP; i++)
 {
 ip = BaseIP + i.ToString();

 System.Net.NetworkInformation.Ping p = new System.Net.NetworkInformation.Ping();
 var task = PingAndUpdateAsync(p, ip);
 tasks.Add(task);
 }

 await Task.WhenAll(tasks).ContinueWith(t =>
 {
 stopWatch.Stop();
 ts = stopWatch.Elapsed;
 MessageBox.Show(nFound.ToString() + " devices found! Elapsed time: " + ts.ToString(), "Asynchronous");
 });
}

private async Task PingAndUpdateAsync(System.Net.NetworkInformation.Ping ping, string ip)
{
 var reply = await ping.SendPingAsync(ip, timeout);

 if (reply.Status == System.Net.NetworkInformation.IPStatus.Success)
 {
 lock(lockObj)
 {
 nFound++;
 }
 }
}

Asynchronous pings are light years faster! Half a second and we’re rocking with all the pings we needed.

Verification with Benchmark

To make sure our numbers add up, let’s compare it with a known tool that is much more mature than this C# application.

Verifying scanned result with Angry IP scanner because apparently nmap isn’t fully functional in Windows Subsystem for Linux yet.

Same number of hosts alive, but a little slower. However, Angry IP Scanner is more robust in its pings and thread handling, so the few extra seconds is likely put to good use. I found my application to be somewhat inconsistent in finding all the alive hosts, which may be mitigated with multiple ping packets to send (instead of just one).

Comparing with Synchronous Ping Sweep

Because pinging is such an interesting past-time, let’s see how the very first synchronous solution performs using Ping.Send().

private string BaseIP = "192.168.1.";
private int StartIP = 1;
private int StopIP = 255;
private string ip;

private int timeout = 100;
private int nFound = 0;

Stopwatch stopWatch = new Stopwatch();
TimeSpan ts;

public void RunPingSweep_Sync()
{
 nFound = 0;

 stopWatch.Start();
 System.Net.NetworkInformation.Ping p = new System.Net.NetworkInformation.Ping();

 for (int i = StartIP; i <= StopIP; i++)
 {
 ip = BaseIP + i.ToString();
 System.Net.NetworkInformation.PingReply rep = p.Send(ip, timeout);

 if (rep.Status == System.Net.NetworkInformation.IPStatus.Success)
 {
 nFound++;
 }
 }

 stopWatch.Stop();
 ts = stopWatch.Elapsed;

 MessageBox.Show(nFound.ToString() + " devices found! Elapsed time: " + ts.ToString(), "Synchronous");
}

Result of 255 pings using a synchronous method. Nobody has 2 minutes to wait for a complete scan.

Wow. A full two minutes compared to less than one second. Life truly is better when you live asynchronously.

Closing Thoughts

This concludes another adventure through asynchronous methods in C#. It’s amazing how much information is available on the internet, and I truly would not be able to get this far without it. Google and Stack Overflow, what would I be without you?

Thanks for reading, and I hope you learned a ping or two about these methods.

Check out the full source code of this project on Github!

According to Stephen Cleary, one of Microsoft’s Most Valuable Professionals and top answerer for async/await questions on Stack Overflow. I’d trust him if I were you. ↩︎

Measuring the Spectral Characteristics of a Light Therapy Lamp

Fri, 12 Jan 2018 09:42:57 -0800

Disclaimer: Light therapy is one method of easing seasonal affective disorder (SAD); some people swear by it whereas others remain unaffected. This blog post does not intend to refute the effectiveness of light therapy, but rather to dig deeper into the technology behind these light therapy lamps to better educate fellow consumers.

Ah, the winter blues of Vancouver, BC. While some days bring bluebird skies and fresh pow for skiing, other days are downright gloomy. Spending this past Christmas in Singapore meant warm, sunny, and +25°C weather, which is quite atypical of how most Canadians spend their winters.

However, coming back to the overcast, bone-chilling, gray, and wet Vancouver winter was a climate shock my body had yet to feel before. A week or so of jetlag-ridden sleep cycles and my circadian rhythm was back to normal, but I couldn’t scratch the itch of yearning for sun and truly despising this city’s gloomy winter weather.

Thanks to Amazon Prime’s same/one day shipping, a solution to this problem was less than 24 hours away. Introducing my new light therapy lamp!

Verilux HappyLight Compact Light Therapy Lamp with the UV filter/diffuser installed.

Ultraviolet difuser/filter removed from the lamp. Wait a minute, is that just a CFL bulb?

The Suspicions

But wait just one moment; upon unboxing the lamp, I noticed something familiar. The light source behind the box looked strangely similar to a standard CFL bulb… Had I just been duped into spending $70 for a plain old light bulb in a fancy plastic enclosure?

Left: $14.99 Verilux Full Spectrum Bulb. Right: $1.50 Standard CFL bulb.

Fortunately for me, I work at MistyWest and we have a fancy spectrometer that will allow me to confirm/deny my suspicions. It’s used to measure the spectral power density at each wavelength of light (or in layman terms, it measures the intensity of each specific colour of light). The two main questions I was curious to answer:

Are the spectral characteristics of this bulb any different than a regular CFL bulb?
Is the filter/diffuser actually made of a special UV-blocking material?

But before we get into the details of spectral characteristics, let’s shed a bit of light on the different lighting methods and how they came to be. (Stay with me; it’s worth it.)

The Research

Illumination Nation

Overview of Light Sources

Different sources of light can have significantly different spectral characteristics. The figure below shows how six different light sources vary greatly at each wavelength.

Typical spectral characteristics and corresponding colours of various lighting. (Source: HouseCraft)

Daylight
- Rounded peak around 450 nm
- Broad spectrum light intensity
- 5000 - 6500K
Incandenscent
- Increases from 400 nm to 700 - 800 nm
- Burning of tungsten filament inside a vacuum bulb produces a very warm glow (in addition to a great amount of heat!)
- 2400 K
Fluorescent
- Distinct peaks around 420 nm, 490 nm, 550 nm, and 610 nm
- Peaks are due to the fluorescence of excited phosphor within the glass tube
- 2700 - 6500 K

Electricity ionizes the mercury and argon gas, producing UV light which hits the phosphor coating and finally fluoresces white. (Source: Natural Blaze)

Halogen
- Rounded peak around 600 nm
- Also burns a tungsten filament, but the gas is at a higher pressure for brighter illumination (basically a more advanced incandescent bulb)
- 2800 - 3400K
Cool White LED
- Sharp peak around 450 nm, rounded peak around 550 nm
- Gallium nitride on a sapphire substrate produces blue (first peak), and a phosphor coating produces yellow light through fluorescence (second peak)
- 3500 - 4100 K
Warm White LED
- Shallow peak around 450nm, large peak around 550 nm
- Combining different phosphors change the fluorescent characteristics of the emitted light
- 2700 K

When an electron flows from anode to cathode across the band gap, it falls into a lower energy level and releases energy in the form of a photon (light). (Source: Lumen Electronic Jewelry)

A Walk Down Memory Lane

Now, slapping some dates to the evolution of lighting technologies (thanks Wikipedia) gives us a pretty good understanding of how this all played out:

1904 - Alexander Just and Franjo Hanaman invent the tungsten filament for incandescent lightbulbs.
1926 - Edmund Germer patents the modern fluorescent lamp.
1927 - Oleg Losev creates the first LED (light-emitting diode).
1953 - Elmer Fridrich invents the halogen light bulb.
1962 - Nick Holonyak Jr. develops the first practical visible-spectrum (red) light-emitting diode.
1981 - Philips sells their first Compact Fluorescent Energy Saving Lamps, with integrated conventional ballast.
1995 - Shuji Nakamura at Nichia labs invents the first practical blue and with additional phosphor, white LED, starting an LED boom.

Through this brief overview of light sources, we can see how the different illumination methods developed from sending current through a tiny strand of wire making it glow bright, to using fluorescence as an energy efficient way to produce light (and also at different colour temperatures).

So What?

The takeaway through this stroll through history lane is this: even before taking the spectral measurement of this bulb, we already know that the likelihood of this light therapy bulb being different is quite low. The History of Light(TM) paints a clear path of what’s both economically and physically possible in creating a light source, so this mystical light therapy bulb is almost certainly just your everyday compact fluorescent bulb (with a GU10 base).

Let's mix and match different bases and shapes and label it as a proprietary bulb! The marketing team will love it. (Source: Tomic Arms)

But since we’re already here, let’s measure the data anyway! It’s not everyday we get to learn about the physics we’re surrounded by, especially when it involves expensive ~~toys~~ instruments.

Let’s Get To It: Playing with the Spectrometer

With our Ocean Optics visible light spectrometer placed about 10 cm in front of the light therapy lamp and set with a 10 ms integration time, we can capture the spectral characteristics of the bulb. To recap, the two things we’re interested in are the following:

Are the spectral characteristics of this bulb any different than a regular CFL bulb?
Is the filter/diffuser actually made of a special UV-blocking material?

To CFL or To Not CFL? That is the Question

Looking at the figure below, the trace in blue is the response with the frosted plastic in front of the bulb (ie. the lamp fully assembled). Surprise, surprise: the distinct peaks suggest that it is in fact a compact fluorescent bulb.

Spectral responses with and without the UV diffuser/filter.

However, a typical fluorescent bulb has sharp peaks and a flat baseline. What’s up with the rounded peak around 450 nm?

Standard fluorescent tube. (Source: HouseCraft)

Digging a bit deeper, it appears that the additional peak is due to the different colour temperature. Based on the spectral data collected by Advanced Aquarist shown below, the rounded peak is likely from an additional/different phosphor inside the tube. This creates additional fluorescence to shift the visible colour temperature towards a more daylight-esque tone (by adding more blue content). On the other hand, the soft white bulb shows a flat spectral base, as expected from our previous graphs.

Spectral response of a daylight CFL bulb, 6500K (left) and soft white, 2700K (right). (Source: Advanced Aquarist)

The takeaway: CFL bulbs have similar spectral characteristics to daylight in terms of how our eyes interpret it. This particular light therapy lamp uses a standard 6500K CFL bulb, so you’re kind of paying for the marketing hype.

True or Alternative Fact: UV Blocking Filter

So now that we have our first question answered, let’s get on with the second. The frosted filter/diffuser that comes with this lamp is advertised to ensure it is a UV-free light source. Is this an actual optical filter that transmits light beyond the UV wavelengths (> 400 nm), or does it just a piece of plastic?

Let’s look closely at the spectral characteristics. Zooming in to the 350 - 400 nm region and changing the y-axis to a log scale (to better visualize the data), we see that the peak around 365 nm is indeed attenuated by a factor of ~10. However, looking back at the original figure (above), we see that the entire spectral response is attenuated.

Detailed look at the UV exposure on a logarithmic scale.

To put a number to this, we can numerically integrate the spectral power density at each wavelength and sum up the values to calculate irradiance (in units mW/cm²). Over the entire measured spectrum of 350 - 800 nm, we see that the diffuser reduces the total brightness by ~48%. So instead of removing just the UV wavelengths, they diffused the entire spectrum until the UV exposure is low enough to meet the “no UV exposure” threshold.

Comparison of total irradiance with and without the diffuser.

Integrating the UV region between 350 - 400 nm shows that there’s still a bit of UV light that gets through. However, it’s of insignificant amount compared to the UV exposure from other sources (ie. sun, TV, computer screen, etc.) that they can say that there’s “effectively no UV radiation” from this lamp.

Comparison of irradiance in the UV range, with and without the diffuser.

The takeaway: UV is removed by attenuating the entire spectral response of the CFL bulb until the UV light is removed (for practical purposes). Yes, the frosted diffuser also makes the lamp more user friendly (as staring into a bare bulb is rather jarring), but now you know that you can simply replace it with another diffuse material should you ever lose that part.

The Light at the End of the Tunnel

I hope this journey through light and its spectral characteristics was both entertaining and educational. With any luck, you may have learned a thing or two (and gained an appreciation) about all the different light sources surrounding you everyday.

Although you can build your own light therapy lamp for a fraction of its retail purchasing cost, that may not be a viable (or convenient) option for everyone. Additionally, light therapy is apparently more dependent on the brightness output rather than the actual wavelengths; as long as it’s bright and white(-ish) then it should work for light therapy!

Whatever you choose to do with this information is up to you, so light up your own path to mitigating your SAD-ness :)

Acknowledgements: Thanks to MistyWest for letting me use their spectrometer to measure this data!

Debugger Setup with GDB + OpenOCD in Visual Studio Code

Sun, 29 Oct 2017 14:24:52 -0700

Visual Studio Code’s combination of functionality, customizability, and aesthetics makes it one of my favourite code editors. As such, I was set on making it work with embedded development since I was getting started with the STM32 line of microcontrollers. I was following the steps outlined in Mastering STM32 by Carmine Noviello (which is an excellent resource) until it said to use Eclipse, because life’s too short to use software with unnecessary bloat. Enter VS Code and someone wanting to use the latest and greatest in code editors.

The only thing that was keeping me away was the lack of out-of-the-box debugging compatibility. Fortunately, with VS Code’s debugging capability and my sunk-cost pain of figuring this out, getting this workflow going is fairly straightforward!

Prerequisites

Optional

STM32CubeMX - For setting up projects. Now making makefiles!
PlatformIO - A one-stop shop for setting up common hardware kits.

Note: The following was set up on Windows 10 for STM32 Nucleo F303K8. The Nucleo F030R8 was also tested and confirmed working with this setup.

Configuring VS Code

Open the Debug panel (CTRL + SHIFT + D) and select “Add Configuration > GDB” through the top left dropdown arrow. Create a GDB configuration in launch.json and add the following. Note: Change the paths in “target”, “gdbpath”, and “autorun” to the correct locations.

{
 "name": "GDB",
 "type": "gdb",
 "request": "launch",
 "cwd": "${workspaceRoot}",
 "target": "${workspaceRoot}/.pioenvs/nucleo_f303k8/firmware.elf",
 "gdbpath" : "C:/STM32Toolchain/gcc-arm/5.4 2016q3/bin/arm-none-eabi-gdb.exe",
 "autorun": [
 	"target remote localhost:3333",
 	"symbol-file ./.pioenvs/nucleo_f303k8/firmware.elf",
 	"monitor reset"
 ]
}

GDB debugger entry added after successfuly entry in launch.json file.

Starting a Debug Session

Before entering debug mode (F5), you need to launch the OpenOCD server. Open Terminal in VS Code (CTRL + ` ) and type:

openocd -f board\st_nucleo_f3.cfg

Adding breakpoints and stepping through code on an STM32 through VS Code!

Unfortunately you must always start the OpenOCD server before hitting F5, but fortunately you can just hit Up in the terminal to recall the last command.

Troubleshooting

When running OpenOCD, the following error might come up:

Error: libusb_open() failed with LIBUSB_ERROR_NOT_SUPPORTED

Use Zadig to upgrade ST-Link Debug to the correct WinUSB driver. Enable through Options > List All

Fixing faulty USB drivers on Windows 10.

Happy debugging!

Annealing 3D Printed Plastics: Sous Vide Style

Tue, 13 Jun 2017 15:56:53 -0700

Featured on Hackaday, 3D Printing.com, All3DP, and Fictiv Blog. Also cited in a few research papers¹^,².

Yep, you heard it right. With all the craze surrounding cooking sous vide these days, it was only a matter of time before someone decided to venture using it outside of the culinary world. Turns out that someone also had a 3D printer, and you won’t believe what happened next!

Click-bait headlines aside, this post is quite lengthy and not everyone may have the patience to get through it all. Feel free to read the TL;DR below, or if you’re feeling inclined, follow along my adventure through the land of 3D printing, materials science, and modern cooking.

Measuring maximum force before material failure, recorded on an iPhone 6 at 240 fps.

The Short Version

Heat treatment was carried out on 3D printed parts using a temperature controlled water bath (aka sous vide) instead of being baked in an oven.

What the goal was:

To determine if extra strength can be squeezed out of PLA filament by annealing the parts and testing how much force can be applied before the test piece breaks in half.

How it was tested:

Printed some small test blocks (7mm x 7mm x 30mm)
Submerged them in water at 70°C for 30 mins
Half of the blocks cooled to room temperature in air (70°C to 18°C in < 10 mins), and the other half were regulated to cooled much slower (70°C to 18°C in > 4 hours)
Applied a point load on the test block
Recorded the maximum load before failure

What the results suggest:

Using a temperature controlled water bath provides a more stable, uniform, and controllable heat source (so parts are less prone to warping from uneven heating)
Annealing PLA yields an increase in mechanical strength under some circumstances
Testing was fairly inconclusive due to the inconsistency/human error in rate of applied force

What the takeaways are:

Annealing printed parts by sous vide is a plausible method for annealing PLA
Further testing is required to conclusively determine the balance between post-processing time and the resulting performance gains

The Long Version

Let’s say we wanted to make a highly functional part using a 3D printer. Maybe you need a replacement gear or a weight-bearing mounting bracket, and 3D printing would be the easiest way to fabricate the part.

You carefully select your slicer settings to optimize shell thickness, infill density, and layer height based on your application. You select a material that adequately suits your needs based on material strength or flexibility and environmental factors like UV exposure and heat resistance. You even take layer geometry into consideration to maximize strength in the loading direction.

Optimal layer orientation with respect to direction of primary load.

With all settings configured and high hopes for success, you load up the filament and hit “PRINT”. Hours later and your part is complete and, giddy with excitement, you pop it off the printer bed and finally test the part.

Fence bracket repaired (left), and broken after a heavy windstorm (right).

But it still fails. Tears ensue. Aspirations crumble. Is there any hope for the humanity of functional, home-made, plastic designs?

The Problem With Printed Parts

3D printing shines in the rapid creation of designs with moderately complex geometries. From trinkets to tool holders and enclosures to gear trains, 3D printing has found its way into a variety of purposes. However, at the end of the day, they’re only plastic parts having limited practicality in more demanding applications.

The inherent property of these parts is that they’re built layer upon layer, with different areas being rapidly heated and cooled at different rates. This causes internal stresses and they end up acting like perforated lines that are prone to snapping apart.

Thankfully there’s an entire industry dedicated to squeezing every ounce of performance out of material properties, so that will be an adequate starting point. Let’s begin!

The Solution: Heat Treatment

In metallurgy (the study of physical and chemical behaviour of metallic elements), annealing is a heat treatment process that alters the material’s physical (and sometimes chemical) properties³. For common metals such as copper, steel, silver, and brass, the process looks something like:

Heat material until glowing
Maintain at desired (recrystallization) temperature
Slowly let cool to room temperature

The Importance of Reducing Internal Stresses

Typically with any material, internal defects are evident (notably on a microscopic scale) and create internal stresses which weaken its overall strength. When creating metal parts, the initial metal-forming processes create these defects and as a result, the metal will crack under stress along these stress-forming juncture lines called “grains”.

To minimize the effect of these grains, annealing can be done to soften the material, relax the grain structures causing the internal stresses, and allow new, strain-free grains to form as replacements.⁴

Diagram showing the effect of heat treatment on the material's microstructure. (Source: 3DSourced)

So to recap our newfound knowledge in maximizing material performance in metals:

Internal stresses are bad
Internal stresses are created when a material is pushed, squeezed, and formed into a part
Internal stresses can be reduced by reheating, softening, and re-hardening the part

How does this relate to our 3D printed plastic?

With 3D printed parts, these internal defects occur on a more macroscopic scale. Plastic is heated, pushed through the extruder nozzle, and quickly cooled to form a layer of a printed part. Since plastic is poor conductor of heat, it cools unevenly and result in a mishmash of internal defects and grains. When an entire part is fabricated with this method, there’s really no surprise that parts usually break fairly easily! Each printed layer forms a juncture line of non-ideal bonding, and within each layer yields internal stresses due to rapid and uneven cooling.

A Review of Current Research

Fortunately, a similar heat treatment process can be applied to plastics to remove these nasty stresses and allow internal harmony to coalesce. I came across a research paper by Lih-Sheng Turng and Yottha Srithep, which discusses the relationship of crystallinity (ie. the degree of structural order in a solid) and mechanical properties of injection molded polylactide, commonly known as PLA⁵.

… Or, in plain English: they took a bunch of plastic sample pieces, performed some heat treatment on them, stuck them back in an oven to see if they still deform, and measured how much better the annealed samples hold up in the heat. Let’s dig in and see what they found!

The Nitty Gritty of Crystallinity

To go into a bit more detail, increasing a polymer’s crystallinity is good because it can lead to an increase in stiffness, strength, heat deflection temperature, and chemical resistance. However, this is difficult to do with PLA because of its low crystallization rate and its required slow cooling rate.

The Findings

The figure below shows the outcome of non-annealed (clear, first from bottom) and annealed (at varying temperatures and times) PLA test pieces. After heat treatment, they were placed in an oven to test their heat resistance. The annealed samples held up well in comparison with the non-annealed sample, suggesting that post-molding heat treatment results in better heat resistance and potentially mechanical performance as well.

The first sample (non-annealed) had the lowest degree of crystallinity and highest deformation at 65°C. (Source: Turng and Srithep, 2014)

Looking at the graph below, we see that the PLA samples had a maximum crystallinity of about 49%. Maintaining the oven/annealing temperature at 80°C led to the fastest rate of crystallization, whereas 65°C had the slowest rate.

Degree of crystallinity versus annealing time. (Source: Turng and Srithep, 2014)

This shows that maximum crystallinity can be achieved even at lower temperatures, as long as the material is given enough time to sufficiently undergo recrystallization.

The Takeaway

Unfortunately, this paper only tested the heat resistance of the annealed samples, as it would have been interesting to see them evaluate other mechanical properties such as tension/compression and getting a stress/strain curve out of it all. But this at least sheds some insight on performing heat treatment on PLA; if it improves heat resistance, then it should also improve other (potentially related) mechanical properties.

A Review of Current Methods

From a cursory search in the 3D printing community, annealing PLA seems to be a common, known method in squeezing a bit of extra mechanical performance out of printed parts. YouTubers Thomas Sanlader⁶ and Joe Mike Terranella⁷ have shown both quantitative and qualitative results in strength improvements by annealing.

The Oven Bake Method

Thomas’ approach in testing oven-baked samples was nicely scientific, and warping was shown to be an issue since ovens aren’t great at providing even, uniform heating.

Annealing various 3D printed plastics in an oven. (Source: Thomas Sanlader, YouTube)

The Boiling Water Method

Joe’s approach with boiling PLA was a good proof of concept, but it was only qualitative, his parts were floating in the water, and most importantly, no data was collected (savage).

Boiling PLA for 10 minutes for extra strength. (Source: Joe Mike Terranella, YouTube)

Areas of Improvement

Using water as a heat source is advantageous because it provides fairly uniform heating, but temperature control is fussy to maintain a specific temperature. Ovens are convenient since it provides a (moderately) temperature controlled chamber, but heat transfer from the heating element to the part is less than ideal and still leads to uneven heating.

If only there was a way to combine the temperature control of an oven and uniform, stable heating of a water bath…

Annealing PLA with… Sous Vide?

Yes, that’s right. Sous vide is the ultimate hero of this story.

A while back, I made a sous vide controller to get in on the cooking fad. A few months later and the novelty wore off, but I still had a modular, capable temperature controller ready for its next task (coffee roasting comes to mind, but I digress). In comes my 3D printer, and the combination of cooking and tinkering lead to the idea of performing heat treatment with a kitchen gadget.

DIY sous vide controller hooked up to a kettle.

Ladies and gentlemen, welcome to the meat and potatoes of this post.

To recap, we’ve learned why annealing is desirable to reduce internal stresses (ie. increase crystallization), what previous research has identified, and what current heat treatment processes have already been tried. Although the presented information has helped in answering our preliminary questions, we still have unanswered ones that are left for us to uncover and test.

The Questions

Will annealing PLA in a temperature controlled water bath improve its mechanical properties?
What effect does layer height have on annealed parts?
Do we really need to cool the samples slowly, or can we get away with (quicker) cooling in room temperature?

The Setup

We can learn from Joe’s mistakes by making sure our parts are submerged to reduce any risk of non-uniform heating. I put the samples in a Ziploc bag with a few Canadian pesos to keep them underwater. Two batches of 6 samples each were cooked sous vide, one set at 0.2625 mm layer height and the other at 0.175 mm.

Deprecated pennies were used to keep the samples submerged in the temperature controlled water bath.

Test samples lined up for carnage.

Designing the Tests

Wait, hold the phone: we still have many other questions! What annealing temperature is going to be maintained? How long are the samples going to be annealed for? Why are the samples so small?

All great questions, but unfortunately not all have great answers.

Q: What annealing temperature are we going to maintain?

PLA melts around 180-220°C, and its glass transition temperature is between 60-65°C⁸. We’re interested in the latter since that’s the temperature where recrystallization occurs. However, lower quality PLA requires higher temperatures due to more impurities in the material. To be safe, we’ll set the temperature to 70°C. According to the graph of crystallinity vs annealing time (a few page scrolls above), we’ll hit the maximum 48-49% crystallinity at around 6 hours.

Q: Wait, 6 hours of annealing time? Are we really going to wait that long?

Ain’t nobody got time for that! I’m an impatient guy, and waiting for my 3D prints to finish is painful enough. If we want to be robust in our test methods, sure we can wait 1/4 of an entire day to extract a bit more performance out of a plastic part. But I’m also a practical guy, so I want to see how little time I can get away with to achieve a meaningful increase in strength. My threshold for this is about 30 minutes; anything longer and I would question if it’s worth it for everyday printing, so we’ll go with that.

Q: Sounds good. But why are the test samples so small? Other people seem to test with much larger parts.

Since we’re applying (what we’ll assume to be) a point force, the sample doesn’t actually need to be that long. In terms of cross-sectional area, wall thickness has a much larger impact on a part’s strength than infill. Thus, these samples were designed to be hollow with a 2.0 mm wall thickness, which is actually a reasonable thickness for standard printed parts. Since those are the criteria that needs to be met, the sample just needs to be big enough to be able to test with (ie. long enough to span the gap). And going back to my impatience, 8 minutes is about the longest I want to wait for these samples since I’ll be printing multiple of these.

The Procedure

Now that that’s out of the way, we can finally start testing and breaking things!

Annealing the Samples

Print 9 rectangular prisms as the test samples
Remove 3 samples as the control set (ie. unmodified and directly off the printer)
Fill kettle with room temperature water
Submerge the remaining 6 samples in water bath
Set desired temperature of water bath
Maintain temperature for 30 mins
Remove 3 samples (1st sous vide set) and allow to air cool at room temperature
Turn off heat and allow the remaining 3 samples (2nd sous vide set) to slowly cool with the water bath

Breaking the Samples

Place test jig on top of bathroom scale
Apply vertical point force on the sample
Record maximum load before sample catastrophically explodes

Overview of test setup. Camera is used to capture the scale measurement at peak force.

The Results

To recap our hypotheses:

Annealing PLA in a temperature controlled water bath will promote crystallinity (and thus lower internal stress) in comparison with using conventional ovens.
Samples printed at a 0.175 mm layer height will have higher internal stress than those at 0.2625 mm.
Samples cooled at room temperature in air will have higher internal stress than samples cooled with the water bath.

The figures below tell the story. However, the results were not 100% as expected! And that’s why we test our assumptions.

Measuring the Maximum Applied Load

For the 0.2625 mm layer height, there was virtually no change between the control and annealed (sous vide, slow/fast cooled) samples. On the other hand, the samples printed at 0.175 mm demonstrated ~53% increase in resistance to shear force when compared with the non-annealed samples.

Max applied force before failure at 0.2625 mm and 0.175 mm layer heights.

Averaging the data paints a clearer picture. Interestingly, the sous vide samples when cooled quicker (ie. at room temperature instead of in the water bath) showed a slightly higher maximum force. Granted, three data points is hardly enough information to make any conclusions, but it provides some indication to the characteristic trend of these scenarios.

Averaged data with error bars showing standard deviations.

Measuring the Changes in Physical Dimensions

One of the hypothesized benefits of annealing with sous vide instead of in an oven is the uniform temperature control and reduced risk of warping. These parts were small and relatively thick, so warping wouldn’t normally be an issue anyway, but I measured the dimensional changes in the part from heat treatment anyway.

Quantifying the changes in each dimension after annealing.

We see that the Z dimension (ie. the sample height and the longest dimension) increased the most, whereas both expansion and contraction occurred with the X and Y dimensions. However, these changes result in less than 2% dimensional change (and in most cases it was less than 1.5%), which is fairly acceptable for 3D printed parts.

The Discussion

So after all this, what does it all mean? Like it or not, it means we don’t have a conclusive answer (but we can at least still talk about it).

Different Results for Different Layer Heights

This was probably the most unexpected observation. Annealing didn’t seem to have an effect on the samples printed at 0.2625 mm, but it did at 0.175 mm. What gives?

After doing a bit of Googling, other tests have shown that larger layer heights provide greater part strength⁹. 3D Matter wrote a great article on how layer height, infill percentage, and infill pattern affects the maximum strength of a printed part.

Detailed results showing the stress-strain curves for samples printed at varying layer heights. (Source: 3D Matter)

Their results: test samples printed at 0.3 mm had a maximum stress of about 36 MPa, whereas the samples at 0.1 mm topped out around 29 MPa. Backing up a bit, we can safely say that a part made of solid piece of plastic would be significantly more robust than two pieces of solid plastic bonded together. Since the material itself has imperfections, adding the bonding layer creates another source of imperfection and instability. Extrapolating this to a 3D printed part with hundreds of layers, we can guess that the increase in sources of imperfection will not bode well for the part’s own well being.

So why did only the smaller layer height benefit from annealing? My guess is that the annealing helps massage these imperfections out, so more imperfections means more room for improvement. It’s possible that with the larger layer height, my test sample was too small and short to really benefit from the annealing. That, in addition to the many sources of error in my testing (which I’ll get to later), may be the reason for the lack of improvement in mechanical performance with the 0.2625 mm parts.

Slow Cooling vs Fast Cooling

For annealing to be effective, the material typically needs to be cooled uniformly back to room temperature. If it’s cooled too rapidly, different areas of the part may cool at different rates and cause either warping or internal stress to form. To slowly cool the test samples, they were left in the water bath (with heat turned off) such that the rate of part cooling matched that of the water bath. Since water has a high heat capacity of 4.181 J/g/K (and for comparison, solid aluminum is only 0.897 J/g/K)¹⁰, it would reach room temperature at a significantly slower rate than if the parts were just left on the counter (a difference of about < 10 mins versus > 4 hours).

The results suggest that faster cooling yielded a greater improvement in strength over the slow cooling, which goes against the above logic. However, since the test parts were small in size and having low thermal and physical masses, the benefits of slow cooling may not be evident. Additionally, the individual 0.175 mm results for the “SV, Slow Cool” set (in turquoise) had a fairly large spread, so more data should be collected to determine if it’s actually statistically significant from the 0.2625 mm data.

The Sources of Error

During the testing process of breaking the samples, it occurred to me that I was most likely applying the force at different rates. As I got into the rhythm of testing and got a feel for when the samples failed, I likely changed the applied force profile. I would have more confidence in the results if I applied force at the same speed for all trials, but alas human error got the best of me. This was probably the greatest source of error in the testing.

The digital bathroom scale was also not ideal since I don’t know how quickly the measurement actually updates on the screen. Since the event of failure occurs so quickly, I may have easily missed the actual maximum force. This in tandem with the inconsistency in applied force means that I’m unfortunately unable to draw meaningful conclusions from the collected data.

Another source of error includes part to part variation on the same printer, which would be reduced by collecting more data points (ie. more than a measly 3 tests per set). The test piece also may not have been perfectly centered between the two wooden supports, which may also affect the load distribution and thus required breaking force.

Future Work

The silver lining to all of this are the learnings from mistakes. If I were to carry out the tests again, I would change the following:

Apply the point load at a consistent rate (maybe attach a stepper/DC motor to drive the force down into the sample)
Use an analog instead of digital scale, or
Set up a dedicated load sensor in addition to displacement measurement to quantify the stress-strain characteristics

Annealing, PLA, and You

So does this mean annealing is worth the effort? Is sous vide really necessary over a regular oven? Was this just a waste 20 minutes reading an article with inconclusive results?

Maybe, probably, and it depends.

As discussed earlier in this post, annealing plastic has tangible benefits in increasing its mechanical strength. However, plastic (and especially 3D printed plastic) is not the end-all material for home projects. It can only go so far, and sometimes 3D printing may not be the most suitable manufacturing method. Yes, people have printed gearboxes and mechanical vises which are undoubtedly impressive, but sometimes it’s cheaper in material cost and/or time to look at alternative manufacturing methods (or even off-the-shelf components).

Despite sous vide being home in the kitchen, I’m still adamant that it also has a place in heat treatment applications. The benefits of having an easily regulated, uniform temperature controlled environment is advantageous. However, limitations are evident in scaling as it may not be feasible to have a large tank of heated water in industrial settings when an oven may be cheaper and achieve similar results.

As for this excessively long article, the least you can do is learn from my mistakes and apply the knowledge to your own future projects. Hopefully you found it interesting and enjoyable to follow along this technical deep dive, and maybe even learned a thing or two in the process.

Until next time!

References

Identifying a commercially-available 3D printing process that minimizes model distortion after annealing and autoclaving and the effect of steam sterilization on mechanical strength, April 2020 ↩︎
The effect of annealing on deformation and mechanical strength of tough PLA and its application in 3D printed prosthetic sockets, August 2021 ↩︎
Annealing (metallurgy), Wikipedia. ↩︎
How to Anneal Your 3D Prints for Strength, 3DSourced. ↩︎
Annealing conditions for injection-molded poly(lactic acid), Plastics Research Online. ↩︎
Bake your PLA and have it outperform everything else!, Thomas Sanladerer. ↩︎
Annealing MakerGeeks Raptor PLA - The Boil Method, Joe Mike Terranella. ↩︎
PLA, RepRap Wiki. ↩︎
What is the influence of infill %, layer height, and infill pattern on my 3D prints?, 3D Matter. ↩︎
Heat capacity, Wikipedia. ↩︎

Battery Power Protection and Regulation PCB

Tue, 09 May 2017 11:30:56 -0700

Objective:

Design a custom PCB to protect and regulate a 14.8V LiPo battery for use with an autonomous RC car

Features:

12V undervoltage and 10A overcurrent protection
8.4V, 7.4V, and 5V regulated outputs

Main Components:

LT6109 High Side Current Sense Amplifier
RT8288AZSP Synchronous Step-Down Converter

Skills:

Reading and understanding datasheets
Schematic capture and PCB layout in Altium Designer
Board bring-up
Hardware debugging

Achievement Unlocked:

A mechanical engineer doing electrical engineering

Acknowledgements:

This project was completed under MistyWest with the guidance of Dave MacLeod, Div Gill, and Ryan Walker.

Completed PCB in all its glory, like a newly erected city skyline.

Schematic capture of the designed circuit.

PCB layout of top overlay (right) and bottom overlay (left).

3D render of PCB in Altium Designer.

Boards ordered and received from OSH Park.

Stencil cut out and prepped for applying the solder paste.

Solder paste applied and components placed.

PCB cooking in the reflow oven.

After much debugging, the board finally works! Red LED indicates protection circuit is active; Green LED indicates each regulated output is active. Multimeter shown is connected to the 8.4V output.

Overhead Robotic Gantry for Tethered VR Headsets

Tue, 04 Apr 2017 09:26:50 -0800

Project Overview

Objective: Create an autonomous gantry to follow the HTC Vive headset around, keeping its cable behind the user at all times.

Motivation: An extravagant party prop for an evening at CES 2017, hosted by MistyWest.

Features:

CoreXY planar gantry design
System built with 8020 aluminum extrusions and laser cut acrylic components
Stepper motor control through Teensy 3.2
HTC Vive pose tracking through C++
Patent pending

Skills:

Mechanical design with rapid prototyping methods
C++ software development
Arduino-based firmware development

Sources:

MistyWest Github
Medium blog post titled Tired of cables in VR? We are too.
CES Founders and Friends 2017 Photo Gallery by Natalia Leva

Acknowledgements:

This project was completed under MistyWest with the help of:

Derek Disanjh - Project Supervisor
Div Gill - Firmware Advisor
Ryan Walker - Software Advisor
Denis Godin - Filming
Madison Reid - Video Editing

Demo video of the gantry in action.

The gantry being used at a penthouse party hosted by MistyWest during CES 2017. (Photo by Natalia Leva)

The Development Story

We wanted a solution to increase and maintain presence in virtual reality; having to worry about tripping over the cable takes away from it. What we needed was a way to mimic a cable sherpa, following our every movement to prevent tension in the cable and entanglement around our legs. Whether we were moving forward, backward, left, right, or turning around, we wanted a way to roam freely as if virtual reality was already wireless.

Design and Testing

We wanted to make this as quickly as possible, so I designed the rig using off-the-shelf parts and laser cut acrylic parts.

The gantry frame designed in SolidWorks 2017.

Preliminary tests were less than stellar. I initially used relative position commands, but as the clip below shows, but it wasn’t responsive enough for practical use.

Initial testing of the tracking using relative position commands, almost as if it has a mind of its own.

Changing the motor control to use absolute coordinates instead of relative showed promising results.

Revised tracking using absolute position commands for significantly improved precision.

An overview of the software algorithm to parse the user’s current position and command the stepper motors is shown in the flowchart below.

Flowchart of the software algorithm.

User Testing

The results were so promising, in fact, that we decided to strap it to the ceiling and do some user testing. The results were… Loud due to the rattling and vibrations from the stepper motors. System responsiveness was slow, so more work needed to be done.

First user test with the gantry attached to the lab ceiling (rotation tracking not yet implemented).

Many revisions later, and the rig was finally working as expected!

After many hardware, software, and firmware tweaks, the gantry finally became usable.

Making it “Portable”

Since this was an internal project with MistyWest, the value was in showing this prototype around. The social media content was one thing, but physically bringing it around was thought to have additional “wow” factors. Since CES 2017 was coming up and MistyWest would be hosting a penthouse party during one of the evenings, it was time to make this thing free standing.

Continuing the theme with 8020 extrusions, the external frame was assembled and tested.

Construction of the free standing frame in preparation for bringing the rig to CES.

Testing the free standing setup to gain confidence in its sturdiness.

Yes, it is in fact portable!

Mission Complete

On January 6, 2017, the rig was set up in Las Vegas and (after some headache and remote debugging) the rig came alive at the Founders and Friends 2017. It was a huge hit amongst the tech enthusiasts and ran beautifully through the night.

People showing interest in the high-tech marriage between robotics and virtual reality. (Photo by Natalia Leva)

Founders and Friends 2017. (Photo by Natalia Leva)

The rig is now retired and sleeps soundly in its storage container. On to the next project!

Hakko-Style Solder Fume Extractor

Sat, 25 Mar 2017 21:14:29 -0700

I’ve been slowly getting into the land of electrical pixies. So now, not only will I be inhaling sawdust, I can also fill them with leaded solder (because we all know that the performance of leaded solder takes higher priority than the safety of lead-free solder)! However, because I still value living, I figured I should get something to help get the solder fumes away from my lungs.

At my current job with MistyWest, we have a nice, bench top solder fume extractor by Hakko. But I’m cheap, so heck no was I going to pay over $100 for blowing away fumes when I can blow them away myself! Despite it being a nifty design with multiple configurations (shown in the images below), I would resist the temptation to lighten my wallet on this. Oh wait, I just bought a thing that I can use to make other things…

Hakko solder fume extractor.

Look at all the lead I won’t be inhaling!

Taking the Hakko FA400-04 as a (heavily-inspired) design reference, I set off to make my own. Given my limited desk size and work area, I wanted to make it small and flexible. I had a 12V 40x40mm fan from eBay kicking around, which fortunately fit my needs quite well. See below for detailed images of the final product.

Image kind of showing it working. Configuration 1: Upright

Configuration 2: Low profile airflow

Configuration 3: Third hand

Carbon filters cut to size.

Pseudo-exploded view of assembly.

Uses M3 fasteners and a 40x40mm 12V fan.

It may not be the greatest fume extractor out there, but at least it works. Maybe I’ll make another version which has a proper power connector and its own supply. Maybe I’ll also add some LED strips to help illuminate my work area. Maybe I’ll even add wifi to it, because you know, the Internet of Things and everybody’s doing it.

Kidding.

Resources for 3D Printing with the MP Select Mini

Thu, 23 Mar 2017 19:38:27 -0700

What’s better than an inexpensive 3D printer? Free sources of information! The items below are a resource list for 3D printing with the Monoprice Select Mini, all thanks to a lively community of users around the interwebz.

General Info

Product Page:
- Monoprice Select Mini V1
Official User Manual:
- MP Select Mini User Manual
Reddit:
- /r/MPSelectMiniOwners
Facebook Group:
- MP Mini User Group
Unofficial Wiki:
- MP Select Mini Wiki
Community Knowledge Base:
- Google Doc

Materials and Accessories

Tested PLA Brands:
- Hatchbox
- AMZ3D
Alternative Bed Surface:
- PEI Sheet, see RepRap Wiki for details

Initial Set Up

CAD Modeling Software (free for hobbyists and enthusiasts):
- Autodesk Fusion 360
Slicing Software:
- Ultimaker Cura
Onboard WiFi for wireless printing:
- Connecting to WiFi through G-code
Using Raspberry Pi and Octoprint for wireless printing:
- OctoPi

Machine Settings

General

The machine settings below are based off the ones in the official user manual. However, the start and end gcode was taken from the community-driven Google Doc (see above) and slightly modified to include an initial nozzle wipe/primer and to remove the delay in turning the fans off.

Machine settings for MP Select Mini.

For the lazy, you can simply copy and paste the code boxes below (ignore the End Gcode in the image above).

;; Start Gcode
G21 ;metric values
G90 ;absolute positioning
M82 ;set extruder to absolute mode
M107 ;start with the fan off
G28 X0 Y0 ;move X/Y to min endstops
G28 Z0 ;move Z to min endstops
G1 Z15.0 F9000 ;move the platform down 15mm
G92 E0 ;zero the extruded length
G1 F200 E3 ;extrude 3mm of feed stock
G92 E0 ;zero the extruded length again
G1 F9000
;Put printing message on LCD screen
M117 Printing...

;; End Gcode
M104 S0 ; turn off hotend heater
M140 S0 ; turn off bed heater
G91 ; Switch to use Relative Coordinates
G1 E-2 F300 ; retract the filament a bit before lifting the nozzle to release some of the pressure
G1 Z1 ; raise Z 1mm from current position
G1 E-2 F300 ; retract filament even more
G90 ; Switch back to using Absolute Coordinates
G1 X20 ; move X axis close to tower but hopefully far enough to keep the fan from rattling
G1 Y115 ; move bed forward for easier part removal
M84 ; disable motors
G4 S600 ; keep fan running for 600 seconds to cool hotend and allow the fan to be turned off
M107 ; turn off fan

OctoPrint

To connect OctoPrint through Cura, go to “Settings > Printer > Manage Printers”, select your printer from the list, then click the “Connect OctoPrint” button on the right side of the menu.

Setting up OctoPrint through Cura.

This way, you can send gcode files directly from Cura to Octoprint with a click of a button!

Print Settings

When fiddling with layer heights, it’s recommended to use the optimal values below:

0.04375 (results may vary)*
0.0875
0.13125
0.175
0.21875
0.2625
0.30625

Numbers courtesy of Michael O’Brien on Hackaday. Explanation:

So that motor [Z-Axis] is a 7.5°, 48 step motor as I just listed. Since the motor is attached to a M4 rod, which has a 0.7 mm thread pitch, then in one revolution makes the Z-Axis travel up or down 0.7 mm. Since it took 48 steps to turn that rev, each step is 0.0014583… mm. To avoid rounding errors, you can use multiple of 3 of this number, which is a nice and pretty 0.04375 mm. That is a nice and handy number that effectively represents the layer heights that mathematically work the best for layer heights for this printer.

The settings below should serve as a decent starting point in dialing in your own print settings for PLA.

PLA material settings.

Design Guidelines

Want parts to fit together? 0.25mm is usually works well.

For mating parts, a general guideline of 0.25mm is sufficient.

Troubleshooting

Hole Features on First Layer Not Adhering

Make the following setting modifications, in descending order of preference (I try to avoid using rafts to reduce print time and material waste).

Slow down print speed to ~10mm/s
Increase build plate temperature and/or use external adhesion methods (ie. glue/hairspray)
Increase initial layer height such that more material sticks to the bed
Disable fans for initial layer
Use raft

Stringy Parts

Fiddle with retraction distance and speed (start with 4.5mm and 40mm/s)

Last edited: February 10, 2018

New Toy Tuesday: Monoprice Select Mini 3D Printer

Tue, 07 Mar 2017 15:56:53 -0700

I’ve been bit by the 3D printing bug.

Like any tech-enthusiast, I have toyed with the idea of owning a 3D printer for quite some time. For my final year mechatronics design project, my partner and I designed an extravagent contraption to measure the yolk done-ness of a soft-boiled egg, which we called The PerfEGGct. We borrowed my roommate’s 3D printer to create the enclosure, and it turned out to be an indispensable tool during our prototyping process. At the time, reliable 3D printers were upwards of $500 which was just out of my threshold for nice-to-have gadgets.

Your everyday solution for perfect soft-boiled eggs: The PerfEGGct

Fast forward to today, and we have the Monoprice Select Mini 3D printer. This bad boy is a mere $200 USD, putting it right into impulse buy territory. This price range of sub-$500 printers typically puts you with Chinese clones that are inconsistent, unreliable, and dangerous (no thermal regulation = setting fire to your house/rental suite). However, Monoprice is a reputable company in the electronics industry, putting ease to the reliability concern. More notably, just look at the thing: sheet metal enclosure and no assembly required. No muss, no fuss. At the time of writing, this is the best entry-level 3D printer due to its great price to performance ratio.

Let’s get printing! See below for a collection of printed models off Thingiverse.

Calibration cube to test dimensional accuracy. A little off of 20mm due to bulging at the corners.

Overhang test looking good.

Pebble charging stand.

Zoomed in on the base of the stand. Quality is quite impressive.

Hard drive cable organizer.

Raspberry Pi case.

Fancy desk organizer.

Fancy desk organizer: Nerdy style.

Extravagant business card dispenser

Stylus holder for Surface Pro 4 pen.

Overall, I’m quite impressed with this little guy. Setting up was a breeze, and it printed well right out of the box. Fun times ahead with this 3D printer.

ESP8266 Sous Vide Controller

Sun, 05 Feb 2017 12:28:27 -0800

Project Summary

Objective: Create a small, modular controller to regulate the temperature of a water bath.

Motivation: To get in on this cooking fad without dropping fat stacks of cash on an immersion circulator.

Features:

Crisp 0.96" OLED display
Pushbutton rotary encoder provides simple user interaction
Removable temperature plug through standard 3-pos audio connector
Temperature controlled outlet to be used with any heating element (ie. rice cooker, slow cooker, etc.)
Always on outlet for water circulator
Temperature logging over wifi

Source: Github

Modular sous vide controller powered by everyone’s favourite WiFi chip, the EPS8266.

Push button toggles between three states: main monitoring screen, set cooking time, and set cooking temperature.

The Long Version

The Sous Vide Story

You may have heard people rant about this fancy new cooking method that’s all the rage right now, but of course you shrug it off as a fad. But here you are, intrigued enough to be reading this article about the very thing you previously rolled your eyes at. So what the heck is sous vide anyway? The peeps over at ChefSteps have an answer:

Imagine you’re cooking a steak. You probably know exactly the color and texture—the doneness, in other words—you’d like, right? With sous vide (say “sue veed”), you simply set a pot of water to the corresponding time and temperature, and you can get that perfect doneness you desire, every time.

Simply put, you stick the steak in a ziploc bag, stick the bag in the temperature controlled water bath, and wait for the steak to reach thermal equilibrium. After an hour or so, the steak will be the same temperature of the water bath. Need to wait an extra hour before you cook? Not to worry, your steak will be safe since it’ll never exceed the temperature of the bath. With sous vide, say goodbye to overcooked food; physics just won’t let it happen.

Talk nerdy to me about sous vide. (Source: ChefSteps)

Once immersion circulators came to the consumer market, sous vide became accessible to home kitchens. (Source: Anova Culinary)

The Controller

Wanting to get in on this craze without spending a fortune (especially if it turned out to be a novelty), I set off to make my own. If all a sous vide device does is temperature control of a heating element, then it wouldn’t be anything my recent mechatronics degree couldn’t handle.

Equipment:

Rice cooker to act as the heating element and water vessel
80 GPH Acquarium pump to circulate the water
DS18B20 waterproof temperature sensor
ESP8266 WiFi Chip

DIY sous vide setup with home-made temperature controller.

Front face of the controller.

Backside of the controller. Note the switch and cord grip for that back-me-on-Kickstarter finish quality.

Little space was left unused to minimize the physical footprint of the circuit.

Reveal of perfboard craftsmanship using the gobs-of-solder trace method.

I designed the panel in SolidWorks, printed out the layout, taped it to a store bought enclosure panel and used a Dremel to carve out the holes.

Dry fitting after a bit of filing to clean up the edges and corners.

Programming the ESP8266.

The Data

No project is complete without analyzing data! The main unknowns thus far:

Rise time and stability comparison between various cooking vessels
Temperature stability improvements with a water circulator
Is a bang-bang controller sufficient or is PID required?

To assess these questions, I setup the ESP8266 to push live temperature data to an online web server. Through MQTT, I was able to send live temperature data from the controller to Adafruit.IO. This service (in open beta at the time of writing) allows dashboards and data feeds to be easily created for real-time monitoring of anything.

The ESP8266 logs temperature data through Adafruit servers. Data is displayed through their live dashboard feed.

A Comparison of Cookers

Four trials were conducted with three different cooking vessels: a 3 cup rice cooker, 7 cup rice cooker, and a 14 cup (3.3 L) slow cooker. Two cups of water were used in each vessel for these tests. As shown through the graph below, the rice cookers get up to 55°C significantly than the (very) slow cooker. Although it would be better in maintaining temperature due to its ceramic makeup, it took way too long to heat up for practical purposes. You can fill it with hot/boiling water to help it warm up, but it still wouldn’t be as fast or convenient than the rice cookers.

Slow cooker, you are the weakest link. Goodbye.

Comparison of rice and slow cookers with circulated and uncirculated water baths.

Looking at the rice cookers in more detail, we see that the small rice cooker has a much lower overshoot than the large. This makes sense since the heating element is smaller, thus it retains less heat “momentum” upon shutoff. The addition of a water circulator significantly reduced the steady state temperature oscillation as well as the amplitude of overshoot. With the added benefit of greater temperature uniformity within the water bath than relying on natural convection to mix the water, using a circulator is proven to be a necessity.

Moving forward, I decided to use the large rice cooker with a circulator. Although the small rice cooker reaches steady state more quickly, its small vessel size s unfortunately not practical for cooking anything but a feast for ants.

Comparison of the remaining cooking vessels.

Small aquarium pump used to circulate the water.

Time to actually cook with this thing! After trying out various foods, the data shows that the temperature stability is surprisingly stable given the simple on-off temperature control.

Log of cook times.

Quantifying the ripple with some simple analysis shows that the average standard deviation for these four cooking trials was ±0.4°C, where the steak had the highest deviation of ±0.6°C.

 Trout Fillet at 48C for 59 mins
 Mean: 48.35°C (+1.15, -0.73)
 Std: ±0.34°C

 Sirloin Steak at 53C for 84 mins
 Mean: 53.03°C (+1.78, -2.65)
 Std: ±0.63°C

 Chicken Thighs at 70C for 99 mins
 Mean: 70.03°C (+0.59, -1.53)
 Std: ±0.30°C

 Burger Patty at 60C for 31 mins
 Mean: 60.16°C (+0.72, -0.66)
 Std: ±0.23°C

 --> AVERAGE STANDARD DEV: ±0.38°C

Thus, using a simple temperature control implementation with a water circulator yields quite satisfying results.

The Verdict

They say a picture is worth a thousand words. The one below might not be worth quite that many, but that pink uniformity definitely speaks for itself.

A uniformly cooked sirloin steak finished on a cast-iron pan.

Tired of Cables in VR? We Are Too.

Tue, 15 Nov 2016 23:02:43 -0700

Originally posted on Medium under MistyWest.

Virtual reality pushes the envelope of bleeding edge technology, allowing us to explore and experience worlds beyond our mortal imaginations. It gives us immersion in another dimension, providing an unprecedented medium for communication and story telling. If a picture is worth a thousand words, then virtual reality must be worth millions. Except there’s one thing keeping it grounded to reality: cables.

Since the first virtual reality headset that was hacked together in 2011, immersive head-mounted displays have progressed far and quickly. Oculus Rift, HTC Vive, and PSVR are some of the main contenders of virtual reality hardware, and between them lie stark differences in performance and usability. However, cables are the common denominator in these high performance virtual reality headsets. They are tethered to a host computer to provide the computationally intensive processing power required for high-fidelity content. Although mobile solutions are available such as Microsoft’s HoloLens, Samsung VR, Google Daydream, and even Google Cardboard, wired headsets will continue to be the vanguard of high performance virtual reality.

Or so we thought.

But before we begin, let’s backtrack to a mere two months ago. We found ourselves growing tired of worrying about tripping over cables while using the HTC Vive in room-scale experiences. This was (and still is) a common problem: our eyes tell us we have freedom of movement, but our hardware reminds us we don’t. Wouldn’t it be great if there was a way to break free from cables and improve the immersivity of virtual reality?

Humans are hackers at heart and always strive to make things better. As a team of curious and hungry engineers, our answer was yes.

We wanted a solution to increase and maintain presence in virtual reality; having to worry about tripping over the cable takes away from it. Some would argue that subconsciously stepping over the cable is a minor problem that one gets used to. However, despite its small inconvenience, it’s an inconvenience nonetheless which detracts from the otherwise immersive VR experience. Solutions do exist: at event demos, people hire cable sherpas to hold the wires behind the user and follow them around. Unfortunately, not everyone has access to their own personal sherpa.

We have the ability to be transported to completely different worlds, yet we are tethered by cables that have existed longer than we can remember. We shouldn’t stop innovation for VR at just the headset; the entire experience needs to be unique and immersive.

If this is such a common problem, what have people done to manage these notorious cables?

A handful of HTC Vive owners have mounted retractable identification badge fobs or dog leashes to the ceiling to create an overhead holding point of the cable. However, certain player movements will result in tension between the mounting point and the headset. We tried this ourselves, and we found the cable tugging more distracting than having to step over the cable.

Free standing cable boom. [SteelSeries Tech Blog]

Ceiling mounted dog leashes. [YouTube]

What we needed was a way to mimic a cable sherpa, following our every movement to prevent tension in the cable and entanglement around our legs. Whether we were moving forward, backward, left, right, or turning around, we wanted a way to roam freely as if virtual reality was already wireless.

Cue the engineers.

Combining our expertise in rapid prototyping, software development, and the strong desire to make things better, we had ourselves a solution: the autonomous robotic gantry.

Achievement unlocked: Freedom of movement with wired VR.

The autonomous robotic gantry is an overhead cable management system which allows the user to roam freely within the play area. Using aluminum extrusions and laser cut acrylic parts as the base materials, we designed and constructed a lightweight, planar motion system driven by two stepper motors through a a CoreXY timing belt configuration. Minimizing the number of moving components and weight was the key in optimizing acceleration and speed. By using off the shelf components and laser cut parts, we were able to design, build, and assemble the gantry in a short two weeks. Thanks to OpenVR, we were able to use the positional data to control and automate the overhead gantry. No muss, no fuss, no more tripping over cables.

Cable-free gameplay of Space Pirate Trainer.

Over-engineered? Maybe. Overkill? We think not. This was an exercise not in cost-effectiveness for the average user, but to explore how we can integrate existing technology with bleeding edge products in order to improve its efficacy and functionality. As the world has become a community of hackers, we don’t have to rest on our laurels and wait for companies to come up with solutions to our problems. The resources are available to us to come up with these solutions ourselves.

Full demo video of the robotic gantry.

Fast forward to present day. Just last week, HTC announced a kit for wireless virtual reality. That’s right: the devs behind the HTC Vive have already been working on a consumer ready solution to cables. We didn’t anticipated this wireless movement for another 5–10 years since latency and frame rate is still an issue with current wired headsets. However, despite the rapid research and development to unlock the potential of true freedom of movement, wireless performance will always be succeeded by wired headsets.

Is our robotic gantry dead on arrival? It was a fun, quick project to work on and we believe it still holds value. Potentially not as a product, but as an exercise to explore how we can use existing technology to improve upon new ones. In any event, we are excited to see the progression of wireless headsets and will continue to watch the rapid changes in industry that are happening before our very eyes.

Virtual reality is still a nascent industry with large room for growth, and solutions for wired, wireless, and mobile media will continue to fill the stage. As new products emerge, older ones may fall. Regardless of what happens in the realm of virtual reality, we don’t want to just sit back and watch it happen. As mentioned before, we are all hackers at heart and continually strive to make things better. Whether it be virtual reality in wired, wireless, or mobile form, we are all capable of taking part in the forward progression of technology and will constantly seek opportunities to improve its performance and usability.

Wooden Phone Stand

Sun, 13 Nov 2016 21:35:28 -0700

Phone stands are something that seem a little unnecessary, but once you have one then you’ll wonder how your desk lived without one. I’m a firm believer in having a dedicated place for everything as a solution to keep things tidy, so making a phone stand for my desk at work just made sense.

I started off on Google to look for existing designs I could imitate, but most fell short of not fitting my aesthetic needs. I wanted a balance between simplicity and form, without requiring too much effort to make. I eventually came across one design by Woodenlife on Artfire and deemed it the winner for my copy-cat woodworking. I sketched out the side profile in SolidWorks, pasted it on to the wood block, and proceeded to cut it out on my bandsaw.

Searching Google images for inspiration.

Design from Woodenlife on Artfire.

Cutting template created in SolidWorks.

Template pasted on to the wood stock.

Smoothing the curves with a mini plane.

Cleaning the cutout with a Dremel router attachment.

Stained, finished, and ready for display.

For such a simple hunk of wood, I’m quite pleased with how it turned out.

Making Do With a Bargain Bin Bandsaw

Sat, 22 Oct 2016 20:58:10 -0700

I’ve always wanted a bandsaw. Being able to cut wood, plastics, etc. quickly and with flexibility has been a dream in my home shop. When I found an ad on Craigslist for a portable bandsaw for $50, I just couldn’t resist. The main caveat was that it was missing a table, but that wasn’t a big enough issue to turn me away from this killer deal.

Fresh off Craigslist in its table-naked glory.

Due to the lack of proper tools in my garage suitable to make a decent table, I needed to improvise with what I had. Enter the inverted jigsaw. Sure, I could have just used the jigsaw by itself, but it’s always easier to move the material instead of moving the tool. More importantly, this allowed me to easily cut smaller parts out.

I needed something to cut parts for the table support. Thus, the inverted jigsaw was birthed.

Cutting the blade insert was much easier with the inverted jigsaw.

Cutting the table center out on the bandsaw stand-in.

Blade insert cut out…

… and seated nice and snug.

The first set of brackets I cut out weren’t great, but they did the job. However, it was good enough to get the bandsaw rolling and cut a second, revised set of brackets for better stability.

Attempt #1: First version of the table supports. It wasn’t extremely stable, but it was good enough to cut sturdier parts as replacements.

The second set was much beefier and even included angle adjustment. Unfortunately it doesn’t work well since bandsaw tables typically use trunion brackets to keep the pivot point at the center of the blade. I didn’t envision myself using this bandsaw to cut angles (since the thing can’t even cut a damn straight 2x4…, which I’ll get to later), so I didn’t bother with it this time around.

Cutting out the slotted curve.

Attempt #2: Completed table support with angle adjustment from the slotted hole. Also note the modified knob handle above to provide easier adjustment.

Bandsaw fully functional and ready to rip.

Below are a few woodworking projects I used the bandsaw for. Turns out you get what you pay for, with this bandsaw being no exception. It retails for $249 at Canadian Tire, and many reviewers complained that cutting straight is virtually impossible without the blade bowing, likely due to the inadequately designed blade tensioner. It performs okay for cutting thin stock like plywood, but I wouldn’t rely on this for any mating joints.

First bandsaw project was making a keyholder and to increase stoke for the winter ski season.

Also made a butcher’s block out of an old fireplace mantle.

Stay tuned for more woodworking projects!

Engineer's Diary

Sat, 01 Oct 2016 11:34:47 -0700

Background: Between paper notebooks, post-it notes, OneNote, Evernote, and so many more, there is no shortage of ways to write things down. Each has its strengths and weaknesses, but none satisfied my requirements to act as a daily work log to record key events, thoughts, and milestones during my work day.

My paper notebook is excellent for free-form thoughts, sketches, and calculations, but I would want to keep a separate notebook to keep track of these sequential events. We use OneNote at work, but where the infinite blank canvas is a strength in applications such as for research or brainstorming, I found it to be a weakness in record keeping since the document is too easy to edit and “fragile”.

One day after learning about Git and SourceTree, I knew wanted to create a similar commit-style application to record activites and “freeze” them in time. Not web-based, internet connected, or cross platform; just a no frills desktop application that does one thing and one thing only.

Objective: Develop a simple application to log daily activities at work.

Features:

Entries for each day are saved in a date-stamped text file
Each commit is time-stamped in the text file
Archive of log entries is accessible through the list in the left column
Text files show up as read only in the application, but are editable and searchable through the Windows File Explorer

Framework: C#

Source: Github

Screencap of the Engineer’s Diary. Write in the ‘Description’ and ‘Project’ text boxes, then press ‘Commit’ when complete.

After the ‘Commit’ button is pressed, the entry is written or appended to a text file and displayed on screen.

MECH 458: Capstone 3D Optical Well Imager

Thu, 02 Jun 2016 12:57:32 -0700

Introduction

Objective: Develop an optical scanning system to be used for generating 3D models of well liners and casings with sub-millimeter accuracy.

How: As a team of five mechatronic and mechanical engineering students, we designed and constructed an alpha prototype of the optical 3D well imager.

Project Details

Requirements and Specifications

Optical

Device must have a tolerance on distance measurement of 1 mm
Device must have a camera circumferential resolution of 1.42°
Device must have a 3D scan circumferential resolution of 1.42°
Device must have an axial resolution of 1 mm

Mechanical

Device must be submersible
Device must fit in a 3.5” ID pipe
Device must have a minimum operating velocity of 10 mm/s
Device must be operable using the existing test apparatus

Design Overview

The final design of the prototype includes a camera module, laser module, and two conical mirrors. The basic functionality of the device is shown in the figure below. A red laser ring is generated by the laser module which is reflected by 90° on to the surface of the well casing. The camera’s vision is then reflected by 150° to capture the projected laser ring.

Simplified diagram of scanning functionality.

Combine Modules

Overview of the optical imaging device.

Camera Module

Diagram of the camera module.

Laser Module

Diagram of the laser module.

System Evaluation

Our prototype was successful in recreating a 3D model of the pipe internals during unsubmerged operation, as shown through the figures below. However, we were unable to test the device in submerged conditions due to time constraints of the project.

Preliminary Testing and Calibration

Setup of the calibration grid with a green laser ring (left); Captured image (right)

Device Integration

Our device installed with the DarkVision lab equipment (left); Close up of the laser ring inside the well liner (right).

Post-Processing and Analysis

Four different exposure times and traversing speeds were tested to determine which camera parameters would generate adequate images for post-processing and analysis. From our tests, we determined that the imaging parameters resulting in an effective frame rate of 0.13 fps at a forward jog speed of 7 mm/s showed the most promising image slices of the well liner. To increase the slice resolution of the scan, the frame rate was increased to 1.3 fps which yielded visually poorer images, but our processing algorithms were still able to generate a 3D model of the well liner. A hue saturation value filter was used to detect the outer edge of the raw image.

Captured laser ring by the camera through the conical mirror (left); Processed image (right)

Generated 3D model from the image slices.

New Workshop + Plywood Bench Stool

Wed, 01 Jun 2016 15:39:56 -0700

My family moved into a new house, meaning that we finally have an actual garage instead of a measly carport. But what this really means is that I get a chance to claim space to set up an actual workshop! First project was naturally a workbench, which I put together using 2x4’s and a 3/4" MDF board for the work surface (see above). Second project was a bench stool, as shown in the following images below. The builders left some 1/8" plywood in the yard, and I wanted to see how sturdy of a chair I can make with it given the inherent flimsiness of the thin sheet stock. By adding a reinforcing cross-brace at the base of the legs, I was able to significantly increase its sturdiness.

I used a jigsaw to cut out each side, and fastened together with glue and nails (since screws would likely split the wood).

Final panel being attached.

Blocks of wood being clamped, which will be used to attach the seat. note the cross-brace near the feet of the stool.

Gluing and clamping three sheets of the same plywood to get a more comfortable contour for sitting. This is a similar method to how skateboards/longboards are given their complex curvatures.

Stool sanded and clear coat applied.

For a stool made with the cheapest possible wood, it actually turned out half-decent and it does what it needs to do!

Adjustable Portable Power Supply

Wed, 25 May 2016 17:29:58 -0700

Objective: Build a cheap, portable, variable DC power supply.

Motivation: It was finally time to get my hands on a variable power supply for my electronics projects. Previous projects mainly involved Arduino, which was able to supply 5V and 3.3V with ease. However, the need for a supply with higher voltage, current, and flexibility eventually arose, resulting in the birth of this ghetto (but functional) power supply.

Limitations:

Only DC voltages available
Max current is a function of input power and desired voltage (I=P/V)
Current limiting feature is non-existent, so must be careful to not let the genie out of circuits

Internals of the power supply.

MECH 423: Mechatronics Product Design

Fri, 22 Apr 2016 22:38:31 -0700

Background: When a hard boiled egg is spun on a table, it rotates freely since the inside is completely solid. With a raw egg, the liquid yolk sloshes around and resists rotation. By using math and physics, we can analyze the rotational oscillation of an egg and determine the yolk consistency.

“If it’s worth doing, it’s worth overdoing.” - Jaime Hyneman

Objective: Design and build a device to determine how cooked a boiled egg is using non-invasive techniques.

Motivation: As part of our final year engineering course, we were tasked to create a mechatronics device of our choosing. This project was heavily inspired by Matthias Wandel and his boiled egg hardness tester.

Functional Requirements:

Measure egg mass from strain gauge
Measure oscillation from accelerometer
Control LCD screen
Mechanical design and fabrication
Integration and user interaction study
C# interface
MATLAB data analysis

Limitations: This prototype was developed in 3 weeks during school and has approximately 80% repeatability.

Partner: Justin Liang

Sixty second demo video of the device in action.

CAD model of electronics inside the enclosure (transparency shown for clarity)

PCB diagram.

PCB assembled.

Dry fitting of components inside handle enclosure.

Annotated diagram of the final device.

Example data from five egg-jiggling trials.

The PerfEggct stored in its natural habitat.

Kitchen gadgets aren’t complete without a fancy photoshoot!

Truly perf-egg-ct.

Interested in reading more about this project? Head over to Instructables for more details! And if you’re really interested, feel free to read our glorious 35-page final report here.

Boo-Boo with the Bamboo Bike

Fri, 18 Dec 2015 20:50:37 -0700

As you probably already know, I made a bamboo bike which I’m very proud of. However, what you probably don’t know is that I made some mistakes in the process. The photo above shows the result of one of these progress speedbumps, and yes, I did indeed cry the day I had to cut those tubes out.

I started this project in November 2013 but failed to finish due splitting bamboo. Progress was moving forward: I had the bamboo, carbon fibre, and epoxy ready for frame building, and the poles were in the process of being cut and shaped to size. However, winter was quickly approaching and the weather was becoming less and less ideal for carbon fibre layup. The epoxy I was using had a working temperature above 10°C, but the outside temperature was beginning to drop below that. I was able to keep it in its liquid state by using my mother’s hair dryer (don’t tell her though since she still doesn’t know…), but the epoxy wasn’t the main issue. Apparently bamboo is a living thing and doesn’t like drastic changes in temperature. Turns out doing the layup in colder temperatures, then bringing the frame inside to dry in warm, livable temperatures wasn’t good for the bamboo. Moisture is the devil here, and causes the wood to expand and contract based on the current environmental conditions.

A week of thermal cycling like so, and I had myself some cracked bamboo. I was heart-broken. It was the end of December at this point, so I decided not to continue with this project until the weather warmed up.

Once spring came around, I set off to Chilliwack to get myself much higher quality materials from Bamboo World. I was significantly more meticulous in heat treatment this time around, conducting an oven bake as well as direct heat treatment to remove as much moisture as possible and to cure the bamboo.

Bamboo poles after heat treatment in the oven, preparing for the next heat treatment.

I weighed my options in what to do about the frame, whether to start from scratch or try to salvage it. Being both optimistic and lazy, I chose the latter. I figured I should be able to cut out the compromised tubes and simply replace it with new ones. The joints would be a bit bulkier from the leftover carbon fibre, but that would merely be an aesthetic issue. To replace the entire front triangle, I had to be strategic in the order of extraction and replacement to preserve the steerer tube geometry. Luckily I didn’t mess that up, so it became a matter of starting off where I left off. Seven steps back, 3 steps forward!

Eventually I managed to finish the bike, and everything else worked out in the end…

… Except not. Turns out bamboo really doesn’t like moisture. Two winters later and this happened:

Another split down the seatpost :(

Despite all my efforts in heat treatment, wood sealing, and surface finishing, the bamboo still managed to crack. Initially I was only going to ride this bike in dry weather, but come on, it’s Vancouver aka the “Wet Coast”. From the photo above, all the road grime clearly didn’t do the frame any favours. The dirt/sand slowly withered the clear coat away (5 layers of it, by the way) and moisture slowly made its way into the bamboo.

But to heck with it, I’ll keep riding it to the ground and see how long it lasts.

TL;DR Wood doesn’t like moisture, and if you don’t listen then you’re going to have a bad time.

Update: As of May 2017 and no more riding in the rain, the bamboo managed to dry and the split sealed up. The ride is much less spongy/flexy than before, so I think it will now officially become a fair-weather bike.

The Making of Project Haikuza: Part 2

Thu, 06 Aug 2015 23:49:59 -0700

The format of this series is an outline of my thought process during the development of @thehaikuza.

Poetry is hard
To write when algorithms
Are extremely dumb.

I dont want my haiku generator to be a vegetarian chef. There’s nothing wrong with always making word salad, but eventually it’ll have to learn to make fancier things. A poetic risotto would be nice from time to time.

Leaving @thehaikuza to make complete gibberish wasnt what I had intended. I envisioned my algorithm to be able to reconstruct bad haikus, but definitely not as crappy as the ones it actually made. My idea of bad had more to do with this xkcd comic:

iOS Keyboard Predictions (Source: xkcd.com/1427)

In any event, I needed to try something a little more sophisticated than shoving words into slots where they didn’t really fit.

The Lesser Known Cousin of 2 Chainz

Every time I Googled a new topic I didnt know about, five more topics were thrown on my plate. Notable keywords that popped up:

Computational linguistics
Neural networks
Sentiment classification
Bayesian inference
Tree kernels for semantic role labeling
Markov chains

By no means am I suggesting that I understood all of those search terms after reading about them. In fact, I still don’t and am constantly trying to wrap my head around those polysyllabic words. (Fun fact: anything above three syllables starts to scare most people.) The takeaway is the last item on the list, which coincidentally is the one I did manage to (somewhat) understand: the Markov chain.

A Primer for Markov Chains

In technical terms, a Markov chain is any random process that undergoes transitions from one state to another. It’s also a memoryless process, meaning that it only cares about its current state and not the states it has previously occupied.

In less technical terms, a Markov chain is like your tokens position during a game of Monopoly. The next property that your token lands on is random (ie. its a random state transition), and every dice roll is independent of the previous roll (ie. its a memoryless process). Probability determines what the next dice roll is: theres a higher chance of rolling a 7 since there are 6 possible combinations (6/36 = 16.7% probability), whereas theres only one way to roll a 2 or 12 (1/36 = 2.8% probability).

The marvelous aspect of Markov chains is that they can work with any item, not just numbers! Lets say you wanted to form sentences using Markov chains. Given a corpus of phrases (like the entire works of Shakespeare), a flowchart like the one below can be formed and used to create new phrases. Interestingly enough, this is the basis of prediction engines used on smartphone keyboards! It will learn from your phrasing habits and try to guess what your next word will be, based on how frequently youve typed similar words or phrases (just like the above xkcd comic).

Visualization of words forming a Markov chain. (Source: Andrew Cholakian’s Blog)

Harnessing its Raw, Indisputable Power

Using Markov chains would allow me to use song lyrics as a training ground for creating alternate phrases. This would consequently form a probability-based flowchart like the one above, allowing me to generate new sentences by walking through each word state and letting probability determine my next word.

To understand how to turn this into a programmable scenario, let’s take a look at an example. Given a phrase like:

Mo butter, mo better, mo slipperier

A Markov chain algorithm will take triplets of the phrase, use the first two words as the dictionary word, and the third word as the definition. If you’re familiar with using dictionaries in Python, the former is known as keys and the latter as values. This would result in the above phrase being translated to:

('mo', 'butter') : ['mo']
('butter', 'mo') : ['better']
('mo', 'better') : ['mo']
('better', 'mo') : ['slipperier']

Okay, but this isn’t very captivating because there’s a 1:1 ratio of keys to values, which means that there will never be any mixing and matching of words since the probability of the next word is always 1. Alternatively, if we have a sentence such as:

Living in a land of butter is like living in a paradise with flying unicorns

Then the resulting dictionary will look like:

('Living', 'in') : ['a']
('in', 'a') : ['land', 'paradise']
('a', 'land') : ['of']
('land', 'of') : ['butter']
('of', 'butter') : ['is']
('butter', 'is') : ['like']
('is', 'like') : ['living']
('like', 'living') : ['in']
('living', 'in') : ['a']
('a', 'paradise') : ['with']
('paradise', 'with') : ['flying']
('with', 'flying') : ['unicorns']

The interesting part of this phrase (other than the wildly imaginative scenario) is the second line item, where the key (in, a) has two possible values, either land or paradise. So if I’m wandering around a word-based flowchart and come across the pair of words in a, my next word can either be land or paradise. Now, this was just for a 15 word phrase, so imagine the magnitude of choices if lyrics for an entire song was used (where more repetitions are prevalent), or all lyrics from an artists full discography, or even a body of text with a +1,000,000 word count meant for this purpose! More phrases and more words result in more possible combinations, which is excellent news for attempting to create pseudo-random sentences.

The code below for the Markov chain algorithm was adapted from Agiliqs Blog:

import random

class Markov(object):

 def __init__(self, string):
 self.cache = {}
 self.words = string
 self.word_size = len(self.words)
 self.database()

 print self.cache

 def triples(self):
 """ Generates triples from the given data string.
 """
 if len(self.words) == 3:
 return

 for i in range(len(self.words) - 2):
 yield (self.words[i], self.words[i+1], self.words[i+2])

 def database(self):
 for w1, w2, w3 in self.triples():
 key = (w1, w2)

 if key in self.cache:
 self.cache[key].append(w3)
 else:
 self.cache[key] = [w3]

 def generate_markov_text(self, size=25):
 seed = random.randint(0, self.word_size-3)
 seed_word, next_word = self.words[seed], self.words[seed+1]
 w1, w2 = seed_word, next_word
 gen_words = []
 for i in xrange(size):
 gen_words.append(w1)
 w1, w2 = w2, random.choice(self.cache[(w1, w2)])
 gen_words.append(w2)
 return ' '.join(gen_words)

Teaching Markov the Art of Haikus

Time to make some progress in making some beautifully robotic poetry! The plan for @thehaikuza V0.2 is to create a Markov chain dictionary using song lyrics and create randomized phrases from the resulting keys and values. Although this implementation still doesn’t involve a proper grammar model, it’s an improvement from the previous method because of its more structured approach. By simply mixing and matching phrases that once made sense before, there’s a much higher probability that the resulting phrase will also make some level of sense.

The success rate should now go from laughable to respectfully laughable! Progress is progress.

Check out the full repo on Github!

The Making of Project Haikuza: Part 1

Sun, 12 Jul 2015 23:49:52 -0700

The format of this series is an outline of my thought process during the development of @thehaikuza.

Haikus are simple
Even children can write them
maybe programs too?

Nothing is cooler than algorithmic poetry. Except for maybe Carl Sagan. I heard he was a pretty cool guy.

I was listening to the radio while driving home one Sunday evening, and an ad came up for a university that was submitting computer-generated poetry to a literature competition. I’ve recently been fascinated by the many intricacies of the English language, so the thought of somehow teaching a computer how to construct proper phrases seemed like an elusive task. But hey, if Google is also working on natural language processing, then how hard can it be?

The idea of computer generated text is often a humourous one. Spamming your smartphone’s predictive keyboard is usually enough justification to avoid trying to computationally bang out the works of Shakespeare. As laughable as the monkeys at a typewriter idea is, maybe weve become smart enough to at least get close to this literary pipe dream…

Prequel: The Prudence of Pivoting

Needless to say, it can take a few tries to end up at a decent idea, and @thehaikuza was no exception. After hearing about the poetry contest, I wanted to develop something related to computational linguistics, but I was partial to run-of-the-mill poetry. High school English classes had given me a negative bias for this type of literary art, so naturally I was in search for a different end product.

Rap songs were the first to come to mind. After all, songs are simply poems accompanied by musical harmonies and melodies, so I wasn’t too far off from the realm of poetic computations. However, I eventually realized that writing rap songs from scratch (not to mention by an algorithm) would have mountains of difficulty, especially since my knowledge of natural language processing was currently at ground zero. I decided to scale back and start off small with the most simple form of poetry known to (probably) every North American student: the haiku.

Haikus are great because they are short in length and only have one simple rule: don’t break the 5/7/5 syllable count per line. One of my coworkers suggested using Twitter as my platform for haiku generation, which would prove to be much easier than opening another can of worms of trying to develop my own web app. Since there’s already a Twitter API library for Python, and with each average haiku being under the threshold of 140 characters, my decision was made before I even had to think about it.

With the advent of using Twitter, I had a few more options I could play with. My first idea was to make haikus based on the trending topics on Twitter. This would give my Twitter-bot the dynamic quality that’s inherent in the fast-paced nature of the internet, rather than just creating stale haikus about nature. It could retrieve the trending topics, gather all the relevant tweets, and then use the collected phrases to formulate a relevant haiku. In theory, this seemed like a great idea; in practice, not so much! After some prototyping, I quickly learned that the broken English, frequent spelling mistakes, and chaotic nature of < 140 character messages wouldnt serve as an adequate corpus for phrase generation.

Okay, well forget about that idea. But I still wanted a venue to stay relevant on the internet. I figured songs can also be relevant, especially if I would be grabbing ones that are currently playing on the radio. Adding an interactive element to @thehaikuza by making it able to respond to song requests would just be icing on the cake. Thus, @thehaikuza was born!

Or so I thought. Turned out it was still an embryo.

Makin Bacon Haikus V0.1

The first research topic on my ever-growing list was to figure out what makes a sensible sentence. In elementary school, we’re taught that the most basic phrase consists of a subject and verb. In the sentence below, for example, the boy is the subject and is sweating is the action.

The boy is sweating.

Okay, this evidently isnt new knowledge to anyone. But what if we break down the phrase into types of words? Referencing a universal part of speech (POS) tagging system, we get:

[The, determinant] [boy, noun] [is, verb] [sweating, verb]

Interesting. Maybe I can create a Mad Lib-based language model that can learn the sentence structures of existing haikus, then use an external wordset to substitute the corresponding words. For poetry, and notably haikus, grammar isnt as important as a passive aggressive letter to your not-so-favourite mobile carrier, so I might be able to get away with this rudimentary language model. (Spoiler alert: I couldnt.)

Haiku Training

The plan was to create a pseudo-smart script (of sorts) that did the following:

Import a collection of human-written haikus
Extract the POS tags for each line
Construct a database of 5 and 7 syllable POS phrases

The implementation looked something like this:

import nltk

class LearnHaiku(object):
 def __init__(self, fname):
 self.haikus = self.parse(fname)
 self.pos_five, self.pos_seven, self.log = self.analyze()

 @staticmethod
 def parse(fname):
 haikus = []
 with open(fname) as inputfile:
 for line in inputfile:
 haikus.append(line)

 return haikus

 def analyze(self):
 line5s = []
 line7s = []
 errors = []

 is_line7s = 1
 numrows = len(self.haikus)

 for i in range(numrows):
 """Only parse if not a blank line"""
 if self.haikus[i] != '\n':
 try:
 tags = nltk.pos_tag(self.haikus[i].split(' '), tagset='universal')
 if i == is_line7s:
 line7s.append([x[1] for x in tags])
 is_line7s += 4
 else:
 line5s.append([x[1] for x in tags])
 except UnicodeDecodeError:
 errors.append(i+1)

 log_str = "Couldn't parse line(s) %s." %(', '.join(str(x) for x in errors))

 return line5s, line7s, log_str

if __name__ == '__main__':
 hk = LearnHaiku("[REF] Haiku Training Examples V0.1.txt")
 print hk.pos_five
 print hk.pos_seven

From this, I could then randomly generate haiku POS templates that I would be able to use with an external set of words. From the (rather awfully juvenile) example haikus below:

An old silent pond
A frog jumps into the pond,
splash! Silence again.

The autumn moonlight
a large worm digs silently
into the chestnut.

Lightning flashes bright
what I thought were white faces
are plumes of plain grass.

The POS tag retrieval for the phrases were:

Five Syllable Lines:
 [u'DET', u'ADJ', u'NOUN', u'NOUN'],
 [u'NOUN', u'NOUN', u'NOUN'],
 [u'DET', u'NOUN', u'NOUN'],
 [u'NOUN', u'VERB', u'NOUN'],
 [u'VERB', u'ADV', u'ADP', u'NOUN', u'NOUN']

Seven Syllable Lines:
 [u'DET', u'NOUN', u'VERB', u'ADP', u'DET', u'ADJ'],
 [u'DET', u'ADJ', u'NOUN', u'VERB', u'NOUN', u'NOUN'],
 [u'PRON', u'PRON', u'VERB', u'NOUN', u'NOUN']

Almost there to our first haiku!

First Pass Haiku Generation

I wrote another simple script to take in song lyrics, apply a POS tag for each of the lyrics, and create a word list sorted by the corresponding tags. I could then plug-and-chug words from the generated list into an appropriate slot in the haiku template.

For the first test, I thought it was only appropriate to use lyrics from Drake’s classic Started From The Bottom. The output:

Story screaming real
Out uncle own stay worry
Through bottom lips heard

When that standing as
Poppin coming we begging
From started keep if

Fucking ball standing
Niggas explaining guys crowd
Say girl chain rumors

Oh dear lord, what a mellifluous balance of repulsive vocabulary and lackadaisical grammar. Those were definitely some of the worst haikus I’ve read… Maybe if I try using Blank Space by Taylor Swift:

Half bottom boy flames
Feel what bottom you darling
crying was from is crying

Explaining queen lights
Stays leave could cherry stay
Town cause lets dress the

Crystal blank starts: flames
thats explaining funny hand
Cherry at god im

I lied. Pretty sure those were worse.

Obviously, the generated haikus were complete word salad. With absolutely no concept of context or proper grammar, @thehaikuza V0.1 was light years away from reliably generating poems that made at least some remote level of sense. There was definitely more to constructing phrases than stuffing in random words into corresponding POS slots, so I needed another approach.

Hm, I keep hearing about this Markov Chain… I wonder if he’s related to 2 Chainz?

Click here to read The Making of Project Haikuza: Part 2!

Project Haikuza

Fri, 10 Jul 2015 11:42:26 -0700

Objective: Develop an algorithm to generate haikus using song lyrics.

Motivation: Because computational linguistics are cool.

Project: twitter.com/thehaikuza

Features:

Scrapes Virgin Radio’s broadcast history to find recently played songs
Creates a song-based haiku queue in Google Sheets
Generates a haiku using the queue as a reference and posts it on Twitter
Checks for new tweets every 5 minutes and generates a relevant haiku, if requested
Finds all song lyrics from Lyrics Wikia
Runs on a Raspberry Pi

Challenges:

Formulaically counting syllables
Developing a context-free language model
Creating phrases that actually make sense

Framework: Python 2.7.9

Source: Github

Raspberry Pi 1 Model B running @thehaikuza, 24/7!

Interested about the development of @thehaikuza? Follow it on The Making of Project Haikuza: Part 1!

Places To Be: Howe Sound Crest Trail

Wed, 08 Jul 2015 23:14:15 -0700

Originally posted on The Ubyssey.

Oh boy, some beginner trip this turned out to be.

As a new member of the VOC, it quickly became apparent to me how “beginner” can take on drastically different meanings. In retrospect, I should have read the wiki page on “The 5 Definitions of Beginner Friendly” before agreeing to this little excursion, but a bit of Type II fun (not fun during, but makes for a great story later) never killed anyone, right? To be fair, Mike, the trip organizer, did specify this was a “beginner plus” trip, but my willingness (or call it ignorance) got the better of me.

The item on this weekend’s menu was the Howe Sound Crest Trail, featuring a 29 km walk-in-the-park and a healthy serving of root gardens, rock faces and boulder fields, garnished with majestic summits along the way. We split into two groups at the pre-trip meeting. Team Slog would travel north to south and Team Carrywater would travel in the opposite direction. Team Slog would be starting from sea level with a climb of 1500 metres and Team Carrywater would have no water refill spots during the first day. Team Slog was the more popular choice at a ratio of 3 cars to 2; obviously elevation was preferable to thirstiness.

Team Carrywater arrived at the Cypress trail head around 8 a.m. With our packs on our backs and boots on the roots, we began our adventure. The first viewpoint was St. Mark’s Summit, about 5.5 km from the trailhead. It was a relatively easy-going trail even with the constant elevation gain and handful of log hurdles. We enjoyed the pleasant view of Howe Sound while loading up on some snacks. Based on the presence of touristy folks, I concluded we still had quite a ways to go. The most difficult part for me was trying to figure out how to stay balanced while carrying a 60-pound backpack; being top heavy was definitely something I wasn’t used to. Not to worry, I still had a few kilometers to get used to it before the real fun began.

St. Mark’s Summit

As we continued northward toward Mt. Unnecessary the trail began to narrow and evolve into a slightly trickier, but manageable, beast. The forested area began to thin out, and a couple forehead sweat-wipes later we were hiking out along the mountain ridge. While I was too busy focusing on my footing and trying not to tip over like an inverted pendulum, I hadn’t even noticed how spectacular the view was. Hiking through lush greenery was one thing, but traversing a rocky slab surrounded by an expansive view of the mountains and sea was on a whole different level. We approached a fairly steep climb along the ridge, but with the dry conditions and help of our trusty hiking boots, we didn’t leave anyone at the bottom. A few more sweat-wipes later and we were soaking in the glorious view of Howe Sound. A herd of mildly ominous-looking clouds peered from the east, covering portions of the Lions in the distance. The Weather Network reported a dry, sunny weekend, so why should we have anything to be concerned about? It’s not like they’ve been wrong about the weather in the past…. Did I remember to bring a rain jacket?

Continuing on from Mt. Unnecessary to the Lions was mostly harmless, aside from losing sight of the next trail marker and the rest of my group, accompanied by the reassuring sound of complete silence in the wilderness. Fortunately a pair of hikers (who were ironically lost themselves) coming from the opposite direction helped locate the next trail marker. Team Carrywater somehow managed to regroup along the trail and we collectively made it to the next summit. We enjoyed the gorgeous view while consuming our much deserved lunches.

There was some discussion about scaling up the West Lion, but glancing over to what looked like the future death of me, I took the train back to Nopeville and happily continued eating my PB&J pudding. The more adventurous members of Team Carrywater quickly finished their meals and began their ascent up the West Lion. The three of us who held back watched from a comfortable distance, as if we were filming a National Geographic series on the spider man-like abilities of mountain goats. Shortly after sub-team Sketchy Scramblers were out of sight, sub-team You Only Live Once continued along the the trail toward our destination for the evening.

The terrain was magnificently diverse, ranging from switchbacks through flowery meadows to White Walker-esque boulder fields. Step by step, I began to fully understand why this was a “beginner plus” trip. Having decent cardio was the only thing keeping me alive. We arrived at James Peak to a panoramic view of our days work. Seeing the mountains from this perspective was something else, and infinitely better than the view from the city that I’m accustomed to.

Many sweat-wipes and bushwhacks later, we finally arrived at Magnesia Meadows, our tent site for the night. Some of us relaxed while others took a side trip up Mt. Harvey. We met up with Team Slog and shared the evening with the rest of the VOC party. As the night grew older, we gathered on a slab of rock and bundled up in sleeping bag burritos to watch the sunset and stars.

As beautiful as the evening was, the fact that it was the summer solstice literally made it the worst day in the year to be stargazing. The extended sunset sure was splendid though!

The morning came and we parted ways with Team Slog after breakfast. Team Carrywater marched through the meadows until reaching the base of Mt. Brunswick. We dropped our packs and hiked up the surprisingly steep trail. Have I mentioned how pleasant it was to walk up a waterfall of loose rocks? Again, 100 per cent beginner plus but the scramble was totally worth it, especially the 360-degree view.

Going down a loose, steep trail was a challenge. I considered trying to use gravity and momentum to my advantage, but common sense suggested otherwise (with good reason). We eventually all made it down back to our packs where we enjoyed the remains of our lunch and continued on the latter half of our trip.

Woo hoo, it was all downhill from here! Literally, not figuratively.

As a group, the hardest decision was which of the three beautiful lakes to swim in. We skipped the first lake with some hesitation, but the second was just as marvellous. The lake was refreshingly cold and the weightlessness of water was a treat to our tired bodies. We relaxed on the stranded logs and dried off in the sun before returning to our journey.

The trail was much more leisurely than the previous day, and we all made it to Porteau Cove in one piece. We finished early so we ended our trip at Elwood’s and enjoyed their Sunday special, a $5.50 meal of steak and fries.

Co-op: Air Quality Calibration Chamber

Tue, 23 Jun 2015 21:30:17 -0700

Background: Welcome to TZOA (pronounced ‘zoa’) - the world’s most advanced environment tracker. TZOA uses internal sensors to measure your air quality, temperature, humidity, atmospheric pressure, ambient light, and UV (sun) exposure, all in one wearable device.

Objective: Design and build a purified test chamber with zero air particulate to calibrate the TZOA devices against the DusTrak DRX 8533. An aerosolized cloud of ultrafine (0.3 - 20 μm) test dust was used to create the particle events.

Source: Indiegogo Campaign Page

Client: TZOA

Acknowledgements: This project was completed under MistyWest with the guidance of Taylor Cooper.

The TZOA paired with a smartphone to demonstrate the environment tracking dashboard.

Close up of the device in the test environment.

Overview of the test equipment inside the test chamber.

The Dylos air quality monitor showing zero particulate measurements. A HEPA filter was used to provide the filtration.

Correlation data between the TZOA vs DustTrak devices.

External Ballistics Simulation

Thu, 19 Feb 2015 11:38:35 -0700

Background: including (but not limited to) drag, gravity, air density, altitude, rotation of the bullet, and rotation of the Earth.

One solution is to connect a computer to a manual targeting system (ie. a scope) and estimate the corrected target location by accounting for these external factors. The user may then line the manual targeting system up with the corrected target location, hoping to the high heavens that the target will be hit upon releasing the projectile.

Objective: Determine the required release angle given the muzzle velocity and target distance, accounting for external factors.

Limitations:

Only the corrected vertical distance is calculated
Values have not yet been validated for accuracy
Purely mathematical simulations lack the accuracy provided by simulations backed by empirical data

Framework: Python(x,y) 2.7.9.0

Visualization of release angles between 0 and 90° given the same muzzle velocity.

Flowchart of the algorithm to determine the required release angle.

Visualization of the algorithm on the release angle increment. The magenta trajectory is 5° higher than the red trajectory; the green is 2.5° higher than the magenta, and so on.

Predicted trajectory on flat ground.

Predicted trajectory when target is uphill from user.

Predicted trajectory when target is downhill from user.

MECH 328: TrailRider Design Project

Mon, 24 Nov 2014 12:37:55 -0700

Background: The TrailRider™ is a specialized device to provide accessibility to the wilderness for those with limited mobility. It is intended for a seated rider propelled and balanced by assistants.

Objective: Expand the range of both riders and assistants who can ride safely use the Black Diamond TrailRider™.

How: As a team of seven mechanical engineering undergrads, we explored a range of areas for improvement and developed a new design to incorporate these features.

Existing Problems:

Device relies entirely on assistants for balance, which may be physically demanding for some assistants
Riders generally find the current device too passive
Single-wheel design provides difficulty in rolling over abrupt trail features
Single-wheel provides limited cushioning over rough terrain
Device is difficult to transport and store when space is limited

Our Improvements:

Incorporation of a 255 W electric power assist feature
Implementation of lever-propulsion handles for rider engagement
Integrated frame and suspension design for increased shock absorption
Increased portability

Client: British Columbia Mobility Opportunities Society (BCMOS)

Original TrailRider in use. (Image courtesy of BCMOS)

How the original TrailRider folds for transportation.

Our TrailRider in the folded configuration, 56% smaller than the current design.

Diagram showing the how the kickstand is integrated with the assistant’s push handles.

Overview of our redesigned TrailRider.

Close up of the integrated frame and suspension system.

Co-op: Drill Cover User Study

Sat, 27 Sep 2014 12:39:34 -0700

Background: Drill availability is a major limitation in developing countries, where surgeons turn to using manual hand-crank drills to perform surgeries. A common hardware drill equipped with a waterproof, sterilizable fabric cover provides a cost-effective solution in the operating room.

Objective: Develop, conduct, and analyze a study to quantify the drilling performance of the drill cover solution and compare it against a commercial surgical drill and a manual hand-crank drill.

Abstract: Surgical Device Innovation for Low-Resource Settings: An Alternative for Bone Drilling

Company: Arbutus Medical

The Drill Cover Solution

From the Arbutus Medical product page:

The DrillCover Hex is a sterilizable and reusable fully-sealed barrier that transforms a hardware drill into surgical grade drill. Our kit includes a drill and cover to provide the precision, sterility, and adaptability needed in low-resource settings.

Detailed view of the drill cover.

The User Study

Owing to limited access to surgical power drills, orthopedic surgeons in low-resource settings commonly use manual drills or nonsterile hardware drills. We propose the use of a drill cover solution: a sterilizable fabric bag plus sealed surgical chuck adaptor to permit the safe use of hardware drills for orthopedic surgery. The purpose of this study was to compare the drilling performance of covered hardware drills with manual drills and standard surgical power drills.

From left to right: commercial surgical drill, drill cover solution, manual hand-crank drill.

Operation of the test apparatus.

Method to easily measure the plunge depth through the surrogate bone. The platform moves down as the drill bit pushes against it. The calipers measure the resulting depth.

Stamping method to consistently mark targets for the drilling tests

Comparison of hole quality. A well drilled hole face (left) compared to two holes drilled off axis (right).

Test apparatus, modeled in SolidWorks.

Product photos by Masashi Karasawa

Bowling With car2go

Fri, 29 Aug 2014 12:37:42 -0700

Objective: Design a roof-mounted bowling rig for a car2go marketing event.

Constraints: Use a non-permanent method to securely attach the apparatus to the roof.

Venue: 2014 Vancouver Fringe Festival

Bowling rig set up and ready to roll.

Car2go bowling in action.

Award Winning Bamboo Bike

Mon, 18 Aug 2014 22:12:57 -0700

Objective: Exceed the Vancourite-hipster threshold by building a fixed-gear bicycle using bamboo and carbon fibre.

Challenges:

Constructing a frame jig to allow freedom around joints for carbon-fibre layup
Accounting for the non-uniformity of bamboo poles at the frame lugs
Maintaining compatibility with standard bicycle components
Ensuring the frame is treated for typical"Wet" Coast weather

Awards:

First Prize - Hand Tools Only Instructables Contest
First Prize - Teach It! Instructables Contest, Sponsored by Dremel

Track bicycle custom geometry.

Frame dimensions with tubes shown to determine the required cut lengths of bamboo.

Frame jig modeled in SolidWorks.

Bamboo poles after heat treatment in the oven, preparing for the next heat treatment.

Dry fitting of bamboo poles after mitering.

Bottom bracket junction with mitered bamboo.

Slotted bamboo to fit the dropouts.

Frame wrapped, sanded, and ready for finishing details.

Applying the final touches to the bottom bracket.

Aaaand we’re done!

For a detailed step-by-step guide, head over to my Instructable.

MATLAB Photo Editing Script

Sun, 12 Jan 2014 11:45:14 -0700

Objective: Develop a script to find and remove any differences in a series of photos.

Motivation: How to Remove People From Your Travel Using Photoshop

Framework: MATLAB

Script takes any number of images (preferably taken on a tripod) and combines them into one cleaned-up image.

Flowchart of the implemented algorithm.

Co-op: The Age Effect on Bicycle Helmets

Sat, 28 Dec 2013 12:37:10 -0700

Objective: Determine if age has an effect on the mechanical properties of bicycle helmets.

Tasks:

Conducted 1630 helmet impact attenuation tests to measure the effect of age and use on bicycle helmet foam
Developed scripts in MATLAB to autonomously post-process the impact data and organize information according to existing helmet databases
Wrote the Methods section of the research report to be submitted to the Accident Prevention & Analysis Journal
Assisted in protocol development of the helmet and foam core impact attenuation tests

Company: MEA Forensic Engineers & Scientists

Annotated diagram of the helmet drop tower.

MECH 223: Hovercraft Competition

Sat, 20 Apr 2013 12:22:15 -0700

For our final MECH2 design project at UBC, we were given three weeks to research, conceptualize, test, and build a functional hovercraft. The end goal was a class-wide competition of speed, cargo capacity, maneuverability, and driving skill. Budget was limited since costs came out of our own pockets, so we set out to build an inexpensive hovercraft using parts from the hardware and dollar store.

Our result: 5th place out of 30 teams. Not too shabby for a hovercraft made with foam and garbage bags!

Various fan and chassis designs to provide lift.

Intermediate prototypes prior to our final design.

Close up of our rooster-themed hovercraft.

Mech2 Hovercraft competition showcase at the Robson Skating Rink in Vancouver, BC.

Diagram outlines our well-implemented design features and recommendations for future revisions.

Men in Massage: Research Paper

Thu, 01 Dec 2011 17:51:38 -0700

Originally written as a research paper for ENG 112 at UBC. Republished here for archival purposes.

Research Proposal

According to Tom Delph-Janiurek, a “mis-performance of voice” may be necessary in order to gain authority in specific interactional spaces by creating a display of control and assertiveness (277). He proposes that the vocal performances in teaching spaces adapt to their social roles of authority by opposing the hegemonic heterosexuality in voices and perform vocal features of “drag” without giving up their gendered identities, contributing to and disrupting the “heterosexing of [teaching] space” (277). In spite of his conclusion regarding voice as “drag”, he fails to thoroughly assess the aspects outside of vocal performances that also relate to the projections of gendered bodies such as embodiment. A performed gender role relies on the physical aspects of body language and gesture in addition to voice in order to create a complete portrayal of a sexualized identity.

The greatest panic of heterosexual men may be the perceived inability to conform to the hegemonic image of masculinity, especially in a female dominated environment (Delph-Janiurek 265). As a result, men may not be willing to enter a setting of close relations and intimacy such as massage therapy possibly by reason of homophobia, alongside the belief that it is solely a woman’s profession (van Meter, 2006). Female dominance is evident in massage therapy because of the nurturing and caring qualities that are typical of hegemonic femininity and this particular profession (Dobson, 2005). However, not all men share this same necessity of adhering to an idealized, unachievable masculinity, as there is still a presence of men (albeit a scarcity of them) in female-dominated work environments (van Meter, 2006). In consequence, men are forced to adopt effeminate qualities to gain authority in the interactional space of massage therapy. Thus, how are men’s gendered identities performed when they cross the boundary of conventional gender roles and enter a female-dominated work setting?

Men In Massage

The Performance of Hegemonic Femininity by Male Bodies in the Female-Dominated Environment of Massage Therapy

Introduction

Studies have shown how sexualized bodies adopt characteristics of other gendered identities to better perform their occupational roles. According to Tom Delph-Janiurek (1999), a “mis-performance of voice” may be necessary in order to gain authority in specific interactional spaces by creating a display of control and assertiveness (277). He proposes that the vocal performances in university teaching spaces are adapted to their social roles of authority by performing vocal features of “drag” without sacrificing their gendered identities (277). However, he fails to thoroughly assess the aspects outside of vocal performances that also relate to the projections of gendered bodies such as embodiment. As previous evidence suggests, performed gender roles rely on the physical aspects of body language and gesture in addition to voice in order to create a complete portrayal of a gendered identity.

Body

Delph-Janiurek proposes that the greatest panic of heterosexual men may be the perceived inability to conform to the hegemonic image of masculinity, especially within a female-dominated environment (265). As a result, van Meter theorizes men may not be willing to enter a setting of close relations and intimacy such as massage therapy possibly by reason of homophobia, alongside the belief it is solely a woman’s profession (48). Moreover, Dobson illustrates how female dominance is evident in massage therapy because of the nurturing and caring qualities that are typical of hegemonic femininity and this particular profession (165). McCarthy notes how “men… are constructed by gender ideologies that discourage their participation in… caretaking activities that involve body intimacy” (12). However, van Meter suggests not all men share this same necessity of adhering to an idealized, unachievable masculinity, as there is still a presence of men (albeit a scarcity of them) in female-dominated work environments (48). Thus, men are forced to perform femininity in order to gain authority in the interactional space of massage therapy.

According to Sullivan, people are drawn to massage as a form of health care therapy because of the psychosocial processes relating to contact, comfort, connection, and caring (182). He also claims the “culture of care” affiliated with massage therapy provides clients with “benefits such as relaxation, feelings of well-being, improved circulation, and reduction in anxiety and pain” (181). A consumer survey conducted in 2007 by the American Massage Therapy Association (AMTA) found 25 percent of men, relative to 43 percent of women, had a massage within the past five years; 30 percent of the total recipients had massage therapy for medical reasons, whereas 22 percent did so in search for relaxation and 13 percent for indulgence. These statistics suggest men and women mainly seek massage therapy for (medical) maintenance of the body.

The profession of massage therapy is recognized as women’s work because of its associations with qualities of mothering. According to AMTA, approximately 82-84 percent of the 258 000 total therapists in the United States are female. Both Dobson and McCarthy agree the nature of intimacy in massage is inherently related to the stereotypical characteristics of nurturing and caring perceived in hegemonic femininity. They emphasize how massage is a sensual and intimate profession, where these “natural properties” of femininity parallel the nurturing and caring mothers typically display to their children, relative to the infrequent physical interaction children receive from their fathers (171; 180). Dobson speculates that the similarities between the natures of work in massage therapy and being the caretakers of the family are intrinsic precursors to the societal idea of massage being a profession of femininity, where women are typically located in the “domestic sphere of work” in a heteronormative society (9). However, she also suggests women are drawn to massage because of its self-employability and flexible scheduling, which ultimately permit women to achieve a balance between work and family (9).

The preference for female massage therapists is determined by the construction of gender roles in a heteronormative society. Through van Meter’s interviews with licensed therapists, she found their clients tended to have the preference of female therapists over male ones when scheduling appointments, some to the extent where clients would rather wait for the next available female therapist (when no others were available) and decline the offer for a male therapist, even after reassurance by the receptionist concerning the proficiency and experience of the male therapist (48). Notably, for women having had experience(s) with violent men, Klein reveals the preference for female therapists may develop from their fear of being sexually assaulted, as female therapists provide the safety and comfort male therapists typically may not be able to contribute to (13). Dobson continues by suggesting the difficulty for male massage therapists to reach out to female clients is a result of the conventions of a heteronormative society implying “all heterosexual men must want sex with women” (180).

Male clients who choose to embrace massage therapy will request female therapists to pronounce their masculinity in this setting of femininity (Klein, 186). Socially constructed homophobia functions as a constituent in determining the masculinity of men in massage therapy; Klein suggests homophobia has been internalized to the degree where the notion of being massaged by another male is perceived as unconventional and irrational in society (13). Furthermore, van Meter illustrates how the sexualisation of touch heightens the difficulty for male therapists in gaining male clients as Dave Murdock, a nationally certified massage therapist, notes, “Your potential male clients are afraid you might be gay and your potential female clients are afraid that you’re not” (48). However, as noted by Nevels through an interview with Pete Whitridge, a licensed massage therapist and former assistant director of the Florida School of Massage in Gainesville, there is minimal gender difference once the client has overcome the initial barrier of sexualisation and enters the massage room and gets on the table (108).

Massage therapy is not openly accepted in a heteronormative society because of its implied associations with sexuality and intimacy. Nevels identifies men as being hesitant in accepting massage as a form of health care therapy due to the perception that having massage means being “pampered” (which relates to indulgence and luxury), rather than having the body being medically “maintained”, ultimately leading to the disruption of the “hegemonic normative prescriptions of gender” (104). Moreover, she believes the conventional performances of male gendered bodies are illustrated through the omission of femininity and therefore, massage, because therapy in this form does not recognize their masculine needs (104). To Connell, hegemonic masculinity forces men to disregard qualities of femininity in body language and speech (cited in Delph-Janiurek, 265). Similarly, Pascoe suggests any qualities of effeminacy in the behaviour of men results in being labelled as non-heterosexual, consequently “powerfully undermin[ing] a man’s masculinity and social value in a misogynist, heteronormative culture” because of the constant judgement by society (cited in Klein, 186).

Males interpret touch differently than females as a result of the physical interactions they, as children, have with their parents. According to Klein, young children must have physical interaction as a form of “social nutrition” in order to grow and develop in society, despite human sexuality being restrained throughout childhood (49, 50). Kiely’s study of fathers in families illustrates how fathers rarely (if at all) have physical interaction with their children: 59 percent of respondents were never hugged by their fathers whereas 61 percent were occasionally hugged by their mothers (cited in McCarthy, 180). McCarthy proposes this evidence leads to the belief that touch is solely for women in a heteronormative society, as patriarchal ideologies insist men have minimal physical interactions (of intimacy) with each other (180). Thus, while women interpret touch as a sign of friendliness, men interpret it as being intrusive (180).

The impartiality of a male’s performance in massage therapy deprives them of their own embodied experience of healing from massage (van der Riet, 153). Seidler emphasizes “learning to be a man means learning to be impersonal” by reason of the “hegemonic scripts of conventional gender roles” (cited in van der Riet, 153). Chodorow follows by stating men prefer not to be handled or touched because masculinity is threatened by intimacy, which subsequently gives rise to, as McCarthy describes, the implications of a reduced status as a gendered body (cited in van der Riet, 151; 180). Van der Riet’s study of a students’ massage workshop finds the intimacy and sexual implications of massage therapy cause males to feel uncomfortable, which is shown through a lack of concentration by the male students in practice (e.g. constantly looking around, not focusing on the massage), and most notably, their body language of awkward movements, rigidity and tense facial expressions throughout the duration of delivering their massage (151).

The sexual implications of massage force therapists to continuously desexualize the massage atmosphere in order to establish a professional relationship with the client. Van der Riet states massage is perceived as “socially fragile” because of the evident risks in embarrassing the client through physical and/or emotional invasion of the client’s body and private space from “unpleasant” procedures (151). Klein emphasizes it is especially inherent for male therapists to discourage the socially constructed “sexual misrepresentations of massage” by avoiding terminology associated with sexual connotations (i.e. “massage table” vs. “massage bed;” “disrobe” vs. “take your clothes off”) and by presenting themselves professionally through their clothing and personal grooming (78). Furthermore, it is essential for male therapists to respect the personal boundaries and comfort zone of the client. However, Dobson suggests this becomes difficult when the therapist is sexually attracted to the client and is forced to manage his “sexual responses to ‘beautiful women’ or ‘pheromones’” (171). In order to create a “non-sexual interaction” with the client, Klein speculates desexualisation is achieved through the therapist partitioning the body (by draping the areas not being worked on) and avoiding “sensitive” areas (inner thighs, abdomen, sides of women’s torso, buttocks) of the client (81). Moreover, van der Riet proposes therapists should be emotionally detached from the massage by setting emotional boundaries between the client and themselves (153).

Discussion

From analysing past studies on massage therapy, it is evident the difficulty for males to become effective therapists lies in the construction of heteronormativity in society. Male therapists are faced with the issue of being judged in society through their choice of profession, thus deterring other men from entering massage therapy. As previously stated by van der Riet, the continuous effort by men to prove their masculinity causes male student therapists to perform poorly, where the display of mothering qualities of nurturing and caring are necessary in order to enhance the atmosphere of massage for the client (153). Thus, males entering massage therapy should disregard the socially constructed pressures to conform to the hegemonic image of masculinity in order to gain authority as a proficient therapist. Moreover, as Nevels illustrates, it is recommended for male therapists to cohere to the characteristics of hegemonic femininity by embracing the caretaking qualities evident in the “culture of massage,” possibly allowing them to attract a client base of both males and females (104). Male therapists are still able to thrive as a male gendered body despite massage being a female-dominated profession by desexualizing the environment and relationship between the therapist and client, consequently producing a healthy atmosphere for healing and maintaining the body for their clients receiving massage. Additionally, the preference for female therapists may be refuted once male and female clients allow themselves to disregard what a heteronormative society forces them to believe concerning the sexual misrepresentations of massage therapy.

Conclusion

In spite of the adequate evidence to suggest the performance of femininity is necessary for male therapists to thrive in massage therapy, this does not directly translate into the acceptance of massage as a suitable form of health care therapy for males and females without facing repercussions by a heteronormative society. Although male therapists are able to stylise their sexualized identities to present themselves in a more attractive light for both male and female clients, it is ultimately the decision of clients, with respect to their perception of massage therapy, whether or not to embrace massage as their desired form of medical healing. While this paper explains how male therapists perform their gendered body in a female-dominated setting to gain authority, it fails to answer how massage therapy may be displayed without the negative connotations of sexuality that hinders its growth as an effective method of healing. Future research should include further evidence concerning why sexual interaction is frowned upon in society, possibly achieved through studying the influence of mass media on parents and their children, as the notions of non-sexual interactions are primarily induced during childhood. Regardless, it is fundamentally necessary for masculine bodies to adopt effeminate qualities in massage therapy where authority is predetermined by female gendered identities.

Works Cited

Dobson, Marnie. "Professionalizing Touch: Gender, Sexuality, the Law and Massage Work." University of California, Irvine, 2005. United States -- California: ProQuest Dissertations & Theses (PQDT). Web. 9 Nov. 2011.
Klein, Noa. “Loving Touch: Therapeutic Massage, the Socialization of the Body, and the Healing of US Culture.” University of California, Santa Barbara, 2010. United States – California: ProQuest Dissertations & Theses (PQDT). Web. 9 Nov. 2011.

McCarthy, Michael. “Skin and touch as intermediates of body experience with reference to gender, culture and clinical experience.” Journal of Bodywork and Movement Therapies 2.3 (1998): 174. Science Direct Journals. Web. 4 Nov. 2011.

Nevels, Amanda. “Getting him through the door: How to market massage therapy to males.” Massage Therapy Journal 47.1 (2008): 104. Health Reference Center Academic. Web. 4 Nov. 2011.

Sullivan, S John. “The culture of massage therapy: Valued elements and the role of comfort, contact, connection and caring.” Complementary Therapies in Medicine 17.4 (2009): 181. Science Direct Journals. Web. 4 Nov. 2011.

van der Riet, Pamela. “Massage and sexuality in nursing.” Nursing Inquiry 2.3 (1995): 149-156. Wiley Online. Web 4 Nov. 2011.

van Meter, Ryan. “A touchy subject: is there gender bias in massage therapy?” Massage Therapy Journal 45.4 (2006): 48. Health Reference Center Academic. Web. 4 Nov. 2011.

LEGO Headphones

Sat, 10 Jul 2010 12:37:00 -0700

Background: I decided to combine the two things that I love most: music and LEGO. This whole project took me about a month and a half, very on and off. I’m quite pleased with how the headphones turned out in the end, as they are sturdy enough to be tossed around in a bag. My favourite part is that I can take it apart any time to make repairs that may be needed on the go. No tools necessary!

Objective: Use LEGO to create the external assembly of a pair of headphones.

The world’s finest LEGO headphones in all its glory.

Easily deconstructed, staying true to the LEGO roots.

Visit Instructables to learn more about the project!

A Minty-Fresh Power Pack

Thu, 16 Apr 2009 22:52:25 -0700

Before the ubiquity of powerbanks, there was nothing on the market to provide longer battery life for portable electronics. Connecting a female USB connector to four rechargable AA batteries opened up a new world of extended iPod playback. I built a simple, rechargable battery pack to keep the tunes playing on long road trips.

The iPod is alive!

President’s Choice rechargable batteries and some janky wiring is all it took.