Jacques Kaiser

Bayesian beliefs app

2020-12-05T00:00:00-08:00

I have developed a light web app in vue.js using the quasar framework. It is available as a website as well as an app in the android play store.

This app helps you keep rational beliefs about the world using the Bayesian approach. It works by listing down competing hypothesis, and maintaining their probability of being true. This probability is re-calculated with the Bayesian formula everytime a new observation about the world is made.

This is a pure frontend app: all data stays in local storage on the device, there is no backend. The code is open-source on github.

PhD on Synaptic Learning for Neuromorphic Vision

2020-08-18T00:00:00-07:00

After over five years of researching in the field of biological learning for robotics, I have successfully completed my PhD. The full dissertation is published on the KIT bibliothek. In this blog post, I introduce the field of neuromorphic computing in layman’s terms, and present some of the results.

Silicon retina for a silicon brain

The Dynamic Vision Sensor is a very special type of camera. Conventional cameras take images at a given rate, around 30 times per seconds. Standard computer vision algorithms have to read and process these images, one by one. Modern smartphones are capable to perform such processing, but it consumes a lot of energy. On the other hand, the Dynamic Vision Sensor will only emit an individual event, when the light received by one pixel has changed. An event consists of the address of the pixel which changed, and whether light increased or decreased for this pixel. In other words, when the scene does not change, and the sensor does not move, no events are emitted. A computer vision algorithm can now run more efficiently: instead of processing full images regularly, it only has to process individual events, when the scene changes.

How the Dynamic Vision Sensor works

Until now, most computer vision algorithms were tailored to process images, not events. This also includes deep neural networks, which are used today in almost every computer vision problems. A deep neural network consists of neurons connected to each other in layers. Every neuron has a value, which is computed from the neurons in the previous layers and communicated to the neurons in the next layer. The actual computations depends on some values, called weights, which are learned. To process an image, the neurons of the first layer simply take the value of the pixels in the image. The values are used to compute the activation of the second layer of neurons and so on, until the last layer of neuron which is the output. Therefore, for every processed image, all neurons of a layer have to communicate their values synchronously. This is another expensive process (although it fits the GPU very well).

Conventional deep learning neuron

Spiking neurons are biologically-plausible version of conventional artificial neurons. They are particularly suited to processing events rather than images. Unlike a conventional neuron, a spiking neuron maintains a dynamical state which is not communicated to other neurons. Communication between neurons only consists of spikes, emitted when the state reaches a threshold. Spikes are spontaneous, stereotypical events: the information resides in the time at which the spike was emitted. Spikes and address events from the Dynamic Vision Sensors are very similar. In fact, to process address events with a spiking network, the first layer of neurons simply emit spikes for every address event.

Spiking neuron

Intuitively, processing a stream of events with a spiking network requires far less computations/communication than processing a sequence of images with conventional neural networks. The reason is, instead of processing the full information about the visual scene at every time-step again and again, we only process and communicate differences. Less computations implies potentially saving energy while running faster. However, this style of event-based computations do not fit conventional hardware very well: it is a paradigm shift. Taking advantage of these saved computations requires dedicated hardware, called neuromorphic hardware. Designing and manufacturing neuromorphic hardware was initiated in academia (Caltech, Zürich, Manchester, Dresden, …). Since then, both Intel and IBM started developing neuromorphic chips.

Deep learning networks are used everywhere today, because they can be easily trained with an algorithm called backpropagation. But how can deep networks of spiking neurons be trained? Are such networks capable of learning directly from the events of the Dynamic Vision Sensor? These are research questions I have investigated as part of my PhD.

Gradient descent in spiking networks

Over all the methods I have evaluated during my research, approximations of backpropagation turned out to provide the best performance. Not only was it learning more accurately, these approximations also solve some of the biologically implausible aspects of backpropagation, yielding a new theory of learning in the brain.

To go further, let’s formalize what is learning. Machine learning is a neat technique to solve complex problems, whenever data is available. At its core, learning requires:

a parameterized model: some mathematical operations, which is a function of some parameters. In deep learning, the neural network is the model, which is parameterized by the synaptic weights $W$.
a loss function $L: \mathbb{R}^n \rightarrow \mathbb{R}$, a metric assessing the performance of the synaptic weights on the task, using the available data. The goal of learning is to find the weights which minimize this loss function: $W^\star=argmin_W L(W)$.

To find the best weights, a naive approach would be to try random weights, evaluate the loss function, and simply keep the ones where the loss function is the smallest. This is the strategy of black-box optimization methods such as evolutionary algorithms. However, evaluating the loss function is expensive, and we would like to do it as few times as possible. To this end, the best method to date is gradient descent. The gradient of the loss by the weights $\frac{\partial L}{\partial w_{ij}}$ for all weights $w_{ij} \in W$ is computed at the same time the loss $L(W)$ is evaluated. Unlike black-box optimization, this requires to know what computations are performed by the model. By definition, the gradient of the loss by a weight $w_{ij}$ means how an infinitesimal change in $w_{ij}$ affects the loss. Therefore, by taking an (arbitrary small) step in the opposite direction of the gradient, we have good chances that the loss function will decrease. Gradient descent repeats this operation for all weights $w_{ij} \in W$ until the loss function stops decreasing:

\[w_{ij} \leftarrow w_{ij} - \alpha \times \frac{\partial L}{\partial w_{ij}},\]

with $\alpha \in \mathbb{R}$ the learning rate: an arbitrary small step.

Now back to neural networks. Backpropagation is the method used to compute the gradient $\frac{\partial L}{\partial w_{ij}}$ in deep learning. To explain how it works, we need to dive in the equations of artificial neurons. A conventional deep learning neuron starts by summing up the values of its pre-synaptic neurons, weighted by the synaptic weights. It then applies a non-linear function, usually simply nulling-out negative values (ReLU). These are the $u_i$ and the $s_i$ terms in the following neural equations:

\[\begin{aligned} u_i &= \sum_{j\in \textit{pre}} w_{ij} \times s_j \\ s_i &= u_i \; \text{if} \; u_i > 0 \; \text{else} \; 0 \end{aligned}\]

Using this definition, the step from a conventional neuron to a spiking neuron is small. In fact, there are only two important differences with spiking neurons:

$u_i$ carries a trace of the previous activity (it is a state which needs to be stored and updated). It is called the membrane potential.
$s_i$ is either 0 or 1, denoting spike and absence of spike, unlike conventional neuron which have output in $\mathbb{R}$).

Mathematically, a spiking neuron can be expressed as:

\[\begin{aligned} u_i &= \sum_{j\in \textit{pre}} w_{ij} (\epsilon \ast s_j) + \eta \ast s_i \\ s_i &= 1 \; \text{if} \; u_i > 0 \; \text{else} \; 0 \end{aligned}\]

The $\ast$ operation denotes a temporal convolution, with kernel $\epsilon$ and $\eta$ respectively. In words, this means that $u_i$ also depends on the previous values of $s_j$ and $s_i$. A spike $s_j$ is instantaneous, but it leaves a trace in $u_i$: this is the first convolution. The second convolution denotes the decrease of the membrane potential of a neuron, right after it emitted a spike, commonly called the refractory period. A simple way to model this is simply by having the neuron inhibiting itself, with $\eta \ast s_i < 0$.

Example dynamics of a spiking neuron

Let’s calculate the gradient $\frac{\partial L}{\partial w_{ij}}$ for both an analog neuron and a spiking neuron. We can use the chain rule to express the gradient of a loss function $L$ with respect to the weight $w_{ij}$:

\[\frac{\partial L}{\partial w_{ij}} = \frac{\partial L}{\partial s_i} \times \frac{\partial s_i}{\partial u_i} \times \frac{\partial u_i}{\partial w_{ij}}\]

Using this formulation, we see that the gradient is a multiplication of three terms. Let’s discuss these terms one by one.

Calculation of $\dfrac{\partial u_i}{\partial w_{ij}}$:

This term captures how a change in synaptic weight affects the weighted sum of inputs $u_i$. For a conventional deep learning neuron, this term is simple to calculate, it is simply $s_j$. For a spiking neuron, the calculation is more involved, especially because of how a change in weight $w_{ij}$ affects the refractory period. Here comes the first approximation to backpropagation for spiking neuron: we ignore the refractory period in the gradient calculation. Note that if neurons do not spike often (in the brain neurons spike at maximum 60Hz) then refractory periods won’t play a big role in the dynamics. Therefore ignoring it in the gradient computation won’t affect learning too much. Using this approximation we can write $\frac{\partial u_i}{\partial w_{ij}} \approx \epsilon \ast s_j$.

Calculation of $\dfrac{\partial s_i}{\partial u_{i}}$:

This term captures how a change the membrane potential $u_i$ affects spikes $s_i$. With conventional deep learning neuron, this is simply the derivative of the ReLU function. For a spiking neuron however, the threshold function is not differentiable (it is undefined at the threshold value). Here comes the second approximation. We simply pretend that the threshold function was a similar, differentiable function – this is called a surrogate. For example, a function that linearly increases with slope $1$ around the threshold value. We therefore have: $\frac{\partial s_i}{\partial u_{i}} \approx 1$ around the threshold value, 0 otherwise.

Calculation of $\dfrac{\partial L}{\partial s_i}$:

This term captures how a change in this neuron activity affects the loss of the whole network. The calculation of this term is the same for conventional neurons and spiking neurons. Note that $s_i$ might not affect the loss directly if $i$ is not an output neuron. Instead, it affects the activity of the neurons in the next layer, which themselves affect the next next layer, and so on until the output layer. Therefore, an exact computation of $\frac{\partial L}{\partial s_i}$ also depends on the synaptic weights in the following layers. In conventional deep learning, this is not a big deal: when the error at the output layer becomes available, it is simply sent backward through the whole network. Hence the name backpropagation. For spiking networks, this is a problem: biological synpases are not bi-directional. This is where the third and last approximation jumps in. For spiking networks, we instantiate dedicated feedback synapses with random weights, distributing the network error to all neurons in all layers. This method is called feedback alignment, because the whole network will learn to align with the randomly chosen feedbacks. The actual choice of the loss function, and how feedback alignment is applied, is currently how modern synaptic learning rules for spiking networks differ.

Putting it all together

Injecting the three terms back into the gradient equation yields the synaptic learning rule for spiking neural networks:

\[\frac{\partial L}{\partial w_{ij}} \approx e_i \times \epsilon \ast s_j \; \text{if} \; u_i \; \text{around threshold, 0 otherwise},\]

with $e_i$ the loss-dependent error. Like the dynamics of a spiking neuron, note that this gradient also has a temporal dynamics, due to the presence of the temporal convolution. In practice, this allows the spiking network to learn temporal correlations at no additional cost in memory, since the dynamical state for learning can be shared with the forward dynamics. Additionally, weight updates can be performed online, while sensory input is being streamed inside the network.

There are multiple instances of synaptic learning rules which relate to this derivation. As part of my PhD, we developed the DECOLLE learning rule [paper, code] together with Prof. Emre Neftci during a stay at the University of California, Irvine, funded by a DAAD scholarship. This learning rule achieves state-of-the-art accuracy on the popular gesture recognition dataset from IBM, DvsGesture. It is closely related to other rules such as SuperSpike and e-prop.

Conclusion

The CPU in your phone or laptop will not be replaced by a neuromorphic chip, but it might be complemented by it. Neuromorphic computing opens the door to novel applications where computing speed and low power consumption matters. With modern synaptic plasticity rules, such chips will also be capable of learning temporal correlations efficiently using approximations of gradient descent.

ROS Tensorflow

2020-03-22T00:00:00-07:00

In this blog post I go through some useful techniques to nicely integrate Tensorflow (TF) in ROS nodes. The related code can be found in the following ros_tensorflow github repo.

What this blog post covers:

Creating a TF model and saving the session
Training a TF model in a ROS action
Making predictions from callbacks

Integrating Tensorflow 2 in ROS nodes

Tensorflow requires python 3, but ROS melodic and ubuntu 18.04 still rely on python 2 by default. You can get away by running the Tensorflow node with python 3, using the shebang #!/usr/bin/env python3. You will have to install some missing ROS python 3 libraries, which you can do with conda or pip. If you plan to work with images, you will encounter an additional issue with cv_bridge, which you can solve by following this tutorial.

Model creation

Since rospy callbacks are called in a separate thread than the main, it is important to handle the Tensorflow session accordingly. When you create your model at initialization phase in the main function, your code is running in the main thread. Here, you should store the Tensorflow session so that you can restore it later in your callbacks:

`#!/usr/bin/env python3`

import tensorflow as tf
import rospy

class ModelWrapper():
    def __init__(self):
        # store the session object from the main thread
        self.session = tf.compat.v1.keras.backend.get_session()

        self.model = tf.keras.Sequential()
        # ...

class RosInterface():
    def __init__(self):
        self.wrapped_model = ModelWrapper()

def main():
    rospy.init_node("ros_tensorflow")
    ri = RosInterface()
    rate = rospy.Rate(1)
    while not rospy.is_shutdown():
        rate.sleep()

if __name__ == "__main__":
    main()

Model training

If you are recording data at runtime, you may want to re-train your model at runtime too. The best way is to perform training in a ROS action server, so that a client can also cancel training if needed. Since the action callback is called in a separate thread, you have to restore the session from the main thread. Here is the training function for the tensorflow model:

import tensorflow as tf

class ModelWrapper():
    # ...
    def train(self, x_train, y_train, n_epochs=100, callbacks=[]):
        with self.session.graph.as_default():
            tf.compat.v1.keras.backend.set_session(self.session)

            self.model.fit(x_train, y_train,
                epochs=n_epochs,
                callbacks=callbacks)

The line tf.compat.v1.keras.backend.set_session(self.session) restores the session object to the main thread. If you omit this line, you we’ll get a strange error that will look like the following:

Error processing request: Error while reading resource variable model/dense_1/kernel from Container: localhost. This could mean that the variable was uninitialized. Not found: Container localhost does not exist. (Could not find resource: localhost/model/dense_1/kernel)
     [[]]

We will call the train function from an action server. Note that we declare a callbacks parameter, which will allow us to provide action feedback while the model is training and cancel training when the action is cancelled. Here are some nice wrappers to achieve this:

import tensorflow as tf

class StopTrainOnCancel(tf.keras.callbacks.Callback):
    def __init__(self, check_preempt):
        super(tf.keras.callbacks.Callback, self).__init__()
        self.check_preempt = check_preempt
    def on_batch_end(self, batch, logs={}):
        self.model.stop_training = self.check_preempt()

class EpochCallback(tf.keras.callbacks.Callback):
    def __init__(self, cb):
        super(tf.keras.callbacks.Callback, self).__init__()
        self.cb = cb
    def on_epoch_end(self, epoch, logs):
        self.cb(epoch, logs)

Now let’s implement the ROS action server in our RosInterface class:

import rospy
import actionlib
from ros_tensorflow_msgs.msg import TrainAction, TrainFeedback, TrainResult

class RosInterface():
    def __init__(self):
        self.wrapped_model = ModelWrapper()
        self.train_as = actionlib.SimpleActionServer('train', TrainAction,
                                                     self.train_cb, False)
        self.train_as.start()

    def train_cb(self, goal):
        stop_on_cancel = StopTrainOnCancel(
            check_preempt=lambda : self.train_as.is_preempt_requested())
        pub_feedback = EpochCallback(
            lambda epoch, logs: self.train_as.publish_feedback(
                TrainFeedback(i_epoch=epoch, loss=logs['loss'], acc=logs['accuracy'])))

        # ... load x_train and y_train
        self.wrapped_model.train(x_train, y_train, callbacks=[
            stop_on_cancel,
            pub_feedback])

        # Training finished either because it was done or because it was cancelled
        if self.train_as.is_preempt_requested():
            self.train_as.set_preempted()
        else:
            self.train_as.set_succeeded()

Model predictions

There are two ways you can integrate a Tensorflow model in a ROS nodes:

Making predictions in a loop
Making predictions on callbacks

If you want to perform prediction on callbacks, you will have to restore the Tensorflow session of the main thread, just like training:

class ModelWrapper():
    # ...
    def predict(self, x):
        with self.session.graph.as_default():
            tf.compat.v1.keras.backend.set_session(self.session)
            out = self.model.predict(x)
        return out

Restoring the session is not necessary if you perform predictions in the main loop, but it doesn’t hurt. Since predictions are fast compared to training, you can implement them with rostopic subscribers or rosservice servers, no need for actionlib in this case.

In the provided ros_tensorflow repo I demonstrate both making predictions in a service callback and making predictions in main loop. The repo also makes a clean separation between Tensorflow code and ROS code. If you plan to use Tensorflow in your ROS project you can simply fork the repo and start from there.

Gleeo time plotter

2019-12-31T00:00:00-08:00

I have been using the Gleeo time tracker app on my phone to track my time at work for multiple years. To make sense of the data, I implemented a jupyter notebook. The notebook reports:

Total time spent per project
Time spent per project [per unit of time]
Over(under)hours aggregation, accounting for holidays

You can fork the jupyter notebook and try it with your own data.

The notebook

This is what the notebook outputs with some scrambled time tracking data for 2019. We first read the Time Tracker data export.csv file with pandas, which you can export from the Gleeo app:

import pandas as pd
import numpy as np
time_entries = pd.read_csv("Time Tracker data export.csv")
time_entries['Start'] = pd.to_datetime(time_entries['Start'])
time_entries['End'] = pd.to_datetime(time_entries['End'])
projects = time_entries['Project'].unique()
tasks = time_entries['Task'].unique()
time_entries

	Domain	Project	Task	Details	Start	...	Decimal Duration	Project-Extra-1	Project-Extra-2	Task-Extra-1	Task-Extra-2
0	work	administrative	mail	NaN	2019-01-02 09:23:00	...	0.250000	NaN	NaN	NaN	NaN
1	work	research	writing	NaN	2019-01-02 09:38:00	...	2.766667	NaN	NaN	NaN	NaN
2	other	break	lunch	NaN	2019-01-02 12:24:00	...	0.683333	NaN	NaN	NaN	NaN
3	work	research	writing	NaN	2019-01-02 13:05:00	...	5.033333	NaN	NaN	NaN	NaN
4	work	administrative	mail	NaN	2019-01-03 10:13:00	...	0.116667	NaN	NaN	NaN	NaN
...	...	...	...	...	...	...	...	...	...	...	...
1506	work	research	hardware	NaN	2019-12-20 09:14:00	...	0.266667	NaN	NaN	NaN	NaN
1507	work	administrative	meeting	NaN	2019-12-20 09:30:00	...	1.083333	NaN	NaN	NaN	NaN
1508	work	research	hardware	NaN	2019-12-20 10:35:00	...	1.666667	NaN	NaN	NaN	NaN
1509	other	break	lunch	NaN	2019-12-20 12:15:00	...	1.083333	NaN	NaN	NaN	NaN
1510	work	research	hardware	NaN	2019-12-20 13:20:00	...	0.733333	NaN	NaN	NaN	NaN

1511 rows × 13 columns

Total time spent on projects

Let’s print the total time spent on projects:

project_total = time_entries.groupby('Project').sum()
project_total["Decimal Duration"]

All times are displayed in hours:

Project
administrative    434.516667
break             158.566667
industry          530.433333
public            195.533333
research          806.933333
Name: Decimal Duration, dtype: float64

Proportion of time spent on projects:

ax = project_total.plot.pie(y='Decimal Duration', autopct='%.1f')
ax.set_title('Total time spent on projects')
ax.set_ylabel('')
ax.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)

Time spent on projects per unit of time

Let’s plot the time spent on project over time of the year. We define a function that reorganizes the tracked time in regular time interval:

start = time_entries.iloc[0]['Start'].round('D')
end = time_entries.iloc[-1]['End'].round('D')
years = range(start.year, end.year+1)

def get_time_spent_per(time_unit):
    plt_time_index = pd.date_range(start, end, freq = time_unit)
    range_df = pd.DataFrame(plt_time_index, index=plt_time_index, columns=['Date'])
    range_df['End'] = range_df['Date'].shift(-1)
    range_df = range_df.iloc[:-1] # drop last range

    series = [ time_entries.loc[(time_entries['Start'] > row['Date']) & (time_entries['End'] < row['End'])]\
        .groupby('Project')['Decimal Duration'].sum()
     for _, row in range_df.iterrows() ]
    organized_durations = pd.DataFrame(series, index=range_df['Date']).fillna(0)
    return organized_durations

We can use this function to plot the time spent on project per month:

df_month = get_time_spent_per("BM")
ax = df_month.plot.area(title="Time spent on projects every month")
average_hours_per_month = df_month.mean().sum()
print("Average hours/month: {}".format(average_hours_per_month))
ax.axhline(average_hours_per_month, linestyle="--", linewidth=3)

The average hours/months for all projects (including the break project):

Average hours/month: 181.71833333333333

Or per week:

df_week = get_time_spent_per("W")
ax = df_week.plot.area(title="Time spent on projects every week")
average_hours_per_week = df_week.mean().sum()
print("Average hours/week: {}".format(average_hours_per_week))
ax.axhline(average_hours_per_week, linestyle="--", linewidth=3)

Average hours/week: 41.968707482993196

Over(under)hours

To calculate the amount of overhours, it is required to know the public vacations as well as personal vacations, so that we can estimate the expected work time. Fortunately, the holidays library can help us with the former. Sadly there is no library to help us with the latter. You need to manually provide your personal vacation in the file personal/holidays.yaml. It should have the following format:

-
  name: Snowboard in the alps
  from: 2019-01-14
  to: 2019-01-19
-
  name: WOOFing in an organic farm
  from: 2019-05-20
  to: 2019-05-27

Do not worry about precise dates if your vacation ends or begins on a weekend/day off, the script handles this. You can check that the script counted the right amount of taken vacation days:

import holidays
import yaml

state_vacation = holidays.Germany(prov='BW', years=years)
state_vacation_dates = list(state_vacation.keys())

vacation_dates = [] + state_vacation_dates
personal_vacations = yaml.load(open('personal/holidays.yaml', 'r'))
vacation_count = 0
for vacation in personal_vacations:
    date_range = pd.bdate_range(start=vacation['from'], end=vacation['to'], freq='C',
                                name=vacation['name'], holidays=state_vacation_dates)
    print('{} used up {} vacation days'.format(vacation['name'], len(date_range)))
    vacation_count += len(date_range)
    vacation_dates += date_range.tolist()
print('Total number of holiday days taken: {}'.format(vacation_count))

Snowboard in the alps used up 5 vacation days
WOOFing in an organic farm, Denmark used up 6 vacation days
Total number of holiday days taken: 11

Let’s now compute the expected hours per business day, accumulate them over time and subtract the actual worked hours. In Bade-Wurtemberg, expected work-time per business week is 39.5h. Here I explicitely take the break project out of the worked hours.

business_dates = pd.bdate_range(start, end, freq='C', holidays=vacation_dates)
business_df = pd.DataFrame(39.5 / 5., columns=['Expected hours'], index=business_dates)

worked_days = get_time_spent_per('D').drop('break', axis=1)
worked_df = pd.DataFrame(worked_days.sum(axis=1), columns=['Worked hours'])

overhours_df = pd.merge(business_df, worked_df, how='right', left_index=True, right_index=True).fillna(0)
overhours_df['Overhours'] = overhours_df['Worked hours'] - overhours_df['Expected hours']
overhours_df['Accumulated Overhours'] = overhours_df['Overhours'].cumsum()
overhours_df.plot(title="Accumulated Overhours", y=['Accumulated Overhours'])

In this plot, positive means overhours, negative means underhours.