• [1] F. Jiang, X.-Y. Liu, H. Lu, R. Shen. Efficient multi-dimensional tensor sparse coding using t-linear combinations. AAAI, 2018.

• The version of Python is in the repository ./2DSC. Please run main.py to start. And the version of Tensorflow is in the repository ./2DSC/tensorflow. Please run train.py to start.

• The codes are in the repository ./TGAN.

Basicly, we select MNIST dataset as an example.

• Firstly, run GAN.py to train a generative model which is used to generate low-quality tensors with size 14 * 14 * 7 from a random distribution.
• Secondly, run gen_training_data.py, which is used to generate training data consists of pairs of low-quality images and high-qulity images. As downscaled from the original MNIST images with size 28 * 28, the low-quality image is of size 14 * 14 * 7, which is concatenated by single images shifted with different pixels.
• Then, run ./SR/D_Training.m to train a combined tensor dictionary.
• Lastly, run gen_test_data.py to use GANs to generate low-quality images. And run ./SR/Demo_SR.m to generate high-qulity images.

• Dataset: In order to adapt the data to our tensor theory, the MNIST dataset are transformed to the image tensors (HT) with size 28 * 28 * 7, concatenating 7 images which are shifted from the same single image with different pixels. Then downscaling the HT with 2x, we get low-resolution image tensors (LT), which is of size 14 * 14 * 7.

• Low-resolution images generating: A DCGAN with gradient penalty is applied to generate low-resolution images (LS) with size 14 * 14 * 1 from a latent vector with size 128 * 1. Similarly, we concatenate 7 shifted LIs into tensors with size 14 * 14 * 7, named LTS.

• Tensor dictionary training: HT and LT are used to train a high resolution dictionary (HD) and a low resolution dictionary (LD) which have a same shape. With the same sparse coefficients caculated by the LD, a high-resolution image tensor (HTS) with shape 28 * 28 * 7 is generated from the low-resolution image tensor sample (LTS). For example, we set the parameters patch_size=4, step=1, n_samples=10000, r=128. As is sampled from HT by small tensor blocks with strides, the high-resolution input data (HI) is of size 16 * 10000 * 4. Extracting 6 kinds of features from LT and reshape it by blocks, the low-resolution input (LI) data transformed from LT is of size 96 * 10000 * 4. According to the input data shape, the HD is of size 16 * 128 * 4, and the LD is of size 96 * 128 * 4. In order to keep the consistensy between high- and low-resolution sparse coefficients with convenience, concatenating the LI and HI to training data with size 112 * 10000 * 4, as well as concatenating HD and LD to the dictionary with size 112 * 128 * 4, we alternately train the dictionaries and coefficients, which are of size 128 * 10000 * 4.

• Dimensional variation: latent vector (128 * 1) →(DCGAN) LS (14 * 14 * 1) → LTS (14 * 14 * 7) →(SR) HTS (28 * 28 * 7)

We have tested the high-quality image generation from a low-resolution image via tensor combined dictionary.

And the following pictures are the generated handwritten numbers the model. The above pictures are generated from latent representations by a generator, which is composed of an adversarial autoencoder networks. The pictures at the bottom are generated by our tensor superresolution (essentially tensor sparse coding) from the above pictures.