Use Notepad++ to edit files on VM

Very similar to the previous post on working with GoPiGo files. This example is for the Robotics Nanodegree VM on VirtualBox and VMWare.

1) Install openssh-server on the VM.

robond@udacity:~$ sudo apt-get install openssh-server

2-VMWare) Open a terminal in the VM and use ifconfig to find inet addr.

2-VirtualBox) Add a port forwarding rule in the Oracle VM VirtualBox Manager.

  1. Select the VM and click settings.
  2. Select Network and click Port Forwarding.
  3. Set new rule with the following values:

Screenshot 2017-07-22 23.36.04.png

3) Open Notepad++ and add NppFTP plugin if you don’t already have it.

  1. Plugins -> Plugin Manager -> Show Plugin Manager
  2. Find NppFTP and install it.

4) “Show NppFTP Window” in the Plugins dropdown.

5) Click the gear symbol and then “Profile settings”

6) Add a new profile. Call it “Robo-ND-VM” or any other name and edit the following fields (Use inet addr for Hostname on VMWare):

Hostname: 127.0.1.1   (or VMWare inet addr)
Connection type: SFTP
Port: 2222
Username: robond
Password: robo-nd (or your new password)

Screenshot 2017-07-22 23.42.22.png

7) Connect and click yes on the warning pop-up and you are ready to edit. You might need to chmod the file permissions on the VM first.

 

Setup Notepad++ to remotely edit GoPiGo2 files

The Dexter website has instructions on how to do this but I want to summarize it again here with some small changes. https://www.dexterindustries.com/BrickPi/brickpi-tutorials-documentation/program-it/hints-and-tricks/

1) Open Notepad++ and add NppFTP plugin if you don’t already have it.

  1. Plugins -> Plugin Manager -> Show Plugin Manager
  2. Find NppFTP and install it.

2) “Show NppFTP Window” in the Plugins dropdown.

3) Click the gear symbol and then “Profile settings”

4) Add a new profile. Call it “GoPiGo” or any other name and edit the following fields:

Hostname: dex.local
Connection type: SFTP
Port: 22
Username: pi
Password: robots1234  (or your new password)
Initial remote directory: /home/pi  (or your preferred directory)

5) Authorize remote editing on Raspberry Pi.

On Raspberry Pi, open a terminal and enter this:

sudo find /home/pi -type d -exec chmod 777 {} \;

This will make everything editable. If you only want to set specific files then replace “/home/pi” with another file or folder.

Scanse.io Sweep Lidar installation on GoPiGo2

I’m writing this the day after completing the setup so I may have forgotten some part of the process. I believe it went like this:

 Image

Clone the Sweep-SDK Repository

Log in to the Raspberry PI OS. Open a terminal and clone the sweep repo.

git clone https://github.com/scanse/sweep-sdk

Install CMAKE

Install CMAKE on Raspberry Pi in order to build `libsweep.so` in *sweep-sdk*.
Replace the version number with the latest, which in my case was 3.8.1 (https://cmake.org/download/).

INSTRUCTIONS FROM: http://osdevlab.blogspot.tw/2015/12/how-to-install-latest-cmake-for.html

1. Create a folder

pi@raspberrypi ~ $ mkdir Download
pi@raspberrypi ~ $ cd Download

2. Download the compressed file and extract it

pi@raspberrypi ~/Download $ wget https://cmake.org/files/v3.8/cmake-3.8.1.tar.gz
pi@raspberrypi ~/Download $ tar -xvzf cmake-3.8.1.tar.gz

3. Compile and install cmake. This will take several minutes.

pi@raspberrypi ~/Download $ cd cmake-3.4.1/
pi@raspberrypi ~/Download/cmake-3.4.1 $ sudo ./bootstrap
pi@raspberrypi ~/Download/cmake-3.4.1 $ sudo make
pi@raspberrypi ~/Download/cmake-3.4.1 $ sudo make install

Install libsweep

INSTRUCTIONS FROM: https://github.com/scanse/sweep-sdk/tree/master/libsweep

1. Enter the libsweep directory

cd sweep-sdk/libsweep

2. Create and enter a build directory

mkdir -p build
cd build

3. Build and install the libsweep library

cmake .. -DCMAKE_BUILD_TYPE=Release
cmake --build .
sudo cmake --build . --target install
sudo ldconfig

Install SweepPy

INSTRUCTIONS FROM: https://github.com/scanse/sweep-sdk/tree/master/sweeppy

1. Go to the `sweep-sdk/sweeppy` folder where `setup.py` is located.
2. Install sweep for python. (I don’t know if both versions is necessary but that’s what I did.)

python3 setup.py install --user
sudo python setup.py install --user

Run Sample Code

1. Connect Scanse Sweep to USB. I used the center top USB port for the Scanse lidar.

2. Run some code. (I skipped trying to add the `sweep` module globally and just wrote my test file inside the `sweeppy` folder.)

from sweeppy import Sweep

with Sweep('/dev/ttyUSB0') as sweep:
    print(sweep.get_motor_speed())
    print(sweep.get_sample_rate())
    sweep.start_scanning()

    for scan in sweep.get_scans():
        print('{}\n'.format(scan))

Note:

You cannot “get” or “set” while scanning. Do it before “start_scanning()” or after “stop_scanning()”.

String formatting with dictionary-like objects

In the past, whenever I needed to format a datetime.date string, I would do something like this:

date = datetime.date.today()

#Chinese year, month and day.
u'{}年 {}月 {}日'.format(*str(date).split('-'))

#Output: u'2014年 05月 02日'

It worked but I always had a nagging feeling that it’s not clean or pretty enough. So I finally took a moment to rethink it and remembered that you can also access attributes of an object when formatting:

u'{0.year}年 {0.month}月 {0.day}日'.format(date)

#Output: u'2014年 5月 2日'

(Zero is the index of the argument passed to the format method.)

In this case, the two ways are about the same length but the second one is more readable. These are not exactly the same though. Using split in the first method will give you two-digit strings for month and day (i.e. “05”), whereas accessing the attribute only retrieves a number. To get the two-digit output from attributes, add a bit more formatting code:

u'{0.year}年 {0.month:0>2}月 {0.day:0>2}日'.format(date)

#Output: u'2014年 05月 02日'

“0>2” orders up a 2 character long string with the main value served on the right with a side of zeros (if there’s room).

Chinese in matplotlib

I recently needed to display Chinese characters in a plot legend. I’ve only ever changed font size in the past so this was new to me.

Before adding a legend, you need to create a font properties object.

from matplotlib import font_manager
...
fontP = font_manager.FontProperties()
fontP.set_family('SimHei')
fontP.set_size(14)
plt.legend(prop=fontP)
plt.show()

Working with PIL and EXIF

PIL has a way to look at a simple set of EXIF tags but cannot read the myriad of file types out there. I needed something to get everything I could out of a NIKON *.jpg file.

The solution I found was running exiftool.exe with the subprocess module.

Exiftool.exe supports many file types. Download at http://www.sno.phy.queensu.ca/~phil/exiftool/

Loading EXIF from a file into Python
To load EXIF data from an image file, use the check_output method in the subprocess module. This method will return the exiftool.exe output as a string. Then splitlines and add each line/tag to a dictionary. I prefer using OrderedDict from the collections module, but I use a regular dictionary in the code below.

import subprocess

# filename is the path to your target image file.
exifdata = subprocess.check_output(['exiftool.exe',
                                    filename], shell=True)
exifdata = exifdata.splitlines()
exif = dict()
for i, each in enumerate(exifdata):
     # tags and values are separated by a colon
     tag,val = each.split(': ', 1) # '1' only allows one split
     exif[tag.strip()] = val.strip()

You can review all the available tags from a file with exif.keys().

Getting a preview image or thumbnail from EXIF into Python
You can get a preview image from a NEF or JPG file also. Not all files will have a preview so you’ll need to handle the error. You can also retrieve the full-size jpeg that is embedded in the file.

import subprocess
import StringIO
from PIL import Image

def get_preview( filename ):
    try:
        im_binary = subprocess.check_output(['exiftool.exe',
                                             filename,
                                             '-previewimage',
                                             '-b'], shell=True)
        image = Image.open( StringIO.StringIO(im_binary) )
        return image
    except:
        return None

def get_thumbnail( filename ):
    try:
        im_binary = subprocess.check_output(['exiftool.exe',
                                              filename,
                                              '-thumbnailimage',
                                              '-b'], shell=True)
        image = Image.open( StringIO.StringIO(im_binary) )
        return image
    except:
        return None

def get_jpeg( filename ):
    try:
        im_binary = subprocess.check_output(['exiftool.exe',
                                              filename,
                                              '-JpgFromRaw',
                                              '-b'], shell=True)
        image = Image.open( StringIO.StringIO(im_binary) )
        return image
    except:
        return None

Preserving the EXIF after PIL manipulation and save
PIL does not automatically save EXIF and PIL’s exif methods do not grab everything, particularly NIKON MakerNote data.
My solution is to save a manipulated image separate from original (do not overwrite original JPG) and copy the EXIF from old to new with exiftool.

# After editing/manipulating image with PIL
savename = filename[:-4]+'_new.jpg'
image.save( savename, quality=95 ) # Default quality is 75
subprocess.call(['exiftool.exe',
                 savename,
                 '-tagsFromFile',
                 filename], shell=True)

After copying the EXIF data, delete the old image or change its name to be a backup. Use os.rename or os.remove.

Cython workflow

1)
cython_file.pyx
Create a *.pyx file containing Python or Cython code. The *.pyx file contains the code that will be compiled to C.

2)
cython_setup.py
The setup *.py file contains the code that runs the compile process.

The setup *.py file:

import numpy
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

pyx_filename = "cython_file.pyx"
new_module_name = "cython_module"

ext_modules = [Extension(new_module_name,
                         [pyx_filename],
                         include_dirs=[numpy.get_include()]
                        )]
setup(
  cmdclass = {'build_ext': build_ext},
  ext_modules = ext_modules
)

Lines 1 and 11 are necessary if you are using NumPy in the pyx file, otherwise they can be commented out.

3)
C:\...>cython_setup.py build_ext --inplace
In the command prompt from the setup file directory execute the code above. This will create a cython_file.c, a build directory, and a cython_module.pyd file.

3b)
error: Unable to find vcvarsall.bat
If you get this error. If may be because you have Visual C++ 2010 installed instead of the 2008 that it looks for. In the command prompt enter: SET VS90COMNTOOLS=%VS100COMNTOOLS%
This will redirect the search to the 2010 version.

3c)
The build directory and *.c file are not necessary and can be deleted. The *.pyd file is what you keep and import into projects.

4)
import cython_module
In a new project the C code module can be imported like any other module. The methods can be individually imported or accessed by cython_module.method()

PyCUDA fail and Cython win

I tried yet again to get PyCUDA working and failed. Getting an error that says the _driver module in pycuda is not there. The tips on installing that I’ve seen failed to solve the problem, so I’ll put this off for another time.

I can get CUDA working in C++ but I’m not too familiar with C++ and seem to always hit a wall when trying to use it. Always some package or library not found.

I did get Cython and scipy.weave working. If I need a speedup I guess this is the way to go for now.

When compiling the cython *.pyx file I got a error: Unable to find vcvarsall.bat
Typing SET VS90COMNTOOLS=%VS100COMNTOOLS% fixed this problem. A tip I found on stackoverflow that guides cython to my Visual C++ 2010 install instead of looking for 2008. After doing this, I got a different error, error: command ‘mt.exe’ failed with exit status 31 but it completed building the *.pyd file and I was then able to import and use it.

It works just like the tutorial at http://docs.cython.org/src/quickstart/build.html says except for having to enter the SET command before building.

Online backup try outs

I tried MediaFire and BitCasa recently and after a few days of waiting for them to back up a couple gigabytes of data I decided to give up. There was no indication that a single file was uploaded, so forget it. Delete.

Then I found a deal with box.net that gives you an extra 5 GB if you download the iPhone app. That’s a total of 10 GB with folder syncing. This deal ends at the end of Oct 2012.

So now I’ve added another 10 GB of free syncing to my other services:

  • 9.25 GB on Dropbox (+25 GB “temporary” SpaceRace space)
  • 25 GB on SkyDrive (previous user bonus)
  • 5 GB on Google Drive
  • 5 GB on SpiderOak
  • 10 GB on Box

I’ve been using Dropbox the longest and save my programming work and GitHub folders there. The version history/recovery has already saved me a couple times when a program file I’m working on blanked out during a crash. And they give extra space to students – https://www.dropbox.com/spacerace?r=NTQ5NzIyNjg4OQ

I use SkyDrive for photos mostly. I like the look of the online image viewer. I also use hotmail, so it is easy to send friends a link to my photos. Service comparisons that I’ve seen online don’t mention that SkyDrive’s versioning is only for MS documents. It doesn’t store old versions of any other documents.

SpiderOak allows you to sync any folder so I have it backing up my desktop and a couple program config files.

I haven’t found a specific use for Google Drive yet. I like Google’s online spreadsheet more than MSN’s online Excel just because it will auto-scroll if you are highlighting cells while dragging the mouse off the page. It’s a subtle but important feature that MSN overlooked.

I don’t know what I will do with Box yet. (I think the best way to use multiple syncing services is to have a different purpose for each one.)

These are all syncing services. I decided to stay away from the backup services for now, especially the free unlimited space variety. You have to actively manage your backups and you never know when they may delete your account due to inactivity or a change in their payment policy, because they don’t declare that any space is permanently yours like the other services I use. If I really need a lot more space, I will probably go with SkyDrive and just use the extra space for pictures anyway. Also, SkyDrive had the best pricing the last time I compared services.