Showing posts with label The-UNIX-Programming-Environment. Show all posts
Showing posts with label The-UNIX-Programming-Environment. Show all posts

Friday, October 24, 2014

Print selected text pages to PDF with Python, selpg and xtopdf on Linux

By Vasudev Ram


Image

In a recent blog post, titled My IBM developerWorks article, I talked about a tutorial that I had written for IBM developerWorks a while ago. The tutorial showed some of the recommended techniques and practices to follow when writing a Linux command-line utility that is intended for production use, and how to write it in such a way that it can easily cooperate with existing UNIX command-line tools, when used in a UNIX command pipeline.

This ability of properly written command-line tools to cooperate with each other when used in a pipeline, is, as I said in that IBM article, one of the keys to the power of Linux (and UNIX) as a development environment. (See the classic book The UNIX Programming Environment, for much more on this topic.)

The utility I wrote and discussed (in that IBM article), called selpg (for SELect PaGes), allows the user to select a specified range of pages from a text file. At the end of the aforementioned blog post, I had said that I would show some practical uses of the selpg utility later. I describe one such use case below, involving a combination of selpg and my xtopdf toolkit), which is a Python library for PDF creation.

(The xtopdf toolkit contains a PDF creation library, and also includes some sample applications that show how to use the library to create PDF output in various ways, and from various input sources, which is why I tend to call xtopdf a toolkit instead of just a library.

I had written one such application of xtopdf a while ago, called StdinToPDF(.py) (for standard input to PDF). I blogged about it at the time, here:

[xtopdf] PDFWriter can create PDF from standard input. (PDFWriter is a module of xtopdf, which provides the core PDF creation functionality.)

The selpg utility can be used with StdinToPDF, in a pipeline, to select a range of pages (by starting and ending page numbers) from a (possibly large) text file, and write only those selected pages to a PDF file. Here is an example of how to do that:

First, build the selpg utility from source, for your Linux OS. selpg is only meant to work on Linux, since it uses some Linux C standard library functions, such as from stdio.h, and popen(); but you can try to run it on Windows (at your own risk), since Windows does have (had?) a POSIX subsystem, from Windows NT onward. I have used it in the past. (Update: I checked - according to this section of the Wikipedia article about POSIX, Windows may have had POSIX support only from Windows NT up to Windows 2000.) Anyway, to build selpg on Linux, follow the steps below (the $ sign is the shell prompt and not to be typed):

1. Download the source code from the sources section of the selpg project repository on Bitbucket.

Download all of these files: makefile, mk, selpg.c and showsyserr.c .

2. Make the (shell script) file mk executable, with the command:
$ chmod u+x mk
3. Then run the file mk, with:
$ ./mk
That will run the makefile that builds the selpg executable using the C compiler on your Linux box. The C compiler (invoked as cc or gcc) is installed on most mainstream Linux distributions. If it is not, you will need to install it from the repository for your Linux distribution. Sometimes only a minimal version of a C compiler is installed, which is only enough to (re)compile the kernel after making kernel parameter changes, such as for performance tuning. Consult your local Linux expert for help if such is the case.

3. Now make the file selpg executable, with the command:
$ chmod u+x selpg
4. (Optional) You can check the usage of selpg by reading the IBM tutorial article and/or running selpg without any command-line arguments:
$ ./selpg
which will show a usage message.

6. (Optional) You can run selpg a few times with some text file(s) as input, and different values for the -s and -e command-line options, to get a feel for how it works.

Now download xtopdf (which includes StdinToPDF) from here:

xtopdf on Bitbucket.

To install it, follow the steps given in this post:

Guide to installing and using xtopdf, including creating simple PDF e-books

That post was written a while ago, when xtopdf was hosted on SourceForge. So you need to make one change to the instructions given in that guide: instead of downloading xtopdf from SourceForge, as stated in Step 5 of the guide, get it from the xtopdf Bitbucket link I gave above.

(To make xtopdf work, you also have to install ReportLab, which xtopdf depends uses internally; the steps for that are given in my xtopdf installation guide linked above, or you can also look at the instructions in the ReportLab distribution. It is easy, just a couple of steps - download, unzip, configure a setting or two.)

Once you have both selpg and xtopdf installed, you can use selpg and StdinToPDF together. Here is an example run, to select only pages 2 through 4 from an input text file:

I wrote a simple Python program, gen_selpg_test_file,py, to create a text file that can be used to test the selpg and StdinToPDf programs together.

Here is an excerpt of the core logic of gen_selpg_test_file.py, omitting argument and error handling for brevity (I have those in the actual code):

# Generate the test file with the given filename and number of lines of text.
    try:
        out_fil = open(out_filename, "w")
    except IOError as ioe:
        sys.stderr.write("Error: Could not open output file {}.\n".format(out_filename))
        sys.exit(1)
    for line_num in range(1, num_lines + 1):
        line = "Line #" + str(line_num).zfill(10) + "\n"
        out_fil.write(line)
    out_fil.close()
I ran it like this:
$ python gen_selpg_test_file.py selpg_test_file_1000.txt 1000
to generate a text file with 1000 lines, in the file selpg_test_file_1000.txt .

Then I could run the pipeline using selpg and StdinToPDF, as described above:
$ ./selpg -s2 -e4 selpg_test_file_1000.txt | python StdinToPDF.py p2-p4.pdf
This command extracts only the specifed pages (2 to 4) from the input file, and pipes them to StdinToPDF, which converts those pages only, to PDF, in the filename specified at the end of the command.

After doing the above, you can open the file p2_p4.pdf in your favorite PDF reader (Evince is one PDF reader for Linux), to confirm that it contains all (and only) the lines from page 2 to 4 of the input file selpg_test_file_1000.txt (considering 72 lines per page, which is the default that selpg uses).

Read the IBM article to see how that default can be changed - to either another number of lines per page, e.g. 66 or 80 or whatever, or to specify form feeds (ASCII code 12) as the page delimiter. Form feeds are often used as a page delimiter in text file reports generated by programs, when the reports are destined for a printer, since the form feed character causes the printer to advance the print head to the top of the next page/form (that's how the character got its name).

Though this post seemed long, note that a lot it was either background information or instructions on how to build selpg and install xtopdf. Those are both one time jobs. Once those are done, you can select the needed pages from any text file and print them to PDF with a single command-line, as shown in the last command above.

This is useful when you printed the entire file earlier, and some pages didn't print properly because the printer jammed. Just use selpg with xtopdf to print only the needed pages again.


Image

The image above is from the Wikipedia article on Printing, and titled:

Jikji, "Selected Teachings of Buddhist Sages and Son Masters" from Korea, the earliest known book printed with movable metal type, 1377. Bibliothèque Nationale de France, Paris

- Enjoy.

- Vasudev Ram - Dancing Bison Enterprises

Click here to get email about new products from Vasudev Ram.

Contact Page

Monday, October 28, 2013

Python can run .c files


By Vasudev Ram

Python can run .c files - I'm not joking.

I was taking another look at the vr_debug function mentioned in my previous post:

A simple Python debugging function.

To run the program, I was going to type:

python vr_debug.py

Using tab completion (similar to what bash on UNIX has) at the Windows command prompt, I accidentally tabbed twice after typing "python vr_", which expanded to vr_debug.py~ (note the ~ character at the end of the filename) instead of vr_debug.py (the file ending in .py~ is a Vim backup file - I use GVim for Windows as my text editor whem programming on Windows).

Since I type fast, I hit Enter before I realized it was the wrong filename. Was surprised to see that Python ran the file, with no error message (like 'not a .py file) and gave the same output as the original .py file.

To check things a bit more, I copied vr_debug.py to a file called just vr_d and ran it with python. Again, it worked.

To check more, I copied vr_d to vr_d.c and ran that .c file with python. It also worked.

The output of the two runs (using Python) of non-dot-py files (one with no extension, one with a .c extension) is below. You can see that there is no error message, and the output is the same as from running vr_debug.py, as shown in my earlier blog post.

C:>python vr_d
z
a : 1
b : 1 'hi'
c : 1 'hi' True
d : 1 'hi' True [2, 3]
d : 1 'hi' True [2, 3] {'a': 'apple', 'b': 'banana'}

C:>copy vr_d vr_d.c
1 file(s) copied.

C:>python vr_d.c
z
a : 1
b : 1 'hi'
c : 1 'hi' True
d : 1 'hi' True [2, 3]
d : 1 'hi' True [2, 3] {'a': 'apple', 'b': 'banana'}

So Python can actually run .c files.

Of course, to be precise, as Thomson and Thompson might say, I should mention that Python can only run .c files if they contain Python code, not C code ... :-)

I'm guessing this works because Python was at least partly influenced by UNIX, and on UNIX, at least shell scripts don't need to have an extension of .sh - they can be called anything. There are many examples of this in the classic UNIX book, The UNIX Programming Environment, by Kernighan and Pike (K&P), which is one of my favorite computer books of all time. You don't need to have the K&P book to check that, of course; just create any simple shell script on Linux (without a .sh or any other extension in the filename) and then run it, after making it executable.

Interestingly, while Googling for "the unix programming environment" (in order to get the Wikipedia link to the book, which I used in the previous paragraph), I noticed that the moment I typed "the unix", Google showed me a list of commonly searched phrases beginning with those two words, and "the unix programming environment" was at the top of that list. See the screenshot below. That phrase even comes before the phrase "the unix operating system".

Image
- Vasudev Ram - Dancing Bison Enterprises

Consulting / training inquiry




Free shipping with purchase of Inc or LLC