Showing posts with label javascript. Show all posts
Showing posts with label javascript. Show all posts

Monday, 18 April 2011

piefull - a simple javascript & canvas visualisation tool

OK, so calling piefull a visualisation tool is going a bit over the top, but it is a tool, and it does help with visualisation. It does one thing and one thing only - plot pie-charts indicating a single value. And it's even more restricted than that - the value it plots needs to be a percentage value.

The main use cases for this are things like task completion status (project and outliner applications, test coverage, etc) or resource allocation (disk space, CPU usage).

By choosing contrasting colours (these are configurable in piefull), the the overall outlook can be ascertained by a glance from a distance. By default the charts it generates are fairly small - 24px - which allows them to be used in-line in text, or as entries in tables. The general approach is derived from the sparklines example given in David Flanagan's Canvas Pocket Reference - replacing some textual data with a pictorial equivalent in the hope(!) it will be more quickly understandable. Of course this approach also lends itself well to graceful degradation, as the data is already there in the document itself.

There are plenty of other pie-chart generators - after all it's a fairly trivial thing to write with HTML5 canvas elements. But most of these tend to be fairly complex, with lots of options. I needed something where a fixed size display could represent a single value clearly, and piefull is the result.

Basically piefull turns this:

10%
20%
33%
20%
10%
18%
55%
33%
23%
12%
14%
35%
40%
21%
11%
12%
29%
14%
11%
5%
12%
17%
10%
9%
5%

(which looks like this in code:)

<table class="piefull">
    <tr>
       <td><div>10%</div></td>
       ...
       <td><div>5%</div></td>
    </tr>
</table>                 

into this:

10%
20%
33%
20%
10%
18%
55%
33%
23%
12%
14%
35%
40%
21%
11%
12%
29%
14%
11%
5%
12%
17%
10%
9%
5%

by doing this:

<script>
  window.onload = (function() { piefull.main("table.piefull td div"); });
</script>

where 'table.piefull td div' is a selector passed to querySelectorAll() to locate elements which will be replaced by little canvas piecharts. The contents of the selected elements are matched against a regular expression looking for a percentage value to extract (generally speaking, the first number), and the element is replaced with a canvas element displaying that value. The classes and id of the original element are preserved in the new element, allowing sensible CSS styling, and the canvas title takes on the text which it has replaced. As well as the selector, there are a (small) number of other parameters - size, 'yes' and 'no' colours. A value of e.g. 10 will display a 10% pie-segment in the 'yes' colour - the remainder will be the 'no' colour. (Like this: 10%.) These are all optional - even the selector defaults to '.piefull', which works great for a small number of spans or divs in some prose:

In other news, at least 66% of statistics are made up. No, wait - it should be a little higher now.

Note: If you're viewing this on IE8 or below, this probably makes no sense, as I've not included excanvas here. It is supported for IE8 with excanvas (but not less than IE8). One gotcha with IE is that block-level elements such as canvas don't work inside <p> elements. But you know you want to get a better browser... And why not get one with webgl support while you're at it?

The code for piefull can be found here

Friday, 31 December 2010

HTTP 'streaming' from Python generators

One of the more annoying things about HTTP is that it wants to send things in complete chunks: you ask for an object with a particular URL, and some point later you get that object. There isn't a lot you can do (at least from JavaScript) until the complete resource has loaded. For more fine grained control, it's a bit annoying.

Of course web sockets will solve all of this (maybe) once the spec has gone through the mill a few more times. But in the mean-time there is often an impedance mismatch between how we'd like to be able to do things on the server-side and how we are forced to do them because of the way HTTP works.

The following is an example of one way to manage splitting things up. It allows Python generators to be used on the server side and sends an update to the client on every yield, with the client doing long-polling to get the data. This shouldn't be confused with CherryPy's support for yield streaming single responses back to the server (which is discouraged) - the yield functionality is hijacked for other purposes if the method decorator is applied. Also note that this is only of any help for clients which can use AJAX to repeatedly poll the generator.

Example

Let's suppose that we want to generate a large (or infinite) volume of data and send it to a web client. It could be a long text document served line-by-line. But let's use the sequence of prime numbers (because that's good enough for aliens). We want to send it to the client, and have it processed as it arrives. The principle is to use a generator on the server side rather than a basic request function, but wrap that in something which translates the generator into a sequence of responses, each serving one chunk of the response.

Server implementation using CherryPy - note the json_yield decorator.

import cherrypy

class PrimeGen(object):
    @cherrypy.expose
    def index(self):
        return INDEX_HTML # see below

    @cherrypy.expose
    @json_yield   # see below
    def prime(self):
        # this isn't supposed to be efficient.
        probe = 1
        while True:
           for i in range(2, probe):
               if probe % i == 0:
                   break
           else:
               yield probe
           probe += 2

cherrypy.quickstart(PrimeGen)

The thing which turns this generator into something usable with long-polling is the following 'json_yield' decorator.

Because we might want more than one such generator on the server (not to mention generator instances from multiple clients), we need a key - passed in from the client - which associates a particular client with the generator instance. This isn't really handled in this example, see the source file download at the end of the post for that.

The major win is that the client doesn't have to store a 'next-index' or anything else. State is stored implicitly in the Python generator on the server side. Both client and server code should be simpler. Of course this goes against REST principles, where one of the fundamental tenets is that state should be stored only on the Client. But there is a place for everything.

import functools
import json

def json_yield(fn):
    # each application of this decorator has its own id
    json_yield._fn_id += 1

    # put it into the local scope so our internal function
    # can use it properly
    fn_id = json_yield._fn_id

    @functools.wraps(fn)
    def _(self, key, *o, **k):
        """
        key should be unique to a session.
        Multiple overlapping calls with the same 
        key should not happen (will result in
        ValueError: generator already executing)
        """

        # create generator if it hasn't already been
        if (fn_id,key) not in json_yield._gen_dict:
            new_gen = fn(self, *o, **k)
            json_yield._gen_dict[(fn_id,key)] = new_gen

        # get next result from generator
        try:
           # get, assuming there is more.
           gen = json_yield._gen_dict[(fn_id, key)]
           content = gen.next()
           # send it
           return json.dumps({'state': 'ready',
                              'content':content})
        except StopIteration:
           # remove the generator object
           del json_yield._gen_dict[(fn_id,key)]
           # signal we are finished.
           return json.dumps({'state': 'done',
                              'content': None})
    return _
# some function data...
json_yield._gen_dict = {}
json_yield._fn_id = 0

The HTML to go with this is basic long-polling, and separating out the state from the content. Here I'm using jQuery:

INDEX_HTML = """
<html>
<head>
<script src="http://code.jquery.com/jquery-1.4.4.min.js">
</script>
<script>
$(function() {

  function update() {
    $.getJSON('/prime?key=1', {}, function(data) {
      if (data.state != 'done') {
        $('#status').text(data.content);
        //// alternative for appending:
        // $('#status').append($('
'+data.content+'
')); setTimeout(update, 0); } }); } update(); }); </script> </head> <body> <div id="status"></div> </body> </html> """

Uses

The example above is contrived, but there are plenty of possibilities if the json_yield decorator is extended in a number of ways. Long running server side processes can send status information back to the client with minimal hassle. Client-side processing of large text documents can begin before they have finished downloading. One issue is that a chunk of the data should be semantically understandable. Using things like this on binary files or XML (where it is only valid once the root element is closed) won't have sensible results.

There are plenty of possibilities in extending this; the decorator could accumulate the content (rather than the client) and send the entire results up-to-now back, or (given finite memory) some portion of it using a length-limited deque. Additional meta-data (e.g. count of messages so far, or the session key) could be added to the JSON information sent to the client each poll.

Disclaimer: I've not done any research into how others do this, because coding is more fun than research. There are undoubtedly better ways of accomplishing similar goals. In particular there are issues with memory usage and timeouts which aren't handled with json_yield. Also note that the example is obviously silly - it would be much faster to compute the primes on the client.

Download Files

Sunday, 26 September 2010

Non-evil SEO?

I've always thought SEO was evil (in the non-theological sense!), but this effort (via @rem) seems well worth supporting. According to my Facebook page my favourite quote is 'It is amazing what you can accomplish if you do not care who gets the credit' - so said Harry Truman. Perhaps amazing is overstating the goal, but maybe it could be paraphrased 'It is amazing what you can make Google do if it is actually helpful and not (directly) self-serving'. I guess we'll find out...

JavaScript JS Documentation: JS Function apply, JavaScript Function apply, JS Function .apply, JavaScript Function .apply

Tuesday, 17 August 2010

New website - codedstructure.net

I've not got round to putting much on here for a while, but I have been busy, honest! I've finally got my own website after far too many years of not bothering: codedstructure.net.

I'm probably not the first to use scrolling like that, but I wasn't copying anyone and I think it's quite funky, especially the nice text shadows. Me likes the way web development is going, after years of shunning it...

Oh, and while I'm at it, I've got a twitter account, and hope to be churning out iPhone apps in the near future - though Objective C looks horribly like some terrible collision of Java and Smalltalk - in a bad way. Hope I can grow to like it.

Tuesday, 8 June 2010

Mandelbrot Canvas...

This Mandelbrot Plotter is something I worked on a little while ago. One of the things I want to do in any language I'm learning is know how to get a pixel on a page. A long time ago it was function 0Ch of INT 10h, then memory mapped displays, and now the wonderful ImageData object underlying HTML5's Canvas element. It's so much more fun displaying a picture than 'Hello World' in text...

I'm also fascinated by fractals, and the Mandelbrot set in particular (since I sort-of understand the maths behind it). It makes me wonder whether it is an invention or a discovery, and just why it is what it is - and whether other inscrutable patterns are just waiting to have some new visual representation cast a whole new light on them...

Of course with it being hosted (albeit slightly oddly on Blogger), I can now claim to have written a 'web app', for better or worse. Anyway, it's nice to be able to put random JavaScript up within Blogger...