Sunday, 4 March 2012

Ruby: Exceptions and Continuations

Years ago, I stumbled across a paper describing user interface continuations. At the time, the concept of continuations seemed like nothing short of wizardry: the program would be at one place, processing instructions, and then would suddenly be somewhere else to collect a piece of data, and the back at the original place, with that collected data available to the computation that was happening originally.


A nice theoretical exercise, but that could never be useful, right?
Well, since then, I've learned Common Lisp, and then its sibling, Scheme. Continuations aren't a part of Common Lisp, but they're readily available in Scheme and, looking back, CL's awesome condition system starts to look like a regular exception framework with continuations thrown in for some fun.


So far, this is all sounding somewhat academic: papers and arcane languages don't have anything to do with what programmers do on a day to day basis, right?


Ruby is a modern scripting language, and supports continuations out of the box: they're built right into the Kernel module as the callcc method (the method name has been lifted from Scheme, where it's called call/cc, or call-with-current-continuation, if you like typing).


This method takes a one-arg block, where the argument is supplied by the system and represents the current continuation, which is a representation of where the program is at the moment that it is created. So what can you do with that? Well, Continuations can be call'ed, and when they are the value of the callcc call becomes the value that the continuation is call'ed with, regardless of where in the program the call was made. The continuation is a regular object that can be stored in data structures, passed around, etc., but when it's invoked, program flow resumes from the site of the callcc.


This needs an example.


I mentioned Common Lisp's condition system earlier. It's analagous to the exception mechanism in languages like Java and Python, with one notable difference: when a condition is signalled, the stack is not unwound to an enclosing exception handler. Instead, the stack is searched for a handler, which then gets to look at the condition and, if it determines that there is remedial action that can be taken, can provide information to the exception site that can tell the code there how to proceed. These are called restarts.


Where would this be useful?
A simple example from Peter Seibel's Practical Common Lisp is a log file parser. Imagine you're writing this, and you've sensibly layered the different functions: from the abstract 'parse a log file with this filename', through 'process all log entries in the file', into 'extract a single log entry', and then 'analyse a single log entry'.


But what if something goes wrong at the analysis level? What do you do with a malformed entry? If you're working with functions, you just have to pass something into the function that tells it how to handle that. If it's an object, set some property on the object for this situation. But what if it is, as in this case, several layers down from the application's interface? Well, you can pass some property dictionary or other miscellaneous contextual information into either the intervening functions or objects.


This kind of action indicates a break in reasoning: you're setting or passing properties on something that really shouldn't need to care about their existence. This kind of clutter makes maintenance programming difficult, as objects and functions are littered with things that they don't use themselves, but instead are made aware of for the sole purpose of handing over to something else. This has adverse effects upon reusability, as it's now assumed that these objects are part of a particular call chain.


Looking at it, the only layers that need to know about the problem are the bottom one, where the problem occurs, and the top one, where the business logic lives.


With continuations, we can make this happen. Here's an example.


  class Condition < Exception
    attr_accessor :continuation, :payload
    def initialize(continuation, payload)
      self.continuation = continuation
      self.payload = payload
    end
    def continue(value)
      @continuation.call(value)
    end
  end
  
  def topLevel
    begin
      intermediateLayer1
    rescue Condition => c
      c.continue(0 - c.payload)
    end
  end
  
  def intermediateLayer1
    intermediateLayer2
  end
  
  def intermediateLayer2
    intermediateLayer3
  end
  
  def intermediateLayer3
    fragileLayer
  end
  
  def fragileLayer
    (1..5).each { |i|
      i = callcc { |cc|
        begin
          processEntry(i)
        rescue Exception
          raise Condition.new(cc,i)
        end
      }
      puts i
    }
  end
  
  def processEntry(entry)
    (entry % 2 == 1) ? (raise "Can't deal with odd numbers!") : entry
  end
  
  topLevel()


The purpose of this program is quite simple: a top-level caller gets some work done (in this case, printing the numbers from 1 to 5) by asking a lower layer. The intermediate layers exist to demonstrate that there's no direct linkage between the raiser of the exception and its handler.


In this example, the bottom layer refuses to work with odd numbers, and raises an exception when given one. This is caught, but the decision as to what to do next is not appropriate for that low level: the business logic needs to make that decision, but it's several layers up in the stack.


At this point, a continuation is captured with callcc, and a new ContinuableException is raised. There's nothing special about these objects: they just encapsulate the continuation and the data that caused the error.


Normally, an exception propagating up the stack causes the intermediate stack frames to become inaccessible and therefore eligible for garbage collection. However, the continuation captured in the exception that's just been thrown refers the stack frame in which it was created, so the stack remains live, even if control flow is being unwound through it.


Now, the wizardry: the top level handler has access to the continuation and the problematic value, so it can decide what to do next. It can re-raise the exception, or it can provide a new value to the original source of the exception to be used in its place. Continuations can be call'ed, and they take a value to treat as the return value of the callcc call. Lower-level processing can continue as though it hadn't been interrupted; the intermediate layers are not unwound or invoked again.


The output of the above is just:

-1
2
-3
4
-5


Now, there's no need to follow this precise pattern. The value that's returned could instead be a Symbol that indicates which of a range of choices should be executed. It could be a Proc, which the receiver is expected to call.
Pretty neat, huh?

Saturday, 25 February 2012

Visual Studio 11's New Look

Microsoft recently unveiled a proposal for a new look on Visual Studio 11. Now, I'm hardly a graphic designer, but one thing repeatedly strikes me as a bit bonkers in software written in the past ten years, and it's not restricted to Visual Studio: the 'save' and 'save all' icons.


Do you see that? They're floppy disks. I don't know if their continued representation is maybe indicative of an aging population of computer programmers, but floppy disks haven't been relevant since the turn of the millenium.

I wonder if new up-and-coming programmers even know what they're clicking?

Saturday, 18 February 2012

Mac DVDRipper Pro

I have a Mac mini under my TV, with an NFS mount to 2TB of RAID-1 space on the other end of a gigabit ethernet home network. So it makes perfect sense to get rid of physical DVDs, putting them in a box in the loft after ripping them into digital form. So I've been looking for software that does this well, and stumbled across Mac DVD Ripper Pro a few days ago. It even managed to rip/transcode some troublesome DVDs that seem to resist other means of doing the same.

MDRP offers an on-the-fly transcode feature which looks to be built on HandBrakeCLI.


All things considered, it's a pretty good converter: simple, slick and reliable. That said, I'm not sure how its proprietary license works with the GPL'd HandBrake underneath it.

Update 19-Apr-2012: I ended up not really bothering with transcoding my DVDs: the storage on the NAS is such that I could avoid the re-coding overhead and just rip the VOBs directly with MPlayer:

mplayer dvd://1//dev/sr0 -dumpstream -dumpfile "Some DVD Title.ps"

This way, I get all the audio and subtitle streams, and with the help of lsdvd can pick out just the movie track (which MDRP can do as well).
One potential gotcha is to remember, particularly with subtitled/multi-audio movies, is to grab the .INF files from the DVD, too. They contain the names of the tracks and the palette used in rendering the subs; the subs may render strangely without this information.

Sunday, 12 February 2012

Shell Scripting: Counting Occurences of a Character

I recently found myself needing to know the occurrences of a letter in a line from a shell script. I was working with a delimited file, and needed to know how many columns were in a given line (it would be consistent within the same file, but could differ between files, depending upon the version of the code that produced it).

Surprisingly, it took a bit more digging than I thought would be needed for such a simple task. Counting lines is easy, but characters within a line? It's not a difficult thing to do in Perl or awk, but launching their respective interpreters seemed a bit heavyweight.

tr to the rescue. This under-used command line utility translates from one character set to another (a set can just be a single character). It can also delete characters from its input and, using the -c (complement) switch, can work on a set that's the inverse of the one specified. Tying those loose threads together, you end up with this to pick out occurrences of the letter 'e':


danny@khisanth ~ [2] % echo one two three four five | tr -cd 'e'
eeee%


That % is my shell indicating that the line didn't terminate with a newline, so it was just the 'e's. Once you have those, wc -c can do the rest:


danny@khisanth ~ [3] % echo one two three four five | tr -cd e | wc -c
4


Annoyingly, wc on some Unixes indents its output, so if you just want the number itself, you might need to play with your shell's string manipulation functions to get something neater. e.g., in zsh:


danny@khisanth ~ [4] % echo ${$(echo one two three four five | tr -cd e | wc -c)// /}
4

Wednesday, 29 June 2011

Gran Turismo 5 (eventually, I'm assuming)

I'm occasionally a bit behind the curve when it comes to gaming. Given the appalling state of some titles on release day, I've found that it's quite often sensible to wait a while until the first few patches have come out to address the glaring problems that the initial 'gold' build shipped with regardless.

So it's not until today that I got a copy of Gran Turismo 5 for the PS3 (pre-owned). I shove the disk in, and immediately have to download the better part of three quarters of a gigabyte of patches. As soon as this non-backgroundable download starts, I fire up the PS2, switch to component input, restore a save of Shadow of the Colossus, and then spend the next half hour beating the 9th colossus. I save that game, and switch back to the PS3 on HDMI in. The patches still haven't finished downloading, so I switch to DVI in and boot the Linux box, do some simple updates and routine administration, then check in on the PS3 again. Hey, all the patches are down! Maybe I'll get the play the game.

Or maybe not: right at the start, the game carries the advice that installing 8GB to hard disk can speed up loading times. Now, if the game developers felt the need to admit that, you know that there are going to be problems, so I click 'yes', and see an estimated time of a paltry 14 seconds. Good! Oh, wait: that's the preparation time before it can start installing: the actual estimate is more like 30 minutes.

So, I take the time to write this blog post, and it's still installing.

What the hell happened to console gaming?

Saturday, 25 June 2011

Console Gaming, Rebooted

I've become a bit disillusioned with console gaming over the past few years. Many hours of my late teens were spent on the original PlayStation on classics like Final Fantasy VII & VIII and the various incarnations of Resident Evil. The upgrade to the PlayStation 2 in my early twenties was a foregone conclusion, offering much the same experience, but with better graphics and DVD playback as an added bonus. But the PS3 has never quite captured me in the same way. Part of that is undoubtedly me simply getting older. I'm married, have a daughter, and run my own consultancy company: it's only natural that some of the less important things like computer games slip a bit.

Still, throughout each generation of console, I've always had at least a background flirtation with PC games. Some games (first person shooters springing immediately to mind) are just out-and-out better on a PC. Something a simple as mouse vs. control pad become critically important when you want to be able to turn quickly to find out what's shooting you. Other games (RPGs like Dragon age) are again just better on a PC largely because navigating the intricate menu systems of such games with a d-pad is infuriatingly slow. Thus, while games like Infamous and God of War 3 on the PS3 are undoubtedly best on their home platforms, the shiny black console spent most of its time languishing unused under the TV, while its forgotten dad, the PS2, was in a damp-proof box with its games in the attic.

Anyway, I bought a new monitor/TV earlier this week: finally time to retire the 4:3, 1280x1024 and catch up with the 1920x1080 high-def kids. One of the major selling points for this particular monitor was the sheer number of connectivity offerings on its back. My PC displays via DVI; I have a small, VIA-based server that has a VGA port for the occasional time (i.e. backups) when I need local terminal access; and there's an HDMI port for the aforementioned PS3. But there's also component input, which made me think of the old PS2. Indeed, why not hook both it and the PS3 up to the same display, just for kicks?

Well, since I've done that, I've played the PS2 more than its younger, more powerful offspring, and it's reminded me of why console gaming used to be fun: it was just so simple. Shove in the disk, and go.

With the PS3, turning it on assaults you with a menu of menus, each of which contains a dizzying number of options. If you're an infrequent player like me, it seems like you're being prompted for a firmware update just about every time you turn the damn thing on, and if you play a game that you've not touched in a while, there's every possibility that you'll be looking at anything been 100MB and 1GB of patches to fix the problems caused by rushing the original version out the door.

One of the great things about the PS1 and PS2 is that they didn't have internet connectivity, so game studios had to be very, very sure that the software they committed to CD or DVD was as bug free as possible. With the PS3 (and Xbox360—don't think I'm picking on the PS3 in particular), just about any old crap can be thrown together, to be patched later when the early adopters—devoted fans that a studio should treat especially well—are used as unpaid acceptance testers.

So I'm holding off on purchasing Infamous 2 for the PS3, and instead having fun with Shadow of the Colossus on the PS2. I think I should be able to get through that and Silent Hill 2 before I see if I can stomach the hassles of modern console gaming, or whether I just ditch the platform and stick to the PC.

Thursday, 10 March 2011

Continuously Learning Lisp

I like to think that I 'know' Perl. I also 'know' C, Java, Smalltalk, Ruby and a handful of other languages. But Lisp is the only one that continually makes me feel like a newbie. Just when you think you've got it pegged... BLAM— you read something that shows you a completely new dimension.

A good example of this for me is Nikodemus Siivola's post on "Optimizing Lookup Functions Using LOAD-TIME-VALUE", basically demonstrating a way of altering the language such that hash table lookups with keys that are known to be constant at compile time have no lookup overhead at runtime.

I feel like I have a whole lot to learn, and it's good :-)