Friday, December 23, 2011

lein uberjar ClassNotFoundException

This is probably documented somewhere, but it didn't come up in my searches and it took me forever to find out the cause of the problem so I'm posting it here for the greater good, general advancement of humankind and other worthy reasons.

I had a normal leiningen project with the :main attribute set in project.clj

(defproject clojure-diff "1.0.0-SNAPSHOT"
  ; Dependencies etc omitted for clarity
  :main clojure-diff.server
)

It runs fine using lein run but when I ran lein uberjar and then executed the jar file, I got a class not found error.

$ java -jar clojure-diff-1.0.0-SNAPSHOT-standalone.jar
Exception in thread "main" java.lang.NoClassDefFoundError: clojure_diff/server
Caused by: java.lang.ClassNotFoundException: clojure_diff.server
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)


The solution is to specify :gen-class when defining the main namespace.  This means a .class file will be generate ahead of time when compiling the jar.  Obvious when you know how!

(ns clojure-diff.server
  (:gen-class)
  ; more namespace setup)

; .. stuff

(defn -main []
  (run-jetty #'clojure-diff.server/app {:port 8080}))


Tuesday, August 23, 2011

Rawk

I've created a gem called rawk.  It's a ruby DSL for command-line processing stream processing based on the classic AWK utility.   I wrote it because I like using awk, but I find it's lack of useful data structures a bit of a pain.  Rawk provides the same block-based processing, but using good old ruby.

Let's take an example.  I want to pretty-print the name and last modification date from a directory where the  filename starts with 'd'


$ ls -l /bin | rawk '
  start do 
    puts %q{Files n /bin starting with "d"}
    @hits = 0
  end


  every do |record|
    if record.cols[8] =~ /^d/
      puts %Q{  "#{record.cols[8]}" modified #{record.cols[6..7].join(" ")}}
      @hits += 1
    end
  end


  finish do 
    puts "#{@nr} files were processed"
    puts %Q{#{@hits} files start with "d"}
  end
'
Files n /bin starting with "d"
  "date" modified Dec 2010
  "dd" modified Dec 2010
  "df" modified Dec 2010
  "domainname" modified Jun 2010
38 files were processed
4 files start with "d"


You can install rak by running "gem install rawk"

See my github page for more information and detailed documentation on how to use rawk.

Thursday, June 9, 2011

PRY - Improved Ruby REPL

In other news from GFunc, PRY is an improved Ruby REPL


One more for the TODO list.  How I wish for an extra 5 hours in the day... maybe 10 :-)

Learnings from gfunc meeting 8 June 2011

The Glasgow Functional Programming Group (gFunc) held it's second meeting last night.  We were working on the bowling kata in clojure.

Here are a few notes to remind myself and share the things I learned.

(defn- ...) creates a private method

The replicate function does what it says on the tin: 
user=> (replicate 2 10) 
(10 10)
user=> (replicate 2 [1,2])
([1 2] [1 2])

Lein is a good (the preferred?) build tool for clojure https://github.com/technomancy/leiningen

Clojure does not provide (in ruby terms) each_with_index out of the box.  You can code it as map_with_index - https://gist.github.com/17283 - but it seemed last night to promote an imperative style of coding because your solutions end up indexing into a sequence.  Clojure seems much happier iterating over a (potentially infinite) sequence.  I suspect this is more functional style.

The JetBrains IDE has a very nice Clojure plugin.  I must get round to learning it sometime, but I can't really face learning a new IDE.  I'm still mourning Oracle's newly mandated Java focus for NetBeans :-(





Tuesday, June 7, 2011

JavaScript: Creating and Chaining calls to setTimeout() - Part 1

The excellent book DOM Scripting by Jeremy Keith includes and example of how to animate a simple page element around the screen using setTimeout() to move the element incrementally every n milliseconds.  There's nothing especially remarkable about the example.  It seems straightforward enough, but I ran into problems when I tried to extend it.

I would like to be able to chain movements so I could run the page element around the edges of a rectangle as shown below.



I found the movements seemed to conflict with one another and only one movement occurs.  This post explores setTimeout(), the reason for the problem I found and how to produce the animation sequence I want.

Starting with a simple page that shows the first movement in the rectangle.  The important elements are highlighted bold.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
    <head>
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <link rel="stylesheet" href="styles/typography.css" type="text/css" media="screen" charset="utf-8"/>
        <script type="text/javascript" charset="utf-8" src="scripts/addLoadEvent.js"></script>
        <script type="text/javascript" charset="utf-8" src="scripts/animate.js"></script>
        <title>
            Animation Example
        </title>
    </head>
    <body>
        <p id="message">Whee!</p>
    </body>
</html>

The JavaScript is based on the example from the book.

addLoadEvent.js (manages a queue of calls to window.onload)

function addLoadEvent(func) {
var oldOnLoad = window.onload;
if (typeof window.onload != 'function') {
window.onload = func;
} else {
window.onload = function () {
oldOnLoad();
func();
}
}
}

animate.js

addLoadEvent(animateMessagePosition);

function animateMessagePosition() {
if (!document.getElementById || 
!document.getElementById("message")
) return false;
var elem = document.getElementById("message");
positionElement(elem, 10, 10);
moveMessage(elem, 250, 10);

function positionElement(elem, left, top) {
elem.style.position = "absolute";
elem.style.left = left + "px";
elem.style.top = top + "px";
}


function moveMessage(elem, xtarget, ytarget) {
if (!setTimeout) return false;
   var xpos = parseInt(elem.style.left);
var ypos = parseInt(elem.style.top);
if (xpos == xtarget && ypos == ytarget) return true;
if (xpos < xtarget) xpos++;
if (xpos > xtarget) xpos--;
if (ypos < ytarget) ypos++;
if (ypos > ytarget) ypos--;
positionElement(elem, xpos, ypos);
setTimeout(function () {moveMessage(elem, xtarget, ytarget)}, 5);
}


 This works fine.  The "whee!" text moves across the screen.  Let's add the 4 movements we want in sequence and uncomment the second step.


function animateMessagePosition() {
if (!document.getElementById || 
!document.getElementById("message")
) return false;
var elem = document.getElementById("message");
positionElement(elem, 10, 10);
moveMessage(elem, 250, 10);
moveMessage(elem, 250, 250);
// moveMessage(elem, 10, 250);
// moveMessage(elem, 10, 10);
}

The second movement is ignored!

The problem occurs because setTimeout() does not block the execution our script  It simply registers a callback function and a timeout value in the JavaScript runtime and returns control to the calling script. The callback runs when timeout expires.  John Resig describes JavaScript timers in much more detail.

This behaviour is OK for a single call to moveMessage, but adding the second call means that two timeouts are ticking down at the same time and the first one registered undoes the changes made by the second one.  We can demonstrate this happening by adding a debug trace to the page.

First, I added an extra div to the markup

...
    <body>
        <p id="message">Whee!</p>
        <div id="statusMessages" style="position: absolute; left: 10px; top: 260px;">
            <h3>Status Messages:</h3>
        </div>
    </body>
...

and a new JavaScript function which writes a position as (x,y) co-ordinates into the div


function writePosition(left, top) {
if (!document.createElement || 
!document.getElementById || 
!document.getElementById("statusMessages") || 
!document.createTextNode ||
!document.body.appendChild 
) return false;
var msgNode = document.createElement("p");
var textNode = document.createTextNode("position: (" + left + "," + top + ")");
msgNode.appendChild(textNode);
var statusMessages = document.getElementById("statusMessages");
statusMessages.appendChild(msgNode);
}

finally, I updated the function that performs the movements to write a debug line

function positionElement(elem, left, top) {
elem.style.position = "absolute";
elem.style.left = left + "px";
elem.style.top = top + "px";
writePosition(left, top);
}

Refreshing the page starts the animation and displays a list of all the movements made.  Here are the first few positions shown:




The first line is the initial position.  Then the first call to moveMessage kicks in an shifts the x position by 1.  Next the second call starts and moves the y position by 1.  However, when the first call picks up again, it resets the y position and increments the x position.  Each call looks at the current position of the message and shifts it's x and y position to move it closer to the target.  This means that the message is actually wobbling up and down the y axis by 1 pixel as it moves right but first call eventually gets the message to x == 250.  At that point, the 2 function calls enter an infinite tug of war where the first call cannot complete because the second call keeps moving the message 1 pixel down the y axis!

How do we fix it?

The moveMessage() function manages it's own callbacks once it starts.  I like this design because JavaScript programs run in a single thread and it's important not to block the thread while animating because that would lock the page.  There's no reason to change the design pattern.  We can fix the problem by make moveMessage() call the next animation when it is completed.  This is achieved by passing a callback function to moveMessage that runs once the animation is completed.  

function moveMessage(elem, xtarget, ytarget, nextMove) {
if (!setTimeout) return false;
  
var xpos = parseInt(elem.style.left);
var ypos = parseInt(elem.style.top);
if (xpos == xtarget && ypos == ytarget) {
if (typeof nextMove == 'function') nextMove();
return true;
}
if (xpos < xtarget) xpos++;
if (xpos > xtarget) xpos--;
if (ypos < ytarget) ypos++;
if (ypos > ytarget) ypos--;
positionElement(elem, xpos, ypos);
setTimeout(function () {moveMessage(elem, xtarget, ytarget, nextMove)}, 10);
}

And where it is called...

function animateMessagePosition() {
if (!document.getElementById || 
!document.getElementById("message")
) return false;
var elem = document.getElementById("message");
positionElement(elem, 10, 10);
moveMessage(elem, 250, 10, function () {moveMessage(elem, 250, 250)});
// moveMessage(elem, 10, 250);
// moveMessage(elem, 10, 10);
}

nextMove is an optional argument to moveMethod and if it is undefined the animation sequence will simply end.

This works but there is still a problem.  The call to moveMessage is becoming a little hard to understand and it will become a tangled mess of round and curly brackets if the remaining movements in the sequence were to be added.  Really, I would like to be able to setup a chain of animations and then run it.  Something like this would do the trick...

sequence = createAsyncSequence();
sequence.add(moveMessage, [elem, 250, 10]);
sequence.add(moveMessage, [elem, 250, 250]);
sequence.add(moveMessage, [elem, 10,  250]);
sequence.add(moveMessage, [elem, 10,  10]);
sequence.run();

Before I wrote this, I noticed that the moveMessage function could move any element, while the variable 'elem' points to our message.  We should refactor these naming problems out before they become confusing.  Here are the updated functions

function animateMessagePosition() {
if (!document.getElementById || 
!document.getElementById("message")
) return false;

var message = document.getElementById("message");
positionElement(message, 10, 10);

sequence = createAsyncSequence();
sequence.add(moveElement, [message, 250, 10]);
sequence.add(moveElement, [message, 250, 250]);
sequence.add(moveElement, [message, 10,  250]);
sequence.add(moveElement, [message, 10,  10]);
sequence.run();
}

function moveElement(elem, xtarget, ytarget, nextMove) {
if (!setTimeout) return false;
   var xpos = parseInt(elem.style.left);
var ypos = parseInt(elem.style.top);
if (xpos == xtarget && ypos == ytarget) {
if (typeof nextMove == 'function') nextMove();
return true;
}
if (xpos < xtarget) xpos++;
if (xpos > xtarget) xpos--;
if (ypos < ytarget) ypos++;
if (ypos > ytarget) ypos--;
positionElement(elem, xpos, ypos);
setTimeout(function () {moveElement(elem, xtarget, ytarget, nextMove)}, 10);
}

Now, I can write createAsyncSequence()

function createAsyncSequence() {
var my = {};
var seq = [];
var emptyEntry = {
name : "",
args : [],
};

emptyEntry.toFunc = function () {
var that = this;
return function () {that.name.apply(null, that.args)};
}
my.add = function (name, args) {
var entry = Object.create(emptyEntry);
entry.name = name;
entry.args = args;
if (seq.length > 0) seq[seq.length - 1].args.push(entry.toFunc()); 
seq.push(entry);
};
my.run = function () {
if(seq.length > 0) {
return seq[0].toFunc() ();
} else {
return true;
}
};
return my;
}

Notice the call to Object.create.  This function creates a new object based on an existing object prototype.  It is added at the top of the source file.


if (typeof Object.create !== 'function') {
Object.create = function(o) {
var F = function () {};
F.prototype = o;
return new F();
}
}

Now, let's dig into the code that creates the asynchronous sequence. First, we create an empty object that we will build and return, and an empty array to hold the sequence of calls

function createAsyncSequence() {
var my = {};
var seq = [];

My strategy is to store the data needed to run the sequence of calls in an array and convert it to actual function calls as and when they are needed.  We, therefore, need a private class to hold the items in the sequence.  We do this by creating an empty object that we can clone using prototype inheritance.  Strictly speaking, it's not necessary to create a prototype object in this case because we could attach attributes to an empty object later.  However, I think it makes the code clearer by stating explicitly what attributes this object holds so I have decided to create it anyway.

var emptyEntry = {
name : "",
args : [],
};

We also need a function to convert our entry objects into function calls.  

emptyEntry.toFunc = function () {
var that = this;
return function () {that.name.apply(null, that.args)};
}

Now we have the plumbing in place, we add two methods to the async sequence object we are building.  One to add items to the sequence and the other to run the sequence.  First the add method.

my.add = function (name, args) {
var entry = Object.create(emptyEntry);
entry.name = name;
entry.args = args;
if (seq.length > 0) seq[seq.length - 1].args.push(entry.toFunc()); 
seq.push(entry);
};

The critical line is highlighted in bold.  Remember that moveElement() takes an optional argument called nextMove that holds a function to run after the movement is completed.  The my.add method appends the 'functionized' version of the entry we are adding as the final argument to the previous method call in the sequence.    The run method simply needs to start the first movement in the sequence and the chain of methods will look after itself.

Notice also the call to Object.create described above.

Here is the run method. It simply converts the first entry to a function and then runs it.

my.run = function () {
if(seq.length > 0) {
return seq[0].toFunc() ();
} else {
return true;
}
};


Finally, we return the asynchronous sequence object. 


return my;
}

Well, that's covered a fair amount for now.  The next step is to add a way to stop the animation mid-flight when the user clicks a button but I'll leave that until part 2.

Saturday, May 21, 2011

Clojure Lists and Forms

I'm working through Programming Clojure by Stuart Halloway.  I am at the middle of chapter 2 (exploring Clojure).  The syntax is different to anything I have used before and it's little hard to understand at times so I have decided to take time to do some background research.  


As usual, Google throws up a page from the fountain of all knowledge - Wikipedia's article on Lisp.


Lists

Clojure is a Lisp implementation (aka "a Lisp").  The name Lisp derives from "LISt Processing" because in a Lisp, everything is represented as a list like this one.
(foo 1 2 3)


The list is enclosed in parenthesis and the items in the list (known as atoms) are separated by spaces.  The first atom is an operator (usually a function call; foo in this case) and the remaining atoms (1,2 and 3) are arguments to the function.  The bracket notation is known as an S-expression and the use of the first atom in the list to represent an operator is called prefix notation.  Prefix notation provides a very regular syntax that make it easy to generate code for metaprogramming.  


So, if we know the str function creates a string from it's arguments, we can write 
(str "hello " " world") 
=> hello world


This looks quite similar to most programming languages, but it looks a bit odd when you see a arithmetic operation for the first time
(+ 1 2 3) 
=> 6


Most languages use infix operators for arithmetic so in Java or Ruby this s-expression would be 
1 + 2 + 3 


Clojure runs on the JVM and it also provides direct access java.lang.  You can call java functions by prefixing the operator atom with a dot.
"foo".toUpperCase();
becomes 
(.toUpperCase "foo")
=> FOO

As you would expect, s-expressions can be nested to any depth:
(str (str "hello" " world") (str " from" " me"))
=> "hello world from me"
(+ (- 3 (* 2 5)) 8)
=> 1

Forms

Lisp code is read by a reader that can be programmed (in most Lisps - see below) using macros.  These macros are pretty sophisticated and go beyond the pre-processor templating features in languages like C.  For example, Clojure implements a macro that tells the reader to ignore everything after a semi-colon  - in other words, a line comment.

(+ 1 2 3)  ; Comment: Adds 3 numbers together

I would need to dig deeper to find out, but I can image there *may* have been a time when everything in lisp was represented as an s-expression with a bunch of reader macros to provide shortcuts and "syntax".  I'm sure that would have been possible, but it would have severely limited portability because I would need to have your macros to run your code, and they might conflict with mine.

Thankfully, Clojure provides proper syntax for important constructs and a limited set of reader macros that cannot be extended.  The syntax elements are known as forms


Form
Usage
Syntax
Example
Boolean
Conditional logic
true, false
true, false
Character
A single character
\<char>
\a
Keyword
An constant value that refers to itself - like a Symbol in Ruby
:<name>
:foo
List
S-Expressions, lists of data
(*<atoms>)
(str "a" "b")
Map
Hashes
{key1 value1, key2 value 2}
{:name "Bill", :age 42} 
Nil
Nil
nil
nil
Number
Integers, decimals
*digits[.*digits]
100, 12.45
Set
A set of non-duplicate values
#{<value1> <value2>}
#{1 "a" :foo}
String
A (java) string
"<text>" 
"foo"
Symbol
A handle on an entity - e.g. a function
*<text>
user/foo, java.lang.String 
Vector
An ordered sequence of data separated by spaces
[<value1> <value2>]
[1 :bar "x" 4.3 \a true]


I don't know whether these are native to Clojure or borrowed from other Lisps, but it's not really important to my learning right now.

The usage for these forms is pretty much as you would expect.  We've already seen the string and number literals in use above.  I won't document them there because there are plenty of Internet resources out there.  

* Chapter 2 of Programming Clojure covers all the forms in detail

Note though that clojure is a dynamic language so you can mix an match different types within a data structure.

The conj function pushes items into a vector
(conj [1 2] 3)
=> [1 2 3]

We can also mix up the types
(conj [] 1 2.3 \s "foo" nil true)
=> [1 2.3 \s "foo" nil true]

Learnings

The important thing to remember is that everything is an s-expression.  Sometimes it can be hard to see in more complex expressions but you can usually figure out what is going on if you break everything down into a list and look for the s-expression.