Sunday, July 14, 2013

Clojure Course in Edinburgh on 7 September


I'm pleased to announce that my Clojure Kickstart course will be held at Neo's offices at TechCube in Edinburgh on the 7th of September 2013.  This course is aimed at beginners and improvers who want to learn Clojure or push their existing skills on to the next level.  

Tickets are now available at http://clojure-kickstart-scotland.eventbrite.co.uk/ for £30.  Just to be clear, the fee is intended to cover costs and anything extra I make will be invested back into making the course even better.

The course will be heavily based on practical exercises.  You will learn...
  • The basic syntax and usage of Clojure
  • The core elements of the Clojure ecosystem
  • What sort of problems are people solving with Clojure in the real world
  • How to make the paradigm shift from Object Oriented to Functional Programming
  • Parallel/multi-threaded Clojure programming and Clojure's transactional memory model
Although Clojure is in its infancy in Scotland, the London community is very strong and I am  delighted to welcome London Clojurian Malcolm Sparks (https://juxt.pro) to Edinburgh as guest facilitator.  

Malcolm will be on hand to help you with the exercises and this is a rare opportunity to get a full-time Clojure programmer's insight on how you can improve your Clojure and Functional programming.  He will also be giving a talk on how Clojure is used in the real-world that will take us all beyond the texbook and into the trenches.

Saturday, February 9, 2013

Finding The Way

Everyone on the Internet is an expert.  Some of them might actually be.

It can be easy to feel like you are chasing an impossible goal when every blog you read makes it look like the author has it all sorted out and their code just flies off their finger tips ready for a quick spot of refactoring and then straight into production.

I don't think it's really like that but I still wonder sometimes if I could somehow make myself more productive by working slightly harder.

An old story  I found on the Aikido FAQ might tell you some of the mindset you ought to apply when studying martial arts and I think it applies to programming too.

A young boy traveled across Japan to the school of a famous martial artist. When he arrived at the dojo he was given an audience by the Sensei
"What do you wish from me?" the master asked. 
"I wish to be your student and become the finest kareteka in the land," the boy replied. "How long must I study?" 
"Ten years at least," the master answered. 
"Ten years is a long time," said the boy. "What if I studied twice as hard as all your other students?" 
"Twenty years," replied the master. 
"Twenty years! What if I practice day and night with all my effort?" 
"Thirty years," was the master's reply. 
"How is it that each time I say I will work harder, you tell me that it will take longer?" the boy asked. 
"The answer is clear. When one eye is fixed upon your destination, there is only one eye left with which to find the Way."



Sunday, January 20, 2013

todo.text


I've been looking for a todo app that fits my workflow.  It must...

  • Work across my devices
  • Allow me to do GTG style task tracking
  • Make it easy to categorise tasks HOW I WANT TO 

You would think it would be simple.  Indeed there are many apps that fit the first 2 categories but the last one is a killer and it seems that most task apps force you to organise your tasks by date (which is pointless when you have a calendar to do that) or priority (which is also pointless because priorities change according to your context).

However, Rob Lally recommended todo.text and it seems to fit the bill nicely.  It uses simple text files to store tasks and dropbox to sync them up between devices.  You also have a command line interface to enter and manage tasks and because it's just text you can sort, search, chop and change using normal UNIX tools.

I used the homebrew package to get started.  It doesn't work out of the box (for version 2.9m at least) but the instructions at http://dangerisgo.com/blog/2012/09/20/setup-todo-dot-txt-cli-on-osx-with-homebrew-and-dropbox/ did the trick.  

One little gotcha - when you change the home location in the cfg file to point at dropbox, make sure you comment out line 5 which sets the location to the install directory.

I'm planning a Clojure course In Scotland

The London Clojurians are running a Clojure dojo on the 29th of January.

This got me thinking that maybe would could do some kind of hands-on course or dojo in Scotland so I have created a survey to gather more information.

http://www.surveymonkey.com/s/WDCTMHM

Any help you can provide by completing and sharing this survey would be greatly appreciated.

Friday, January 13, 2012

Background process management

I ran an impromptu tutorial the other day on background process management from the bash command line with one of my colleagues. I thought it might be helpful to write it up for anyone else who might find it useful.

We were starting a redis server in this case, bit this works for any UNIX process.

So, we can start the server in the background by suffixing the command with an &

$ redis-server /usr/local/etc/redis.conf &

We can then list background processes by running

$ jobs
[1]+ Running redis-server /usr/local/etc/redis.conf &


The [1] indicates a job "handle" (I'm sure there is a proper name for it, but I don't know what it is), that we can pass to the following commands to manage the process:


- fg %1: pulls process 1 into the foreground
- bg %1: pushes process 1 into the background
- kill %1: kills process 1



If you start another process, it will be assigned the next available id which is %2 in this case

$ tail -f project.clj &
[2] 37913
$ (defproject show-grid "1.0.0-SNAPSHOT"
:description "FIXME: write description"
:dependencies [[org.clojure/clojure "1.2.1"]])
$ jobs
[1]- Running redis-server /usr/local/etc/redis.conf &
[2]+ Running tail -f project.clj &


Killing a process will free up it's handle and it will be reused. This means that for most casual uses, you will only need %1 and, occasionally, %2 unless you run a lot of deamons for some reason.

$ kill %2
[2]+ Terminated: 15 tail -f project.clj
$ jobs
[1]+ Running tail -f project.clj &
$ tail -f README &
[2] 37927

## Usage
FIXME: write
## License
Copyright (C) 2011 FIXME
Distributed under the Eclipse Public License, the same as Clojure.
$ jobs
[1]- Running tail -f project.clj &
[2]+ Running tail -f README &


The problem with simply putting things into the background is that the standard output stream for the process remains attached to your terminal and it will continue to write to your session while you are trying to work on other things. Usually, you want to run the process silently and log the output to a file using nohup like this. Notice that we have merged the stdout and stderr streams so nothing gets lost.

$ nohup redis-server /usr/local/etc/redis.conf > ./redis.log 2>&1 &
[1] 21212
$ tail -f redis.log
[21212] 10 Jan 11:16:39 * Server started, Redis version 2.4.2
[21212] 10 Jan 11:16:39 * DB loaded from disk: 0 seconds
[21212] 10 Jan 11:16:39 * The server is now ready to accept connections on port 6379
[21212] 10 Jan 11:16:39 - 0 clients connected (0 slaves), 922288 bytes in use
[21212] 10 Jan 11:16:44 - 0 clients connected (0 slaves), 922288 bytes in use
[21212] 10 Jan 11:16:49 - 0 clients connected (0 slaves), 922288 bytes in use
[21212] 10 Jan 11:16:54 - 0 clients connected (0 slaves), 922288 bytes in use
[21212] 10 Jan 11:16:59 - 0 clients connected (0 slaves), 922288 bytes in use
[21212] 10 Jan 11:17:04 - 0 clients connected (0 slaves), 922288 bytes in use
[21212] 10 Jan 11:17:09 - 0 clients connected (0 slaves), 922288 bytes in use

Friday, December 23, 2011

lein uberjar ClassNotFoundException

This is probably documented somewhere, but it didn't come up in my searches and it took me forever to find out the cause of the problem so I'm posting it here for the greater good, general advancement of humankind and other worthy reasons.

I had a normal leiningen project with the :main attribute set in project.clj

(defproject clojure-diff "1.0.0-SNAPSHOT"
  ; Dependencies etc omitted for clarity
  :main clojure-diff.server
)

It runs fine using lein run but when I ran lein uberjar and then executed the jar file, I got a class not found error.

$ java -jar clojure-diff-1.0.0-SNAPSHOT-standalone.jar
Exception in thread "main" java.lang.NoClassDefFoundError: clojure_diff/server
Caused by: java.lang.ClassNotFoundException: clojure_diff.server
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)


The solution is to specify :gen-class when defining the main namespace.  This means a .class file will be generate ahead of time when compiling the jar.  Obvious when you know how!

(ns clojure-diff.server
  (:gen-class)
  ; more namespace setup)

; .. stuff

(defn -main []
  (run-jetty #'clojure-diff.server/app {:port 8080}))


Tuesday, August 23, 2011

Rawk

I've created a gem called rawk.  It's a ruby DSL for command-line processing stream processing based on the classic AWK utility.   I wrote it because I like using awk, but I find it's lack of useful data structures a bit of a pain.  Rawk provides the same block-based processing, but using good old ruby.

Let's take an example.  I want to pretty-print the name and last modification date from a directory where the  filename starts with 'd'


$ ls -l /bin | rawk '
  start do 
    puts %q{Files n /bin starting with "d"}
    @hits = 0
  end


  every do |record|
    if record.cols[8] =~ /^d/
      puts %Q{  "#{record.cols[8]}" modified #{record.cols[6..7].join(" ")}}
      @hits += 1
    end
  end


  finish do 
    puts "#{@nr} files were processed"
    puts %Q{#{@hits} files start with "d"}
  end
'
Files n /bin starting with "d"
  "date" modified Dec 2010
  "dd" modified Dec 2010
  "df" modified Dec 2010
  "domainname" modified Jun 2010
38 files were processed
4 files start with "d"


You can install rak by running "gem install rawk"

See my github page for more information and detailed documentation on how to use rawk.

Thursday, June 9, 2011

PRY - Improved Ruby REPL

In other news from GFunc, PRY is an improved Ruby REPL


One more for the TODO list.  How I wish for an extra 5 hours in the day... maybe 10 :-)

Learnings from gfunc meeting 8 June 2011

The Glasgow Functional Programming Group (gFunc) held it's second meeting last night.  We were working on the bowling kata in clojure.

Here are a few notes to remind myself and share the things I learned.

(defn- ...) creates a private method

The replicate function does what it says on the tin: 
user=> (replicate 2 10) 
(10 10)
user=> (replicate 2 [1,2])
([1 2] [1 2])

Lein is a good (the preferred?) build tool for clojure https://github.com/technomancy/leiningen

Clojure does not provide (in ruby terms) each_with_index out of the box.  You can code it as map_with_index - https://gist.github.com/17283 - but it seemed last night to promote an imperative style of coding because your solutions end up indexing into a sequence.  Clojure seems much happier iterating over a (potentially infinite) sequence.  I suspect this is more functional style.

The JetBrains IDE has a very nice Clojure plugin.  I must get round to learning it sometime, but I can't really face learning a new IDE.  I'm still mourning Oracle's newly mandated Java focus for NetBeans :-(





Tuesday, June 7, 2011

JavaScript: Creating and Chaining calls to setTimeout() - Part 1

The excellent book DOM Scripting by Jeremy Keith includes and example of how to animate a simple page element around the screen using setTimeout() to move the element incrementally every n milliseconds.  There's nothing especially remarkable about the example.  It seems straightforward enough, but I ran into problems when I tried to extend it.

I would like to be able to chain movements so I could run the page element around the edges of a rectangle as shown below.



I found the movements seemed to conflict with one another and only one movement occurs.  This post explores setTimeout(), the reason for the problem I found and how to produce the animation sequence I want.

Starting with a simple page that shows the first movement in the rectangle.  The important elements are highlighted bold.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
    <head>
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <link rel="stylesheet" href="styles/typography.css" type="text/css" media="screen" charset="utf-8"/>
        <script type="text/javascript" charset="utf-8" src="scripts/addLoadEvent.js"></script>
        <script type="text/javascript" charset="utf-8" src="scripts/animate.js"></script>
        <title>
            Animation Example
        </title>
    </head>
    <body>
        <p id="message">Whee!</p>
    </body>
</html>

The JavaScript is based on the example from the book.

addLoadEvent.js (manages a queue of calls to window.onload)

function addLoadEvent(func) {
var oldOnLoad = window.onload;
if (typeof window.onload != 'function') {
window.onload = func;
} else {
window.onload = function () {
oldOnLoad();
func();
}
}
}

animate.js

addLoadEvent(animateMessagePosition);

function animateMessagePosition() {
if (!document.getElementById || 
!document.getElementById("message")
) return false;
var elem = document.getElementById("message");
positionElement(elem, 10, 10);
moveMessage(elem, 250, 10);

function positionElement(elem, left, top) {
elem.style.position = "absolute";
elem.style.left = left + "px";
elem.style.top = top + "px";
}


function moveMessage(elem, xtarget, ytarget) {
if (!setTimeout) return false;
   var xpos = parseInt(elem.style.left);
var ypos = parseInt(elem.style.top);
if (xpos == xtarget && ypos == ytarget) return true;
if (xpos < xtarget) xpos++;
if (xpos > xtarget) xpos--;
if (ypos < ytarget) ypos++;
if (ypos > ytarget) ypos--;
positionElement(elem, xpos, ypos);
setTimeout(function () {moveMessage(elem, xtarget, ytarget)}, 5);
}


 This works fine.  The "whee!" text moves across the screen.  Let's add the 4 movements we want in sequence and uncomment the second step.


function animateMessagePosition() {
if (!document.getElementById || 
!document.getElementById("message")
) return false;
var elem = document.getElementById("message");
positionElement(elem, 10, 10);
moveMessage(elem, 250, 10);
moveMessage(elem, 250, 250);
// moveMessage(elem, 10, 250);
// moveMessage(elem, 10, 10);
}

The second movement is ignored!

The problem occurs because setTimeout() does not block the execution our script  It simply registers a callback function and a timeout value in the JavaScript runtime and returns control to the calling script. The callback runs when timeout expires.  John Resig describes JavaScript timers in much more detail.

This behaviour is OK for a single call to moveMessage, but adding the second call means that two timeouts are ticking down at the same time and the first one registered undoes the changes made by the second one.  We can demonstrate this happening by adding a debug trace to the page.

First, I added an extra div to the markup

...
    <body>
        <p id="message">Whee!</p>
        <div id="statusMessages" style="position: absolute; left: 10px; top: 260px;">
            <h3>Status Messages:</h3>
        </div>
    </body>
...

and a new JavaScript function which writes a position as (x,y) co-ordinates into the div


function writePosition(left, top) {
if (!document.createElement || 
!document.getElementById || 
!document.getElementById("statusMessages") || 
!document.createTextNode ||
!document.body.appendChild 
) return false;
var msgNode = document.createElement("p");
var textNode = document.createTextNode("position: (" + left + "," + top + ")");
msgNode.appendChild(textNode);
var statusMessages = document.getElementById("statusMessages");
statusMessages.appendChild(msgNode);
}

finally, I updated the function that performs the movements to write a debug line

function positionElement(elem, left, top) {
elem.style.position = "absolute";
elem.style.left = left + "px";
elem.style.top = top + "px";
writePosition(left, top);
}

Refreshing the page starts the animation and displays a list of all the movements made.  Here are the first few positions shown:




The first line is the initial position.  Then the first call to moveMessage kicks in an shifts the x position by 1.  Next the second call starts and moves the y position by 1.  However, when the first call picks up again, it resets the y position and increments the x position.  Each call looks at the current position of the message and shifts it's x and y position to move it closer to the target.  This means that the message is actually wobbling up and down the y axis by 1 pixel as it moves right but first call eventually gets the message to x == 250.  At that point, the 2 function calls enter an infinite tug of war where the first call cannot complete because the second call keeps moving the message 1 pixel down the y axis!

How do we fix it?

The moveMessage() function manages it's own callbacks once it starts.  I like this design because JavaScript programs run in a single thread and it's important not to block the thread while animating because that would lock the page.  There's no reason to change the design pattern.  We can fix the problem by make moveMessage() call the next animation when it is completed.  This is achieved by passing a callback function to moveMessage that runs once the animation is completed.  

function moveMessage(elem, xtarget, ytarget, nextMove) {
if (!setTimeout) return false;
  
var xpos = parseInt(elem.style.left);
var ypos = parseInt(elem.style.top);
if (xpos == xtarget && ypos == ytarget) {
if (typeof nextMove == 'function') nextMove();
return true;
}
if (xpos < xtarget) xpos++;
if (xpos > xtarget) xpos--;
if (ypos < ytarget) ypos++;
if (ypos > ytarget) ypos--;
positionElement(elem, xpos, ypos);
setTimeout(function () {moveMessage(elem, xtarget, ytarget, nextMove)}, 10);
}

And where it is called...

function animateMessagePosition() {
if (!document.getElementById || 
!document.getElementById("message")
) return false;
var elem = document.getElementById("message");
positionElement(elem, 10, 10);
moveMessage(elem, 250, 10, function () {moveMessage(elem, 250, 250)});
// moveMessage(elem, 10, 250);
// moveMessage(elem, 10, 10);
}

nextMove is an optional argument to moveMethod and if it is undefined the animation sequence will simply end.

This works but there is still a problem.  The call to moveMessage is becoming a little hard to understand and it will become a tangled mess of round and curly brackets if the remaining movements in the sequence were to be added.  Really, I would like to be able to setup a chain of animations and then run it.  Something like this would do the trick...

sequence = createAsyncSequence();
sequence.add(moveMessage, [elem, 250, 10]);
sequence.add(moveMessage, [elem, 250, 250]);
sequence.add(moveMessage, [elem, 10,  250]);
sequence.add(moveMessage, [elem, 10,  10]);
sequence.run();

Before I wrote this, I noticed that the moveMessage function could move any element, while the variable 'elem' points to our message.  We should refactor these naming problems out before they become confusing.  Here are the updated functions

function animateMessagePosition() {
if (!document.getElementById || 
!document.getElementById("message")
) return false;

var message = document.getElementById("message");
positionElement(message, 10, 10);

sequence = createAsyncSequence();
sequence.add(moveElement, [message, 250, 10]);
sequence.add(moveElement, [message, 250, 250]);
sequence.add(moveElement, [message, 10,  250]);
sequence.add(moveElement, [message, 10,  10]);
sequence.run();
}

function moveElement(elem, xtarget, ytarget, nextMove) {
if (!setTimeout) return false;
   var xpos = parseInt(elem.style.left);
var ypos = parseInt(elem.style.top);
if (xpos == xtarget && ypos == ytarget) {
if (typeof nextMove == 'function') nextMove();
return true;
}
if (xpos < xtarget) xpos++;
if (xpos > xtarget) xpos--;
if (ypos < ytarget) ypos++;
if (ypos > ytarget) ypos--;
positionElement(elem, xpos, ypos);
setTimeout(function () {moveElement(elem, xtarget, ytarget, nextMove)}, 10);
}

Now, I can write createAsyncSequence()

function createAsyncSequence() {
var my = {};
var seq = [];
var emptyEntry = {
name : "",
args : [],
};

emptyEntry.toFunc = function () {
var that = this;
return function () {that.name.apply(null, that.args)};
}
my.add = function (name, args) {
var entry = Object.create(emptyEntry);
entry.name = name;
entry.args = args;
if (seq.length > 0) seq[seq.length - 1].args.push(entry.toFunc()); 
seq.push(entry);
};
my.run = function () {
if(seq.length > 0) {
return seq[0].toFunc() ();
} else {
return true;
}
};
return my;
}

Notice the call to Object.create.  This function creates a new object based on an existing object prototype.  It is added at the top of the source file.


if (typeof Object.create !== 'function') {
Object.create = function(o) {
var F = function () {};
F.prototype = o;
return new F();
}
}

Now, let's dig into the code that creates the asynchronous sequence. First, we create an empty object that we will build and return, and an empty array to hold the sequence of calls

function createAsyncSequence() {
var my = {};
var seq = [];

My strategy is to store the data needed to run the sequence of calls in an array and convert it to actual function calls as and when they are needed.  We, therefore, need a private class to hold the items in the sequence.  We do this by creating an empty object that we can clone using prototype inheritance.  Strictly speaking, it's not necessary to create a prototype object in this case because we could attach attributes to an empty object later.  However, I think it makes the code clearer by stating explicitly what attributes this object holds so I have decided to create it anyway.

var emptyEntry = {
name : "",
args : [],
};

We also need a function to convert our entry objects into function calls.  

emptyEntry.toFunc = function () {
var that = this;
return function () {that.name.apply(null, that.args)};
}

Now we have the plumbing in place, we add two methods to the async sequence object we are building.  One to add items to the sequence and the other to run the sequence.  First the add method.

my.add = function (name, args) {
var entry = Object.create(emptyEntry);
entry.name = name;
entry.args = args;
if (seq.length > 0) seq[seq.length - 1].args.push(entry.toFunc()); 
seq.push(entry);
};

The critical line is highlighted in bold.  Remember that moveElement() takes an optional argument called nextMove that holds a function to run after the movement is completed.  The my.add method appends the 'functionized' version of the entry we are adding as the final argument to the previous method call in the sequence.    The run method simply needs to start the first movement in the sequence and the chain of methods will look after itself.

Notice also the call to Object.create described above.

Here is the run method. It simply converts the first entry to a function and then runs it.

my.run = function () {
if(seq.length > 0) {
return seq[0].toFunc() ();
} else {
return true;
}
};


Finally, we return the asynchronous sequence object. 


return my;
}

Well, that's covered a fair amount for now.  The next step is to add a way to stop the animation mid-flight when the user clicks a button but I'll leave that until part 2.