Pointers to Functions in C# and Python

30017215757_365cc8720f_z.jpg

After the description of pointers to functions in C it occurred to me that you might like to know how to do function pointers in other languages. Then again you might not, in which case just look at the picture above that I took in Singapore a while back.

Pointers to methods in C#

If you want to make a reference to a function in C# you need to create a delegate type which contains the reference. Instances of the delegate type can then be made to refer to methods. It’s a bit more long-winded than the C way of doing things, but delegates can also be used as the basis of some quite powerful publish and subscribe mechanisms which I might mention later.

class Program
{
delegate void SimpleDelegate();
static void aMethod()
{
Console.WriteLine("Hello from a method");
}
static void Main(string[] args)
{
SimpleDelegate myDelegateInstance = aMethod;
myDelegateInstance();
}
}

This is a tiny example of a program that uses a delegate to make a reference to a method. The delegate type that I’ve created is called SimpleDelegate. Variables of this type (I made one called myDelegateInstance) can be made to refer to a a method and then called.

If you compare this with the C syntax, it’s not that different. The good news is that before the program runs the compiler can make sure a program will never call a method incorrectly using a delegate. Because SimpleDelegate is declared as returning void (i.e. nothing) and accepting no parameters it is impossible to make instances of SimpleDelegate refer to a method that returns a different type or has a different set of parameters. If you worry about these kind of things, this is a good thing.

Pointers to Functions in Python

If you want to make a pointer to a function in Python you just do it:

def a_function():
print("Hello from a function")

function_ref = a_function
function_ref()

The variable function_ref is made to refer to the function a_function and the function_ref variable can then be called as a function. Later in the program function_ref might store an integer, or string, or a reference to an object of type Cheese. That’s how Python is. This means that a Python program can fail in ways that a C# program won’t, because the errors that cause the Python program to fail would be picked up by the C# compiler.

This nicely encapsulates the difference between the cultures of C# and Python. C# is great because it forces you to invest effort to make sure that the objects in your programs can only ever be fitted together correctly. Python takes everything on trust. It lets the programmer just get on with expressing the actions to be performed and assumes that things will turn out OK. If they don’t the program will fail at runtime.

I love the way that C# forces me to think about my code, and I love the way that Python just lets me get on with writing, without me having to worry about adding lots of extra syntax to express what I know I want. You should know about both ways of working.

Pointers to Functions in C

27931484787_633bdb9b2e_z.jpg

I’m re-writing my neopixel animation software for the new light. I’m using a C feature that is sometimes very useful. I’m using pointers to functions.

Note that this is quite strong magic. If you don’t use it properly your program will just stop with no errors or warnings. But then again, that’s pretty much how it is when you write Arduino code anyway, so feel free to have a go. You won’t break any hardware.

The reason why I’m using function pointers is that I want to make the program to select between different animations to control the lights. The update behaviour for each animation is a different function. For example, I’ve got two different displays, one which walks colours around the lights and one which just displays the text in a particular colour. That means I’ve got two update functions:

void update_coloured_message()
{
// update the coloured message
}

void update_text_message()
{
// update the text message
}

Each of these is a function that accepts no parameters and does not return a result. I can create a C variable that can refer to these methods as follows

void (*animation_update) ();

This rather convoluted syntax creates a variable called animation_update that can refer to methods that accept no parameters and don’t return a value. I can make this variable refer to one of my update methods by using a simple assignment:

animation_update = update_text_message;

Note that normally when you want to create a reference to a value in C or C++ you need to put an ampersand (&) in front of the thing you are getting a pointer to. In the case of function references you can leave this out. This is either a neat trick to make C programming easier or a nasty trick to make C programming more confusing. I’ll let you decide.

Anyhoo, once I’ve set animation_update to refer to a function I can call the pointer just as I would the function:

animation_update();

I’ve actually gone one more, in that I’m using function pointers in a structure. Go me. I’ve got a structure that describes an animation in terms of the function that sets up the animation, the function that updates the animation and the number of frames it should run for:

struct Animation_item
{
void(*animation_setup) ();
void(*animation_update)();
int duration;
};

I then create an array of these items that contain all the animations that I want to move between:

struct Animation_item animations[] = {
{ setup_coloured_message, update_coloured_message, 600 },
{setup_walking_colours, update_walking_colours, 600} };

Then just need to count around the array to move between animations.

animations[animation_number].animation_update();

This statement calls the animation update method for the animation that is selected by the value in animation_number. If I add more animations to the array my program just picks them up automatically with no need to change anything else.

Code like an idiot

42745874705_fa1f4bb873_z.jpg

Hmm. No matter how good you are at programming, developers will still put bugs in their programs. Including me. Lost a couple of hours today because of this little beauty.

if ((tickCount % lights[i].colourSpeed)!=0 || lights[i].colourSpeed==0)
    return;

This code has been working inside HullOS for a couple of years. No problems. However, when I tried to port the code onto a Wemos device it kept crashing with an invalid instruction message.

Wah. The idea is that the code will exit if is not time to update the colour animation (tick count is not an exact multiple of colourSpeed) or if no animation is being performed (colourSpeed is zero). 

It took me a while to figure out what was happening. Turns out this code is wrong, but under some circumstances it will work perfectly well. 

It's all to do with the % (modulus) operator. This gives the remainder that you get if you divide one number by another (or example 7 % 4 would be 3). If the modulus is zero that we have an exact multiple on our hands. So, every time tickCount reaches the next multiple of colourSpeed the program will perform an update. 

Snag is that I've been stupid, in that I use a colourSpeed value of 0 to mean that the light is not being updated. And the behaviour of a C++ program when you do x % 0 is undefined. In other words, any number modulus zero is a stupid question to ask.

To make things worse the fact that I'm doing this bad thing as one half of an or expression means that, depending on the whim of the compiler, the modulus calculation may not get evaluated when colourSpeed is zero. If the code produced by the compiler does the test for zero before the modulus my program works. If the tests are done the other way round the program explodes.

It's an easy fix, just break the single test into two tests so that I control the order they are performed:

if (lights[i].colourSpeed == 0)
    return;
if ((tickCount % lights[i].colourSpeed) != 0 )
    return;

Now we never use the value of 0 in a modulus expression.  

Oh well, just goes to show that you can learn stuff by doing stuff.....

C# Quick Question 2 resolution

39093204760_70998883d6_z.jpg

Ha. Nobody seemed to know this. Or nobody cared....

Anyhoo, the question was:

"When would a value type be stored on the heap?" And no, the answer doesn't involve boxing.

A value type just holds a value. Not a reference to something. If I have an object managed by reference I have to be careful about removing it from memory, because I need to be sure that nothing in the program is referring to the object. This means that objects managed by reference are stored in a "heap" of objects. It is the job of the "garbage collector" to find objects that are not referred to by anything, and remove these. 

However, value types are easy to get rid of. There can't be any references to them. 

Think of a value type as a book in a library. We can take the book away and nobody can use it any more. That's sad. But we can be sure that nobody is using the book when we take it away. Think of a reference type as a web site. We can't just delete a web site because we don't know how many people out there have made references to the site. 

So, when do we need to put a value type on the heap, and treat it a bit like a reference type? One answer is when we "box" a value type to convert a value type into a into a reference type, but this this is not what I'm after. 

The answer is when a value type is used in a "closure". We see closures when we use lambda expressions. OK, so what's a lambda expression? Lambda expressions are a pure way of expressing the "something goes in, something happens and something comes out" part of behaviours.

The types of the elements and the result to be returned are inferred from the context in which the lambda expression is used. Instead of writing a method and creating a reference to the method, you can just create a lambda expression.

We get into lambda expressions shortly after we start trying to treat lumps of functional code as we would data. Consider the following method:

static int add(int a, int b)
{
    return a + b;
}

This method is called add, it takes two integers and returns their sum. Maybe, for some reason, I want to my program to select between adding, subtracting and some other mathematical operations that take in two integers and return an integer result. C# lets me create pointers to methods. They are called delegates and we would make such a delegate like this:

delegate int IntOperation(int a, int b);

A delegate is another type I can use in my program. So I can create a variable of type IntOperation. And I can make my delegate refer to a method. Watch carefully:

IntOperation thingToDo;

This statement has made a delegate called thingToDo that can refer to methods that accept two integers and return a single integer result. Now I can make this delegate refer to a method:

thingToDo = add;

Now, if I call thingToDo, the add method will run:

int result = thingToDo(1,10);

This would set the value result to 11, because thingToDo presently refers to the add method. With me so far? Good. We can create our methods "the hard way", or we can use a lambda function instead. Look at this statement:

thingTodo = (a, b) => { return a - b; };

Don't panic. It wants to be your friend. Let's take a careful look at what I've done. I've made thingToDo refer to a lump of code that will perform a subtract operation. The code that does the subtraction doesn't live anywhere and has no name.  It is a lambda expression .Sometimes called an "anonymous function". The sequence => is the lambda operator and it separates the "stuff goes in" from the "stuff comes out" parts of the expression. In this case the parameters a and b to in, and the result of a-b comes out. Because thingToDo accepts integers and returns integers, inside the lambda expression the types of a and b are integers. 

OK. We can put these anonymous lumps of code in our program and pass them around on the end of references, just like any other object. What about closures? Take a look at this strange code:

{
    int localValue = 99;
    thingTodo = (a, b) => { return a + b + localValue; };
}

This is stupid, useless code. The statement inside the lambda function is using the local variable localValue for some silly reason. This is quite legal, compiles fine, and gives the compiler a problem to solve. The problem is that the lifetime of localValue exceeds the block in which it is declared. Remember that local variables are usually stored on the stack and deleted when the program leaves the block of code in which the local variable is declared. However, thingToDo is still referring to a lambda expression that uses the value of localValue. So localValue can't die. It must live on outside the block in which it is declared.

The compiler solves this problem by putting the localValue variable on the heap, rather than the stack. The heap is where we keep stuff that has to stick around for a while. This extension of the life of a local variable is called a "closure". 

Short question. Long answer. Phew. 

C# Quick Question 1 resolution

26030367387_c46d37e916_z.jpg

This was the question:

Can I make some C# that compiles and contains these two statements? What type are d and x?

d = null;
d += x;

I wrote the question as a result of my surprise that you could do this with delegate types. In other words, you could add things to a null delegate.

However, it turns out that it works with other things too, including strings. It seems that a += overload (which is how we get the behaviour that allows us to use += to append strings and add handlers to delegates) is also smart enough to make a new object if it turns out that the original is null. Which makes very good sense I suppose. 

Using the StopWatch class properly

Most people don't really care about the speed that their programs run at. Unless they run too slow of course. 

However, if really do want to know timings, C# provides a rather useful StopWatch class in the System.Diagnostics namespace that you can use to measure the time it takes your program to do something. You use it like this:

Stopwatch stopwatch = new Stopwatch();
stopwatch.Start();
// Do something that takes a while
stopwatch.Stop();
Console.Error.WriteLine("It took: " + stopwatch.ElapsedMilliseconds);

It is especially useful if you're trying to re-write a program so that it uses parallel processing, and you want to find out how long things take to complete. Parallel programming is where your code makes use of all the processors in your computer, not just one of them. It should be faster. 

I'm writing some stuff about parallel features in C# at the moment. For a book that I might have mentioned. So I wrote two versions of the code and then discovered, to my dismay, that the one I'd carefully optimised actually seemed to run slower than the original. I know that for small data sets a parallel solution might not be worth the effort of setting up all the parallel gubbins, and I also know that if there are any shared variables that the parallel code ends up fighting over, this can impact on speed, but whatever I did, the parallel version always took at least as long as the original. Wah. Took me a little while to find the mistake. Here's my code:

Stopwatch stopwatch = new Stopwatch();
stopwatch.Start();
// Do single threaded version
stopwatch.Stop();
Console.Error.WriteLine("It took: " + stopwatch.ElapsedMilliseconds);
stopwatch.Start();
// Do parallel version
stopwatch.Stop();
Console.Error.WriteLine("It took: " + stopwatch.ElapsedMilliseconds);

The parallel version always, always takes longer than the serial version. Have you found the mistake yet?

Turns out that stopping and starting a stopwatch doesn't reset it. So the time for the parallel version is added onto the time for the original. And I'm an idiot. The method: 

stopwatch.Restart();

- resets the stopwatch and starts it, so that my second operation is timed correctly. I'm now getting sensible speedups, which is nice...

"In-band" error signalling is dangerous

38848188484_28844cceab_z.jpg

I've been working on the Hull Pixelbot for what seems like ages (You probably think I've been blogging about it for roughly as long. Don't care. My blog.)

Anyhoo, today I learned the perils of "in band error signalling". The phrase "In-band" is a radio term. It means that control information is sent in the came channel as the data. My "in-band" errors worked like this:

int findVariablePos (char * name)
{
    int result;
     /* Stuff happens in here to find the variable  */
    return result; // return the offset of the variable */
}

The function findVariablePos returns the offset into the variable name table of the variable with the given name. In a program a variable is a named location that you can use to store data that is used as the program runs. If my program does this:

i = 99

- this is an attempt to store 99 in a variable called 'i' for no readily apparent reason. The thing running the program needs to have a way of finding out where in the computer the variable i is actually stored. That's the job of findVariablePos. Deep inside the program that actually runs the Hull Pixelbot script there is a statement like this:

int iPos = findVariablePos ("i")

The statement above (we're writing C by the way) would get me the position in the variable table of the variable with the name 'i'. "Aha", you say. "What if the program doesn't contain a variable called 'i'. ". Well, in that case the findVariablePos function returns the value -1 to indicate that the name was not found. This is kind of sensible, because you can't have anything at the position -1 in a table, negative numbers are meaningless in this context. All good. To make things clearer I even did this:

#define VARIABLE_NOT_FOUND -1

This gives meaning to the value, so that I can write tests that make sense:

if (iPOS==VARIABLE_NOT_FOUND)
{
    Serial.println("Variable not found");
}

All good. Works fine. Then I re-factor the code and add a bunch of new error codes. I then decide it would be nice to have a set of numbers that the user (and other programs) can use to make sense of error messages. And I make the following change:

#define VARIABLE_NOT_FOUND 2

This makes sense in the context of fiddling with my error numbers, but it means that if the program ever puts a variable in location 2 in the variable table, the program will completely fail to find it because every time it gets the offset value this will be regarded as meaning that no variable was found. 

Which is of course what happened. The problem in caused by a bad design decision (using the data value as a means of signalling errors) and then doing something without considering the consequences. The latest version of findVariablePos looks like this:

int findVariablePos (char * name, int * position)
{
    int result;
     /* Stuff happens in here to find the variable  */
    
    *position = result;
    return FOUND_OK;
}

The result of the call is returned via one channel (the result of function) and the position value is returned by the method setting the value of the second parameter, which is a pointer to the variable. The call is a bit more complicated:

int iPos;
if (findVariablePos ("i",  &iPos) == FOUND_OK)
{
    Serial.println("Found the variable");
}

However, it now doesn't matter what the error numbers are, and whether or not they clash with any valid variable positions. 

"In-band" error handling is great if your'e in a hurry and you're trying to keep the code simple and quick. But they also leave you open to problems further down the tracks. 

I'm an Idiot

I'm an idiot. Spent half a day chasing a bug that isn't really there. I've been writing the code that lets you download new programs into the HullPixelbot over the network. It worked fine for tiny programs, and then failed for larger ones.

I can hear you thinking "Buffer overrun". Except that I'm not that kind of idiot, and my code is carefully protected against too much data arriving when it shouldn't. And the fault didn't appear at a consistent point in the conversation, which it would if the program was hitting a hard limit somewhere. The failure threshold moved about a bit. Sometimes the program failed with a ten line program, other times it failed with an eight line program.

Any ideas? Took me a little while to figure it out. And of course it was my own stupidity.

It turned out that my over enthusiasm for code instrumentation was my undoing. When you are writing embedded programs you need code instrumentation to stay sane. In simple terms code instrumentation is "putting in print statements to see what is happening". My little program was notifying me each time a data byte arrived so I could convince myself that all was well.

And therein lies the stupidity. It was sending out three bytes of diagnostic information for each incoming byte. Since the send and receive rates are the same, this meant that after a while the program became unable to deal with incoming data because it was taking too long to send out the debug information.

The Arduino will do some buffering of data in and out of the device, but at some point this will fill up, at which point bad things happen. It's not going to happen at a consistent point in the program because the serial timings will vary slightly as other events occur on the processor.

So, I took out the debug code and the program worked perfectly. Strange but true.

And I'm an idiot.

Programming and DIY

29813578533_d434171e2d_z.jpg

I'm not very good at DIY. But I'm the cheapest labour I know, and so I've had a fair bit of practice over the years. As I was laying down each of the strips of flooring yesterday I got to thinking about how the experience of my decorating efforts maps onto writing code. I think there are some parallels here. 

When I started doing DIY I'd measure everything and then cut to precisely the dimensions that I had measured. Then I'd find that none of the corners in the room were square and that lots of my carefully measured items were the wrong size. I'd also find that, because I'd sometimes misinterpreted the numbers from my careful measurements, I'd make things the wrong size anyway. Unhappy times. 

Nowadays I don't measure anything. I hold up the next piece of flooring against the hole that I'm filling at that time, mark it off and then cut it directly. If the size of the room changes as I move down it (and it does) then my process will automatically take this into consideration. And because I don't write things down there's no chance of any errors creeping in.

You can have exactly the same experience when you write software. If you establish the specification at the start and then just build the thing you can be pretty much sure that it will be wrong. Much better to make something, play with it, and then build on top of that. By iterating and refining as you go you'll end up with a much better product. 

Oh, and one other thing I do is work "with" the job rather than against it. If I cut a piece of flooring a couple of millimetres too small there'll be a tiny gap that hardly anyone will see and nobody will care about. If I cut the flooring two millimetres too big it won't fit, which is much more of a problem.

In a programming situation you need to find out what things about the project are critical and make sure that you get those things right. It's also useful to know where you have some flexibility in the specification. Remember that there is no such thing as a perfect solution, just ones that make the user happy and ones that don't. 

At the moment (i.e. nothing has fallen off) the results of my job make me quite happy. 

Interrupts Interrupted

I've spent some time writing smooth movement routines for the motors for the HullPixelBot. I want the motor movement to run in the background, so that the program can do other things as the robot moves. This will make the drive very like the behaviour that you get from a traditional "servo" motor that starts turning when the program switches it on.

Thing is, the stepper motors only turn if the program sends the appropriate set of signals to the coils in them. The solution to this problem is to use the timer on the Arduino processor. This can repeatedly trigger a bit of program code at regular intervals using a hardware technique called an interrupt.

Interrupts are used by a computer to respond to real world events. Tapping a touch sensitive screen, moving the mouse or pressing a key are all things that interrupt a running program and cause the computer to run a special piece of code called an interrupt handler which deals with the event. In the case of a key press, for example, the operating system (perhaps Windows 10) responds to the interrupt by sending a keyboard message to the currently active program.

In the case of the HullPixelBot I'm using a timer to produce a sequence of interrupts at regular events which are then used to trigger code which moves the stepper motors to their next position. The code required to do this is quite tiny, and so once the move has been completed the "normal" program can be resumed.

I don't reckon you can call yourself a programmer unless you've written an interrupt handler or two. I've done quite a few and they are rather fraught bits of code. You're not allowed to touch any of the things used by the "normal" program (in case you confuse it), and you must complete your action in the shortest possible time, otherwise you'll bring the computer to a halt. Programming ninjas can write code that uses multiple interrupts from different sources at the same time. 

Which is what I want to do. 

I want programs to be able to use the distance sensor and the motors, and for both of these devices to work using interrupts. Now, I think of myself as a bit of a programming ninja, so I knock out the code and fire it up. And it doesn't work. 

I can have motors, or distance sensor, but not both. 

I'm not surprised by this because it can be tricksy if interrupts occur during interrupt handlers. So I check the hardware manual for the processor and get myself into a state where I'm convinced it should work, but it doesn't. Hmm. Perhaps I'm not a ninja after all. 

So it was back to the start, and a careful re-write of the code to make it as smooth as possible, and remove any ways in which the two handlers (one for the timer, one for the distance sensor) could interact. And it still didn't work. No fair. 

Of course, I eventually figured it out (otherwise you would not be reading this blog post). It turns out that it had nothing to do with the interrupts. It was just that when the program turned the motors on it dropped the voltage level in the robot power supply to such a low level that the distance sensors stopped working. I was testing the code on a machine on my desktop and I'd left the motor power supply turned off so that the robot wouldn't try to run away. That lack of power was my undoing. 

The good news is that I'm now pretty confident that the code I've written is solid, and the HullPixelBot can move in a very nice, precise way, measuring distance values as it goes. And I've learnt something about motor power and distance sensors, which is useful. 

I'll be showing the code off at the hardware meetup on Thursday this week.