C# Quick Question 2 resolution

/

Ha. Nobody seemed to know this. Or nobody cared....

Anyhoo, the question was:

"When would a value type be stored on the heap?" And no, the answer doesn't involve boxing.

A value type just holds a value. Not a reference to something. If I have an object managed by reference I have to be careful about removing it from memory, because I need to be sure that nothing in the program is referring to the object. This means that objects managed by reference are stored in a "heap" of objects. It is the job of the "garbage collector" to find objects that are not referred to by anything, and remove these. 

However, value types are easy to get rid of. There can't be any references to them. 

Think of a value type as a book in a library. We can take the book away and nobody can use it any more. That's sad. But we can be sure that nobody is using the book when we take it away. Think of a reference type as a web site. We can't just delete a web site because we don't know how many people out there have made references to the site. 

So, when do we need to put a value type on the heap, and treat it a bit like a reference type? One answer is when we "box" a value type to convert a value type into a into a reference type, but this this is not what I'm after. 

The answer is when a value type is used in a "closure". We see closures when we use lambda expressions. OK, so what's a lambda expression? Lambda expressions are a pure way of expressing the "something goes in, something happens and something comes out" part of behaviours.

The types of the elements and the result to be returned are inferred from the context in which the lambda expression is used. Instead of writing a method and creating a reference to the method, you can just create a lambda expression.

We get into lambda expressions shortly after we start trying to treat lumps of functional code as we would data. Consider the following method:

static int add(int a, int b)
{
    return a + b;
}

This method is called add, it takes two integers and returns their sum. Maybe, for some reason, I want to my program to select between adding, subtracting and some other mathematical operations that take in two integers and return an integer result. C# lets me create pointers to methods. They are called delegates and we would make such a delegate like this:

delegate int IntOperation(int a, int b);

A delegate is another type I can use in my program. So I can create a variable of type IntOperation. And I can make my delegate refer to a method. Watch carefully:

IntOperation thingToDo;

This statement has made a delegate called thingToDo that can refer to methods that accept two integers and return a single integer result. Now I can make this delegate refer to a method:

thingToDo = add;

Now, if I call thingToDo, the add method will run:

int result = thingToDo(1,10);

This would set the value result to 11, because thingToDo presently refers to the add method. With me so far? Good. We can create our methods "the hard way", or we can use a lambda function instead. Look at this statement:

thingTodo = (a, b) => { return a - b; };

Don't panic. It wants to be your friend. Let's take a careful look at what I've done. I've made thingToDo refer to a lump of code that will perform a subtract operation. The code that does the subtraction doesn't live anywhere and has no name.  It is a lambda expression .Sometimes called an "anonymous function". The sequence => is the lambda operator and it separates the "stuff goes in" from the "stuff comes out" parts of the expression. In this case the parameters a and b to in, and the result of a-b comes out. Because thingToDo accepts integers and returns integers, inside the lambda expression the types of a and b are integers. 

OK. We can put these anonymous lumps of code in our program and pass them around on the end of references, just like any other object. What about closures? Take a look at this strange code:

{
    int localValue = 99;
    thingTodo = (a, b) => { return a + b + localValue; };
}

This is stupid, useless code. The statement inside the lambda function is using the local variable localValue for some silly reason. This is quite legal, compiles fine, and gives the compiler a problem to solve. The problem is that the lifetime of localValue exceeds the block in which it is declared. Remember that local variables are usually stored on the stack and deleted when the program leaves the block of code in which the local variable is declared. However, thingToDo is still referring to a lambda expression that uses the value of localValue. So localValue can't die. It must live on outside the block in which it is declared.

The compiler solves this problem by putting the localValue variable on the heap, rather than the stack. The heap is where we keep stuff that has to stick around for a while. This extension of the life of a local variable is called a "closure". 

Short question. Long answer. Phew. 

C# Quick Question 1 resolution

/

This was the question:

Can I make some C# that compiles and contains these two statements? What type are d and x?

d = null;
d += x;

I wrote the question as a result of my surprise that you could do this with delegate types. In other words, you could add things to a null delegate.

However, it turns out that it works with other things too, including strings. It seems that a += overload (which is how we get the behaviour that allows us to use += to append strings and add handlers to delegates) is also smart enough to make a new object if it turns out that the original is null. Which makes very good sense I suppose. 

Using the StopWatch class properly

/

Most people don't really care about the speed that their programs run at. Unless they run too slow of course. 

However, if really do want to know timings, C# provides a rather useful StopWatch class in the System.Diagnostics namespace that you can use to measure the time it takes your program to do something. You use it like this:

Stopwatch stopwatch = new Stopwatch();
stopwatch.Start();
// Do something that takes a while
stopwatch.Stop();
Console.Error.WriteLine("It took: " + stopwatch.ElapsedMilliseconds);

It is especially useful if you're trying to re-write a program so that it uses parallel processing, and you want to find out how long things take to complete. Parallel programming is where your code makes use of all the processors in your computer, not just one of them. It should be faster. 

I'm writing some stuff about parallel features in C# at the moment. For a book that I might have mentioned. So I wrote two versions of the code and then discovered, to my dismay, that the one I'd carefully optimised actually seemed to run slower than the original. I know that for small data sets a parallel solution might not be worth the effort of setting up all the parallel gubbins, and I also know that if there are any shared variables that the parallel code ends up fighting over, this can impact on speed, but whatever I did, the parallel version always took at least as long as the original. Wah. Took me a little while to find the mistake. Here's my code:

Stopwatch stopwatch = new Stopwatch();
stopwatch.Start();
// Do single threaded version
stopwatch.Stop();
Console.Error.WriteLine("It took: " + stopwatch.ElapsedMilliseconds);
stopwatch.Start();
// Do parallel version
stopwatch.Stop();
Console.Error.WriteLine("It took: " + stopwatch.ElapsedMilliseconds);

The parallel version always, always takes longer than the serial version. Have you found the mistake yet?

Turns out that stopping and starting a stopwatch doesn't reset it. So the time for the parallel version is added onto the time for the original. And I'm an idiot. The method: 

stopwatch.Restart();

- resets the stopwatch and starts it, so that my second operation is timed correctly. I'm now getting sensible speedups, which is nice...

"In-band" error signalling is dangerous

/

I've been working on the Hull Pixelbot for what seems like ages (You probably think I've been blogging about it for roughly as long. Don't care. My blog.)

Anyhoo, today I learned the perils of "in band error signalling". The phrase "In-band" is a radio term. It means that control information is sent in the came channel as the data. My "in-band" errors worked like this:

int findVariablePos (char * name)
{
    int result;
     /* Stuff happens in here to find the variable  */
    return result; // return the offset of the variable */
}

The function findVariablePos returns the offset into the variable name table of the variable with the given name. In a program a variable is a named location that you can use to store data that is used as the program runs. If my program does this:

i = 99

- this is an attempt to store 99 in a variable called 'i' for no readily apparent reason. The thing running the program needs to have a way of finding out where in the computer the variable i is actually stored. That's the job of findVariablePos. Deep inside the program that actually runs the Hull Pixelbot script there is a statement like this:

int iPos = findVariablePos ("i")

The statement above (we're writing C by the way) would get me the position in the variable table of the variable with the name 'i'. "Aha", you say. "What if the program doesn't contain a variable called 'i'. ". Well, in that case the findVariablePos function returns the value -1 to indicate that the name was not found. This is kind of sensible, because you can't have anything at the position -1 in a table, negative numbers are meaningless in this context. All good. To make things clearer I even did this:

#define VARIABLE_NOT_FOUND -1

This gives meaning to the value, so that I can write tests that make sense:

if (iPOS==VARIABLE_NOT_FOUND)
{
    Serial.println("Variable not found");
}

All good. Works fine. Then I re-factor the code and add a bunch of new error codes. I then decide it would be nice to have a set of numbers that the user (and other programs) can use to make sense of error messages. And I make the following change:

#define VARIABLE_NOT_FOUND 2

This makes sense in the context of fiddling with my error numbers, but it means that if the program ever puts a variable in location 2 in the variable table, the program will completely fail to find it because every time it gets the offset value this will be regarded as meaning that no variable was found. 

Which is of course what happened. The problem in caused by a bad design decision (using the data value as a means of signalling errors) and then doing something without considering the consequences. The latest version of findVariablePos looks like this:

int findVariablePos (char * name, int * position)
{
    int result;
     /* Stuff happens in here to find the variable  */
    
    *position = result;
    return FOUND_OK;
}

The result of the call is returned via one channel (the result of function) and the position value is returned by the method setting the value of the second parameter, which is a pointer to the variable. The call is a bit more complicated:

int iPos;
if (findVariablePos ("i",  &iPos) == FOUND_OK)
{
    Serial.println("Found the variable");
}

However, it now doesn't matter what the error numbers are, and whether or not they clash with any valid variable positions. 

"In-band" error handling is great if your'e in a hurry and you're trying to keep the code simple and quick. But they also leave you open to problems further down the tracks. 

I'm an Idiot

/

I'm an idiot. Spent half a day chasing a bug that isn't really there. I've been writing the code that lets you download new programs into the HullPixelbot over the network. It worked fine for tiny programs, and then failed for larger ones.

I can hear you thinking "Buffer overrun". Except that I'm not that kind of idiot, and my code is carefully protected against too much data arriving when it shouldn't. And the fault didn't appear at a consistent point in the conversation, which it would if the program was hitting a hard limit somewhere. The failure threshold moved about a bit. Sometimes the program failed with a ten line program, other times it failed with an eight line program.

Any ideas? Took me a little while to figure it out. And of course it was my own stupidity.

It turned out that my over enthusiasm for code instrumentation was my undoing. When you are writing embedded programs you need code instrumentation to stay sane. In simple terms code instrumentation is "putting in print statements to see what is happening". My little program was notifying me each time a data byte arrived so I could convince myself that all was well.

And therein lies the stupidity. It was sending out three bytes of diagnostic information for each incoming byte. Since the send and receive rates are the same, this meant that after a while the program became unable to deal with incoming data because it was taking too long to send out the debug information.

The Arduino will do some buffering of data in and out of the device, but at some point this will fill up, at which point bad things happen. It's not going to happen at a consistent point in the program because the serial timings will vary slightly as other events occur on the processor.

So, I took out the debug code and the program worked perfectly. Strange but true.

And I'm an idiot.

Programming and DIY

/

I'm not very good at DIY. But I'm the cheapest labour I know, and so I've had a fair bit of practice over the years. As I was laying down each of the strips of flooring yesterday I got to thinking about how the experience of my decorating efforts maps onto writing code. I think there are some parallels here. 

When I started doing DIY I'd measure everything and then cut to precisely the dimensions that I had measured. Then I'd find that none of the corners in the room were square and that lots of my carefully measured items were the wrong size. I'd also find that, because I'd sometimes misinterpreted the numbers from my careful measurements, I'd make things the wrong size anyway. Unhappy times. 

Nowadays I don't measure anything. I hold up the next piece of flooring against the hole that I'm filling at that time, mark it off and then cut it directly. If the size of the room changes as I move down it (and it does) then my process will automatically take this into consideration. And because I don't write things down there's no chance of any errors creeping in.

You can have exactly the same experience when you write software. If you establish the specification at the start and then just build the thing you can be pretty much sure that it will be wrong. Much better to make something, play with it, and then build on top of that. By iterating and refining as you go you'll end up with a much better product. 

Oh, and one other thing I do is work "with" the job rather than against it. If I cut a piece of flooring a couple of millimetres too small there'll be a tiny gap that hardly anyone will see and nobody will care about. If I cut the flooring two millimetres too big it won't fit, which is much more of a problem.

In a programming situation you need to find out what things about the project are critical and make sure that you get those things right. It's also useful to know where you have some flexibility in the specification. Remember that there is no such thing as a perfect solution, just ones that make the user happy and ones that don't. 

At the moment (i.e. nothing has fallen off) the results of my job make me quite happy. 

Interrupts Interrupted

/

I've spent some time writing smooth movement routines for the motors for the HullPixelBot. I want the motor movement to run in the background, so that the program can do other things as the robot moves. This will make the drive very like the behaviour that you get from a traditional "servo" motor that starts turning when the program switches it on.

Thing is, the stepper motors only turn if the program sends the appropriate set of signals to the coils in them. The solution to this problem is to use the timer on the Arduino processor. This can repeatedly trigger a bit of program code at regular intervals using a hardware technique called an interrupt.

Interrupts are used by a computer to respond to real world events. Tapping a touch sensitive screen, moving the mouse or pressing a key are all things that interrupt a running program and cause the computer to run a special piece of code called an interrupt handler which deals with the event. In the case of a key press, for example, the operating system (perhaps Windows 10) responds to the interrupt by sending a keyboard message to the currently active program.

In the case of the HullPixelBot I'm using a timer to produce a sequence of interrupts at regular events which are then used to trigger code which moves the stepper motors to their next position. The code required to do this is quite tiny, and so once the move has been completed the "normal" program can be resumed.

I don't reckon you can call yourself a programmer unless you've written an interrupt handler or two. I've done quite a few and they are rather fraught bits of code. You're not allowed to touch any of the things used by the "normal" program (in case you confuse it), and you must complete your action in the shortest possible time, otherwise you'll bring the computer to a halt. Programming ninjas can write code that uses multiple interrupts from different sources at the same time. 

Which is what I want to do. 

I want programs to be able to use the distance sensor and the motors, and for both of these devices to work using interrupts. Now, I think of myself as a bit of a programming ninja, so I knock out the code and fire it up. And it doesn't work. 

I can have motors, or distance sensor, but not both. 

I'm not surprised by this because it can be tricksy if interrupts occur during interrupt handlers. So I check the hardware manual for the processor and get myself into a state where I'm convinced it should work, but it doesn't. Hmm. Perhaps I'm not a ninja after all. 

So it was back to the start, and a careful re-write of the code to make it as smooth as possible, and remove any ways in which the two handlers (one for the timer, one for the distance sensor) could interact. And it still didn't work. No fair. 

Of course, I eventually figured it out (otherwise you would not be reading this blog post). It turns out that it had nothing to do with the interrupts. It was just that when the program turned the motors on it dropped the voltage level in the robot power supply to such a low level that the distance sensors stopped working. I was testing the code on a machine on my desktop and I'd left the motor power supply turned off so that the robot wouldn't try to run away. That lack of power was my undoing. 

The good news is that I'm now pretty confident that the code I've written is solid, and the HullPixelBot can move in a very nice, precise way, measuring distance values as it goes. And I've learnt something about motor power and distance sensors, which is useful. 

I'll be showing the code off at the hardware meetup on Thursday this week. 

Programming 2 Stealth Videos

/

I've started recording some of the first year lectures as screencasts, Just to see how it turns out. I joked with the class that I'd only tell the people that came to the lectures where the videos are, but this is probably not very helpful. Anyhoo, you can find the first one here and I've set up a channel here for the rest.  They're a bit rough and ready, being recorded straight from the Surface using Camtasia and the built in microphone, but they might be of interest to C# students at Hull. 

I'll keep posting them for the rest of this semester. The only slight snag for me is that it takes around four hours to render each captured session so I've got to create a workflow that I can use to do this. 

Say Hello to Dit Lexaragez

/

I've been writing the next lab assignment for our programming course. We are creating a Bank, rather like the one in the Yellow Book.  Anyhoo, we've got as far as making some test data for the system that we are building. Demonstrations of massive account databases are all very well, but it is best if they have more than two or three accounts in them. 

It's best if all the fake accounts have different names, so I started playing with ways to make names programatically. I ended up using a method based on a very popular daytime TV show in the UK. See if you can guess the name of the show. 

My method works because names are frequently made up of the sequence “consonant – vowel –consonant” – for example “Rob”. All my method does is pick a random consonant, followed by a random vowel and then a random consonant. For longer names it just adds more vowel-consonant pairs on the end. 

static Random testRand = new Random(1);

static string vowels = "aeiou";

static string consonants = "bcdfghjklmnprstvwxz";

static string pickRandomLetter(string input)
{
    return input[testRand.Next(input.Length)].ToString();
}

static string makeTestName (int parts)
{
    string result = pickRandomLetter(consonants).ToUpper();

    for (int i=0; i < parts; i=i+1)
    {
        result = result + pickRandomLetter(vowels);
        result = result + pickRandomLetter(consonants);
    }
    return result;
}

static string makeFullTestName()
{
    return makeTestName(testRand.Next(1, 4)) + " " +
        makeTestName(testRand.Next(2, 6));
}

This is the C# that we are using for the lab. I fired it up and the first name that came out was "Dit Lexaragez" which I think is a great name. The names have a vaguely "off-planet" feel that I really like. I've suggested that people might like to fiddle with the letter sequences to modulate how much of each letter you get (you probably don't really want 'z' to appear as frequently as 't' and maybe allow some names to start and/or end with vowels.

I love it when a really simple bit of code produces really interesting behaviours.