Formatting Code Listings in Kindle Books using the Html Agility Pack

Well, that could have gone better. I got an email from Amazon suggesting that I buy a copy of the C# Yellow Book.  I get these from time to time, and this time I thought I'd tweet about it as above.  It turned into a very popular tweet (for one of mine).

Anyhoo, with my ego nicely built up I thought I'd took a look at the Amazon page for the book. And I found a 1 star review which noted that they would have liked the book a lot more if all the code samples were properly formatted....

Oh dear. Turns out that if you view the book on your iPad or iPhone the code samples all get printed on one line. I thought I'd checked this, but apparently I hadn't. So I did some digging. 

Kindle books are basically HTML documents, a bit like web pages. Like web pages, if you want to tell the renderer that you have already formatted the document you can use the <pre> </pre> enclosure to mark text that already has a layout. You can put code samples into text without their layout being damaged. 

But there is another way to do this.  You can add "white-space: pre-wrap; " to your styles for the pre-formatted text. This works fine on Kindle devices, Android devices. But not iOS devices.  Guess which technique I'd gone for. My reasoning was sensible enough, I wanted to add other stylistic touches to the code samples (a grey background for example) and it made sense to do it all in once place. But it didn't work.  Stupid me.

I had around 200 pages of text with lots of code samples, all of which were wrong. And a broken text up on Kindle that I really, really, needed to fix quickly. So I did some digging and came across the Html Agility Pack on CodePlex. This is completely wonderful. 

It provides a way of reading in a large HTML text and then traversing the notes in the document and fiddling with them.  Turns out all I needed to do was load each of the chapters and then do this:

void processNode(HtmlNode node)
{
    foreach (HtmlAttribute attribute in node.Attributes)
    {
        if(attribute.Name == "class" && attribute.Value == "CodeExplained")
        {
            node.Name = "pre";
            attribute.Value = "CodeExplainedPre";
            debugString.AppendLine(node.InnerHtml);
        }
    }
    if (node.ChildNodes != null)
    {
        foreach (HtmlNode childNode in node.ChildNodes)
        {
            processNode(childNode);
        }
    }
}

This starts at the base node and then looks for anything with the class CodeExplained. It then changes the name of the node to pre (for pre-formatted) and changes the attribute to CodeExplainedPre. 

It is not very elegant, but it does use recursion. If a node contains any child nodes it calls itself to sort those out too. I was going to figure out the structure of the document and only target the page for fixes, but I was in a hurry and this code meant I could reformat the document and make it to the coffee break in time. 

Note: There are probably lots of much cleverer ways of doing this using the Document Object Model or regular expressions or something. But at least this worked and got I was able to get the fixed version up on Kindle within the hour. 

HTML Wrestling

I'm presently working in a much improved Kindle version of the World Famous (in my world) C# Yellow Book. Because the book has been written and upgraded over a number of versions the text formatting is a bit of a mess behind the scenes. It's taking a while to sort out.

I changed the style-sheet to make it a bit easier to see the code embedded in the text and got a preview page that I quite like above. With a bit of luck it should be finished in a week or so.

Programming At Hull Web Site

Over the last couple of days I've spent some time adding content to our WhereWouldYouThink web site. This is a site that has lots of stuff that might be useful to students in general and computer science students in particular.

I'm trying to rationalise some of the support materials that we have for programming and I've attacked the problem in the only way I know how.

I've bought another domain name.....

I've set up a microsite at the address programmingathull.com. This brings together all the stuff that we've put together to help folks learn to program. This includes the wonderful C# Yellow Book and Arduino and Python content. If we make anything else useful we'll put it there too.

Updating the C# Yellow Book

Yesterday I got an email from Amazon telling me that readers had spotted some spelling errors in the Kindle edition of the C# Programming Yellow book. They are quite right. The errors crept in when I used the Amazon system to convert the word file into an eBook. They've been bugging me for a while.

Not that the errors have stopped my book becoming a best seller......

Anyhoo, I've decided to fix these pesky errors once and for all. I'm doing a new conversion of the text, moving it by hand from Word to HTML, and thence to Kindle.

Lots of things about the layout of the first version of the book didn't translate well into eBook format, hopefully the updated version of the text will be easier to read. It should be out in a week or so, once I've reformatted the text and tidied it up.

There's also some shiny new content that I've added to bring things up to date. 

Birthday Bluetooth

It's my birthday today. I had my treat over the weekend and am now saving up for a Segway (just about doable) and a twenty acre estate that I can use it in (pretty much impossible). So I'm here in the office, eating chocolates and writing C#. Which counts as a pretty good situation in my book. 

Anyhoo, I've been playing with a present I bought myself last week. It is a Texas Instruments SensorTag. You can pick these up for a very reasonable sixteen pounds or so and they are enormous fun (if you like connecting devices to sensors). It talks Bluetooth BLE and I've fancied having a go at this for a while.

Turns out to be easy to get it to connect to a Windows 8.1 device. Just remember that for Windows you have to pair the tag over Bluetooth. It just works with iOS and Android - I hope that they remove the need for pairing with Windows at some point. 

If you fancy having a go I've written a very simple universal app and put it on GitHub. You can use this to connect to the accelerometer in a SensorTag and get events fired in your program when a new reading is produced. 

I used a superb post from Dan Ardelean to get started, and just built a little wrapper class around methods that he described.  Great fun.

Scary Things You Shouldn't Do

One of my programming rules is that "just because you can do something, doesn't mean that you should". Take this piece of magic for example:

This is a lump of XAML, the code that you use to design the user interface for a Windows application. It tells the computer what the display will look like and defines display objects which link to the underlying code that provides the behaviours.

Except that it turns out that you can actually put code into your XAML, along with the display markup.

You should never do this. But I guess that knowing about it is cool. Find out more here.

Finding out my "Hacked to Properly" Ratio

Spent quite a big chunk of this evening discovering that code you write during a GameJam is not actually the most solid of stuff. I've reached the point where I'm piling dirty hacks on top of horrible code to try and get something almost working. Oh well, it looks like the time for a total rewrite is here.

The good news is that I can take all the ideas and stuff over to the new design. The bad news is that I'm going to have to do it. Mind you, at the end I can probably work out my "Hack to Properly" ratio. This is the ratio of time it takes to hack something together vs the time to do a proper job. The code took around a day to write first time. Lets see how long it takes to write the new one.....

Windows Phone, Wedding Lights and Bluetooth

Windows Phone connected Wedding Lights. Eight synchronized lights which can be controlled over Bluetooth. Each light contains an Arduino processor and a 16 NeoPixel ring.

I've finally finished it. I was going to write an article about my Windows Phone Controlled Wedding Lights. But instead I thought I'd do something different. So I fired up Adobe Premier and I made a video about them instead. It only lasts a couple of minutes, but boy was it complicated to make. Anyhoo, feel free to take a look and let me know what you think.

I've done something else I've not done before (and I feel a bit guilty about this one). I've put the Bluetooth code for Windows Phone 8.1 and Windows 8.1 up on Codeplex. I'm ashamed to admit that this is the first code that I've ever posted there. I really should have been posting stuff up there earlier. I'm determined to post more stuff as I come up with it.  

You can find a sample project (my Bluetooth Printer) and the BluetoothManager class that I used to communicate with the embedded Bluetooth controller. There are also details of how to configure the Bluetooth device and send and receive data. 

Wrestling with Pygame and Cheese

This pictures is entitled "Unsuccessful Cheese Movement Number 1"

The "Wrestling with Python" sessions on Pygame are progressing well. Today we got some cheese moving around the screen under keyboard control. Next week we will add some more sprites and get some proper game action going. You can see on of the first attempts at moving things above. Perhaps we should clear the background after each draw.....

It occurred to us (in a rather nervous-making way) that we are exactly four weeks away from the big event on campus when folks are going to come in and make and show off games. But everyone is learning stuff and having fun, so actually I'm quite looking forward to it. 

If you want to find out what we have been doing you can download the content from here. 

Making Screencasts the Hard Way

Having the house to myself this morning I thought I'd record a video of the 08120 Programming 2 Exam walkthrough. I try to remember to make these when I've finished marking the papers. I can cover lots of the questions that folks might have about the right answers and I like to do this when the marking issues are still fresh in my mind. 

Of course it wasn't as simple as I expected. I hadn't done a video for a while, so I had to find the headphones and the microphone, install Camtasia (my favourite program for recording stuff), find that the headphone and microphone weren't working and that sound playback was also broken. Fix all that, set the screen size to work properly, create a PDF that I can browse through and finally, after half an hour of faffing around, get to sitting down and recording something. 

And I thought that modern technology was here to make life easier....

Yellow Book now in Spanish

I love it when Amazon send me recommendations for things that I've written....

I saw on Twitter that folks are working on a version of the C# Yellow Book in Spanish, which is rather nice.  

Good luck folks. 

Making a quick Batch File

It's not often that I get to show off my Command Prompt smarts. But now and again they are useful. The Command Prompt is the place in Microsoft operating systems where you type, er, commands. In the days of MS-DOS it was the only way that you told the computer what to do and it has survived, mostly unscathed, all the way into Windows 8.1 

Of course, in these days of touch and windows you don't often have to give commands at the console, but every now and then you do, and today I got to show off a trick that I've used for years. We were typing in long commands to start up services in the Microsoft Robotics Framework and I suggested that we make a batch file rather than type everything in each time. You can use the "up arrow trick" (press up arrow to get the previous command) but that doesn't work if you close the command prompt and re-open it, as this clears the command buffer. 

A batch file is simply a sequence of console commands that are stored in a text file and can be executed just by giving the name of the file. It has the language extension ".bat". You can run them from within Windows too just by clicking on them. 

What you can see above is a way that you can take a long and complicated command that you have typed in and convert it into a batch file.  You use a few tricks. First trick:

copy con doit.bat

This connects the console (your keyboard and screen) to a file called doit.bat. Essentially, everything I type from now on will go into this file. 

Second trick:

(press up arrow to get your command)

Remember that I said you can press up arrow to get back things you have previously typed. This works when you are copying into a batch file too. So to my recover my echo command in the above screenshot I just pressed up a few times. Once I had the command on the screen I just had to press Enter to put the text into the batch file. Note that if you want to enter a sequence of commands that you have typed into your batch file you just have to recover each one in turn. 

Final trick:

^Z

This is CTRL+Z. It tells the command prompt you've finished entering stuff into the file and want to return to typing commands. The destination (in this case the file doit.bat) is closed and then you can just run the batch file by giving the name. 

The console is a powerful thing in the command prompt. You can copy things to it (which can gives a behaviour similar to typing a file) and you can copy things from it. You can also do stupid things:

copy con con

This works, I'll leave you to figure out what it does. And how to get out of it. 

C# Code Snippets on Demand

This is kind of neat. Rather than have to jump out of your IDE to find samples of how to use language and API features, you can just put the question into a comment and hit tab. 

It is a bit hit and miss at the moment (and doesn't know anything about XNA) but it would be a very interesting future development of my favourite place to work.... Have a play yourself here. 

What does ?? mean in C#

I sent out a tweet asking folks if they knew what the ?? operator in C# does. Quite a few people did, which was nice. For those of you that haven’t heard of it, here is my explanation.

If you are a C# programmer you will know about references. The way that C# works, a variable can be managed by value or reference. A variable of value type is stored in a location in memory. For example:

int i;

This creates an integer variable called i. When the program runs the system will decide where in memory the variable should live. You can think of computer memory as a whole bunch of numbered locations. Because that is what it is. Perhaps the variable i could live at memory location 5,000. Then when we write the code:

i=99;

The effect of this is to put the value 99 in memory location number 5,000

So, if I write code like:

j = i;

This would copy the value held in memory location 5,000 into the location where the integer variable j is stored.

This is how value types work. Whenever we do an assignment of a variable managed by value what happens is that the value is copied from one memory location to another.

So, what about values managed by reference? Well, in these there is a level of indirection between the variable and the actual data. Suppose we have a class called Account, which is managed by reference.

Account a = new Account();

This statement makes a new account variable and then sets the variable a to refer to it. The variable a will hold a reference to the new Account. Perhaps the new Account will be located at memory location 10,000 which means that the variable a (which might be stored at location 6,000) will hold the number 10,000 – because that is where the Account instance is stored. The Account class might have a Name property, so I can write code like:

a.Name = "Rob";

When the program runs it will go to location 6,000 (where a is stored) and read the number out of there to find out where the Account is. In this case the variable a holds the number 10,000 and so the program will go to the Account there and set the name.

So if write code such as:

Account b = a;

This creates a new Account reference called b which refers to the same Account instance as a, in other words it will refer to location 10,000.

So, in the case of the value the information is just there in memory, but for a reference we have to go where the reference refers. With references you can also set them to null:

a = null;

This has the effect of putting a “magic value” in the variable a that indicates it really points nowhere.  It is a way of saying “this reference does not point to any object”.

The null reference is often used in programs to indicate that the thing you asked for could not be found, or hasn’t been made yet, or doesn’t matter.

This “nullability” is so useful that people wanted to be able to make values “null” as well. So they invented one.

int? ageValue;

The addition of the question mark makes an integer variable (ageValue) that can be made null. For example, the bank might store the age of a customer when it is important (when the age is less than 20 say) but once a person reaches a certain age, from then on the age is completely irrelevant to the system. They can mark the ageValue as null to indicate this.

ageValue = null;

Programs can also test for null

if (customerAge != null)
{
    // Stuff you do if the age matters
}

In XNA you often find nullable value parameters being sent to method, so that the method can know to ignore them.

So, I’ve been writing for what seems like ages, and I’ve still not explained what ?? does.

Well, ?? provides a convenient way that I can map the null value of a nullable variable onto a specific value

int actualAge = customerAge ?? -1;

It saves us having to write code that tests for null and sets a value appropriately. The above statement sets the value of actualAge to the value in customerAge unless the value in customerAge is null, in which case it sets it to –1.

if (customerAge == null)
    actualAge = -1;
else
    actualAge = customerAge;

In other words ?? is a short form of the above test.

Going to Hacked

Next weekend we are off to Hacked. I’ve managed to get hold of a couple of tickets so myself and number one son will be taking our places with lots of other developers who want to make something silly/awesome/fun. I’ll be taking my lovely Lumia 920 along and targeting the Windows Phone platform, along with a bunch of gadgets and other things that we can use if mood takes us.

I’m really looking forward to this. The plan is to try and last through Saturday night. But I’ve booked a room in a nearby hotel just in case…..

Yellow Book Translations

Every now and then someone emails me and asks if it OK to translate my C# Yellow Book into their native language.

Of course it is.

I’ve added some notes about translation to the home page for my programming resources. Essentially you can translate into any language you like as long as you keep the colour of the cover yellow.

If you want to sell your translation that's fine too, but you are not allowed to charge more than the price of a translation of the novel "Pride and Prejudice" by Jane Austen in your language.

If you do manage to get all the way to the end of the book, let me know and I’ll link through from my pages to your translation.

What use is a structure?

It is exam season here in Hull. Later in the week we have our programming exam. And I’m getting quite a few questions through about C# and stuff. Today I got asked the question “What use is a struct?”. I’ve been spending a lot of time talking about classes and references and how clever they are, and someone wants to know why, if classes are so wonderful, some objects in a program are managed by value. The answer is simple enough. It is because sometimes you really want a value which you can just copy around easily.

Consider the Rectangle type in XNA. We can use this to represent the position and size of something on the screen. And this is a struct, not a class. Take a look at this code:

a = b;
a.X = 99;

If a and b are both Rectangle variables the effect would be to set a to the value in b and then move the X position of a to 99. This would not affect the position of b.

If the Rectangle type was a class then we would have two references, a and b, that both refer to the same object in memory, so moving a would affect b as well. If we want to place objects in lots of different positions on the screen, and we don’t want them to be linked in any way, then structures managed by value is the way to do it.

Note that things like Textures are managed by reference. This makes a lot of sense too. An image is a large thing, and it is often very useful to be able to share one image for a whole bunch of things. Think multiple sprites in a space shooter. They will have a Rectangle value to give them a unique position and then a reference to a texture that they all share, to give them an appearance.

The XNA framework is full of objects that are actually structures because they work better this way. For example Color, Point and the Vector objects are all structs so that we can manage them by value. In fact, now that you know how it works, you should be able to look at any XNA type and figure out whether it is a class or a struct, just based on how it is used.

Class Design for Shops

At the moment lots of our students are working on their first year coursework. This year they can make either a game, which I’ve called “Space Killers” or a management system for a record shop, which I’ve called “Vinyl Destination” (turns out that there actually is a shop with that name – which only goes to show that great minds think alike sometimes). By the way, a record is what very old people used to buy music on. They are pressed out of plastic, usually 7 or 12 inches in diameter, easy to scratch and becoming fashionable again, which is nice.

Anyhoo, one of the central design decisions that has to be made is just how to store information about the records that the shop has in stock. We have been discussing this in the lectures, and it is interesting to reflect how the business needs of the customer affect the organisation of their data. Lots of shops have things to sell, but the way that you would hold their stock information varies from one business to another. I’ve broken these down to three broad categories which I’ve called “Art Gallery”, “Car Showroom” and “Supermarket”.

Art Gallery: In an art gallery each item of stock is unique. Or at least it better had be. Turning up with a second copy of the “Mona Lisa” would probably raise one or two eyebrows. Every item in the gallery has properties such as the artist or artists that made it, the date, the description and the price, but this information would be held for each individual piece in stock. To store the data for this application we would design a class, perhaps called “StockItem”, and then create an instance of this class for each of the pieces that is held in stock.

Car Showroom: This situation is slightly different. The showroom will hold a large number of cars, but they will be of particular models. They will hold a large number of “Ford Fiestas”, but each of them will have different trim levels, colours, ages and prices. If we held the data about our cars in the same way as we hold the data in the art gallery we would store a lot of the same data multiple times. Every individual StockItem would hold its own copy of the name, the manufacturer and lots of other information common to all Ford Fiesta models. A better way to design the system might be to have a class, perhaps called “CarModel”, which holds all the information about the type of car (all the “Ford Fiesta” parts) and then have the “CarModel” class hold a list of all “StockItems”. The StockItem class would hold all the information about a particular car, including things like colour, mileage and price. The CarModel class could hold a list of the StockItems of that particular model. For example, if the garage had a red Ford Fiesta and a blue Ford Fiesta there would be a CarModel object for Ford Fiesta which holds a list containing two items, a StockItem for the red one and a StockItem for the blue one.

Supermarket: In the case of a supermarket which is selling cans of beans, all the cans of beans of a particular type are identical. In this case we don’t need to store information about any individual can, instead we just want to know how many we have in stock. So for the supermarket we just need an object that holds a description of the item and also contains a counter that tells the system how many of that item are in stock.

If you are building a system for a customer, it is worth considering which of the three arrangements makes most sense. In terms of Vinyl Destination I reckon that because the you can get records of different quality it is more like a garage or an art gallery. The shop may have several copies of “Abbey Road” by The Beatles, but some will be in pristine condition (and therefore worth more) than others.

Making Sample Data Can Be Fun

I’ve just released the first year coursework specification. We are going to be creating a system for a record shop which will track customer orders and suggest new records to buy. I think the only reason I went for a record shop was so I could call it “Vinyl Destination”, but that is by the by.

The program needs to be able to track huge numbers of records and customers, along with their orders. The problem is that, at the start, we don’t have any customers or orders, or anything. And anyone that thinks the way to solve this problem is to sit down and enter some details when the program has been built is very, very wrong.

I make the test data right at the start. I use a program to do it. All it takes is a few loops. Now, I could give the customers names like “Customer1”, “Customer2” and so on. But I can do better than that. All I need are some names to start with:

string [] firstNames = new string [] { "Rob", "Fred", "Jim", 
  "Ethel", "Nigel", "Simon", "Gloria", "Evadne" };
string [] surnames = new string[] { "Bloggs", "Smith", 
 "Jones", "Thompson", "Wooster", "Brown", "Acaster", 
  "Berry", "Ackerman" };

Here we have two arrays, one of first names and the other of surnames. I can now make myself a whole bunch of reasonable looking customers:

foreach (string surname in surnames)
{
    foreach (string firstname in firstNames)
    {
        Customer c = new Customer(firstname + " " 
                                  + surname);
        result.Customers.Add(c);
    }
}

This makes us a whole bunch of customers, from “Rob Bloggs” all the way to “Evadne Ackerman”. We can do something similar with the names for each of the recording artists:

string[] artist1 = new string[] { "Pink", "Flying", "Random", 
    "Singing", "Uptown", "Family", "Floating" };
string[] artist2 = new string[] { "Chicken", "Circus",
    "Banana", "Kitchen", "Groats", "Monkey", "Collective", "Pyjamas" };

I can combine the words to make random artist names.

string artist = artist1[rand.Next(artist1.Length)] + 
    " " + artist2[rand.Next(artist2.Length)];

This picks a random word from the first list and adds it to the second one (rand is just a random number generator that I use to make my test data). So I can have “Pink Pyjamas” and “Singing Groat”, among others. I do something similar for the track names, except that I have three parts. As you can see in the screenshot, this makes for some quite funny combinations, that liven up testing no end.

Don’t Make the Recipe Static

Today it was time to talk about static class members in the first year programming course. I always have problems with static. If you are not careful you start thinking that static means “doesn’t change much”. This is reasonable in real life, but not in C# programming. In C# programming static means “always present, part of a class and not of an instance”. At this point I thought an example would serve well, so we started talking about contexts where static would make sense.

Consider that you are implementing a system for a fast food store. You have a class called “Dish” and the system stores an instance of Dish for each of the items in the menu. A Dish instance holds the list of ingredients for the dish, the sale price, and the name of the dish. For example, there might be a dish called “Chicken and Chips” which held two ingredients (chicken and chips) and a price value (perhaps four pounds 50 pence). I asked everyone for some suggestions for static class members for the Dish class. Someone suggested that recipe would be a good candidate for a static member.

Not so. The idea of a static member is one that exists as part of the class. Which means that recipe could only be static if every dish was cooked in the same way (i.e. the recipe exists once for the entire class). Each dish needs its own recipe, so this must be a non-static member of the Dish class. However some things, for example the maximum and minimum price of dishes, the maximum number of ingredients that a dish can have all make sense as static members of the Dish class.

Simon has a sign on his office wall, “Consider the context”. When trying to work out what makes sense as static and what doesn’t, this is very good advice. And a good starting point is that “a recipe should not be static”.