A long time ago I mentioned I’d added some (opt-in) usage tracking for FreeGuide. Here are the latest stats. I think they’ve held up pretty well given how much I’ve neglected the project…
I’ve made a new release of FreeGuide today, which contains minor bug fixes.
I changed the stability rating on sourceforge to “Production/Stable” (previously it was “Alpha”). Maybe I should bump the version number next time to 1.0. Certainly all this 0.10.x stuff is crazy.
I managed to get rid of quite a bit of unused and unworking code this release, so I’m feeling somewhat more positive about developing FreeGuide. My next plan is to spruce up the ant build so you can easily use it in Eclipse, and convert the whole lot to a single JAR, which may improve startup time.
The program seems to work for quite a few people, and I don’t change it much these days, so it might be time to call it 1.0. The only major feature I am planning (one day…) is to combine the Find dialog with the vertical view and possibly the favourites editor to make them different views onto the same underlying implementation. I’d like to make all of them show filter results as you type.
Other than that all I really want to do it fix user pain in terms of bugs and UI wrinkles.
Of course, I am bound to get a bug report immediately I have posted this telling me my fix breaks everything, but for the moment I am chuffed that I found, tested, and fixed a genuinely difficult bug.
I am particularly proud because I wrote an automated test to ensure it can never happen again, and I used that test to make the debugging process much easier than it otherwise would be. The code that reads, processes and stores listings in FreeGuide is a spider’s web of interfaces and helper classes (because of the arguably over-engineered plugins framework used for all the moving parts), and tracking this down with plain old-fashioned debugging would have been a huge job.
Anyway, I bet you are dying to hear what the bug was, aren’t you?
When you have a programme already stored in FreeGuide, and then a new one comes along that overlaps it, the old programme is deleted. For example if we start off with:
... 19:00 Top Gear ................... 20:00 Charlie and Lola .............
but then later download listings again and get these programmes:
... 19:00 Pocoyo .. 19:15 Round the World ......
Then Top Gear will disappear, and be replaced by the 2 new programmes. In fact, any old programme that overlaps any new incoming programme will be automatically deleted.
At least, that is what is supposed to happen. In fact, the real situation is a little more complex because the programmes are stored in separate files (.ser files) for different days and times. Actually, there are 4 files for each day, named things like “day-2008-09-15-A.ser”, where the suffix A, B, C and D indicate which part of each day a file is for.
So imagine what happens when the first set of programmes comes in looking like this:
19:00 Programme 1A ......... 21:15 Programme 1B .. 21:30 Programme 1C ........... 22:00
and then the second comes in like this:
19:00 Programme 2A......................................... 21:45 Programme 2C .. 22:00
So obviously the old 3 programmes should be completey deleted, and the new 2 should be what you see.
But you don’t. In fact what you see is programme 1B and programme 2C, before 1B and between the two. Weird huh?
“Why?” I hear you ask. Well, it’s simple when you consider how the programmes are split into files.
Programme 1A goes into file day-2008-09-14-D.ser, and programmes 1B and 1C go into day-2008-09-15-A.ser.
[Side note: this is true in this case because the bug reporter is in the GMT -0400 timezone and the file boundaries are quarters of a day in GMT.]
Then, when the new programmes come along, 2A goes into 14-D – wiping out 1A, and 2C goes into 15-A – wiping out 1C but not 1B.
Then, when the files get read back in again later, 2A is read from 14-D, but then 1B is read from 15-A, wiping out 2A, and finally 2C is read in as well from 15-A, so we end up with 1B and 2C.
How to fix it? Well, what I did was leave everything as it is, and then do the final read in the reverse order. This means we read in 1B and 2C, but then we read 2A later, and it wipes out 1B, leaving 2A and 2C as we would expect.
Neat fix eh? It works because this kind of wrongness in the .ser files will only exist when a programme hanging off the end should have wiped out something in a later file. Because programmes are classified into files by their start time, they can only hang off the end of a file, not the beginning, so reading the files in backwards will always read the hanging-over file last, wiping out anything which should have been wiped out earlier.
There is a little bug/feature remaining, but it only applies when you get some really weird listings from your provider. If you had a programme like 1A (19:00 – 21:15), and downloaded new listings, which ONLY contained a programme overlapping it, but falling into a later file (so maybe it starts at 21:00), and didn’t contain any programme starting at 19:00, then the backwards reading would mean you would never see your new programme because it would be wiped out by 1A.
This is a very unusual case though, since normally if you get a new programme at 21:00, you will also get new programmes leading up to it, if only to reflect the fact that 1A is now a different length. So this is really a theoretical bug, which explains why I’ve decided not to fix it…
Anyway, by the time I’d fiddled with my test for this to get the bug to trigger (which took a long time – working out which bits to fake out and which to test at all was tricky), the actual fix was easily implemented (1 line of code I chose to break out into 3), and then validated in a single click.
Just in case I hadn’t mentioned it, I love tests.
So let’s quietly forget FreeGuide 0.10.8, which basically didn’t work.
I fixed the problem (which was some nasty threading thing) by making the initial “Choose Channels for XMLTV” call from the first time wizard not do the clever stuff of displaying errors to the user. Instead it dumps any errors to the console (which is an improvement over swallowing them completely as it did before), and when you run “Choose Channels for XMLTV” from the menu, or “Configure” from the Options screen, you still get the errors displayed in a nice dialog as in 0.10.8.
More excitingly, new in this release: some unit tests!
You can run them by doing this in the src directory:
They test the fixes I made in 0.10.9 for XMLTV files that have certain attributes missing, and have different date formats.
The date formats were supposed to be already working, but a fix I made several years (yes, years) ago broke them. Why didn’t I catch the problem? No tests.
The second age of FreeGuide is coming. I shall call it The Time of Testing.
I am still working slowly on moving FreeGuide forward. Somehow it seems my itches for FreeGuide are all about making it less annoying for people who are trying it the first time. I guess this is motivated by my desire for world domination.
Anyway, we are one small step closer to my mum being able to use FreeGuide – when the “Choose channels” step (i.e. the XMLTV grabber configuration) goes wrong, you can now see a real genuine error message, and hopefully figure out what went wrong.
Actually, it always used to work that way but the error-catching got refactored away at some point. Anyway, I am slowly taking the ground back…
As I do more and more test-driven development at work I am becoming completely addicted. For this FreeGuide code I wrote a couple of unit tests but they are not within a proper framework, and can’t be launched easily as a test suite. I am considering JUnit.
I also want to set up some component-level tests e.g. for downloading listings for each country and checking everything works as expected. It’s brilliant fun having tests in place, but when you have as little time as I have for FreeGuide at the moment, it’s difficult to decide to spend a long time working on a test framework when I could be fixing a “real user problem” or adding a cool new feature.
But I’ve got the testing bug badly, so watch this space.