Andy Balaam’s Blog

Dear LJ,

I’d like to start by saying thank you very much for the years of quality articles and features that your magazine has provided. I eagerly await the arrival of each month’s copy as it jets over the Atlantic to my doorstep. LJ has always provided detailed and informative articles on topics both mainstream and more quirky, but always with the deeply technical edge no other magazine or web site I have found can provide.

I am writing because the April issue proved a grave disappointment. Gone were the technical articles and gone were the interesting projects. In their place I found a plethora of opinion pieces most of which provided no new insight into their subject that I could not have found on Slashdot or OSNews, along with some beginner’s articles on how to install a couple of PHP applications. Even the Doc had nothing interesting to say about the yesterday’s-news trying-to-be-trendy refrain of “blogs, wikis and podcasts” which seemed to be repeated several times on each page. The articles which did contain decent technical content (notably “Work the shell” and “Remote Temperature Monitoring with Linux”) were swamped beneath the river of fluff through which I found myself wading.

When Nicholas Petreley wrote his first uninformed and ill-judged “/etc/rant” I felt I could let it pass since it was just one page of an excellent magazine, but now that his style of writing - opinion rather then content - seems to be spreading to the rest of the magazine, I feel the need to express my disquiet at what he has been doing with this column.

/etc/rant is an article put aside each month for criticising free software projects. The fact that an editor would allow such an article to exist, let alone write it himself, displays a profound and worrying misunderstanding of the community to which he is trying to contribute. Free software projects live and breath through encouragement and just plain getting on with it. As a member of this community, the right way to express dissatisfaction with the way something is being done is to contribute to the project, or (if necessary) launch or encourage an equivalent project that does things differently. By criticising from the sidelines Mr. Petreley reveals himself to be an outsider - as a writer a more appropriate way to contribute to the community (and hence form a useful part of it) would be to praise and encourage projects which are doing the right thing, rather than sucking energy out of valuable and innovative projects which have involved large amounts of volunteer effort such as GCJ and Gnome.

I never thought I would say it, but if this continues I will be cancelling my subscription to LJ. Sadly, I don’t think I’ll ever find another magazine that will reach the level of quality I have come to expect from you.

Yours sincerely,

Andy Balaam

jdub notes that Linux Journal is going to the dogs. If the only change were to the back page I could get over it and move on, but so far I am about half way through the latest issue (March 2006?) and I’ve read nothing but opinion pieces.

I can get all the rantish uninformed opinions I need on OSNews. When I want quality technical articles written by people at the top of their field which cover hardcore problem solving and creative projects, then I turn to LJ. That is why I pay REAL MONEY for a paper journal like this. Because it has substance.

If I have to cancel my subscription, where should I go?

Forgot to mention: I got my first non-anonymous donation to the FreeGuide project - Steve Fuller sent in a donation along with a request (but with no implication of linkage ;). Thanks Steve! It never ceases to amaze me how Americans will donate to projects like this. My feeling about British people (certainly myself) is that I would always think if I were going to donate to something I’d make it something “important” like Christian Aid, but actually I do think Free Software is important too (actually in order to achieve some of the same goals that Christian Aid have…). The idea of mixing philanthropy with self-interest, which seems to be natural to Americans, can feel a little dirty to us Europeans, but actually that is stupid.

Anyway, enough philosophy: Steve, your donation will be used to buy myself something nice (e.g. a GameCube game or some online poker money) and it has significantly encouraged me to get stuck back into FreeGuide development (see the previous post … guilt is rising). You never know, it might even subconsciously encourage me to get to your specific feature request sooner ;).

(The feature request was allowing favourites to match on programme descriptions and other, more specialised fields. I am wondering about how to keep the UI simple for this, but it should be do-able given some time to implement it.)

I have very little time for FreeGuide at the moment. Part of the problem is that there are lots of boring bugs to fix instead of interesting new features, and it’s hard to motivate myself to get into it. I am really keen on making it Just Work every time, but since other people have been working on the code and quite a lot of it is unfamiliar, it’s a lot of work to work out where to look to fix a problem.

Hopefully when I’ve purged this poker thing from my system I will find time to get into the latest TODO list. At the moment all my FreeGuide time is going into trying to keep up with helping people on the mailing lists. Joe Blow is doing a fantastic job of providing an initial response to lots of mails, but often people are encountering weird bugs that need a developer to get involved. It would be great if some of the other people involved in development were able to help out, but it’s quite a thankless task, and takes away time from developing…

It’s frustrating when quite a few of the bugs we have are things I fixed in a previous version that have crept in with new implementations of existing functionality. It’s taken ages to get FreeGuide stable, and recently it seems to have taken some steps backwards. My feeling is that the percentage of people who are able to install it with no problems and download listings straight away has gone down considerably. I _really_ want to fix this.

That’s why the TODO list for 0.10.4 is pure bugs. We need to make it _work_ before we make it do new stuff. Otherwise people are just going to walk away.

Saying that, the addition of a new experimental vertical listings view by Christian is really exciting. I haven’t even found time to try it out yet: I must examine my priorities. FreeGuide has always been my “main” project but I have been neglecting it as time presses in.

The Boy (as I call him) is doing well and has slept well the last 2 nights, giving us a break from the all-over-body pain of extended tiredness. If this is part of a trend towards more sleep, I may get more time to make FreeGuide better.

Forgot to mention that I also have one other optimisation up my sleeve: instead of fully analysing each hand I could only do that when I need it i.e. when the hand type is the same, so I could just return “flush” and only if your opponent also has a flush would I need to say “flush ace high” or whatever. This provision of the detail is actually quite time-consuming, so it may be a significant improvement to avoid it in most cases.

Current timings (without the above optimisation) on a P4 2.4GHz machine:

[andy@andy-p4-linux src]$ time ./poker 5		# Analyse all possible 5-card hands
Straight flush:  0.001539%
Four of a kind:  0.024010%
Full house:      0.144058%
Flush:           0.196540%
Straight:        0.392465%
Three of a kind: 2.112845%
Pair of pairs:   4.753902%
Pair:            42.256903%
High card:       50.117739%

real    0m4.058s
user    0m3.805s
sys     0m0.002s
[andy@andy-p4-linux src]$ time ./poker 7		# Analyse all possible 7-card hands
Straight flush:  0.016811%
Four of a kind:  0.168067%
Full house:      2.555057%
Flush:           3.039766%
Straight:        4.475084%
Three of a kind: 4.873941%
Pair of pairs:   23.512489%
Pair:            43.946865%
High card:       17.411920%

real    3m43.479s
user    3m42.963s
sys     0m0.047s

This is nice, because there are 51 times as many 7-card hands as 5-card ones, and the analysis only takes 55 times as long even though it’s more complex to analyse larger hands, so it means my analysis scales pretty well with the number of cards.

I’ve just requested sourceforge space for my latest mini-project: Poker Analysis. I’ve been a big fan of poker for years, and am pleased to see everyone is catching up with me finally with the boom of Internet poker. I’ve started playing a little online, and it set me thinking about the algorithmic aspects of analysing the game. Obviously, the real skills of poker are about predicting people’s actions, but before you can do that you need to be able to see what hand you’ve got and what possibilities are out there, and that stuff is entirely deterministic.

I like small, self-contained challenges, so I set about writing a program to play every two-person game of Texas Hold ‘Em poker that could ever happen, and ranks the two-card hands you can have in order of how many games they win. Obviously most people have a pretty good idea of which hands are good or bad, but I bet there would be some interesting results to see the full ranking.

Of course, I’m sure lots of people have done this before, but I want to do it too! Also, of course, it doesn’t matter how good your hand is in the grand scheme of things, just whether it beats the other hands out there, but bear with me: I’m interested in the results.

The major challenge is getting it to run fast enough, and that is what I’m finding interesting. I’ve already changed my algorithm several times and made huge improvements in execution time, but to run through all the trillion or so (I think) hands would still take several years to execute on my home machine, so I’ve got some work still to do.

There are 2 major parts to the algorithm: enumerating all the hands you could get, which sounds easy but is in fact slightly tricky, and difficult to do perfectly if you want to avoid repeating redundant analyses, and finding out what hand you have (e.g. pair, flush). I’m pretty happy with my current implementation of the second, but I’ve got quite a lot of work to do on the first.

To analyse what hand you’ve got I’m taking an approach where all the real work is done when you sort the hand, and then looking for pairs or straights is easy since the cards are already next to each other. I then re-sort the cards so that the same suits are next to each other, and that makes it easy to find flushes. The part I’m really happy with is that the way I choose hands to analyse means they are already sorted, and when I transform them into the sorted-by-suit instead of by number order, I have very little work to do the way I’ve done it. (Some slight extra complication is added by the need for aces to be either high or low in straights.) This means that deciding what hand you’ve got is pretty quick, and I am able to enumerate and analyse every 5-card poker hand (2.6 million hands) in 4 seconds on my home machine.

I believe I might be able to speed up the analysis a little by changing my encoding of cards which at the moment is just consecutive integers (e.g. 0=2c, 1=3c, 2=4c, …, 12=Ac, 13=2d, …) but might be better to have each suit starting at a power of two to allow the use of the shift operator instead of %13 or something, but I’m happy for the moment. Where I need to improve is in writing the code that enumerates all the hands. As I have it at the moment, it plays every 2-card hand against every other 2-card hand for every 5-card set of community cards (but obviously preventing repeats of the same card) but this is hugely redundant (something like 24x?) since the suits are symmetrical. Exactly how much redundancy I can squeeze out of it is what I am thinking about now.

Anyway, hopefully soon the code will be in sourceforge CVS, and when I’ve achieved my goal of examining all games I’ll make a release. Hopefully in the future I’ll make it accept as arguments the current situation in a game and output the probabilities of each player winning.