Monday, 26 September 2011

How to discuss our open source projects?

So we now have a collection of Kynesim open source projects:

  • muddle
  • KBUS
  • cdatecalc
  • tstools
  • grump
and another one to be added soon. What we don't have is any way of making announcements about them (other than this blog - not really ideal) or holding discussion on them (issue tracking I reckon is OK, as I quite like the Google Code issue mechanisms).

Mailing lists are a pain, and none of them make using their archives particularly nice.

Google groups are a possibility, although they're not perfect either.

Google+ won't work because I won't use it (so that discussion is moot).

Any good ideas?

Still improooving muddle - some commands slightly changed.

I'm still trying to fix various problems in how muddle handles subdomains, and also to add some more testing. However, I've just pushed a somewhat incompatible change to the way that some of the command line tools work.

In the past, if you did something like muddle fetch at the top level of the build tree, it didn't actually do anything, and you had to do muddle fetch _all. This was, in fact, a bug, and those commands which take checkout arguments will now, in general, always default to the checkouts "below" them, including at the top level (where all checkouts are "below").

Of course, it's also an incompatible change. On the other hand, I don't know how many other people still had the habit of typing muddle fetch (etc.) with the vague feeling that they should be doing something that they weren't...

As normal, please raise any issues on the Google Code issues page (and I get an email at home when you do so).

Friday, 9 September 2011

Finally, muddle has a (VCS) status command

It's been a long wait, but Richard and I both really needed it...

You can now say muddle status and get a report on the subversion/bazaar/git status of checkouts (i.e., the version control systems we currently support). It tries not to say anything for checkouts with no interesting status. See muddle help status for details.

Tuesday, 30 August 2011

KBUS and muddle documentation now on ReadTheDocs

The HTML documentation for both KBUS and muddle is now being hosted on Read the Docs, who are providing a wonderful service to those of us using Sphinx for our documentation.

See and respectively.

The documentation should be rebuilt  each time a push is made to the appropriate repository. Including, of course, a change to the documentation. So I've no excuse, then...

Monday, 29 August 2011

muddle is now stored in git

As of just now, the muddle Google Code repository has been converted to git.

If you're still accessing muddle via subversion, then you are using a "frozen" version, which will not be developed further. The last subversion revision is revision 638. Also note that due to difficulties in converting our subversion repository to git (strange stuff, honestly), only trunk has been converted, and at that only since revision 78 (around when something strange seems to have happened, probably to do with trying to merge stuff in subversion).

Cloning muddle is done as one might expect:

  git clone

Why do this? Well, it makes it easier to development locally (distributed version control systems, yeah), and much easier to branch (last time I tried that in subversion was a nightmare).

Any problems, please contact me and ask.

Friday, 17 June 2011

Muddle dependency visualisation

For about as long as I've been working with muddle, I've felt that it was lacking the ability to get a big-picture view of how a build tree fits together. This led to issue 48. It's not that I have anything against build descriptions being declarative-type programming; just that I sometimes find them hard to follow from scratch and prefer to get an overview.

Over time I hatched a plan to actually do something about this. Well, muddle knows everything it is possible to know about the dependencies; couldn't I put that knowledge to use?

Betraying myself as more of a perl hacker, I initially wrote a perl script which invoked muddle depend user-short, scraped the output and output in dot format. It was grotty and inelegant in a number of ways, but it served as a valid - if slow - proof of concept. Later, I rewrote it properly in python, directly examining the build description. So I had given myself the ability to create dot graphs of a build tree, but it still wasn't all that useful: every label was a separate node, and there were far too many of them, making the output cluttered and hard to understand.

The key insight in reducing the output was that while a muddle package usually gave rise to five labels - preconfig, configured, build, installed, postinstalled - these always appear in strict order. So why not conflate the nodes on the graph? It wasn't quite as simple as unconditionally merging them, as (for example) sometimes one package's built step depends on another package's configured, so I had it search for such dependencies and only reduce the graph as far as it could without throwing away information.

The script was duly committed to the sandbox directory, and languished there for a while. Just recently, Tibs has picked it up again. The graphs output by the script now sport colour-coding to differentially highlight checkouts, packages and deployments - and all of a sudden it's much more legible!

Alongside there is now a wrapper which feeds the output into José Fonseca's xdot viewer script. The wrapper adds the options to choose amongst the various alternative layout filter algorithms (we find fdp often works well), and to pass the graph through tred (transitive reduction of vertices, e.g. if A depends on B, B on C, and A directly on C, you can drop the A-C dependency as it is implicit from the other two).

For example, a build tree I've been working on recently has a beagle-boot deployment, so here is the output from ~/muddle/sandbox/ deployment:beagle-boot/deployed :

If you want to have a play, you'll find it in the sandbox directory. For the time being you will need to install somewhere on your PATH (see the comment at the top of Then invoke, telling it the label(s) you are interested in.

The script is not perfect, being a bit hamstrung by how well dot and friends cope (or fail to cope) with complex graphs: particularly in nontrivial build trees, the output is frequently too cluttered or too sparse. Nevertheless, as you can see from the example above, it's capable of doing a very smart job! We dream of creating a more interactive viewer, perhaps one which allows the user to zoom around a three-dimensional rendering of the graph, but that will have to wait for now...

Muddle logo

Muddle has a new logo!

It came about when I was scrawling notes during a discussion. I found myself instinctively writing an M in a circle as a shorthand for muddle. I showed it around and it was well received.

So a few minutes work in Inkscape cobbling things together, and hey presto! A logo is born. It's a derivative of the FreeSans font ('@' + 'm') so is covered by the FreeFont license, which is GPLv3 with an exception which does not cause documents using it to become bound by the GPL.

Wednesday, 12 January 2011

KBUS surgery in progress

If you've browsed the kbus source on googlecode recently, you may have noticed that it has grown a few more repositories - kernel, cppkbus, jkbus and web. These are in flux at the moment; the eventual goal is to split them and their history out of the 'default' repository. In the meantime, we're going to leave the default repository untouched until we're ready to throw the Big Switch - do not adjust your set!

Mercurial's subrepository support is rather neat. At the moment the plan is to have the kbus 'default' repository check out everything you need to use it (kernel module, libkbus, tools, jkbus and cppkbus) along with the docs source; the built HTML docs, talks and slides will be hived off to their own repository so you don't have to download them if you don't want to.

Monday, 3 January 2011

KBUS Updates

There are now C++ and Java bindings for KBUS. Also, the repository has been moved to Mercurial.

Sunday, 2 January 2011

Gingerbread branch appears to be open..

Those of you who watch the android open-source tree will have noticed a couple of interesting additions lately; specifically, the gingerbread manifest branch and platform/external/chromium.git.

I also notice that someone has been paying attention to the OOM killer and KVM in the android common kernel tree. Sadly, I think KVM is probably a bit heavyweight for this generation of devices, but containers could be a bit of a contender for allowing you to run multiple virtual STBs at the same time - might be a nice way to store your STB (and PVR) in the cloud and have it virtualised onto whatever hardware you happened to be watching at the moment.

For what it's worth, head seems to be building fine for me and the changes thus far seem quite easy to live with. Certainly much nicer to random undocumented API users like me than Linux was at the corresponding stage in its development ..

Also, Happy New Year!