CellBE frameworks

celltask “A clean task interface to Cell programmers to start jobs in SPUs by hiding the tedious context/pthread creation, mailbox/signal/interrupt mailbox communication, etc.” (new)

speutils “…instance initiators for various types of posix threads to run the spe programs as well as a instance oriented message passing interface.”

“MARS (Multicore Application Runtime System) is a set of libraries that provides an API to easily manage and create user programs that will be scheduled to run on various microprocessing units of a multicore environment.”

spumedia “… to provide accellerators for the cell broadband engine processor”

spexms “Simple library for creating spe accelerators for the CELL BE and Playstation 3”

(This post brought to you by I’m Putting These All In One Place So I Don’t Have To Search For Them Again.  And the letter N.)

Fedora 11, PS3 & SPU programming

Some notes on Fedora 11 on PS3

Intro to Cell, part 6 and part 7 the excellent series from NotZed continues.

2D Polygon rendering demo(3067FPS ver.) on PS3 Linux

YUV to RGB on SPU More in-depth spu programming from NotZed.

Insomniac games Nocturnal Initiative and also #nocturnal on irc.freenode.net  (and Insomniac’s R&D page is always a good read)

Computer memory architecture foretold

Ideally one would desire an infinitely large memory capacity such that any particular … word would be immediately available. … We are … forced to recognize the possibility of constructing a hierarchy of memories, each of which has greater capacity that the preceding but which is less quickly accessible.

A. W. Burks, H. H. Goldstine and J. von Neumann
Preliminary Discussion of the Logical Design of an Electronic Computing Instrument (1946)

(Quote found in Computer Architecture : A Quantitative Approach 4th Ed. by Hennessy & Patterson)

Arbitrary problem solving

Spending a few days at the in-laws, I’ve lugged along the PS3 so that I can continue my attempts at hacking on a DirectFB driver.  Unfortunately, I’d left it configured as it had been on my home network – using wired networking.

There’s no way to run network cables around this house without causing massive inconvenience, and there’s actually no cables to run.  Let’s just configure this thing for wireless networking. Except I don’t have a USB keyboard.

Buggr.

Ok, what do we have?  A short piece of CAT5 that connects a VoIP box to the wireless router.  That’s a start.  I can boot it on the wired network, connect remotely, find the IP via the router’s DHCP report (fortunately that has default login credentials), ssh in, make the changes and get on with whatever it was I was going to do.

So, I plug it in (unplugging the VoIP box), boot the PS3, look up the IP, enter username and password and then fail to be allowed access.  What?  Oh, I’m trying to log into the dropbear sshd that comes with petitboot – I’ve not made it past the boot screen because I didn’t specify a timeout for picking a boot option.

Grr.

Ok.  Next option?  Is the CF card that I was booting from recently still in the CF slot? *opens door* Aha!  It is!  Not of immediate help, because it doesn’t have a default option either, but if I can find a Linux machine to plug this into I can set one…

Grab card reader from camera bag, reboot laptop (wifi adapter for laptop is flakey under Linux atm, not sure why), insert card, mount device, add missing boot lines to kboot.conf (which are, I hope, correct)

default=ouros
timeout=5

save, unmount, CF card into PS3, boot and wait…. (five seconds seems like a long time).

And it boots!  ssh login (from putty on restarted laptop), mount /boot, modify kboot.conf in the same way, shutdown, remove card, boot, wait . . . . .

Huzzah!  Problem solved.  Now for some hackery :D

Oh, wait, that wasn’t the problem.  *sigh*.  The PS3 still isn’t speaking wifi.

Boot PS3 (after shutting it down expecting to be able to relocate it further away than the short blue tether that I’d been using), ssh in again and make changes to /etc/network/interfaces.

I’m not normally a wifi user nor a debian user so this is where it gets a little tricky.  The network I’m trying to connect to is WEP, and the debian wiki doesn’t want to tell me how to set that up in case I hurt myself.  Fine.  A little more googling, and I find what appears to be a suitable incantation :

wireless_essid in_laws
wireless_key manycharactersthataddlittlesecurity

Good.  So I uncomment the wlan0 section and add those two lines.

Playing with networking configs over ssh is where things get fun.

/etc/init.d/networking restart

Did it work?  Actually, I think it did – eth0 re-DHCPs, and then wlan0 does, b0rking (it would seem) access via eth0.

Bah.

Reboot.  The same thing happens – eth0 comes up ok, and then wlan0 confuses the matter (the PS3’s two interfaces share a lot in common, so you can’t configure them both like this without hassle.  It would seem)

Ok, so what do I have now?  A machine that won’t boot to having a functional network connection, no keyboard, and I can’t boot it back up from the CF (I think) because the internal HDD is set with a timeout and (I expect) will take precedence over the config on the CF because the HDD was the last used boot device.

Expletive.

Wait, what happens if I unplug the CAT5?  Reboot, wait.  Nothing.  Nothing.  No response to pings.  Nothing.  Nothing.  Ping!  DHCP times out on eth0 and then wlan0 starts up correctly.  Diggity!  So all appears to be (sufficiently) functional for now.

Why was I doing this again?

A new compiler and a visual guide

From http://t-platforms.ru/en/cell/cellcompiler.php :

T-Platforms Cell Compiler is a single source compiler that explores the power of Cell/B.E.™ multicore architecture through auto-parallelization and auto-vectorization of source code of applications written in a sequential manner and language (C/C++, Fortran).

They’re looking for beta-testers – I’ve applied.

Visual guide for SPU instructions Jon has put together a fantastic document illustrating the behaviour of most SPU instructions – very enlightening and a wonderful reference.

Mars 1.1.3 has been released.  “MARS is a multi-tasking runtime system for multi-core processors. The current implementation is for the Cell BE. The system is co-processor centric, in that the co-processors each run a micro kernel that generally runs independent of the host processor.”

The Soul of a New Machine

I’m reading The Soul of a New Machine, which I discovered by accident when looking for some other books.  It’s a fantastic book!  It details the development of the MV/8000 series of superminicomputers by Data General in the 70s/80s – their response to DEC’s VAX.

This book ties in with my own recent work and reading in several ways, which is probably why I’m enjoying it so much.

For my PhD research, I’ve been reading various academic papers going back to the 60s on the topics of automatic memory caching and virtual memory, and the variety of ways that problems have been addressed (hur). This book goes into surprising amounts of detail about the design and implementation of computer architecture, and to read Steve Wallach’s memory segmentation “Golden moment” explained in this narrative history was a delight for me.  The motivation and concepts are quite different, but it brought to mind the explanations that can be found in Neal Stephenson’s Cryptonomicon (and even Cory Doctorow’s Little Brother) – an attempt to teach some computing concepts within the scope of a story.

My reading for research has brought to my attention the Burroughs large systems, computers produced by Burroughs Corporation in the 1960s – particularly the B5500.  I have been fascinated to read how the company’s technology has persisted over time through mergers and renamings and many generations, to the extent that the MCP (Master Control Program) developed by Burroughs back in 1961 has some (vague) support still available in hideously expensive mainframes made by Unisys (including some kind of object-level compatibility, although I’m not sure how far back that goes).

Another reason that this book has been of interest is that it is a story of intra-corporate competition (and conflict) that seems to resonate with the stories found in The Inventor’s Dilemma – companies that grew quickly based on a good product, but that didn’t change as the market required.  Data General spent a lot of time and money fighting to win in (what would prove to be) a superceded product domain, and while they still sold a lot of machines (from what I read, billions of dollars worth of the MV/8000 series), they perhaps changed too slowly and circumstances were against them.  See more on the wikipedia page.

(I should say that I didn’t finish The Inventor’s Dilemma, and I haven’t yet finished The Soul of a New Machine – I’m aware that the first references the idea attributed to Tom West that the internal design of the VAX reflected the internal corporate structure of DEC, but I do not recall reading that part of Inventor’s Dilemma… I guess I should request it again)

At times I wish I knew more about computer history, particularly in terms of how hardware design has changed over time – there is a lot to learn from the past, I think, when it comes to implementing computer solutions now.  For example, algorithms that made sense for managing or controlling dataflow between memory and disk are now often applicable between the processor and the memory, such has the relative speed of each changed.   This is something that I have found to be useful in my research, having implemented a caching system that has more in common with RAM-to-disk virtual memory than traditional cache-to-RAM.

In fact I’d like to see taught (by which I mean “I’d like to teach”) a full-year program-your-way-through-computer-history unit at university :)

With regard to my reading about this book, I found this page where it was fascinating to me to see a post describing the backwards compatibility of the MV/8000 by Michael Meissner on usenet from 1987 – an engineer working for Data General at that time, and a name I recognise as currently working for IBM, making contributions to GCC in the areas of __ea support for the Cell BE SPU, as well as Power7 support.  I also note that DJ Delorie, another name I recognise from the GCC mailing lists, worked at Data General (as mentioned on the DG wikipedia page).  I am continually excited to follow threads connecting computing companies and people to different places and times. (Going back to Burroughs, this story of Don Knuth writing an ALGOL-58 compiler over a summer is quite an entertaining read.  Also, I’d not noticed this interview before).

For further reading, I’m hoping to get a copy of The Race for a New Game Machine – about the development of the PS3 & Xbox360 processors, as well as Racing the Beam, on programming for the Atari VCS.

Mandelbrot, Cell BE and Linux distros

There must be something “special” about the Cell BE, Mandelbrot fractals and Australians – I’ve found another implementation, and this one is really good (not to disparage jk’s implementation, especially when it was so useful).  NotZed at cell.grondklont.nl is up to part 5 in a fantastic in-depth series on implementing a optimised Mandelbrot renderer on the Cell BE – of particular interest to me was the load balancing implemented using the atomic cache unit, which is quite impressive.

A number of #ps3dev residents have decided to put some work into a PS3-targeted Linux distro.  Hanging out in #ouros (on freenode) some progress is being made on getting some basics together, with a rudimentary tarball for (aspiring) developers now available as well as a lot of effort on supporting infrastructure (bots, hosting, planning, etc). If you are at all interested, check out #ouros and/or #ps3dev on irc.freenode.net

From conversations in #ouros yesterday, it would seem that DirectFB is worth investigating as a PS3-Linux friendly way of providing a fast, light-weight graphical user interface for the PS3.  First of all because X is a behemoth and largely undesirable on the limited-resource PS3, but also because DirectFB has a number of design characteristics that seem to make it a good match to Linux on the PS3 – it needs only access to the framebuffer (it already works), and it allows creation of device driver for specific hardware.  It seems likely that DirectFB rendering can be accelerated without much hassle by using the SPEs.  As well as this, SDL supports DirectFB (which means a large number of games and other programs should Just Work) and there appears to be current QT support as well.  All up, this seems quite promising.

Otherwise, I’ve spent time this evening playing the Uncharted 2 beta.  Apart from the eons spent waiting to join a game, it was quite enjoyable, if occasionally buggy and clumsy.

Blindly navigating the PS3 XMB

I move my Playstation 3 between my SD family television and an old 20″ IBM monitor in my office at 1080p (connected via one of these). When moving to the TV, I can do the “long press on the power button” to get to the right resolution, which is simple enough.  Even if it does reset any theme settings.  Setting up the PS3 with the monitor is a bit trickier as it does not support 480i or 576i, so I need to pull out an old 34cm TV I have sitting around, and need to do a cable shuffle and it’s all a lot more hassle than I’m interested in.

To avoid that, here is the combination of controller presses to get to 1080p without being able to see what you’re doing.  It is easiest when you have working audio, so you can hear the clicks as you navigate the menus.

These are done with PS3 firmware 2.70.  It may change in later versions.  You follow these instructions at your own risk.

Pause between each press, particularly X presses as there is often a little delay involved in the screen updating.

To blindly change the resolution :

Turn on the PS3
Press the PS button on controller
All the way up (to the top of the Games menu)
All the way left
1 step right (to the Settings menu)
All the way down (bottom of the Settings menu)
4 steps up (Display Settings)
X (select Display Settings)
X (select Video Output Settings)
1 step up (In my case, select Component / D-Terminal output. 2 steps up for HDMI,  no steps for Composite / S Video, 1 step down for AV MULTI / SCART)
X (select connector type)
X (confirm connector change)
1 step left
X (confirm that it works)
All down (select 1080p.  Back up one for 1080i, up two for 720p, three for 576p)
X (select the desired resolution)
1 step right (move to next screen – all going well, the screen should be working from this point)
1 step left
X (to confirm that it works)

At that stage, you should be able to see what you’re doing :)   If not, turn off the PS3 and start again, hoping you haven’t reformatted the HDD along the way :p

The other circumstance I find myself wanting to navigate the XMB blindly is to get into OtherOS without having to change the resolution first.  This will only work if you have already installed a bootloader (otheros.bld).

To blindly boot into the OtherOS:

Turn on the PS3
Press the PS button on controller
All the way up
All the way left
1 step right
5 steps down
X
All the way down
3 steps up
X
1 step down
X
1 step left
X

Passion, Lessig and Adam Smith

I was inspired by this post of Larry Lessig’s today – it conveys some delightful passion. I also particularly enjoyed this post – Lessig style makes even this self-described “rant” of his a most enjoyable (and educational) read.

In the second post, Lessig quotes Nicholas Gruen who quotes Adam Smith, which brought to my mind a C. S. Lewis quote that I’ve blogged here (yeah, there’s a lot of quoting going on).  From Smith’s Theory of Moral Sentiments :

The man of system. . . is often so enamoured with the supposed beauty of his own ideal plan of government, that he cannot suffer the smallest deviation from any part of it. . . . He seems to imagine that he can arrange the different members of a great society with as much ease as the hand arranges the different pieces upon a chess-board. He does not consider that the pieces upon the chess-board have no other principle of motion besides that which the hand impresses upon them; but that, in the great chess-board of human society, every single piece has a principle of motion of its own, altogether different from that which the legislature might chuse to impress upon it. If those two principles coincide and act in the same direction, the game of human society will go on easily and harmoniously, and is very likely to be happy and successful. If they are opposite or different, the game will go on miserably, and the society must be at all times in the highest degree of disorder.

It’s a beautiful piece of writing (it reads wonderfully aloud) and is, I think, profoundly insightful and worthy of further consideration.  C. S. Lewis’ perspective addresses the same problem, that of ‘well-intentioned’ control (or, rather coercsion), from moral rather than idealistic grounds.

Recently, I’ve come to appreciate some of the limitations of governance through overt coercion, so Lessig’s comments have been particularly timely for me.  As to what he has to say about what is or isn’t socialism, I’m not so sure.  There’s more for me to chew over.