Category: misc

A busy few weeks. A visit from an old friend, a server migration (which isn’t yet complete), and life.

Migrating a bunch of mailman lists to google groups has gone fairly smoothly. The MailMan Archive Migration tools turned out to be very useful — the reposted messages appear with the time they were reposted and the order seems to be a bit mysterious, but everything’s there to the satisfaction of all the interested parties. A more frustrating limitation when migrating lists is that users can be added ten at a time to a maximum of 100 per day. Hopefully I only need two days worth.

The rest of the server migration falls under the headings of domains and databases. Nothing particularly exotic, mostly tedious.

The last two weeks in code…

Opportunistic extraction — wrote a little bit of perl  to make the most of some readily available well-structured data and provide easily readable update information in a particular script. The results are pretty good — and it will probably break as a side effect of some other change at an indeterminate time in the future.

Related thought: dependency tracking. How can we better track reverse dependencies? Not only are unknown consumers of code & data a common source of bugs, that changing some code might break something else becomes a cause for code stagnation. We can do better.

Breaking everyone’s everything — migrating a lot of code to a new shiny settings management system. Worth the effort, but a lot of time spent diagnosing complex interactions between build configurations, target macros, and my own mistakes while working with a third-party’s shiny new code. Related: adding diagnostics. Also related: removing silly old hacks. Both of these can be quite satisfying.

Fixing my own breakage — turns out that bugs added in an unrelated branch can languish a long time before they get noticed. Better testing habits seem desirable. Also, being a less terrible programmer.

Technical debt — hacks put in place to get an app reliably demoable may not be suitable for use outside the demo. I demoed a thing — it  went well (really well). But then I needed days to clean up the frenzy of code hacks, and even then I ended up with bugs that would bite a week later. Probably a lesson there.

Updated the 20+ studio list with ISI

It took far too long, but I finally realized that when trying to filter categories from the front page, the index.php that the howtos are referring to is the index.php of the template.

Coding today: found some bugs, fixed some of what I found.

One was a genuine stack overflow — the stack pointer was being decremented by more than the entire size of the thread’s stack in the prologue to a particular function, landing in the land of unmapped pages. The fix was simple (once I found the right stack size to adjust), but the cause wasn’t clearly obvious to me. In hindsight, I was debugging in the wrong direction.

The way it happened: something like this was causing the crash half way through the function:

mov qword ptr [rbp-4040h], rdx

I was fixated on what that rdx was to try to work out why this was happening (completely overlooking the presence of rbp and the roundish nature of the constant). A quick trip to the ABI docs led me to believe that rdx was a function argument that was entirely unused in the function. And that left me second-guessing my own interpretation — what did this mean? What critical purpose does this instruction serve?

Actually, none.

The data was being pushed to the stack because that’s the sort of code that compilers generate (i.e. the odd instruction that serves no apparent purpose).

It did all become extremely obvious after rebuilding the function in question without optimizations and seeing a cluster of mov [rbp 0x40??], all resolving to unmapped memory.

Other issues: NULL member vars where they were not expected (which was fixed by not me), mismatched sizes for a variable being passed through an opaque interface, a warning at shutdown where some objects were not being explicitly destroyed, more adjustments to filesystem code so that things compile and function [more] correctly on all the necessary platforms.

One rather not-awesome bug that remains unfixed is some serialized game data with alignment problems. A bunch of structs changed size recently — it looks like at least one of them needs some realignment counselling. (Finding the hash of a string inside a counter fills me with delight*)

I spent some time debugging problems with our navmesh generator (aka navbuilder). Right now, we uncritically pass geometry to recast even if the vertical range of the geo is beyond what recast’s heightfield (as configured by us) can represent. Exceeding that range  causes problems – for whatever reason, all generated navmesh ends up being pushed to a low extreme, and then the tools that consume the generated navmesh choke. It’s something that needs to be fixed, but there are more pressing issues right now. And there are workarounds :\

10 Awesome Indie Games of PAX Australia 2014  #straya

Andreas will be doing the shuffle at GDC next year

Completed (“completed”) migration of this blog to the new webhost — DNS has been redirected, and there’s a non-zero chance that someone apart from me can see this. Next: all the other sites & services still on the old host… :\

(I find it interesting how much working on a host with a low ping vs one on another continent makes it genuinely appealing for me to get back to the command line on these boxes.)

Coding today, I was able to port filesystem wrappers & our archive filesystem to a new platform. And it worked second time (not counting the compile & link errors and code shuffling necessary to find the things that needed to be un-if-zeroed/implemented/if-zeroed). The biggest change was due to not using overlapped IO for the archive FS on the new platform — all archive IO is already async to the main thread, but there was support for multiple in-flight operations that can improve overall throughput. The first cut lacks that support for overlapped IO — and the bug I introduced was because I missed the explicit seek necessary to perform accurate reads from archives. Instead, random reads were being serviced sequentially from the start of the archive file. Easy enough to fix, and initial testing indicates that it’s Fast Enough. Until tomorrow, at least.

Found a bug that I’d added to some other code: I had added a new codepath, but failed to set some state in both the new and old codepaths. Fix applied was to hoist the setting of said state above the branch, but in hindsight I think the variable in question is able to be removed and the consumer of said state made a little smarter…

Found some other code that turned out to be not trivially portable, that was also not worth porting. Fix was the delete key.

Spent some time this morning helping to diagnose a weird compilation failure related to linker segment flags not matching in some wonderfully obscure circumstances. XP gained: decoding link flags from the “PE and COFF Specification” http://msdn.microsoft.com/en-us/windows/hardware/gg463119.aspx

Watched Jonathan Blow’s programming language talk (see other post)

Something from that game I worked on: http://www.playerattack.com/news/2014/11/06/sunset-overdrives-most-adorable-secret-uncovered/

Jonathan Blow demoing a programming language for games. Plenty of interesting things.

Thoughts…

compile-time code execution — being able to do codegen at compile time seems potentially really useful (speaking from recent experience wrangling code generators and keeping generated code in sync with the human generated stuff)

scalability — there are things about compilation, dependency management, and the arbitrary compile-time code execution that seem … risky when used in the context of large, complex projects

control — will this language provide the level of control that a low level engine programmer needs/desires? Alignments of all the things, segments, SIMD/intrinsics, inline asm, _penter hooks etc. (to name a few things that I can think of from the past week) It doesn’t need to, but I wonder about these things.