The documents and philosophy I use to get through such projects
It takes an awfully long time to disassemble and annotate every single line of code in a classic 8-bit game like Elite or The Sentinel; for example, it took me three months to create fully documented source code for the latter, and another three months to write the 51 deep dive articles that explain how the game works. Add in another month for creating the project's website, and in total that's seven months of hard (but enjoyable!) work.
I learned how to get to the end of long projects like this by walking from Land's End to John o'Groats in 2003. This 1111-mile walk of a lifetime took me three months, and I wrote the following in my diary on day 1:
I spent most of the day thinking about something that I read on a website a few days ago. 'Take an empty pint bottle,' it said, 'and add one-and-a-half teaspoons of water to the bottle each day; when the bottle is full, then that's how long it takes to walk from Land's End to John o'Groats.' I don't know if I'm going to have the patience, but if it's all as pleasant as today's 11-mile hop, I'm in for a treat.
The point is that you can only finish a project if you keep going, and even if each individual step feels tiny and inconsequential, they soon add up. I apply this philosophy to my disassembly projects, so even if I'm getting bogged down by a particularly difficult bit of code, I still try to record and celebrate the small steps - the individual teaspoons of progress.
So I keep track of everything I do in a dedicated progress spreadsheet. Whenever I finish documenting a subroutine or variable, then that goes into the spreadsheet, and the front page of the sheet automatically updates to show my latest progress. It's the coding equivalent of drawing your day's progress on an Ordnance Survey map, and both are amongst life's great pleasures.
You can see these progress spreadsheets for yourself by using the links below, which will open each one in Google Sheets. I've captured their state from the start of the project, and then at various milestones as I crept towards 100% coverage of the source code. Here are the links:
- See the 0% progress sheet in Google Sheets.
- See the 33.3% progress sheet in Google Sheets.
- See the 50% progress sheet in Google Sheets.
- See the 66.6% progress sheet in Google Sheets.
- See the 100% progress sheet in Google Sheets.
Each spreadsheet has a "Summary" tab that shows current progress at a glance, and as things fill up, the colours go from white to light green to dark green. Here's the very start of the project, where all we have is a basic set of placeholder categories with no progress in any of them:
When we reach one-third progress, things are looking quite a bit better:
And they're even more green at the halfway point:
When we reach two-third progress, the dark green is taking over:
And by the time we reach 100% coverage, things are looking very neat indeed:
The heart of the progress sheet is the "Line numbers" tab, which lists each subroutine, workspace and variable in the source, along with the status, number of lines, type and category (if known). Here, for example, is a snippet of the "Line numbers" tab for the 50% progress point:
At this point I'm in the process of looking at the ExtractTileData routine and have quite a few loose ends to tie up in the various routines, but some are completely done and progress is good.
As I work through the source code, I can change the status column for each part of the code that I document by changing the left column from a blank to either "Explain" (if I've documented it all but haven't quite explained everything) or "Done" (if it's fully documented with no issues). Adding these labels automatically updates the "Summary" tab so I can track my progress, however small.
It also lets me create progress graphs. These help with the inevitable speed bumps, often by proving that progress is steady, even when things appear to be stuck. Here's the progress graph at 50% progress:
And here it is at 100% coverage, which shows how consistent progress is across the whole project.
The progress spreadsheet is the main tool that I use for encouraging me to keep going, but there is another sheet that helps too: the hierarchy sheet. This lists subroutine names in a call-tree format, so you can tell which routines call other routines.
For example, here's an example from the initial version of the hierarchy sheet in Google Sheets, which I created around the 55% progress mark when starting to tackle the landscape-drawing routines:
I've worked out the names of some routines by this point, but overall this part of the code is a mystery; the only thing we know is that it uses angles and deltas, which doesn't exactly narrow it down.
But by the end of the project I've replaced the unknown routine names with proper labels, so we end up with the following in the completed version of the hierarchy sheet in Google Sheets:
This shows that this part of the code draws updated objects on-screen, and that it fetches the object's angles to work out which parts of it are visible. And adding all the routine names is another way to enjoy the progress; that feeling of the penny dropping is a wonderful reward.
This approach of documenting project progress in minute details also extends to the deep dives, which took up the last three months of the project.
When I write, each article goes through a few stages: I write the initial draft, add images and run an initial spellcheck. I then add deep dive headers into the source code, proofread the article once again, run a second spellcheck and publish the results on the project website.
For the project to document The Sentinel, I kept track of this activity in a text file. As I wrote each deep dive, I filled in an "x" for each stage, working through each article in order.
The text file started off looking like this, with the asterisks showing article complexity:
+--------- Write
|+-------- Images
||+------- Spellcheck 1
|||+------ Add headers
||||+----- Proofread
|||||+---- Spellcheck 2
||||||+--- Publish
|||||||
|||||||
vvvvvvv
Memory map | | **
Entry and setup code | | *
Program flow of the main title loop | | **
Program flow of the main game loop | | **
Program flow of the gameplay loop | | **
The interrupt handler | | **
...
As I wrote, I filled it in like this:
+--------- Write
|+-------- Images
||+------- Spellcheck 1
|||+------ Add headers
||||+----- Proofread
|||||+---- Spellcheck 2
||||||+--- Publish
|||||||
|||||||
vvvvvvv
Memory map | xxxxxx | **
Entry and setup code | xxxx | *
Program flow of the main title loop | x | **
Program flow of the main game loop | | **
Program flow of the gameplay loop | | **
The interrupt handler | | **
...
By the end of the project it looked like this, with the plus signs indicating articles with more challenging graphical needs, alongside the dates of publication and article count on the right:
+--------- Write
|+-------- Images
||+------- Spellcheck 1
|||+------ Add headers
||||+----- Proofread
|||||+---- Spellcheck 2
||||||+--- Publish
|||||||
|||||||
vvvvvvv
Memory map | xxxxxxx | ** 1
Entry and setup code | xxxxxxx | * 19/2
Program flow of the main title loop | xxxxxxx | **
Program flow of the main game loop | xxxxxxx | **
Program flow of the gameplay loop | xxxxxxx | **
The interrupt handler | xxxxxxx | **
Pitch and yaw angles | xxxxxxx | ** 7
Cartesian coordinates | xxxxxxx | ** 25/2
Converting angles to coordinates | xxxxxxx | **
Converting coordinates to angles | xxxxxxx | **
Calculating the hypotenuse | xxxxxxx | ***
Trigonometry in 8-bit assembly | xxxxxxx | *
Reusing the geometry routines from Revs | xxxxxxx | **
Tile data | xxxxxxx | ** 14
Tile shapes | xxxxxxx | *** + 5/3
Seed number generation | xxxxxxx | **
Generating the landscape | xxxxxxx | *** +
Adding enemies and trees to the landscape | xxxxxxx | ***
The landscape secret code | xxxxxxx | **
Object management | xxxxxxx | ** 20
Stacking objects | xxxxxxx | ** 11/3
3D object definitions | xxxxxxx | *** +
Calculating angles for drawing 3D objects | xxxxxxx | **
Drawing 3D objects | xxxxxxx | ** +
The custom screen mode | xxxxxxx | * 25
The projection system | xxxxxxx | ** 13/3
The drawing tables | xxxxxxx | **
Drawing filled polygons | xxxxxxx | ***
Colours and palettes | xxxxxxx | *
Screen buffers | xxxxxxx | ** + 30
Panning and hardware scrolling | xxxxxxx | *** + 18/3
Dithering to the screen | xxxxxxx | **
Ray-casting for the tile visibility table | xxxxxxx | *** 33
Calculating quadrants for the landscape view | xxxxxxx | *** 26/3
Drawing the landscape view | xxxxxxx | *** +
Drawing the landscape preview | xxxxxxx | **
Drawing the title screens | xxxxxxx | ** 37
Drawing 3D text using blocks | xxxxxxx | ** 2/4
The energy icons | xxxxxxx | *
The scanner | xxxxxxx | **
The crosshair sights | xxxxxxx | ** 41
Following the gaze vector | xxxxxxx | *** + 8/4
Enemy timers and state variables | xxxxxxx | **
Enemy tactics | xxxxxxx | ***
Sound effects | xxxxxxx | * 45
Music | xxxxxxx | ** 10/4
Random number generation | xxxxxxx | * 47
Anti-cracker checks | xxxxxxx | ** 11/4
The key logger | xxxxxxx | **
Text tokens | xxxxxxx | *
Source code clues hidden in the game binary | xxxxxxx | * 51
After three months, I added the final "x" and put down my keyboard.
So whether it's a status column in a spreadsheet, or a progress graph, or a letter "x" in a text file, I make sure I capture and celebrate every single teaspoon of water... and before you know it the pint bottle is full and we're raising a pint in John o'Groats, celebrating fully documented source code, or publishing an entire book's worth of deep dives.