How I automated my disassembly sites and source repositories
My disassembly websites are generated by a combination of GitHub repositories and Python scripts; every single page, image, web script and download comes from a repository of some form.
In this article I'll give an overview of how this generation process works. You can dive more deeply into the scripts themselves in the articles on generating websites from source code, generating source code repositories for Elite and generating code comparisons for Elite, but here's a quick summary of how the scripts and repositories work together to create the websites you are reading today.
Incidentally, if you want to have a go at running this process yourself, then the bbcelite-scripts repository contains step-by-step instructions on setting up and running the scripts yourself. The process has been built on a Mac, but it wouldn't take much effort to get it working on Linux or Windows.
Generating websites from source code repositories
-------------------------------------------------
First let's talk about how the simpler websites are generated, i.e. the sites for Aviator, Revs and Lander.
For each of these projects, I've hand-built a repository containing the fully buildable and fully documented source code. These are the repositories:
These repositories all have the same folder structure and a similar build process, though the Lander repository is in ARM assembly while Aviator and Revs are 6502 assembly. In terms of building the websites from the source, they all work in the same way, so let's look at just one example: Aviator.
The core script that creates my websites is create-disassembly-websites.py. This was the first script I wrote back in September 2020, and it shows; it has grown organically, shall we say, so don't expect beautiful code. It does work well, however.
The script ingests the relevant source code repository - aviator-source-code-bbc-micro in the case of Aviator - and creates HTML pages that cover the entire source code, all cross-referenced and linked, and with popup information for every operand (i.e. every variable, subroutine, constant and so on). The script also generates indexes to the source, plus a set of statistics broken down by category and type.
The hand-crafted content on the Aviator site comes from the bbcelite-websites repository, which covers the homepage, deep dives, site information and version pages. This gets combined with the output from the Python script, and the result is the complete Aviator website.
Here's a flowchart of how this works:
aviator-source-code-bbc-micro bbcelite-websites | | | | create-disassembly-websites.py | | | | | | | +-------------|-------- aviator.bbcelite.com website -------|-------------+ | | | | | v v | | | | Code pages Homepage | | Indexes About site | | Statistics Deep dives | | Version info | | | +-------------------------------------------------------------------------+
The exact same process builds the Revs and Lander websites, and you can read all about how the script works in the article on generating websites from source code.
Generating source code repositories from the library
----------------------------------------------------
Things are rather more complicated for the Elite website. Not only is there an additional step - that of generating the source code repositories from the library repository - but there is an additional section that compares the code of the different Acornsoft versions, and that's automatically generated too.
Here's the flowchart for the Elite site:
+-- elite-source-code-library ---+ bbcelite-websites | | | | | | create-elite-repositories.py | | | | | | | | v | | elite-source-code-bbc-micro-cassette | | elite-source-code-bbc-micro-disc | | elite-source-code-acorn-electron | | elite-source-code-6502-second-processor | | elite-source-code-commodore-64 | | elite-source-code-apple-ii | | elite-source-code-bbc-master | | elite-source-code-nes | | elite-a-source-code-bbc-micro | | | | | | | | | | | create-disassembly-websites.py create-disassembly-websites.py | | | | | | | | | | +-------------|-- elite.bbcelite.com website --|-------------------|------+ | | | | | | | | v | | v | | | v Homepage | | Code pages About site | | Indexes Code comparisons Deep dives | | Statistics Hacks | | Version info | | Compare info | | | +-------------------------------------------------------------------------+
On the simpler Aviator website we looked at earlier, all of the website content comes from just two sources: the game's source code comes from the aviator-source-code-bbc-micro repository, while the static website content comes from the bbcelite-websites repository. It's similar on the Elite website, except the source code comes from the elite-source-code-library repository.
The library repository is different, though, as it contains source code for nine different versions of Elite, compared to just one version in the Aviator repository. So the first step when building the Elite website is to run the second Python script that I wrote, create-elite-repositories.py, which takes the library repository and generates nine self-contained source code repositories, one for each version of Elite. These nine repositories can then be converted into websites by the same create-disassembly-websites.py script that we use to generate the sites for Aviator, Revs and Lander.
The same script is also used to generate the comparison section of the Elite site. This compares the different versions of Elite in a side-by-side view, showing all the code differences between the five Acornsoft versions (the comparison section omits Elite-A, and Elite on the Commodore 64, Apple II and NES; Elite-A uses a more suitable comment-based approach to highlight its variations from the original, and I felt that adding the non-Acorn versions of Elite into the mix would make the comparison too cluttered). The script generates the comparison section directly from the library repository, as the library contains all the versions in one place.
Finally, the generated content (all 7,482 pages of it) is merged with the 171 hand-crafted pages from the bbcelite-websites repository, which contains the homepage, deep dives, site information, hacks section and version pages. The result is the complete Elite website... all 7,653 pages of it.
For more information on how the scripts work, see the articles on generating websites from source code, generating source code repositories for Elite and generating code comparisons for Elite.