bat gb

a pointless analysis of the graphics compression in batman (gb)

don’t know why i did this but i reverse engineered the graphics compression scheme this game uses. it’s very simple, but it turns out it’s not very well-suited to pixel art.

basically, it takes a byte and uses its bits to keep track of which of the next 8 bytes of the output is a repeat of the previous.

the issue with game boy pixel art is that byte-by-byte repetition doesn’t happen too often. one reason is just the density of detail in the art. another (more technical) reason is that because of the way the game boy encodes its graphics, but if you had 4 squares of solid color, two of them wouldn’t even compress. it’s kinda stupid how sunsoft chose to handle it, in a way.

at the very least, it turns out the rom is very well-organized on the inside. each bank with assets just has a bunch of pointers at the beginning, so once i got the decompressor working i tossed the pointers at it and a bunch of graphics popped out for free (sometimes, technology is good)

anyhow, i configured my script to dump out some nerd stats (higher percent is better):

Bank Resource Unpacked Packed Ratio
3 0 656 665 98.65%
3 1 1344 946 142.07%
3 2 3408 2976 114.52%
3 3 1024 1086 94.29%
3 4 512 507 100.99%
3 5 512 534 95.88%
3 6 512 528 96.97%
3 7 512 534 95.88%
3 8 512 510 100.39%
3 9 512 485 105.57%
3 A 512 505 101.39%
3 B 512 517 99.03%
3 C 512 537 95.34%
3 D 512 536 95.52%
3 E 912 898 101.56%
3 F 224 232 96.55%
3 10 432 479 90.19%
3 11 944 953 99.06%
3 12 880 864 101.85%
3 13 1792 1300 137.85%
- Sub-Total 16736 15592 107.34%
4 0 256 65 393.85%
4 1 1024 783 130.78%
4 2 1280 905 141.44%
4 3 640 630 101.59%
4 4 1536 1296 118.52%
4 5 768 629 122.10%
4 6 1280 1121 114.18%
4 7 1024 867 118.11%
4 8 1152 951 121.14%
4 9 1536 1189 129.18%
4 A 1536 1171 131.17%
- Sub-Total 12032 9607 125.24%
5 0 576 331 174.02%
5 1 1392 1318 105.61%
5 3 1328 1286 103.27%
5 4 576 212 271.70%
5 5 1392 1322 105.30%
5 7 640 558 114.70%
5 8 576 330 174.55%
5 C 576 312 184.62%
5 D 1392 1396 99.71%
5 F 944 972 97.12%
5 10 576 282 204.26%
5 11 576 230 250.43%
5 12 1392 1423 97.82%
5 14 2064 1715 120.35%
5 15 560 207 270.53%
5 16 2064 1688 122.27%
Sub-Total 10144 8225 123.33%
Grand Total 38912 33424 116.42%

(removed some duplicates, may not have caught everything)

the best compressed chunk of graphics (bank 4, file 0) has a compression ratio of 393.85%. it is entirely blank space, so it’s probably mathematically provable that’s the best case scenario for the compression scheme.

here’s what one of the worst compressing ones (bank 3, file 0x03) looks like. it’s 62 bytes bigger compressed:
image

densely packed and detailed, by some standards, but simple-looking enough that you just know a slightly better algorithm could crunch it down easily.

also, i don’t actually recognize these enemies. i think they might actually be unused (!!) (or are they from a different game?)

here’s what one of the better compressing ones looks like (bank 3, file 0x13):
image

lots of blank space, as expected.

bank 5 is full of cutscene graphics. some of the files there have a compression ratio in excess of 200%. i wonder what they look like:

image

i’m pretty sure that’s a tilemap, actually. that would explain it.

maybe i’ll figure out how to recompress this stuff, and that will inspire someone to make batman minus infinity (idk)

23 Likes

batman (gunbat)

2 Likes

well, i managed to get a recompressor working last night. compression performance appears to be the same, though it’s not like there’s really any tricky optimizations one could do with the format

now begins the harder part: refactoring my code so it’s properly reusable in a build system

3 Likes

now to just plug the script into the .bat file…

image

:sickos: :sickos: :sickos: :sickos: :sickos:

(sorry makefile lovers, but i couldn’t resist the pun)

level data appears to use a very similar form of compression, with just a couple minor differences. shouldn’t be too hard to work with.

6 Likes

Levels maps are in bank 7 (the last bank). They almost use the same compression format as the garfics, but the difference is subtle enough to be annoying. Still, it’s clear their compression works much better in this use case:

ID Unpacked Packed Ratio Name
1 1280 897 142.70% Gotham City (1-1)
2 1280 1102 116.15% Chemical Factory (1-2)
3 1280 926 138.23% Chemical Factory (1-3)
4 1280 392 326.53% Batman vs Jack (1-B)
5 1280 955 134.03% Gotham City (2-1)
6 1280 1122 114.08% Flugalheim Museum (2-2)
7 1280 984 130.08% Flugalheim Museum (2-3)
8 1280 597 214.41% Batwing Stage (3-1/3-2)
A 1280 1216 105.26% Gotham Cathedral (4-1)
B 1280 1261 101.51% Gotham Cathedral (4-2)
C 1280 374 342.25% Batman vs Joker (4-B)
Total 14080 9826 143.29%

As compressed, all levels are 20 screens long, with each screen being 8x8 metatiles in size. This applies to every level in the game, even the boss fights — even though those would have ended up much smaller if they only needed to compress 1.25 screens instead of 20. (They could have hidden a secret message in the unused 18.75 screens, but they didn’t.)

As part of the decompression process the level data is also directly converted from 16x16 metatiles into their 8x8 tiles, where they are stored the region of WRAM spanning 0xCC00-0xDFFF (that’s like 5 kilobytes). (Metatiles themselves could be considered a form of compression themselves, but that’s a philosophical discussion I’m not interested in today.)

Anyhow, the levels’ metatiles are also stored in bank 7, though they’re uncompressed:

Tileset Bytes
Gotham 768
Factory 960
Vs Jack 208
Museum 960
Batwing 80
Cathedral 960
Vs Joker 120
Total 4056

Some of the tilesets have a lot of blank entries at the end, so they could have saved a couple hundred bytes there if they needed to.

Adding the metatiles to the previous totals, we get the most important stat: how well did it fit?

Unpacked Packed
Bank Size 16384 16384
Big Total 18136 13882
Margin -1752 2502

If they didn’t use compression one the level data, they would have been over 1.5 kilobytes in the red, forcing them to play data-tetris to cram everything in. As it stands, the compression gave them enough breathing room that they could have almost fit in two more levels.

(I just realized that I forgot the pointers at the beginning of the bank in these totals, which account for 52 bytes. Too lazy to bother fixing that though.)

Anyhow, level compression is now part of the build process (funny what you can do when you take a day off work):

image

Next I’ll see if I can figure out how/where enemy data is stored, what ties particular tilesets/songs to each level, and then probably make this public.

3 Likes

just think how much you’ll save on cartridge costs!

4 Likes

repo’s public now:

https://github.com/alex-west/batGB

(that’s code for: i’ve done as much as i want to for the time being)

hopefully the makefile works idk

might return to this in a few months

2 Likes
devMessage_D: ; 00:2987 - Developer Message
; Shift-JIS Encoded
;  ゲームボーイ ノ テトリス ハ ヤッパリ オモシロイ!! 
;  ツイ コナイダモ、テツヤデ タイセンテトリス ヲ ヤッテシマッタ・・・・。
; Translation per TCRF
;  Tetris on the Game Boy is still interesting!! 
;  Also, the other day, I ended up playing Tetris 
;  through the night...

lol

7 Likes

WIP:

2 Likes

now with a simulacrum of sensible colors:

image

6 Likes

image

oops

image

better

(don’t forget your null terminators, kids)

8 Likes

A city of peace for every one of us

This looks great. So this is upgrading it to a GBC palette rather than a baked in SGB one?

It’s wild how demystifying seeing something basic, like every level is 20 screens long, is. Makes level design feel way more approachable, like a pop song structure or something.

2 Likes

Yeah. My current plan is to give this game two sets of palettes: one for “Day” (like the original game), and one for “Night” (like the NES games). If all goes according to plan, both of these options should be available with the same patch in an options menu.

I have some placeholder hooks for the two modes rn, and they doesn’t seem like they’ll be too hard to add. The actual hardest part feels like it’s gonna be the workflow for the art, and not anything assembly code related at this point.

1 Like

That’s awesome!

With how open the backgrounds are it’s gotta be tricky to find a nice balance for the night look.

I’ve loved this game since before it came out (played it at the Nintendo World Championships pre release!) and I’m excited to see what you do with it!

1 Like

got night mode working. i’m able to modify the graphics independently of day mode, so i made some tasteful edits to bring it closer in line with the nes aesthetic

Comparison (day vs night)

image image

definitely making this project harder on myself, but the results feel like they’ll be worth it

(both of these are running on the same rom (except for a single byte in the new init code (options menu: eventually)))

9 Likes

The night mode looks scrumptious. The hard work is definitely worth it!

1 Like

wowzers this looks incredible

1 Like