Friday, May 31, 2019

Classic99 Debugger

Recently, I've been rather bubbling over how excited I am to have the Classic99 debugger working for ColecoVision software, and in my last post I promised to talk about that some. So I think I will.


While still lacking a number of features I'd really like, it really is my favorite retro-debugger. Since I wrote it, I'm clearly biased, but I also worked to get the features I actually need into it. Some of these features - I'm really baffled that other emulators DON'T have them.

The first feature, and nobody seems to do this, is real-time view. While the system in running, you get a real-time view (updated roughly 10 times per second) of whatever window you are looking at. You can see the disassembly flying by (very useful for seeing tight loops), or you can watch memory in real time (looking for a RAM access pattern - here you go!) All the while, the current system registers are ALWAYS available.

In addition, while less often useful, the commands that let you ALTER memory or registers are available at all times. Usually it's more useful to breakpoint first, but sometimes you just want to end a timed delay early, and you can just write the register directly without freezing execution. Even on the Coleco, which has far fewer registers than the TI, this is useful.

The second feature is flexible breakpoints. Again, there's more to do here, but I can break on PC, or I can break on read or write or either to a particular address in any of the system memories, or I can break on a particular value being written to a target memory. In addition, in case I don't know the exact address, any memory can be a range. I can even look for single bits. Many of these options would be difficult to implement on real hardware, but an emulator doesn't know the difference between real and simulated.

The breakpoint system even allows non-break functions, in this case it can count cycles between two addresses for performance monitoring, or even log all writes to a particular address to a disk file! Want to capture a speech pattern or a song, this can do it.

The ability to single-step and immediately see the result of an instruction is invaluable. This one at least, most debuggers do implement. Classic99 of course lets you alter any register while paused or running, including the program counter, which allows you to go back and retry a piece of code with different inputs, or even change the code on the fly (although you need to enter the hex bytes manually.)

While still rough, Classic99 also allows debug of both the host CPU and the F18A GPU at the same time.

Within an hour of having the debugger available to ColecoVision software, I was able to troubleshoot a problem with the emulator's implementation of NMI, and observe the memory access patterns of a number of Coleco games. I even engaged in a little bit of cheating on a favorite game (Antarctic Adventure: time remaining 0x60e3 (16-bit BCD, maximum value C60E3=9999), distance remaining 0x605e (16-bit BCD, finish level now C605E=0100)).

That's really all I have to say. The main deficiency I see in other debuggers is the lack of real-time view, which really slows me down. Stability seems poor on some, as well. At the same time, there are features in other emulators I also plan to implement, like sprite and tile views, audio waveforms, and more.

I use the Classic99 debugger almost all the time, to debug my own code, to reverse engineer others' code, and even to debug the emulator itself. It's an invaluable tool that too many emulators treat as a secondary add-on.

Tuesday, May 14, 2019

Whoooops, been neglecting you!

My... what... two readers? Oh, you only found this through a Google search? Good enough then!

So, Dragon's Lair is finished and done! I still have about 200 PCBs that need a new project as I can NOT put Dragon's Lair on them (seriously, I will ignore any more questions about just pretending I forget about licensing, unless accompanied with a dump truck of money parked outside the front door when I go to look).

I did publish my followup as a slideshow - if you want to know about the building of Dragon's Lair, have a peek here:

I also put up a Ko-Fi, though I probably don't post here enough to be worth it. ;) But have a peek here:

With all that out of the way, I wanted to talk some about emulation.

For many years now I've had a TI emulator called Classic99. I started it back in 1996 or so so I could have my own emulator to play with, since V9T9 was pure x86 assembly and PC99 was closed source. While development has frequently been slow it has at least been steady, and the emulator has a lot of features that help with debugging software. At first I would breakpoint the source code of the emulator itself, and write C code for whatever I wanted it to dump, but over time most of the reasons I needed to do this were coded into the debugger, and today it's reasonably comprehensive. (It still needs a 'cheat' search and remapper...)

Anyway, I also play around with the ColecoVision a lot. It is, at the basic level, the same machine as the TI except that it has a different CPU. Where the TI has a TMS9900, the ColecoVision has a Z80. There are, of course, some other gotchas - but that's enough to get going.

There are a handful of emulators for the ColecoVision. My favorite is probably BlueMSX, ColEm is probably the best known, and CoolCV seems to be the new hotness. There are many others, including the catch-all MAME. But when I start writing code and I actually need to test it, I need three things:

1) I need a fast way to launch my code
2) I need a debugger that lets me view the system state, single step and observe results, find and set breakpoints, etc.
3) I need a codebase I can modify in case I am using hardware the emulator doesn't support

BlueMSX was my favorite, it came close. True, when I changed the code it broke the debugger for reasons I couldn't guess, and the debugger only works when the emulation is paused, but at least it was in a separate window, let me view memory and set breakpoints.

Last night I loaded up a new project that I was sure worked a couple weeks ago, hit run in BlueMSX, and nothing came up. I loaded the debugger, and a few minutes later I closed the whole thing down. I was tired of dealing with partial debuggers that only let me view some information at a time, only in certain cases. I was tired of code bases that treated complexity like a badge of honor and made building the code a challenge. And I was tired of systems that locked you in to some other systems' way of thinking (I run Windows, I want a Windows interface, thanks.)

This isn't unique to the ColecoVision, by the way, I had the same issue with Sega Genesis debuggers (but the code bases were easier to build.)

I pulled Classic99 up. I've been approaching this point for a while, but I finally went for it. All I needed, in theory, was a Z80. As I'm in the middle of a rearchitecture, I decided to just hack it in for now, since the new code base won't be ready for a while. I ended up creating a new working folder.

Then, I hunted down a Z80 core. I found a nice single-file one that was recently updated, but it required this magic include library. This library is apparently great because it doesn't use any of standard C, and is a pure header library. Personally, I think that's a stupid thing to be proud of. First off, the standard C includes are, you know, STANDARD, and have been for nearly half a century. It's okay to use them! Second, this code didn't do anything weird enough to warrant it.

When I saw that one of the purposes of this library was to redefine its own types for char, int, short, etc (and not using the standard terms, but using its own terms so you're locked in), that was when I discarded it. I created my own include file, and in about 15 lines had everything that this code used.

Yeah... 15 typedefs is why I needed to download this whole other library. Kids these days.

Unfortunately, there was an entire struct not defined anywhere in the code. There was a script that I guess was meant to generate the missing header file, but, you know, Windows loser. So I actually found another project that already had this missing file, dropped it in, and built. F$#%#$ everything.

Okay, now at least the Z80 core itself WAS as simple as I could have hoped, and the documentation adequately described what you needed to do to set it up. (Except for mentioning you had to provide your own copy of the context struct, but I guess that's fair. Would have liked to see it called out).

The memory read/write and IO in/out functions were easy to link to the existing Classic99 functions. I added basic joystick emulation and handled the NMI in the interrupt code. I put the reset calls in with the reset of the 9900 and F18A cores, and then I hacked in some extra code to load the ColecoVision BIOS and copy the loaded cartridge over to Coleco space, which made the Classic99 load functions work. (I gave the Z80 its own memory since the 9900 memory space is quite different, having hardware overtop of the cartridge space). Finally, after considering it for a bit, I threw a hacky call to run Z80 cycles next to the call to run a 9900 instruction. This wouldn't give correct speed, but it would at least provide SOME throttle. Oh! And I disabled the TI CRU code that read the keyboard and joystick, so it would not get input.

Then I shrugged my shoulders and hit run. To my surprise, I got the TI title screen, but corrupted with the wrong colors and graphics. But this was a good thing - it means that it worked!

What do I mean it worked? Well, both the ColecoVision and the TI started at the same time, sharing the hardware. The TI was slower, so it drew the title screen last, but the ColecoVision set the video registers last. In other words, I knew that the Coleco side did SOMETHING! I also knew my original plan, to just leave the TI on the master title page, wouldn't be enough.

When Classic99 boots, it used to start the CPU before the ROMs were loaded, which sometimes caused a startup crash on the emulation side. Back then, I implemented a tiny boot ROM that spins the 9900 until it's ready to go. Since that was still in there, I just reloaded the boot ROM before the reset - now the 9900 was still spinning (and so providing timing to much of the system), but it wasn't doing anything useful.

When I reset it again, the ColecoVision splash screen came up!

I booted up Donkey Kong, but something was still wrong, after selecting the number of players, it just hung on the main screen:

But now I could already start using tools I didn't have just two hours ago - the debugger. Although I didn't have Z80 debug yet, I did have VDP debug. I was quickly able to determine that interrupt processing, which is central to most Coleco actions, was not happening. After checking a couple of other games and finding that everything had similar issues, I finally noticed in my code a little hack I entered. The Z80 core takes the NMI input as a pulse, rather than taking a level (and dealing with the edge itself, like the hardware does). But the flag that I'd used for edge detection was temporary, meaning my code never caught the edge. Once I fixed that...

After roughly three hours, including fighting with the compiler and tracking down foreign libraries, it was running. I had to invert the bits on the joystick, as I'd coded them with the wrong polarity, but that was it. I had a game of Donkey Kong, and got halfway around the world in Antarctic Adventure (a game I'd always wanted to see on the TI, actually...) Everything 32k or less worked, so, I have a functioning base.

I just need to add a disassembly and put up the Z80 registers instead of the 9900 registers, and I can use this for dev. (I also need to add megacart emulation and a couple of other small devices, another evening or two will do).

Sadly, I can't share this code as the GPL licenses conflict with my own, but it's a good learning experiment to see that it DOES come together as expected. Once I get the new architecture out, the multiple CPUs will be a lessor issue and easier to update.

Edit: I guess I forgot to mention why the Classic99 debugger is any better than anyone else's. I'll cover that later, when I update it, but for now, here's an example of the debugger being used to cheat at Alpiner: