June 30, 2012
The Development of Aemula

The Development of Aemula

Today we have an official version of Aemula for Android Tablet on Google play .

Aemula is an 80486 PC emulator for ARM devices that I have been working on since late January 2011, right after iDOS was pulled from Appstore for the second time.

iDOS was actually a DOSBox port to iOS. One of  the problems I had with DOSBox is that it was not fast enough to run civ2 on windows 3.1.  After all, the goal of DOSBox is to run DOS games, and civ2 must be of a lower priority to the authors.  However, this game happens to matter very much to me personally, for those countless nights that I spent on it. I can’t care less for the graphics, it is the inner working of this game that I love. Somehow DOSBox was not fast enough for running windows and I had no idea how to make it faster.

I decided to undertake this project on my own instead of working on the DOSBox port. For me it is always not easy to work on some other people's code.  You have the existing code but have no idea of the decisions made behind the code. You can only do small optimizations on a small part of the code because you have no control of the overall architecture.

It is an ambitious project, far more complicated than I thought at the moment. Had I known the scale of the work I was walking into, I wouldn't start it at all. Since then, I literally worked full time on it, started with a 80486 specification book bought online. I guess a lot of x86 emulators started with real mode implementation. Since my goal is to run civ2 on top of windows, it is very important that I start the design with protected mode in mind.  That is the most important design decision I made at that time. Real mode is not the bottle neck, and I was optimizing for protected mode right from the start.

Another decision is to write the cpu core in assembler.  I learned this from the experience with iDOS.  Looking at the lousy code generated by the compiler even with fastest optimization option, it was an easy decision, although it is very hard to implement. The debugging support for assembler is very inconvenient in xcode.  Between Feb 2011~May 2011, I wrote a lot of assembler for x86 emulation.  When a better idea overrules an old one, old code was replaced with new code. An advantage of coding alone is that you don't have to delete someone else's code.  It was a good decision that I tried to write the whole CPU emulation all together, because that exposed a lot of problems in my original design. There would be a lot of headaches if I started too early on peripheral emulation. It is very important to get the CPU interface correct at the first time even though I must have thrown away 80% of the code written in that period during the following year's refactoring.

Despite of the complexity of CPU emulation, it is not incomprehensible after all. I always loved low level programming ever since my college days, and the experience helped a lot . The really hard part is VGA. I had zero experience with it since I started programming, there was no such thing as VGA IO in my day.  I borrowed books from library (they were so old that it is really difficult to get one online), and started learning every thing from ground up.  It took about two months (Jun and July) before all this stuff made sense to me. It was so frustrating that I even gave up for about two weeks to work on something else.  In the end, with the help of   "Programmer's Guide to the EGA, VGA, and Super VGA Cards", I understood the various VGA modes, and with Michael Abrash's book, I finally had an idea how to implement the core of VGA graphics programming -- the testing of VSYNC signal, and some other tricky features such as the split screen operation.   After that, everything seems to fall into places.

It was in August that the code for VGA emulation was actually getting written. I wrote some unit tests to make sure they were basically working. And in September after I did the PIC/PIT emulation, I began to integrate components together, and got the test programs from Michael Abrash's VGA programs running. The test programs are small ones, with detailed code listing, and can run without the presence of DOS. So it is very useful for testing the correctness of VGA implementation.

After VGA is basically working, then I tried to get DOS running. I decided to use BIOS from Bochs project. It has source code, so it is relatively easy to figure out the cause once it goes wrong.  I implemented a subset of floppy controller, only enough to have the BIOS function well.  Debugging DOS booting process without source code is really hard. You basically have to trace instructions one by one, and remember a lot of critical addresses on paper. I wrote some debug extensions to make the job easier, but still it took days just to get the DOS boot. The moment "A:>" finally appeared on the emulator screen, I jumped from my seat in excitement.

However, it was just the beginning of an even more difficult journey. There are still a lot of things missing. In the following months, I continued to fix bugs (so many of them), put together more new components, like speaker, joystick, etc. And in December 2011, I published Aemula Oldies in Appstore to mark my work of 2011. Some people asked about tricks to hack Aemula Oldies, but the truth is that it was not worthy of hacking at all. It's very limited at that time.

Then came 2012.

In January/Feb hard disk emulation, serial mouse and sound blaster 2.0 was basically done, although there were  big rewrites in the following months as a result of bug fixes. Added Expand Memory in March, and got wolf3d/corr7 working.

In April, started debugging protected mode, beginning with throw away of the code written last year. The idea was basically the same, but the actual design has changed a lot. April and May turned out to be the most difficult part of this project. Spent a lot of time digging in the execution traces to figure what went wrong. There were DOS extenders with source code, and they really helped a lot in the debugging process.  At the end of May, Win3.1/Civ2 is working, though a little bit slow.  There were a lot of bug fixes and performance enhancements in June. VESA1.2 was added to support warcraft2.  There were also FPU bug fixes for XCOM.

Xcode and iOS is where I did all the development. However, since it is very unlikely that Apple would approve a standalone emulator, I had been trying to port Aemula to Android.  The first port was done in May. Native code debugging was so hard on Android that I wouldn't finish this project at all if I did all the development on it. Fortunately it didn't take much debugging for the code to work on Android because it already ran so well on iOS.

Now the most important features of the emulator has been done, I am going to slow down the development a little bit, enjoy Civ2, and wait for the feedback of users.  For how to use Aemula, please visit Aemula Website.