MSX DOS 3 project

For some time already I have been playing with the thought of creating a multitasking module-based operating system, tentatively named DOS 3. I have done much thinking about it, and written down a lot of ideas (which I will try to put on this page when I have time), but at the time of writing the amount of actual code written is still hardly impressive.

In any case it would initially be a project targeted to my personal needs. This means that I’d create only the core functionality (scheduler, driver manager, etc), and add extra features to it as I go when build projects upon it. If at some point this shapes into something useable, I will release it to the public, probably as open-source. Because of its modular nature, anyone should then be able to fairly easily create new driver modules (API’s) for it.

Below are some of my thoughts on the why, what and how:

Motivation

There are several key features I would like to see in an operating system for MSX…

Driver manager

Right now, I have a certain standard set of functions duplicated in all my projects, and I think that is kind of a waste of space, and also a bother to have to copy everytime I start a new project. My basic set of VDP-related functions could then for example form the basis for a graphical API. Because of this potentially huge (though not bloated) API, you should be able to more rapidly develop applications for MSX, and it should also be easier for beginners as they can make use of an already existing set of functions.

Speed

Don’t get me wrong, I love DOS 2, but there are definitely points where it could use improvement :). Faster console I/O for example, which is abysmally slow in DOS 2. You can of course not access the hardware directly anymore in DOS 3 which would make things theoretically slower, but I intend to make up for that by providing a lot of block-level functions and optimizing them highly in various ways such as by unrolling critical loops and using table indexing, etc. Also, specific constructs for high-speed access will be created where needed.

Multitasking

Back when I was working on a TCP/IP stack for MSX (don’t ask - sad story :)) at many occasions I wished for a multitasking OS. Multitasking is very feasible on MSX, as most of the time you will only have one task actually running, the others are usually waiting for user input, or just inactive. It is quite convenient, because you could be unpacking a file while editing a text file in the meanwhile. Even more convenient is to be able to keep that editor in memory while quickly going to another shell with a commandline where you could do various other stuff (kind of like what Compass can do).

Features / ideas

Note that we are talking about a product in design phase here, so these are not final in any way…

• DOS 2 compatible.

  • This would be achieved with some kind of emulation layer. Although at many points radically different, DOS 3 will be ‘based on’ the DOS 2 philosophy, and designed to be flexible enough to create an emulation layer for it.

  • Note that due to its multitasking nature it will only be compatible with software not accessing the hardware directly.

  • The system will ensure that you can only run ‘good’ software by creating a whitelist of working applications, using CRC’s. Of course you can (at your own risk) add applications to this list.

  • A special ‘single-tasking’ mode for legacy software directly accessing the hardware and (some) games needing full system dedication is also under consideration, I’ll have to see whether it is possible.

• Initially I will create the core functionality. That is, scheduler, driver manager, mapper routines (incl. malloc), and console I/O. After that, file system support, graphics libraries, etc.

• The kernel will be some kind of hybrid between a microkernel, because of its good design philosophy, and a monolithic kernel, for speed considerations. Note that I don’t fully understand the exact difference between those two concepts yet, so I can’t be much more specific on which of the two it will resemble most. There will also be some real-time aspects to the kernel (line splits should be a possibility).

• Employing driver system for creating API’s.

  • Drivers can be loaded on-demand, and will be unloaded if they are not in use anymore.

  • There would for example be a core VDP driver providing only the raw access features, and an abstracting one on top of that with more advanced stuff. Internally, these might be integrated to gain additional speed, although that probably conflicts with the microkernel idea.

• Can be run stand-alone or from within DOS2 (initial target). Will also be able to boot from disk and from ROM. Thought: ROM could also reduce the memory used by the kernel? Depends on good code/data seperation and not using self-modifying code.

• Recovers from crashes. After a reset it will trash the crashed program and continue with the others.

• Except for kernel drivers, software may NOT use any of the following instructions: OUT, OUTI, OUTD, OTIR, OTDR, IN, INI, IND, INIR, INDR, HALT, IM, DI and EI.

• Memory related stuff:

  • Memory requirement: 128kB RAM.

  • The mapper driver includes two methods, one like the DOS2 one, the other offering a higher level of abstraction like MemMan does.

  • The mapper driver also includes a malloc for allocating user memory, high memory and system (shared) memory.

  • There would be an adaptive RAM-disk, which does not allocate a fixed amount of memory, but instead allocates more memory as more files are put on it.

Technical details

DOS3 extensible .COM file format:

Value(s)	Description
3 bytes: #C3,#00,#00	These three bytes are for DOS2 backwards support. They do a JP #0000, which quits the program immediately so that nothing weird will happen. You may also jump to a handler elsewhere in the file (in an ignore block for example), which shows a describing text (like LD A,#85 : LD C,#62 : JP 5). On x86 CPU’s, #C3 is RET, so this will also prevent you from running (and crashing) this code on a PC.
1 byte: version	File format version number. Current version is #00.
4 bytes: "DOS3"	DOS3 file ID
word: block-type	type of the following block #0000 = ignore #0001 = program code #0100 = allocate additional memory data: #xxxx = allocate memory up to address (e.g. #8000 or #D000) #0101 = allocated stack space data: #xxxx = stack space (recommended: 256 for small stack, 512 for large stack (default)) #1000-#1FFF = user defined #FFFF = end bit 15 = 1: 32-bit block length
word: block-length	length of following block
… data …	data block

More will follow later…

Other sections

• Misc. ideas