Logo Pending

Some personal notes on the case of byuu v. Google

Put simply: if the warnings were to escalate for not being addressed, I could see my entire website potentially blocked, which is something I’ve personally seen happen to a friend of mine who also hosted a completely safe binary: EliteMap, which was an editor for Pokemon games.

— byuu, Google Safe Browsing

That was me. I am the friend. Now, byuu described an issue somewhat different from mine. That article is about the higan multi-console emulator and how its latest release was googleblocked for being an “uncommon” download. No shit, we all cry out, it was just released that day. Now, byuu’s fear of the whole damn site being eventually turned red isn’t unfounded but I’d still like to describe why exactly mine was.

EliteMap was not an uncommon download. I’ve had this very subdomain for years now, and one of the first versions of my site had a custom-made content system with a matching uploader. That uploader happened to use the filebin directory, and I used it to release EliteMap 3.7 way back then. The problem was that DJ Bouché and I made it in Visual Basic 6, and I wanted to keep the download size down a bit so I compressed all the executables that made it up with UPX.

Bad move.

Some antivirus applications were a bit notorious even back then and the contents of elitemap37.zip were considered harmful by over-eager heuristics that thought if a Win32 executable is UPX-compressed, it must be hiding something. Oh well, no biggie. Just put up a note saying Avast sucks and get on with it, right?

I’ve replaced my site several times since then, though I still have backups. The one constant that I never removed for long? filebin/elitemap37.zip and filebin/sappy12.exe. I did that once, and soon enough people contacted me about it. So I put it back. History demanded it.

Then some time back, Google struck. I got word on the search console that elitemap37.zip was considered harmful and should be removed. I appealed, stating that the file is perfectly fine and AV are being silly about it… and it wasn’t long until my entire site was flagged and every page you’d try to open would turn the whole damn window red with danger. Twice, even.

So I heaved a heavy sigh and finally, after what might be ten years or more, deleted the damn file. And Sappy 1.2 along with it just in case. Sent word back to Google about having “fixed” it, and the red flag was lifted.

Happy new year to all of you, and a hearty fuck you to Google 🖕

Leave a Comment

Priorities Revisited

I just spent way too long drawing little hexagons. Here’s why.

As you know by now, SCI0 up to SCI11 use vector-based priority and control screens, while SCI2 introduced bitmap-based priority screens while also removing the control screen entirely. That means when I render a background for The Dating Pool, I have to trace out everything you can stand behind, by hand, as vectors. And a while back I replaced the simple railing on the space station scenes with hexagons, so I had to re-vector the priority screen to match.

I didn’t draw any hexagons where the booths are because you can’t stand close enough for it to matter. Saves a lotta work for me. That’s still a lot of Line and Fill commands though. And that’s just one screen up there. There’s three.

Here’s what it’d look look if we targeted SCI2, hypothetically:

Much easier to produce, perhaps. I could just load it into my editor of choice, select everything that’d be closer, and cut that out onto a new layer. Rinse and repeat. And despite appearances here, the priority layers wouldn’t all be the full screen size either. They’d actually only be… 320×17 pixels for the one and 320×22 pixels for the other. Could be even smaller if I cut the booth layer into three separate pieces, perhaps.

Note that the background layer is completely missing the colors from the railing and booth layers. Why compress that twice or more after all?

[ , ] 2 Comments on Priorities Revisited

Shampoo, Cologne, Mousse

(Edited from a Twitter rant in nine parts.)

I’ve complained about Shampoo’s name and how fanfic writers tend to write her “original Chinese name” often enough. I’d like to discuss Cologne and Mousse this time.

Now then. Knowing that Shampoo is the only one of the three with a name in actual Chinese characters, Cologne and Mousse are only ever written in katakana: コロン and ムース. Koron and Mūsu. Simple, right? That’s basically exactly how the products would be pronounced, just like with Shampoo.

Problem #1: Cologne’s fandom name is usually written as Khu Lon. Sometimes I think without the H. First of all, I can find no romanization scheme where khu is a valid sound, written that way. There’s khu but that’s in a scheme that’s not even used, and kuh isn’t quite it either. Second, it seems to me to be simply the wrong sound. The vowel is all wrong!

Problem #2: Mousse is usually given the name Mu Tsu, Mu Tse, and in this one doorstopper I’m slogging through, Mse Tsu. I already covered how absurdly wrong that third one is. If these were at all right, wouldn’t his name be written ムー, tsu? But it’s not. There is no T sound in either his established katakana writing or in his name as spoken.

Shampoo’s name, as covered before, is shanpū in Japanese, with a lengthened -u. It’s shānpū in Chinese, where I believe the accent marks denote tone? Correct me on that if I’m wrong. So at worst the length of the -u is different. Likewise, Ranma’s name “translates” to Chinese as luanma. Exactly the same kanji (乱馬) and all that. Readings are cool like that. Between the well-known L/R difficulty and an easily drowned out u, that’s also basically the same when pronounced.

Why then, in the name of logic, should Cologne and Mousse have their names so different? At least back when they started, fanfic authors had no way to look this shit up — there was no Wiktionary or such back then. You really have no excuse now. You can do better than this. If you were to tell me “but those are their names”, you basically admit to being both uninformed and being too lazy. “That’s how we’ve always called it in the fandom” is arguably better, but again you can do better.

Since we can look shit up, let’s actually do look this shit up! What is “eau de cologne” in Mandarin Chinese? It’s 科隆香水, kēlóng xiāngshuǐ. “Mousse” as in the hair product? That’s 摩絲, mósī.

Shānpū, Kēlóng, Mósī. Research over.

But wait. Kēlóng? Ke? Didn’t I say earlier that the vowel is all wrong? I did. It’s a romanization difference. The zhuyin is ㄎㄜ. I gave their names just now in hanyu pinyin. In Wade-Giles, it’s written with an o. Simple. Mousse’s ㄇㄛ is mo in both.

Just for fun, I thought I’d look up some common fanmazon names, but reconsidered when I found Chinese has a perfectly good word for perfume. Oh well, I’d made my point already 🤷‍♀️

[ , , ] Leave a Comment

More regarding Interrupt 21

Last time I explained how your standard file rename function as seen in MS-DOS worked. You’d set up two CPU registers with pointers to the old and new names, set AH to 0x56, and called Int 0x21. Easy, right? And then I went into detail on how malformed inputs were handled. They weren’t handled too well, and DOSBox does it differently from MS-DOS on top of that.

But what if we had a file system and rename function that did support spaces? Maybe more than eight characters, even? In mixed case?

That is of course VFAT, an extension to regular FAT16 available in Windows 95, NT 3.5, and later. With a VFAT driver, most of the old file operations available from Int 0x21 had counterparts installed that generally took the same arguments and had the same numbers, but accepted long filenames.

So to rename a file with long filename support, you’d do exactly what you’d do before but instead of setting AH to 0x56 you’d set AX to 0x7156. Assuming Windows is running and we use the same inputs as last time, your file will now be named hello world.txt. And that’s all that takes, even if it’s a pure DOS program doing it.

Which raises a question. How do you make a pure DOS program that handles files that may have long names, may be run from Windows, and should not drop any of those long names if it is in fact running in Windows? Well, it turns out all those LFN functions — the ones starting with 0x71, all reset AX to 0x7100 if they’re not installed. A trick of the system, I suppose. So what you could do for your LFN-enabled rename function is try to use 0x7156, see if AX has reset to 0x7100, and if it has, you try again with AH set to 0x56. In other words, it’s time to bring back the rename function from SCI11… or rather a branch of SCI11+ that I’ve been working on.

rename	proc	oldName:ptr byte, newName:ptr byte
	mov	dx, oldName	; ds:dx = old name
	push	ds
	pop	es
	mov	di, newName	; es:di = new name
	mov	ax, 7156h	; LFN Rename
	int	21h
	.if	ax == 7100h	; LFN failed, try DOS 2.0 version
		mov	ah, 56h
		int	21h
	.if	carry?
		xor	ax, ax
rename	endp

It’s that easy. Of course, this is old-school MASM code which has some nice things like .if but that’s just sugar to avoid having to write compares and branches — the concept should be clear enough. An attempt to rename a file to Introduction.txt will result in exactly that on Windows, or transparently collapse to introduc.txt on plain DOS.

Note that in the actual SCI11+ code, if you’re crazy enough to look it up, there’s an extra function I made that’s called right before the DOS 2.0 rename call that replaces all spaces with underscores, which renders them about 100% not as confusing and untouchable as the one shown last time. I left that part out for brevity.

[ , ] Leave a Comment

This is why you sanitize your inputs, 1983 edition

(This is heavily expanded from a few Twitter posts of mine.)

When you write an application that has to rename a file, you have your chosen language and platform’s standard library to do the heavy lifting for you. For example in C it’s usually int rename(const char* oldName, const char* newName), and a bunch of other languages follow suit. Why not, it’s a good function! But what does rename actually do?

In MS-DOS, this’d be handled by Interrupt 0x21, subfunction AH 0x56. By which I mean it’d set two specific processor registers (as mentioned in Save Early, Save How) to point to the old and new file names, set the AH register to 0x56, and execute the INT 0x21 instruction. A function installed by MS-DOS will then take over, doing the actual renaming, possibly returning an error value which the C function can immediately use as its’ return value. Since SCI has its own “need-to-use” library…

rename	proc	oldName:ptr byte, newName:ptr byte
	mov	dx, oldName	; ds:dx = old name
	push	ds
	pop	es
	mov	di, newName	; es:di = new name
	mov	ah, 56h
	int	21h
	.if	carry?
		xor	ax, ax
rename	endp

(Full disclosure: the SCI code actually includes a dos macro to save the programmers some typing. I unrolled it here for illustration purposes.)

All of this pretty much matches what you can find on Ralph Brown’s list. Given a suitable function prototype in C such as the one in the second paragraph, SCI can now call its own rename function as it desires.

Enough about SCI though, its function as a practical example is at an end.

But what if you gave it bad inputs? Sure, if the old name doesn’t refer to an existing file it will return 2 “file not found”, but what if the new name isn’t quite valid? Remember, this is MS-DOS; we don’t have the luxury of long file names here. It’s 8.3 or bust. I don’t see any sanity checks in the above function, and Brown’s documentation only speaks of splats.

So what happens if we have a file boop.txt and call rename("boop.txt", "hello world.txt")?

In DOSBox, you’d end up with a file hellowor.txt. You are free to further manipulate this file in any way you please. The command line won’t choke on it, file managers won’t get confused. If you wanted to manually rename it back to boop.txt from the command line, ren hellowor.txt boop.txt will work perfectly fine.

This is actually not true in real MS-DOS. If your program were to run on a real MS-DOS installation, you’d end up with hello wo.txt, an 8.3 file with a space in it. And no contemporary file manager I’ve seen can handle that. The ren command built into command.com can’t parse it — ren hello wo.txt boop.txt is three arguments where ren expects only two, and the first isn’t an existing file’s name that it can change to wo.txt.

In cmd.exe of course you can use double quotes to make it unambiguously two arguments, but this isn’t cmd.exe. What about some file managers though? I have two, Norton Commander and its big brother Norton Desktop.

In Norton Commander, the file list shows hello wo.txt, and its rename function can handle it. So can the built-in editor and viewer. Top marks for Norton Commander!

Norton Desktop on the other hand is not so sturdy. It can show the file in the list but that’s all. Trying to rename it back to boop.txt reveals the incorrectness of the source file’s name quite succinctly:

Technically, this is true. You’re not supposed to have spaces in the middle of a FAT 8.3 file name. If a file has less than eight characters before the dot, it’s secretly padded with spaces, and so are the three extension characters. And the dot isn’t even — the true name as written in the FAT directory would be BOOP    TXT. But that’s just one way Norton Desktop trips. Its viewer seems to be passed the nonexistent hello. It shrugs and asks which existing file we want to open. Its editor is given the same argument(s?) and lets us edit a brand new file named hello. In Norton Desktop’s world, it can see the file, but it can’t do much with it.

What about a contemporary Windows? Can, let’s say, the Notepad from Windows 3.1 handle this file? Okay, so technically this is commdlg.dll talking, but we’re playing for effect here.

Of course not, what did you expect by now!? Norton Commander only worked because it didn’t care enough! Would you really think one of the companies who made the FAT file system would blithely ignore one of the cardinal rules at the time?


Next time, we gettin’ hacky.


…Wait, hold up. Why does it say 1983 in the title? Well, if you notice on Ralph Brown’s site the rename function was introduced in DOS 2, which was first released in 1983. And so was I.

[ , ] Leave a Comment

SCI Drivers, how do they work?

Quite well, thank you.

Now, before I begin I should mention that the extended memory and mouse support in SCI is baked into the interpreter itself, and in SCI32 so are the VGA and VESA video drivers. Oddly, the keyboard is not. But anyway!

An SCI DRV file is a piece of standalone code that is loaded on startup and provides a set of standardized routines for the interpreter to call, even if a given feature isn’t technically supported such as palette rotation in EGA or most things on PC speaker. Their format is actually pretty straightforward.

The first four bytes of the file are a single instruction that jumps to the entry point routine. Whenever a driver function is called for, the BP register is set to the requested function and that jump is called. The entry point routine then uses a lookup table, indexed on BP, to call the requested function, and returns. Easy as pie, really. But that leaves the format.

The next four bytes are the number 0x87654321. Yeah, backwards. I know, wild times. This would be used to check if a given DRV file really is a valid driver, but SCI11 at least seems not to care.

Following is a byte that specifies what type of driver we’re talking about. The install/setup program uses this to determine which list to show it in. It’s otherwise useless, especially with my own install program. From zero to seven, these types are: video, music, voice, input, keyboard, communication, mouse, and memory.

The rest of the data can go basically anywhere — they’re Pascal-style (length-prefixed) strings that should specify their file name and description, but some of them are named dude and one is  . Just a single space. And the descriptions? You wouldn’t be able to tell VGA320.DRV apart from VGA320BW.DRV if you followed this information ‘cos they both say they’re “VGA or IBM PS2, Models 25 & 30 – 256 Colors”. Except one of them is optimized for grayscale-only monitors. So yeah.

Next up is the EXTDRV marker, 0xFEDCBA98, directly followed by a four-byte value whose meaning depends on the type. Again, not actually used, purely for external bookkeeping.

Value Video Keyboard Sound
1 MDA IBM Speaker
2 Hercules Tandy AdLib
4 CGA NEC Sound Blaster/DAC
8 PC Jr. Creative Music System
10 Tandy Tandy 3-Voice
20 EGA Tandy DAC
40 MCGA PS/1 3-Voice
100 CGA two-color Sound Blaster Pro
200 CGA four-color MPU-401
400 Explorer Disney Sound Source
800 CD-Audio
1000 ProAudio FM
2000 ProAudio DAC
4000 Windows Sound Source
8000 No MIDI

These values can be combined, so MTBLAST.DRV for example reports 0x204, or “MPU-401 with Sound Blaster DAC”. Of course, this would only make sense for the sound drivers but what can you do?

After this, the actual driver code resumes with the dispatch table, a list of function pointers to each of the features the driver supports. Some may point to null functions, but none of them are themselves null. Because that’d be bad. Directly after this is all the general-purpose variable memory that the driver may need, all preset. For example VGA320.DRV has a standard palette and “wait hand” cursor of its own that it throws up initially.

There are three functions that all drivers must have. The rest depends on their type. These are Detect, Initialize, and Terminate, in that order. The first simply returns a few metrics. For video it’s the number of colors, for music it’s the device ID (to decide which channels to play) and how many voices it can handle. The Initialize function actually sets up the device, switching video modes and setting up timers or what have you. What Terminate does ought to be obvious.

And that’s how SCI drivers work.

[ ] Leave a Comment

Police Quest’s flashing siren lights

The flashing siren lights in the title screens for Police Quest 1 and 3 are sort of interesting, because they are not quite a simple matter of calling (Palette palANIMATE) once or twice. In fact it’s called eight times each frame! Here’s the final result:

And here’s the Script at the heart of it:

(instance cycleColors of Script
  (method (changeState newState)
    ; Fun fact: the switch isn't actually needed.
    ; Not in this use-case.
    (switch (= state newState)
        (Palette palANIMATE 208 213  1) ;blue in the middle
        (Palette palANIMATE 213 218  1)
        (Palette palANIMATE 218 223  1)
        (Palette palANIMATE 223 228  1) ;blue on the side
        ; Note that we're switching from 1 to -1 now.
        (Palette palANIMATE 229 234 -1) ;red in the middle
        (Palette palANIMATE 234 239 -1)
        (Palette palANIMATE 239 244 -1)
        (Palette palANIMATE 244 249 -1) ;red on the side
        ; Almost immediately do it all over again
        (= cycles 10000)
        (= state -1)

The palette here has a very particular setup. The lowest colors, #208 to #249, are set up like this:

Each of the eight siren colors in the image has its own four-step palette, individually rotated! It looks kinda like this:

If that one black entry wasn’t in the way between blue and red, it’d line up better, but what can you do?

What’s particularly funny about this is of course that no SCI interpreter with fewer than 256 colors implements this feature.

The cycleColors script is still there and is still invoked. Just like with the chronostream animation in Space Quest 4.

[ , , ] Leave a Comment

Save early, delete when you need

There’s one interesting tidbit missing here, which is how deletion (SCI1 and later) is implemented. Namely by manipulating the .DIR file in the script, and not – as any sane person would do – with a kernel call.

So wrote Iskovlun in a comment some time back. Let’s see exactly how insane it really is.

; First we open up the directory file.
; Confusing, I know, to call it a directory file. Perhaps
; "catalog" would be better considering a directory is
; already something else. And in SCI32, they did!
((= fd (File new:))
  name: (DeviceInfo diMAKESAVEDIRNAME @str (gGame name?))
  open: fCREATE
; The format of a save game directory is pretty straight-
; forward -- a word for the index, then the name, terminated
; with an $0A, repeat until done, end with $FFFF.
; (File write:) requires a pointer to the data it is to write,
; so we need to put values into variables, rather than just
; passing them immediately. Well, unless you have SCI11+ with
; the extra file kernel calls I nabbed from SCI32 and a matching
; File class, in which case you could just do (File writeByte:
; $0A) if you were so inclined!
(= ret $0A0A)
; Now we write the number and name of each saved game, EXCEPT
; for the one that was selected for deletion.
(for ((= i 0)) (< i numGames) ((++ i))
  (if (!= i selected)
    (fd write: @[nums i] 2)
    (fd writeString: @[names (* i COMMENTBUFF)])
    (fd write: @ret 1)
; Now we write the terminating $FFFF to finish the catalog
; I mean directory off.
(= ret -1)
  write: @ret 2
; Now that that's done, we can safely delete the actual
; save game file.
(DeviceInfo diMAKESAVEFILENAME (gGame name?) [nums selected])
(FileIO fiUNLINK @str)

I almost feel like doing the so-called sane thing and adding a DeleteGame kernel call to SCI11+.

[ ] Leave a Comment

VGA Versus VESA – A Followup

Back in the days, if you wanted square pixels in your games you had to either switch to 640×480 with only 16 colors or use Mode X, which offered 320×240 with all 256 colors but both of these stored their pixel data in those funky non-packed or non-linear formats.

Sure. Sure. With Mode X you could write a whole bunch of pixels at once. It’s a trade-off, and one that I’d rather not have to make. Give me linear frame buffers or give me death!

But what if there was a third option? Turns out on DOSBox’s approximation of the S3, mode 151h offers 320×240 pixels in 256 colors! Besides the need to switch memory windows to access the bottom 36-something lines, that’s as friendly to work with as anything! And if, unlike me, you write DOS games in 32 bits, even that shouldn’t be a problem!

Too bad whatever VirtualBox exposes on my system doesn’t include one of those — it has no 320×240 at all.

As a bonus for sticking with me for this tomfoolery, here’s DOSBox running a test of mine.

[ ] 2 Comments on VGA Versus VESA – A Followup


We’re all familiar here with the classic 320×200 pixels, 256 color screen mode popularized by the VGA video card, colloquially known as Mode 13h. Most old DOS games from before a particular point in time used it. But what if you want or need bigger? Or more colors? Enter the Super VGA cards with their extended VESA modes.

These VESA modes number 100h and higher, but which exactly are available and what their specs are depends on your exact hardware. As such you can’t rightly assume a certain mode is available and will be that particular resolution and color depth. What you’re supposed to do is ask the system what VESA video modes are available, walk the list to see if you find the one you need, and note its number.

All I have is a copy of DOSBox and a copy of VirtualBox, and of vesachk.exe, available here if you want to try it out yourself. This application gives you that list. Now, the two systems yield vastly different results, mirroring differences in video hardware. DOSBox for example emulates some form of S3 card.

I’ve noticed that when an SCI32 game is made for low-res it runs in plain ol’ Mode 13h, but if it’s meant for higher it’ll use Mode 101h. That’s one of the few in the list that DOSBox and VBox agree on — it has 640×480 resolution at 8 bits per pixel, packed, starting at 0xA0000000.

On the one hand, a regular old 16-bit DOS application wouldn’t be able to address all 307,200 pixels at once the way you can in mode 13h. On the other, a 32-bit application would have direct access to the full linear frame buffer no matter its size. A 16-bit application would need trickery to reach any pixel beyond a certain point, setting the window registers to basically shift the next part of video memory into that same 64 kb block.

This is why SCI32, when switching to mode 13h, just does so:

void Vga::Set320x200()
	union REGS reg;
	reg.w.ax = 0x0013;
	int386(0x10, &reg, &reg);
	SetVideoMode(1);	// clear all video memory
	SetVideoMode(0);	// back to Normal Mode 13
	lenx = 320;
	leny = 200;

But when it switches to mode 101h, it jumps through several hoops. First it checks if VESA is supported at all, then it switches to mode 101h, and then it does some more checks to see if things are as they should be, bailing out if they’re not.

And that’s all good.

But what if you were to find a VESA video mode that was 320×200 with 256 colors? Is there such a thing? A redundancy with mode 13h? As a matter of fact, there is! On the S3 emulated by DOSBox, it’s VESA mode 150h, and once you switch to it things work exactly the same as in mode 13h, except the memory access timing or whatever is different.

; VGA mode 13
mov ah, 0x00
mov al, 0x13
int 0x10
; VESA mode 150h
mov ax, 4F02h
mov bx, 150h
int 10h

But on whatever VBox has to offer, which is a vastly longer list with about a hundred more modes, this could be 146h, 346h, 546h, or 746h. And that’s why you really should ask the system about it!

But SCI32 basically assumes 101h is what it needs to be and presumably the folks at Sierra tested a bunch of contemporary cards and found this to be true.

Fun fact: SCI16 has all its video driver code in files like VGA320.DRV, but SCI32’s VGA.DRV is practically empty. It’s technically a valid SCI driver file but it’s basically just a header. Same with its VESA.DRV. All their code is in the interpreter itself, much like the mouse driver. It’s only there so the installer can offer it and the interpreter can determine which was chosen. And even then, the interpreter for later high-res only games like Gabriel Knight 2 will happily ignore all that.

[ , ] Leave a Comment