Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68
Posted on 25-04-08, 02:21 in (Mis)adventures on Debian ((old)stable|testing|aghmyballs) (revision 2)
Dinosaur

Post: #1341 of 1342
Since: 10-30-18

Last post: 18 days
Last view: 10 hours
I was reviewing my Super Mario 64 for Windows 9x crosscompile notes as someone in IRC had asked me for help (remember: we can't share binaries because Nintendo), when I noticed several things:

- Using prebuilt dependencies is nice, but then you're subject to whatever compiler flags the repo/distro used to build them.
- These msys2 dependencies were clearly built for Pentium 4 or newer CPUs, and it was pure chance that they worked as-is on older CPUs - on my Coppermines, they would crash with a invalid instruction if a gamepad were to be plugged in before running the game.
- Can we get rid of the annoying import table edits?

So I decided to go full crosscompiler here and build not only sm64ex, but also its 3 dependencies for Windows: SDL 1.2, GLEW, and libiconv. Figuring this out wasn't trivial since the procedure to crosscompile things is very poorly documented out there, if at all, especially SDL! But thankfully they use mostly sane-ish build systems that cater for that, so it's not impossible, just annoying.

What do you need:
- sm64ex - the latest git will do.
- The SDL 1.2.15 (the final release on the 1.x line) full source tarball (not the Win32 mingw devel one!) - the official download page will tell you to go away, but the link itself is hidden at plain sight.
- libiconv tarball - as of this post, the last release is 1.18, but I guess any older version will work as well.
- GLEW 2.2 source tarball - get it here. Note that unlike SDL and libiconv (which use good ole' autotools), GLEW has its own build system, made of compiler-and-platform-specific Makefiles.
- On your Debian box, install mingw-w64.

How to build:
- Pick an architecture to build for - the default will be your native host, which in the case of retrocomputing is NOT what you want! If you want to run this on a Pentium 3, then your target arch will be "pentium3". A older Socket 7 Pentium for gits ang shiggles? Then you want "i586" or "pentium" (DO read the caveat at the end of this post). Read the GCC relevant manual page for supported x86 targets. From now on we will call this $TARGETCPU.

- Pick a path to put your crosscompiled dependencies - somewhere under your /home is fine. Let's call this dir $WINDEPS from now on. If you want to build binaries optimized for different CPUs, you may want to have separate dependency dirs, for each architecture.

- Build SDL:
tar xzvf SDL-1.2.15.tar.gz
cd SDL-1.2.15
./configure --target=$TARGETCPU --build=i686-w64-mingw32 --host=i686-w64-mingw32 --prefix=$WINDEPS --enable-static
make
make install


- Build libiconv
tar xzvf libiconv-1.18.tar.gz
cd libiconv-1.18
./configure --target=$TARGETCPU --build=i686-w64-mingw32 --host=i686-w64-mingw32 --prefix=$WINDEPS --enable-static
make
make install

(yes, it's exactly the same procedure as on SDL)

- Build GLEW:
tar xzvf glew-2.2.0.tgz
cd glew-2.2.0
make SYSTEM=linux-mingw32 POPT="-march=$TARGETCPU -O2" GLEW_DEST=$WINDEPS
make SYSTEM=linux-mingw32 POPT="-march=$TARGETCPU -O2" GLEW_DEST=$WINDEPS install


- Before continuing, we need to do a small edit to $WINDEPS/bin/sdl-config or GCC will act stupid and won't find the SDL headers (it's the same second edit from my old guide, the first edit is no longer needed as long as you don't move your $WINDEPS, but you can do it if you wish anyway):
#line 45
echo -I${prefix}/include -D_GNU_SOURCE=1 -Dmain=SDL_main


- Now build sm64ex using your fresh custom crossbuilt libs:
cd sm64ex
make TARGET_ARCH=$TARGETCPU TARGET_BITS=32 RENDER_API=GL_LEGACY WINDOW_API=SDL1 AUDIO_API=SDL1 \
CONTROLLER_API=SDL1 WINDOWS_BUILD=1 CROSS=i686-w64-mingw32- CC=i686-w64-mingw32-gcc CXX=i686-w64-mingw32-g++ \
SDLCONFIG=$WINDEPS/bin/sdl-config

Your resulting executable (assuming you're using a US ROM) will be at ./bin/us_pc/sm64.us.f3dex2e.exe

- Check your built executable with Dependency Walker - there should be no mentions of _strtoi64/_strtoui64 anymore. In fact, you can put that hexeditor (or PE editor) away, there is nothing to fix now as the crosscompiled executable is ready to run as is on Windows 95 (!!!) and later. Oh, and if you're on non-SSE2 CPUs, you can now use GAMEPADS without having the thing crash and burn, glee~!


Caveats:
Because nothing is perfect, there are some problems you might experience during crosscompilation

- Your ./configure script for SDL/libiconv may die with a stupidly dumb error like this:
configure:5508: checking whether we are cross compiling
configure:5516: i686-w64-mingw32-gcc -o conftest.exe conftest.c >&5
configure:5520: $? = 0
configure:5527: ./conftest.exe
./configure: line 5529: ./conftest.exe: cannot execute binary file: Exec format error
configure:5531: $? = 126
configure:5538: error: in '/blah/blah/deps/libiconv-1.18':
configure:5540: error: cannot run C compiled programs.
If you meant to cross compile, use '--host'.
See 'config.log' for more details

I got this report over IRC, but I have no clue on why ./configure would even try to run a foreign binary....
But wait, it gets even more stupid - here is the result on MY machine:
configure:5508: checking whether we are cross compiling
configure:5516: i686-w64-mingw32-gcc -o conftest.exe conftest.c >&5
configure:5520: $? = 0
configure:5527: ./conftest.exe
008c:err:winediag:nodrv_CreateWindow Application tried to create a window, but no driver could be loaded.
008c:err:winediag:nodrv_CreateWindow L"The explorer process failed to start."
008c:err:systray:initialize_systray Could not create tray window
configure:5531: $? = 0
configure:5546: result: no

Yes, on that machine I have Wine installed (and which failed to run because I was compiling over a SSH console), but that was enough to fool ./configure. What the FUCK!? Seriously, Autotools and friends can go die in a bonfire fueled by RMS toenails. If anyone knows how to properly deal with this without installing a frickin' Windows emulator, I'm all ears.

- If you ask GCC to build i586/pentium executables, the resulting binaries will... crash and burn instead. Because GCC now assumes "i586 = it has CMOV, right?", and completely litters your code with those conditional moves that will SIGILL at the first chance on actual period accurate silicon. Apparently GCC is not alone on this, as Clang has equally wrong assumptions on what's is a x86 these days too (apparently "i686 = it has SSE2, NO EXCEPTIONS"), and even Rust is on the same demolition derby for classic CPUs. Someone at #debian told me that this was one of the reasons they had to drop i386 from Trixie as a release architecture: because GCC and friends do not really recognize what's a real 32-bit x86 CPU anymore. So for all practical effects, with GCC 12 and later you're limited to "Pentium Pro and later". I might try crosscompiling from a chroot or VM running older Debian one of these days, but that won't be fun.


Some performance tests:
- PCShits M756LMRT + Radeon VE + Pentium III 750MHz: minor stuttering at times, high CPU usage, but very playable at 800x600
- Compaq/Mitac BMW mobo + GeForce MX4000 + Pentium III 1GHz: smooth as butter, minimal CPU usage, why Nintendo has been leaving money over the table after all these years?!

Bonus screenshots!
98SE Me
"Dear Mario: Your BSoD may be in another castle, yours truly, Princess Bitch."

Licensed Pirate® since 2006, 100% Buttcoin™-free, enemy of All Things JavaScript™
Dinosaur

Post: #1342 of 1342
Since: 10-30-18

Last post: 18 days
Last view: 10 hours
Bought ANOTHER cheap adapter meant for 2.5" drives, but this one also has a 12V input (standard 5.5mm x 2.5mm barrel jack, positive center) so you can attach a external PSU and use 3.5" drives with that (sadly that means it still wants the full 5V load from the USB port!).

Surprise: yet another JMS578 under the hood (made in 2023, but the flash is from another vendor, not identified by the JMicron flasher). On this one UASP seems to be stable, and I hacked up a barrel jack to 4-pin Molex adapter to check the 12V side. Some notes:

- SMART may not work out of the box with smartmontools since it relies on a database of USB VID/PIDs (part of drivedb.h) to know which specific commands to send to the bridgechip (those JMS57x require -d sat). The drivedb.h as shipped on Debian Stable packages of smartmoontools only seems to know the generic JMicron VID/PID (0x152d:0x0578), but the last two generic cables I've bought instead steal the VID/PIDs of some OWC cable! (0x7825:0xa2a4). Run update-smart-drivedb to fix this on Debian, since the latest drivedb.h adds both that OWC VID/PID but also a more generic 0xab12:0x34cd VID/PID for more generic junk). Do be aware that using those "alternate" VID/PIDs will bypass Linux' USB quirks mechanism blacklist, so expect weirdness.

- TRIM support is unreliable, even on the known good Odroid "Hardkernel" firmware. Some SSDs may get TRIM working out of the box, others require manual setup under Linux, and others may not enable the TRIM support at all on the bridge even when it works fine on an actual SATA port!

- Offset 0x0A now has a purpose in life: set it to 0x0A, and the bridge will copy the ATA drive ID string as its own SCSI vendor/product ID. Set it to anything else and instead it will use the generic SCSI vendor strings from offset 0xC8 onwards. Normally this offset is set to 0x01 in such cases, and it doesn't seem to be a case of "tick those two bits" as even 0x0B will revert to the EEPROM vendor string, so instead it's treated as a magic byte. I've already updated that on the EEPROM layout notes post.

- Also managed to confirm the behavior of offset 0x54 - normally this is the USB string descriptor length for the descriptor with the USB serial number. Set this to 0 and the bridge will copy the serial number from the attached drive... most of the times. Sometimes instead it will come up with a made-up serial number.

- Regarding that SCSI vendor string, noticed a weird caching issue which can happen when swapping drives. Consider this case:
0) Ensure your 0x0A offset is set to the magic 0x0A so the bridge uses the attached drive ATA ID.
1) Plug a drive that identifies itself at "StorAge LargeDrive 3000", then plug the bridge to a host. The bridge uses that ID as its SCSI device name.
2) Unplug the adapter, swap drives. Let's use another drive that identifies itself as, say, "HD2000T0001".
3) Plug the adapter again. Your OS will briefly see "StorAge LargeDrive 3000" for a split second (and it may even bitch about it!), then the adapter will reset, and this time "HD2000T0001" will properly be recognized.
4) If you unplug then replug the adapter without swapping drives, this weird reset will not happen since IDs will match.
Here is a example with real world drives (a 80GB Toshiba 2.5" swapped with a 1TB Toshiba 3.5")
Looks like some caching of device IDs may be happening, alas, I can't find any evidence on this on my EEPROM dumps, so most likely the firmware is using some of that extra flash space to do some (pointless) caching. Alas, this seems to be happening on the (normally) undumpable flash area :/

Licensed Pirate® since 2006, 100% Buttcoin™-free, enemy of All Things JavaScript™
Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68
    Main » tomman » List of posts
    Kawa's Github