0 users browsing Engineering. | 1 guest
Pages: 1
| tomman |
Posted on 24-12-26, 02:01 (revision 1)
|
|
Dinosaur
 Post: #1321 of 1326 Since: 10-30-18 Last post: 1 day Last view: 21 hours |
So this is a feature that has been bugging me for AGES with those cheapo SATA to USB bridges: in a effort to cater to the XP crowd, well over a decade ago many bridges found a clever way to bypass the 2TB limitation of MBR: pretend the drive uses 4K sectors! (And this was the early era of AF drives, which also nicely maps to the inner layout of the platters themselves). This comes with the neat advantage of letting you use MBR-partitioned drives up to 16TiB (because one LBA is now one 4K "physical" sector, and the 32-bit limit from MBR means that ~4 billion LBAs * 4096 bytes = 16TiB). Cheap hack that eventually would become a scourge as XP faded into irrelevancy, and instead would bring pain for those regularly relying on USB to SATA enclosures and adapters to juggle with HDDs around: - A drive formatted/partitioned on a "4K" bridge cannot be read on a bridge not enforcing this translation, nor on an actual native SATA port on a PC, and viceversa (either the OS fails to find a vaild partition table, or the partition table extends beyond the native drive capacity leading to mount errors). No, GPT partitioning won't save you from this. - This also makes those "translation-enforcing" bridges unsuitable for cloning disk drives, or any kind of emergency data rescue procedure! - The enforcing of 4K sectors is very inconsistent across bridges, and even among bridges based on the same chip - you may get bridges that enforce 4K sectors only on AF drives, on all drives, or on drives up or past some size threshold (say, 2TiB). JMicron bridgechips fall into this category of data mangling, and sadly they have been wildly popular, dirt cheap, and readily available everywhere in the last decade (and then some). Another common feature of those bridges is an optional EEPROM chip used to store firmware and adapter settings, including the dreaded 4K emulation that can actually be turned on and off, but sadly neither JMicron nor the myriad of Chinese OEMs building enclosures using their bridgechips offer any kind of tool (not even a VSC!) to let users control this undesirable feature. In fact those bridgechips are kinda infamous for this and other reasons, but in practice the fact we have a firmware chip means things CAN be improved! I have several JMicron adapters with this problem: - A bunch of SATA/PATA-to-USB combo JM20337 adapters - almost but ONE (a Vantec-branded kit) enforce 4K on AF drives, but only up to 2TiB. So far I haven't researched on a solution for those, but the fact the Vantec adapter always do 512b sectors is encouraging. - This cheapo Chinesium USB3-to-SATA dongle I managed to buy last year for $9 (I do not like adapters that bring their own proprietary PSU, or worse, 5V-only bus powered). This one is powered by a JMS576 USB3.1 to SATA bridge... or is it? (keep reading!). And of course it enforces 4K emulation on ANYTHING with AF! For this one, after some research, and taking a risk of bricking things, I found a solution~ JMicron chipsets store two things on those EEPROMs - in my case a "Puya" EEPROM, actual capacity may be 64K or 128K but only 49.5K are used on most JMS57x bridgechips: - Firmware code: 49K - "EEPROM" settings: 512 bytes <= this is the part we need to hack! First things first - let's dump our firmware+EEPROM! For this you can pick between the internal JMicron firmware updater for the 2033x series (which also supports the JMS57x series), or this independently made firmware dumper by that Soviet dude that made the tools that can identify your SSD chipset+NAND. Yes, I'm linking to a .ru website - you may be violating one embargo or ten, but who cares when Soviets have the key to all of our lost files? :D And you do need Windows for all this, sorry (I've seen dumpers and flashers for the JMS57x series under Linux, but they're DANGER MINES alpha quality at best so I refrained from even looking at them). I've used my last surviving W7 install for this, but anything later will work fine, and I suspect that even good ol' XP should do the trick too. Be careful - there are reports of certain USB3 chipset drivers *cough*ASMedia*cough* that can interfere with the process! Only use stock MS USB drivers, which means Windows 10 or a USB2 port for older Windows versions. Before starting, make sure to plug the converter to your PC with a drive attached to, otherwise the bridgechip will NOT enumerate to the OS! Any drive will do, and the flashing procedure won't touch your data at all... but if you don't feel safe, just use any ol' scrap drive you have there. To dump firmware using Vadim's JMicron tool: 1) Fire up an Administrator console. 2) cd into the directory where you have extracted the tool, then call it 3) Follow the instructions on screen: 4) Make another dump just to be safe, compare both - if they match and aren't a blob of zeroes or random garbage, you're set. To dump firmware using the JMicron's firmware updater tool: 1) Download the tool - any version should do but I ended using 1.19.17.6. 2) Extract it somewhere and run it - it requires admin privileges so let it elevate. 3) Tick the "RD Version" checkbox - this will enable some extra options on the flasher tool 4) Go tick the "Backup Old Firmware Only" checkbox (this will disable the "Firmware File" line on the dialog - THIS IS NORMAL. Instead, use the Backup File box to specify the path and filename for your backup file. 5) Hit "Run". This will dump your firmware, which will take less than one second. 6) To be safe, specifiy another filename and dump it again - compare both dumps - if they match and aren't a blob of zeroes or random garbage, you're set. You can now close the firmware updater. At this point you will have noticed two things: - Your dump will be exactly 50688 bytes, that is, a 49K firmware and your 512 byte EEPROM config. - The dump identifies itself as a JMicron JMS579 (!), and the JMicron flasher identifies the chip as a JMS577/8 (!!!). Yep, turns out that there are many, MANY mislabeled chips out there, which means my actual chip is _not_ a JMS576 but a 577/8/9 - this is good news since the magical flag for turning off this 4K madness IS documented for those chipsets. If your firmware dump is ~65K or any size but 50688 bytes, STOP - you do not have a 57x but most likely a 58x-series. Now that we know that this $9 Chinesium dongle is not what it pretends to be, the course of action is to figure out where is the magical bit we have to tamper with, and on the 577/8/9 some dude not only reversed that, but even made a neato webapp for generating premodded firmware. But we are not going to use that. Instead I'm going to use a known good firmware supplied by Odroid for the JMS578 that also supports TRIM and fixes some other minor annoyances - you can use whatever firmware you wish, even the one you dumped from your own enclosure/dongle/adapter, but the key here is that we are NOT using a JMS576, but you want 579-compatible firmware which is thankfully easy to find online if you know what to search for. If we read that firmware mod tool code, we will quickly learn where are some of the magical EEPROM flags we can toggle on those bridges. I'm gonna mention relative EEPROM offsets - the EEPROM starts at offset 0xC400 on your firmware dump! - Offset 0xE7: + Bit 7: set = enables USB Mass Storage (you want this!) + Bit 6: set = enables UASP (you also want this!) + Bit 5: set = enables 4K emulation (you do NOT want this!!!!!!) + Bits 4-0: unknown, leave them as-is (in my original firmware dump they were set to 00111) - Offset 0xF2: + Bit 3: set = enables drive sleep timeout (you may want to disable this) + All other bits: unknown, leave them as is (in my original firmware the bitmask was set to 0x78, others set it to 0x38) - Offset 0xF6: high byte of sleep timeout (in seconds) - Offset 0xF7: low byte of sleep timeout (in seconds) Example: 30 minutes = 1800 seconds = 0x0708 => 0xF6 = 0x07, 0xF7 = 0x08 By default those bridges set a 10-minute (600 second) sleep timeout, which many systems dislike. Disable it, or set it to your needs. You can also edit your USB VID/PIDs and vendor strings here, if you want to walk that extra customization mile. Use your favorite hexeditor to do your mods ON A COPY OF THE FIRMWARE YOU ARE GOING TO FLASH. Notice that there are NO CHECKSUMS enforced on the EEPROM, so be EXTREMELY CAREFUL and don't edit any other offsets, and preserve any undocumented bits already (un)set on the offsets you're editing! Save your mods, and now it's time to flash~! To flash your new firmware+EEPROM: 1) Again, ensure your converter is attached to both your PC and to a working disk drive. 2) Run the JMicron firmware updater. 3) Pick your modded firmware file. Do not touch any other checkbox. There is no need to wipe the flash before flashing, so don't bother enabling RD Version this time. 4) Hit "Run". The flash process will take ~2 seconds, so you won't have time to wet your undies. 5) If everything went well, the flasher will tell you to "powercycle" your drive. Just unplug and replug the converter to the USB port of your PC. If it gets detected, congratulations, you've successfully hacked your JMicron JMS57x EEPROM~! Now enjoy your 512-byte sectors: UAS: new and shiny: USB Mass Storage: old and busted, but the only thing that works everywhere: A minor quirk: the Odroid firmware I'm using here will ALWAYS report that your drive has 4K physical sectors, even for non-AF drives. This is a cosmetic flaw, nothing else. --- After some quick research, there is nothing that can be done regarding 4K emulation on JM20337, a 20 year old bridgechip still in production (and probably one of the last PATA controllers still available for sale brand new in 2024!): - There is a datasheet available - not that great but it DOES tell that the chip CAN accept an optional 9346 EEPROM (that is, 128 BYTES), whose intended use is to store USB descriptors (including device name strings), and the EEPROM layout is well detailed on the last page of the datasheet (all 128 bytes of it!). There are 18 reserved bytes, but nowhere near enough room for a full firmware image. - The JMicron flasher/manufacturing tools for all their USB-to-(P|S)ATA bridgechips ARE called "JM2033x $TOOLNAME", even those that actually don't support the JM20337. But it's all moot anyway, as the firmware update tool can't dump anything from those bridgechips, just 64KB blobs of 0xFFs (and some late versions don't even detect the JM20337 anymore!) - Vlad's tool can't do anything with those chips either - it will dump an empty 512 byte EEPROM and that's it. - What it's weird is that the 4K behavior DOES change on some adapters: I have 3 in total powered by the JM20337 - two generic "VCOM" adapters I bought over a decade ago (those enforce 4K emulation only with AF drives <= 2TiB), and a Vantec CB-ISATAU2 bought in 2014 with VERY BRIGHT BLUE/WHITE LEDs that do no emulation at all with any drive! So I suspect that this behavior may be baked into the silicon itself, and several variants of the JM20337 may exist. - The JM20337 had a rather bad reputation in its early days due to data corruption issues because of an oversight related to the implementation on the SATA Hotplug feature (removing one resistor would fix the problem but also break SATA hotplug). FWIW, my adapters were made way after that era, none of then ever experienced any kind of hardware corruption (at least nothing related to the adapter itself - dodgy SATA cables are another issue!), and SATA hotplug works fine. Either JMicron suggested another reference design to OEMs, or they patched the flaw on a later silicon revision, which would also explain the inconsistent 4K emulation on those things. Licensed Pirate® since 2006, 100% Buttcoin™-free, enemy of All Things JavaScript™ |
| tomman |
Posted on 24-12-27, 12:57
|
|
Dinosaur
Post: #1322 of 1326 Since: 10-30-18 Last post: 1 day Last view: 21 hours |
Some more findings: - The JM2033x series seems to be the common ancestor for JMicron SATA to USB bridgechips. The EEPROM/NVRAM format is surprisingly similar on later chips, although by then JMicron extended the NVRAM space from 128 to 512 bytes, and actually started using the reserved bytes for configuration. - Speaking of this NVRAM, at least on the JMS57x series the second half of the 512 bytes not only seems to be unused, it's the same weird garbage across every single firmware dump I've found online (also including the dump I made from my very own adapter), bit per bit. Either they are not using this on generic firmware releases, or it's something else - I'm tempted to blank that area and reflash but I should instead buy other JMS57x-based adapters before wearing down my only sample (the EEPROM is rated for 100k P/E cycles, but I'm not gonna test that as I only own ONE... and of the variety that it's unobtanium in this country!). - Of course if you're a device integrator that can get the relevant documentation from JMicron, you can roll out your own custom firmware - WD does exactly that with their USB HDD hackjobs using the JMS578 (like bridges being complicit on the on-the-fly encryption that renders your data unreadable the day you break that USB port). Those are 8051-based, not ARM. - TRIM works wonders once flashing those things with a suitable firmware - the Odroid's "Hardkernel" release is a popular choice for that. Best $9 ever :P - Customizing those USB device strings is easy - just remember to adjust the string length (it's 2 bytes before the actual string), and terminate them with zeroes. I totally approve of this message: Licensed Pirate® since 2006, 100% Buttcoin™-free, enemy of All Things JavaScript™ |
Pages: 1
[Your ad here? Why not!]