0 users browsing Discussion. | 12 bots  
    Main » Discussion » Looking for a python script in the now defunct archives
    Pages: First Previous 1 2
    Posted on 21-03-31, 22:11 (revision 1)
    Post: #24 of 27
    Since: 08-01-20

    Last post: 1327 days
    Last view: 1226 days
    Got another python question.

    Code:
    import sys, glob, os, struct

    #feed original filename, detect named subfiles, merge
    def mergeIMF(filename):
    with open(filename, 'wb') as file:
    for i in range(1,11,1):
    subfilename = filename[0:8] + "_subfile_" + str(i) + filename[8:]
    print(subfilename, end="\t")
    if os.path.exists(subfilename):
    with open(subfilename, 'rb') as subfile:
    #get subfile size, do bitshift shit, little endian... or encode somehow? google it.
    #write to file in correct order
    #write bytes(size of subfile) of this subfile to file
    #close subfile
    subfilesize = os.path.getsize(subfilename)
    print(subfilesize, end="\t")
    subfilesizebyte = struct.pack('<I', subfilesize)
    print(subfilesizebyte)
    subfile.close()
    else:
    pass
    file.close()

    for arg in sys.argv[1:]:
    for files in glob.glob(arg):
    mergeIMF(files)


    Output:

    python .\imf_merge.py .\IMF00A.IMG
    .\IMF00A_subfile_1.IMG 80 b'P\x00\x00\x00'
    .\IMF00A_subfile_2.IMG 64 b'@\x00\x00\x00'
    .\IMF00A_subfile_3.IMG 124 b'|\x00\x00\x00'
    .\IMF00A_subfile_4.IMG 16 b'\x10\x00\x00\x00'
    .\IMF00A_subfile_5.IMG 1360 b'P\x05\x00\x00'
    .\IMF00A_subfile_6.IMG 2084 b'$\x08\x00\x00'
    .\IMF00A_subfile_7.IMG 78812 b'\xdc3\x01\x00'
    .\IMF00A_subfile_8.IMG 1660 b'|\x06\x00\x00'
    .\IMF00A_subfile_9.IMG 8156 b'\xdc\x1f\x00\x00'
    .\IMF00A_subfile_10.IMG 4328 b'\xe8\x10\x00\x00'


    Files (not actually needed, you can just rename any 10 files as IMF00A_subfile_?.img, where? is a number between 1 and 10, both inclusive):
    https://ufile.io/lkcr8f2y

    Goal of this script right now is to print the file size of each subfile as a little endian byte string with a word size of 4 bytes. As you can see from the output, in all but 3 cases it's printing I think an ascii character for the first (least significant) byte instead of a byte string.

    I'm trying to figure out how to use struct.pack.
    Posted on 21-03-31, 22:35 (revision 1)
    Post: #188 of 202
    Since: 11-01-18

    Last post: 660 days
    Last view: 15 days
    its not gonna escape printable characters.You might want to poke at string formatting.

    to convert between the two you are just reversing the string byte by byte at the end of the day.

    Very few processors use big-endian, so there actually isn't any need to fumble with the byte order, unless you need to muck about with low-level networking.
    Posted on 21-03-31, 23:00 (revision 3)
    Post: #25 of 27
    Since: 08-01-20

    Last post: 1327 days
    Last view: 1226 days
    Posted by funkyass
    its not gonna escape printable characters.You might want to poke at string formatting.

    to convert between the two you are just reversing the string byte by byte at the end of the day.

    Very few processors use big-endian, so there actually isn't any need to fumble with the byte order, unless you need to muck about with low-level networking.

    MIMF/IMF*.IMG are the event scripts, background palettes, and sprite palettes for the corresponding "room" in Saga Frontier. It's a PS1 game - and *.IMG files always have a 10 entry header, with 4 little endian bytes per subfile entry denoting where they start in the file - so it must be little endian.

    The modder I'm working for needs the ability to resize each subfile to add/remove events for each room, and I figured he needs 3 scripts: one to print info about each IMF to tell him how much room he has left in it's sectors (by taking the IMF file size modulo 2048 sector size), a script to split each IMF into its subfiles, and a script to merge a split IMF's subfiles back into a legal IMF file. The first two are done, I'm working on the third now.

    Regardless of what it should be doing regarding escaping printable characters, I looked up the least significant bytes of the file sizes (which are the decimal numbers printed before the byte string) in an ASCII table and that's what it's doing for some reason. The question is why, and why only some of them.

    How could I debug this behavior? Or should I quit trying to learn struct.pack for now and go back to bitwise logical operations and shifts?
    Posted on 21-04-01, 06:28
    Post: #189 of 202
    Since: 11-01-18

    Last post: 660 days
    Last view: 15 days
    you aren't printing a string representation of bytes, you are literally sending the bytes to stdout. stdout isn't equipped to show raw bytes. If you want the hex values for the bytes, you should look in python's string formatting.



    Posted on 21-04-02, 03:38
    Full mod

    Post: #431 of 443
    Since: 10-30-18

    Last post: 1101 days
    Last view: 172 days
    Goal of this script right now is to print the file size of each subfile as a little endian byte string with a word size of 4 bytes. As you can see from the output, in all but 3 cases it's printing I think an ascii character for the first (least significant) byte instead of a byte string.


    You are doing it exactly right.

    In your script, the "subfilesizebyte" is a byte string of length 4, representing the file-size as a 32-bit little-endian integer.

    However, most of the bytes in that byte-string do not map to human-readable characters in ASCII, so if you just "print(subfilesizebyte)" then Python will encode the byte-string in a human readable way.

    Looking at subfile 5 in your example, the length is 1360 in decimal, or 0x00000550 as a 32-bit hexadecimal number. If we convert that to little-endian bytes, we get: 0x50, 0x05, 0x00, 0x00.

    Python's byte-string syntax is designed to express bytes of any value, even ones that don't map to printable characters in ASCII. For example, you generally can't type a 0x00 byte on a keyboard, and even if you could it doesn't display as anything, so Python lets you encode that as the four bytes '\x00'. Python interprets that sequence as a single 0x00 byte, but it can safely be written in source-code and printed out without confusion.

    So, in Python's byte-string syntax, we could write those four bytes as b'\x50\x05\x00\x00'. But since 0x50 is the ASCII code for 'P' we don't have to hex-encode the first byte to make it printable, we can just write b'P\x05\x00\x00' which is what your program outputs.

    The question is - what did you *expect* to see?

    The ending of the words is ALMSIVI.
    Posted on 21-04-03, 21:20
    Post: #26 of 27
    Since: 08-01-20

    Last post: 1327 days
    Last view: 1226 days
    Posted by funkyass
    you aren't printing a string representation of bytes, you are literally sending the bytes to stdout. stdout isn't equipped to show raw bytes. If you want the hex values for the bytes, you should look in python's string formatting.

    Posted by Screwtape
    Goal of this script right now is to print the file size of each subfile as a little endian byte string with a word size of 4 bytes. As you can see from the output, in all but 3 cases it's printing I think an ascii character for the first (least significant) byte instead of a byte string.


    You are doing it exactly right.

    In your script, the "subfilesizebyte" is a byte string of length 4, representing the file-size as a 32-bit little-endian integer.

    However, most of the bytes in that byte-string do not map to human-readable characters in ASCII, so if you just "print(subfilesizebyte)" then Python will encode the byte-string in a human readable way.

    Looking at subfile 5 in your example, the length is 1360 in decimal, or 0x00000550 as a 32-bit hexadecimal number. If we convert that to little-endian bytes, we get: 0x50, 0x05, 0x00, 0x00.

    Python's byte-string syntax is designed to express bytes of any value, even ones that don't map to printable characters in ASCII. For example, you generally can't type a 0x00 byte on a keyboard, and even if you could it doesn't display as anything, so Python lets you encode that as the four bytes '\x00'. Python interprets that sequence as a single 0x00 byte, but it can safely be written in source-code and printed out without confusion.

    So, in Python's byte-string syntax, we could write those four bytes as b'\x50\x05\x00\x00'. But since 0x50 is the ASCII code for 'P' we don't have to hex-encode the first byte to make it printable, we can just write b'P\x05\x00\x00' which is what your program outputs.

    The question is - what did you *expect* to see?

    That's an awful lot of text to tell someone to use *.hex()
    Posted on 21-04-04, 00:18
    Full mod

    Post: #432 of 443
    Since: 10-30-18

    Last post: 1101 days
    Last view: 172 days
    I'm glad to hear you found a way to display integers that conforms to your unstated assumptions.

    The ending of the words is ALMSIVI.
    Posted on 21-04-04, 01:57
    Post: #27 of 27
    Since: 08-01-20

    Last post: 1327 days
    Last view: 1226 days
    Posted by Screwtape
    I'm glad to hear you found a way to display integers that conforms to your unstated assumptions.

    Is it worse than assuming I don't have a wife, family, and full time job; and can indulge an intellectual game of capture the flag more commonly known as "just do the research maaaaaaaaaaaaaaahhhhhhhhhnnnnnnnnnn *massive bong rip*"?

    I'm already extremely burned out as it is, and I'm romhacking as a favor for a friend. Would it kill you to not take the most roundabout way possible of explaining something, just once? Assume IDGAF about intellectual curiosity, and I just want to finish this problem, shit the script out of my brain via my fingers, and go back to something more interesting.
    Posted on 21-04-04, 05:34
    Full mod

    Post: #433 of 443
    Since: 10-30-18

    Last post: 1101 days
    Last view: 172 days
    Sure, I get it, you're busy and you've got other stuff on your todo list, you don't want to study the history of binary decimal conversion systems to get the result you want. But you didn't actually ask for the thing you wanted. Instead, you asked a bunch of other questions ("I'm trying to figure out how to use struct.pack", "The question is why, and why only some of them", "How could I debug this behavior?") and then got mad when people tried to answer them.

    I'm sure you feel like the thing you wanted was perfectly obvious and any sensible person would have recognised it immediately, but evidently it wasn't as obvious as you thought.

    If your first post had been "I have a Python script that prints b'P\x05\x00\x00' but I want 50050000, wat do?" you probably would have saved everybody a lot of time, including yourself.

    The ending of the words is ALMSIVI.
    Posted on 21-04-05, 03:31
    Dinosaur

    Post: #930 of 1315
    Since: 10-30-18

    Last post: 58 days
    Last view: 14 hours
    Posted by MysticLord
    Posted by Screwtape
    I'm glad to hear you found a way to display integers that conforms to your unstated assumptions.

    Is it worse than assuming I don't have a wife, family, and full time job; and can indulge an intellectual game of capture the flag more commonly known as "just do the research maaaaaaaaaaaaaaahhhhhhhhhnnnnnnnnnn *massive bong rip*"?

    I'm already extremely burned out as it is, and I'm romhacking as a favor for a friend. Would it kill you to not take the most roundabout way possible of explaining something, just once? Assume IDGAF about intellectual curiosity, and I just want to finish this problem, shit the script out of my brain via my fingers, and go back to something more interesting.


    Dude, please don't be a JWZ.

    If you want a solution, you need to fully understand the problem, and for that you need to RESEARCH, want it or not.

    I've been there, and the answer is not to be an asshole.

    Licensed Pirate® since 2006, 100% Buttcoin™-free, enemy of All Things JavaScript™
    Pages: First Previous 1 2
      Main » Discussion » Looking for a python script in the now defunct archives
      This does not actually go there and I regret nothing.