anyone working on decode the replay file format?

[chuanhsing]

Code Select Expand



00 22 80 00 00 1D 00
00 21 80 00 00 1D 00
00 22 80 00 00 32 00
00 21 80 00 00 32 00
00 23 80 00 00 1C 00
00 23 80 00 00 1E 01
00 23 80 00 00 25 00
00 23 80 00 00 27 01
00 23 80 00 00 2F 00
00 23 80 00 00 32 00
35 B8 03 83 00 F1 F0 81 01 00 20 01
99 E6 03 83 00 84 93 86 00 3C 1A 06
01 EE 03 83 00 B4 D3 81 00 45 E7 06
01 B5 04 83 01 11 9A 85 00 6C 10 06
01 B8 03 83 01 00 69 84 00 FE 7D 07
09 3F 03 83 00 8E 3A 80 01 27 54 03
3D 67 03 83 01 02 05 82 00 F3 0E 05
11 44 03 83 01 00 69 84 01 08 4B 00
   10 03 83 01 06 D8 83 00 FE 7D 07
41 3E 03 83 00 ED 1D 80 01 05 06 05
05 E3 03 83 00 C1 B0 86 01 00 20 01
19 51 03 83 00 AE 64 82 00 6D 1C 01
59 4E 03 83 01 28 9E 82 00 EF CA 03
29 47 03 83 00 99 7C 80 01 1F 29 04
15 04 03 83 00 AE 64 82 00 6B 79 07
31 71 03 83 00 69 3C 85 00 28 80 05
0D 51 02 00 02 67 67
   8C 03 00 02 67 67
   C0 04 00 02 67 67
09 BD 03 83 00 32 8E 84 00 97 94 06
   28 01 00 1E 68 6F 77 20 64 6F 20 79 6F 75 20 73 74 6F 70 20 6D 61 73 73 20 73 74 61 6C 6B 65 72 73 3F
05 18 03 00 12 6D 61 73 73 20 73 74 61 6C 6B 65 72 73 20 73 75 63 6B
   C8 04 00 15 69 64 6B 20 49 20 64 6F 6E 27 74 20 70 6C 61 79 20 74 6F 73 73
09 0D 01 00 18 79 6F 75 20 74 68 69 6E 6B 20 73 74 61 6C 6B 65 72 73 20 73 75 63 6B 3F
05 1A 03 00 07 74 68 65 79 20 64 6F
01 4B 03 02 14 75 6E 6C 65 73 73 20 75 20 6C 65 74 20 68 69 6D 20 67 65 74
01 7E 03 00 19 75 6E 6C 65 73 73 20 75 20 6C 65 74 20 68 69 6D 20 67 65 74 20 6C 69 6B 65
   30 03 00 03 69 64 6B
   18 01 00 2A 79 6F 75 27 64 20 70 72 65 66 65 72 20 6D 61 73 73 20 70 72 6F 64 75 63 69 6E 67 20 6D 61 72 69 6E 65 73 20 74 68 65 6E 20 3F
   10 03 00 02 36 30
01 8E 03 00 0C 69 74 20 77 6F 72 6B 65 64 20 3B 33
01 5A 01 00 03 6C 6F 6C
   E0 01 00 02 3A 29
01 66 03 83 00 44 3F 82 00 54 9B 03
01 CA 01 00 02 67 67
01 4C 03 00 02 67 67


[newbiz]

Okay.. So there's a new type of header after the 7-bytes-long ones ?
Will work on it.

[chuanhsing]

It's the function of Alt+g (blink on map).


* Unknown Function, 7 bytes for each, always at header
** 0x0, maybe Frame ?
** unknown
** 0x80, maybe OPcode ?
** 0x0
** 0x0
** unknown
** unknown
* Message Function
** Accumulate Frames, 1-N bytes, Big Endian and Frame/64=Second
** Sender, 1 byte
** OPcode, 1 byte, 0x0 to all, 0x2 to alliance
** Message Len, 1 byte
** Message Content, n bytes from Message Len
* Blink Function (alt+g on Map)
** Accumulate Frames, 1-N bytes
** Player, 1 byte
** OPCode, 1 byte, 0x83
** X, 4 bytes
** Y, 4 bytes

[chuanhsing]

Thank you very much ! These are very valuable information ^^
Did you figure it out yourself or are you in a parser project ? If not, you can join my effort @ http://projects.coderbasement.com/projects/show/sc2replay

I'm not in a parser project, just for fun.


I'm also writing these in Sourcepeek.com.

[newbiz]

That's very kind of you
Also, do you know if messages & blinks can be interleaved, or blinks are always before ? if so, it would become a real pain to parse.

[chuanhsing]

messages & blinks can be interleaved in any order.

[newbiz]

This format is sooo ambiguous to parse...

Code Select Expand

09 01 02 83 00 32 8E 84 00 97 94 06

How can you know if this is a blink or a message ?


Blink:

Code Select Expand


byte 0-1   : frames
byte 2     : player
byte 3     : opcode
byte 4-... : X & Y


Message:

Code Select Expand


byte 0     : frames
byte 1     : player
byte 2     : type (all)
byte 3     : length
byte 4-... : message

[chuanhsing]

Check if second byte smaller than 0x10 (max players) and bigger than 0x1 (min player). And check third byte equal to OPcode(0x0, 0x83).

[newbiz]

Dump of the unknown chunk inside replay.info
It seems to be organized in 2 parts, each containing <almost> fixed length structures.

Code Select Expand

// <- ... map cache name length ...
// <- ... map cache name ...

// Unknown chunk
6E D5 18 ED 00 00 50 F8 DF 07 BB 00 10 00 00 F0
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
D0 FB 3F C3 41 02 00 00 80
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

//
//                           0
//                           10
//             0     0  0 0  22
//            01     2 01 20 44
//      ?? ?? 12    03 83 41 88 00 00 ?? ?? ??

         47 00 80 02 81 00 40 00 00 FC 07 C0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

         7E 00 10 00 83 00 08 00 00 E0 1F 38
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

      F0 21 02 00 03 80 01 00 00 00 FF 07
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

      80 4F 00 40 00 83 00 20 00 00 F8 07 E0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

         7C 10 08 00 03 20 04 00 00 C0 3F 1C
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

      E0 43 01 00 03 80 00 80 00 00 FE 07 80
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

         5F 00 20 00 83 00 10 00 00 F0 0F 70
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

         F8 10 04 00 03 40 02 00 00 80 7F 0E
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

      C0 47 00 80 02 81 00 40 00 00 FC 07 C0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

         7E 00 10 00 83 00 08 00 00 E0 1F 38
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

      F0 21 02 00 03 80 01 00 00 00 FF 07
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

      80 4F 00 40 00 83 00 20 00 00 F8 07 E0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

         7C 10 08 00 03 20 04 00 00 C0 3F 1C
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

      E0 43 01 00 03 80 00 80 00 00 FE 07 80
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

1F 28

// ... "s2ma" structs x5

[jdelator]

Anyone still working on this?

[newbiz]

yes

[chriskang]

Hey guys.
I just entered the beta this week and start to put some interest in your work. I think I might be able to help.

Here's my first contribution:
After staring at this post for a few minutes, I noticed that every time the "timestamp" field is 2 bytes long, the first byte ends either with 1, 5, 9 or D.
Also, every time this field is 1 byte long, it ends either with 0 or 8.
This is an obvious indication that the field length is actually encoded in the 2 least significant bits of the first byte.

When you read this first byte, you should put a 0b00000011 (0x03) mask on it and you get your "timestamp" length (first byte not included). This should resolve the ambiguity you were talking about, newbiz.
And btw, as you noticed that the "time unit" inside the replay file was 1/64th of a second, it's possible that the 6 remaining bits of the first byte are actually sub-second unit whereas the other byte(s) are full seconds. Because 6 bits is exactly what you need to count from 0 to 63.

Hope this helps (and hope it's clear too cause I'm not a native speaker).
I'll try to see what I can find with the other files. Any progress with them?

[Noko]

chriskang I wish I'd seen your message before wasting such a shitload of time on figuring timestamps out.
Anyway, if someone is still interested, while I can't claim I figured out the meaning everything in new replay.details file, I at least can parse it. It is very straightforward if you take one thing in consideration: numbers (following 02 bytes, as lengths of strings, and also following 09) are encoded in utf-8 like manner and then shifted one bit right. here's how it goes:Number consists of variable number of bytes. If most significant bit is cleared, then it's the last byte making up a number. Other seven bytes are treated as part of number, most significant bytes coming first. Then the resulting number, made up of groups of 7 bits gets shifted by one bit to the right.

Code Select Expand


      byte sequence   resulting number (dec)
===================   ====================
                00     0
                01     0
                02     1
                03     1
                04     2
                7f     63
                80     <incorrect by itself>
             80 01     64
             80 02     128
             82 01     65
             84 01     66
             F2 14     1337 (actually encountered in every replay, once for
                             each player. Who invited skiddies to blizzard?!)
80 80 8D F1 B0 08     144000000000 (encountered as well, near the end of map.
                                     Number varies but billions are constant)


So, even though it looks like stuff is at different offsets every time, it's only because numbers have different length. Read them properly and everything is very parseable.