• Please review our updated Terms and Rules here

Help with creating binary-identical output from old source

RichCini

RichCini

Veteran Member
Joined
Aug 7, 2005
Messages
547
Location
Long Island, NY
All --

I'm reconstructing a piece of old Microsoft code called 20HAL (project is on my Web site) and I'm getting stuck on one point about displacement sizes. I'm trying to get the code to compile to a binary-exact copy, not just work-alike.


The original code is the following, with a 16-bit displacement:

Code: 
9663:04C0 88 AF 0001 mov ds:1[bx],ch ; (9663:0001=0)

MASM happily reassembles it, but only uses the 8-bit displacement:

Code: 
04C0 88 6F 01 mov ds:1[bx],ch

I can't come up with any combination of modifiers that ASM likes to force it to use a 16-bit displacement. If I enter the bytes into DEBUG, it disassembles as "[BX+0001]" but if I enter it in the assembler, it comes out with the byte displacement size. Right now I just have it coded with "db" so that locations line up.

There are other maddening things, like "jmp location" codes with an absolute address rather than the original negative displacement.


Any help with this would be greatly appreciated. Thanks!

Rich
 
Just spitballing here until someone more knowledgeable shows up…have you tried a different version of masm?
 
It doesn't really matter--AFAIK, there's no way to force MASM to use a specified instruction encoding. Often, there are several ways to encode, say, a MOV; MASM picks one and you cannot override it.

I suppose that's what macros are for. :)
 
If all else fails, you can you can just "hand assemble" that section as literal hex codes with a comment as to what the source code is. Since you're looking for an identical binary, you don't have to worry about 'ch' (in this case) "moving" or being in the wrong place, so literal hex codes should work fine.
 
whartung said:
If all else fails, you can you can just "hand assemble" that section as literal hex codes with a comment as to what the source code is. Since you're looking for an identical binary, you don't have to worry about 'ch' (in this case) "moving" or being in the wrong place, so literal hex codes should work fine.
Click to expand...

I'd still use a macro, since that can convey the intent and meaning of the instruction. As most programmers know, human-to-machine communication is important, but human-to-human communication runs a very close second. One of the gripes I have with a lot of Linux code, as in, "No comments for 50 lines--what the heck is going on here?"
 
Al -- no, I haven't spoke to Rich Alderson about this. If you have his email, maybe PM it to me if you could.

Chuck -- Macros are a good idea. I will also go back in time a bit further and try the Seattle Computer Products ASM and see if it assembles differently. From an historical perspective, not sure if MASM 1/IBM Assembler would have been used for this tool or not.


{later...}
My guess is that this was assembled with the SCP ASM program from the 86-DOS disk. There are some syntactical differences, and with making the syntactical changes, it compiled cleanly. Still need to hand-compare it, though. From a developer's standpoint, kind of makes sense.

Rich
 
Last edited:
Chuck(G) said:
I'd still use a macro.
Click to expand...

I mean, sure, that's fine. Guess it depends on how often this happens, how much DRY (Don't Repeat Yourself) you're trying to achieve. I don't know if it's any more abundantly clear than:

(Note I'm not particularly well versed in MASM syntax)

Code: 
;; Hand assembling to force 16 bit offset: mov ds:1[bx],ch
    .DB 0x88, 0x6F, 0x01

Unless you plan on writing sophisticated enough macros that in the end are actually mini-assemblers in their own right.
 
whartung said:
I mean, sure, that's fine. Guess it depends on how often this happens, how much DRY (Don't Repeat Yourself) you're trying to achieve. I don't know if it's any more abundantly clear than:

(Note I'm not particularly well versed in MASM syntax)

Code: 
;; Hand assembling to force 16 bit offset: mov ds:1[bx],ch
.DB 0x88, 0x6F, 0x01

Unless you plan on writing sophisticated enough macros that in the end are actually mini-assemblers in their own right.
Click to expand...

Assuming that the OP:
  • wants an exact binary copy
  • not using the original assembler
  • doesn't plan on adding any new/different functionality
AND there multiple ways to assemble the same source depending on the assembler, then I would apply whartung's approach to the entire file. It would resemble the output of an assembly pass:

Code: 
.DB 0x88, 0x6F, 0x01      ; mov ds:1[bx],ch

Because: https://youtu.be/nnHmUk_J6xQ?t=32

When I was making some little changes to the CoCo Color Forth to inline the NEXT operation into all the Forth words, I expected the binary size to explode. I used a modern-ish 6809 assembler instead of the simplistic assembler shipped with the CoCo it actually caused the binary to shrink by a great deal. I thought something was wrong but it turned out the newer assembler was just better at it's job.
 
RichCini said:
My guess is that this was assembled with the SCP ASM program from the 86-DOS disk. There are some syntactical differences, and with making the syntactical changes, it compiled cleanly. Still need to hand-compare it, though. From a developer's standpoint, kind of makes sense.
Click to expand...

The SCP assembler is a great little assembler. If you want to force it to generate a 16-bit displacement even though the value is in the range -128..+127, you can "trick" it by using a not-yet-defined symbol -- all code generation (including the selection of instruction encoding) is done during the first pass.

Code: 
  MOV [BX+ONE],CH
  ; ...and then later, give a value to ONE:
ONE EQU 1

Also with jumps, JMP will always generate the 2-byte displacement while JMPS always generates the 1-byte displacement (or produces an error if the destination label is out of range).
 
Ok, I'm done refactoring the source code to work with the SCP version of ASM and I've produced an output file that matches the original. There is a lot of "tweaking" that was needed in making sure that the SEG overrides were right and getting the word size modifiers ("B" or "W"). From a historical perspective, I should have thought about the SCP assembler first, but oh well.

Thanks everyone for the nudging in the right direction. I've posted the source file and listing on my site at the link above for those interested. The "forward equate" trick actually worked.

There are two more encoding anomalies: one with "xchg dx,bx" and one with "cmp cl,0". The xchg can be fixed by reversing it to "xchg bx,dx" (seemingly the SCP ASM does it differently than INTEL, although the notes in the SCP manual don't indicate a difference). The compare seems to assemble using an alias. So, the original binary has "80h, F9h, 00" while ASM compiles it as "82h, F9h, 00". Not entirely sure why just yet.

Al -- feel free to forward whatever on to RichA because he could be a great help on the TOPS-20 side.

Rich
 
Last edited:
Today I decided to try the Intel Assembler ASM-86 (v.3.2) to see how it coded the "CMP CL,0" statement. ASM-86 indeed coded it properly. So, now I'm wondering if ASM-86 may have been used rather than the SCP ASM. Hmmm...

Rich
 
The CMP CL,0 is a byte instruction.

The difference between the 80h and 82h instruction codes is the state of the ‘s’ bit. This is irrelevant for a byte instruction - so it is a don’t care bit (unfortunately). If the instruction was a WORD (say CMP CX,0) then the immediate operand could be a 16-bit quantity (s=0) or an 8-bit quantity (s=1) with sign extension.

Your XCHG instruction is (also) undefined as to the ‘source’ and ‘destination’ operands, as it is an exchange and, therefore, both operands are both sources and destinations. Unfortunately, the implementation is up to the assembler writer.

Your “mov ds:1[bx],ch” is an interesting one. In the original source, I would have expected the ‘1’ to be something like a STRUCT offset. I further suspect that the STRUCT declaration came after the code segment. The assembler wouldn’t have known what the ‘size’ of the operand would be until pass 2. As a result, it should have allocated space to accommodate a 16-bit offset. However, on pass 2, it could still be a smart assembler and code up a byte offset (rather than a word offset) and pad the unused byte (which will now be at the end of the instruction) with a NOP instruction. The result still being no binary equivalence.

EDIT: The ds: prefix is thrown away (as it is the default). MOD is 01 in the case of a 16-bit unsigned offset and 10 in the case of a sign-extended 8-bit offset.

We ran into the same problem when moving code from a cross-assembler on a PDP-11/RSX to one on a PC/DOS. The same source generated different binary.

Dave
 
Last edited:
Thanks Dave. Last night I finished plowing through the source and cleaning it up for compiling with ASM86. The remaining (well, new) issues in comparison to the SCP assembler can be bucketed into the following:

* 15 JMPs. ASM86 assembles 2-byte JMPs (EB) rather than 3-byte (E9) for near.
* Six "mov bx,offset label" which codes with a NOP (so bb, LSB, MSB, 90). Often this is used to grab the address of an error string, or a subroutine location.
* The single mov ds:1[bx]

I guess I'm discovering how much I don't know about the intricacies of the x86 instruction set and tweaking assembler switches.

Rich
 
Most assemblers will be designed to 'do a job' rather than be 100% compatible with someone else's assembler (unless that is the purpose of the assembler of course).

If a jump is known and within -128+127 - the assembler should generate the 2-byte variant. If unknown, space for the 3-byte variant should be allocated on pass 1 - and then it is up to the assembler designer as to use the full 3 byte jump instruction or to reduce it to 2 and NOP pad if the target is within range. There will be no space improvement in this case, but there may be a speed improvement/penalty one way or the other. Some assemblers substitute a 3rd pass (or further multiple passes) in this case to optimise forward references better to reduce the code size.

Some assemblers just go for the 'quick and easy' solution. If a forward reference is found - assume the worst case and use that...

When I used to write assemblers/compilers for a job - it depended upon what the target was. I sometimes started off with the 'quick and easy' and then improved the product. Other times we wanted to 'wring the max' out of the runtime and went to extreme lengths to optimise everything at every stage.

Ruud: Your solution is only one of many though... You can lose the 3E for a start - as DS: is the default segment register for use with that addressing mode. It is only effective as a comment for the human reader. But, there again, 'quick and easy' would possibly include it?

Dave
 
Ruud said:
3E88AF0100 mov [word ds:bx+1h],ch
Click to expand...

Thanks Ruud. I think the problem is how Sourcer decompiled that original instruction. Based on the original object code, Sourcer produces the following, which doesn't have the DS segment override even though it shows in the instructions. It refers to location 1, which is in the DOS PSP. Here's the snippet. CH should probably be 0 when it gets here because it's used in a loop:

Code: 
9663:04BC loc_44:
9663:04BC B1 00 mov cl,0
9663:04BE 88 2F mov [bx],ch
9663:04C0 88 AF 0001 mov ds:data_1e[bx],ch ; (9663:0001=0)


If I use your suggestion, ASM86 barfs with a syntax error, but DEBUG gets it right; removing the DS override also produces an error. Changing it to "mov word [BX+1h],ch" produces 886F03 which is the bit8 operand, not the bit16 version.

Thanks
Rich
 
I've often thought that there is scope for an x86 assembler that would allow some sort of 'instruction modifier' to force it to select a particular instruction encoding (or produce an error if the operands weren't appropriate). Something akin to:

Code: 
  ADD AL,55h        ; generates the "accumulator immediate" form (04 55)
  ADD/80 AL,55h     ; generates the "generic" form (80 C0 55)
  ADD/82 AL,55h     ; generates the "useless" sign-extended immediate form (82 C0 55)
 
kdr said:
I've often thought that there is scope for an x86 assembler that would allow some sort of 'instruction modifier' to force it to select a particular instruction encoding (or produce an error if the operands weren't appropriate). Something akin to:

Code: 
ADD AL,55h ; generates the "accumulator immediate" form (04 55)
ADD/80 AL,55h ; generates the "generic" form (80 C0 55)
ADD/82 AL,55h ; generates the "useless" sign-extended immediate form (82 C0 55)
Click to expand...

From a reconstruction perspective, this would be very helpful for sure. What's interesting is that the Seattle Computer ASM correctly encodes the offending instruction (if tricked into it using a long forward reference to a #define), yet when using the same trick with ASM86 (which is from the Intel APX-86 Platform kit), it produces something different (the #define ONE is defined at the END of the file before the ENDS/END directives).

Code: 
04B9 886F0190 mov [bx+ONE],ch

When using the SCP ASM, there is a similar anomaly with a single "cmp cl,0". The original is coded 80F900 yet the SCP codes it as 82F900. You have to use the same forward reference trick to get this to work too. I can't imagine the code was really done this way, though, but at least it works.

Since I don't really know what assembler was originally used, I'm really just guessing based on what tools I knew were available at the time (1980-1981).

Rich
 
Last edited:
RichCini said:
Since I don't really know what assembler was originally used, I'm really just guessing based on what tools I knew were available at the time (1980-1981).
Click to expand...

If they used e.g. the Microsoft toolchain (which is based around assembling source code modules into relocatable object files followed by a link stage) then you'd get a similar effect as using a forward reference in the SCP assembler; there's no way MASM could know the size of the offset at assembly time if the symbol comes from a different module. So space would be made for a 2-byte offset, and the linker would simply fill in the blanks with the symbol's resolved address. Link-time optimization passes were surely not a thing in 1980s MS-DOS toolchains. :)
 
You must log in or register to reply here.
Back
Top