PDA

View Full Version : Ethernet tools . . .



wiwa64
December 18th, 2009, 02:49 AM
Hello everybody!

After some time of absence i'm here again with some new programs i wrote.

As result of some experiments i made with packet drivers, i wrote four little utilities which you might like :) (or perhaps not? :(). But look yourself:


ETHSEND
Is a program that sends a single raw ethernet packet out to the LAN. As such, it is not terribly usefull for everday (network-) life, but for experimental and testing purposes it can be quite handy. It helped me a lot in writing the other three programs.


ETHDUMP
Is just another packet sniffer. This isn't terribly innovative as well, but for me it was a pre-requisite for writing the other two programs. As it therefore had to be done anyway, why not give it a decent user-interface and turn it into something usefull?

It basicly duplicates the functionality of the program ETHCAPT, written by Yusuf Motiwala, many years ago. However it allows to display the data already while recording, with two levels of verbosity (headers only or full data as raw hex dump). In addition it has some filtering capabilities which its predecessor hasn't got. It can optionally write the captured data to a binary file in the same format as ETHCAPT would do, so it can later be visualized by ETHVIEW (also by Yusuf).


ETHWHO
Is a program that tries to generate a who-is-who of your ethernet LAN (with bus-topology). It lists in tabular form the source and destination ethernet adresses of the packets passing by and counts who is sending whom how many packets. The content of this table is optionally exported into a csv-file (character separated value) from where it can be further processed in what ever way, e.g. be imported into an excel table.


ETHWHAT
Is a program that watches your (bus topology) ethernet and tries to figure out what game they are playing. It analyzes the packets passing by, according to frame-type and protocol used (IP, IPX, ...) and displays statistics about it. In addition, if it finds IP-traffic, it tries to guess the basic configuration parameters from the found data.


All programs require a packet driver but no further configuration. They behave purely passive and do not appear themselves in the LAN (except ETHSEND of course). And sorry, there is no further documentation yet, but each program has a brief help-function, which can be accessed by the /? command-line option ( -h will do as well).

Everybody may feel free to download the attached archive ETHTOOLS.ZIP and try the included programs. And of course i would appreciate many comments. :rolleyes:

Raven
December 19th, 2009, 06:30 PM
These sound nice, especially ETHWHAT. Into the archives of programs and such they go..

mbbrutman
December 28th, 2009, 09:42 AM
Wiwa64 -

Sorry about the delay in responding. I've been in telnet hell recently. :-)

I've tried the Ethtools under DOSBox and they all work well. I will be moving them to my older machines later today.

I have a 'diagnostic' type utility on my to-do list as well - the ability to capture and trace raw Ethernet traffic is really important for debugging. For my TCP/IP development I used TCPDUMP on Linux because it does all of the packet parsing already. For debugging my applications on DOS I have a trace mode in all of the applications that can dump and interpret the packets my code is interested in, but nothing that serves as a packet sniffer.

Are you still coding using Watcom? I would like to see your approach to incoming and outgoing buffer management - Are you doing to post a copy of the source code? I posted my low level packet handling code for Turbo C++ here a while ago: http://www.brutman.com/mTCP/mTCP_tcpacket.html

Good work, and don't be such a stranger. If you want to compare notes on implementing higher level protocols like TCP let me know - mbbrutman@yahoo.com . There are not too many people venturing into this forest, and it is nice to have company.

wiwa64
December 28th, 2009, 11:35 PM
Wiwa64 -
Are you still coding using Watcom? I would like to see your approach to incoming and outgoing buffer management - Are you doing to post a copy of the source code? I posted my low level packet handling code for Turbo C++ here a while ago: http://www.brutman.com/mTCP/mTCP_tcpacket.html


Actually i have never used Watcom. It's all BorlandC (3.1 in my case) and it works quite well, so far. I don't have general reservations against publishing the source code as well, but there is (was?) a limit in file sizes, so i had to make a choice.

As far as the buffer management is concerned, this is essentially how i did it:


#define MAXBUFS 8 /* maximum number of Ethernet buffers */
#define BPMASK 7
#define BUFSIZE 1520

typedef unsigned short word; /* 16 bits */
typedef unsigned char byte; /* 8 bits */

typedef struct {
word pkt_len;
byte pkt_data[BUFSIZE];
} pkt_buffer;

word pkt_get = 0;
word pkt_put = 0;
pkt_buffer pb[MAXBUFS];

/*----- put packet into buffer -----------------------*/

// compiled with Borland C++ 3.1
// with "Compile via assembler" option
// in Options | Compiler | Code generation ...

// this routine is called from the packet driver
void far pkt_callback(void) {
asm {
pop di; // compensate for compiler generated PUSH DI / POP DI
push ds; // save driver's data segment
mov di,DGROUP; // NOT in huge model !!!
mov ds,di; // set C's data segment
}
disable();
if(_AX) {
pb[pkt_put & BPMASK].pkt_len |= 0X8000;
pkt_put++;
}
else {
if(pb[pkt_put & BPMASK].pkt_len || _CX >= BUFSIZE) {
_ES = 0;
_DI = 0;
}
else {
pb[pkt_put & BPMASK].pkt_len = _CX;
_ES = FP_SEG(pb[pkt_put & BPMASK].pkt_data);
_DI = FP_OFF(pb[pkt_put & BPMASK].pkt_data);
}
}
enable();
asm {
pop ds; // restore driver's data segment
push di; // compensate for compiler generated PUSH DI / POP DI
}
}

/*----- get packet from buffer -----------------------*/

do {
if(pb[pkt_get & BPMASK].pkt_len&0x8000) {
process_packet(&pb[pkt_get & BPMASK]);
pb[pkt_get & BPMASK].pkt_len = 0;
pkt_get++;
if(pkt_get >= pkt_put) {
pkt_get &= BPMASK;
pkt_put &= BPMASK;
}
}
} while( . . . );
As you can see, it's mostly "C"-code with only very little in-line-assembler included. It took me some days to find out, how to do that, (and during theese days may cat had to listen to a lot of swearing ;)) but since i found out how to do it, it's no problem anymore. The actual code varies a bit from one program to the other, in ETHDUMP for example, i added some more instructions to precisely detect the position of lost packets in the data stream, while in the other programs i was only interested in their total number.

Sooner or later i will publish an updated version. In the meantime i made a number of minor improvements to most of the grograms and i wrote an additional one to browse the capture file(s) generated by ETHDUMP (or ETHCAPT).

mbbrutman
December 29th, 2009, 03:33 PM
I take it that the cursing occurred while you were trying to figure out exactly what regs to push and pop to make the linkage work with the C code? That is where I had my trouble.

I'm curious about your procedure though - it is conceptually simpler to use the 'interrupt' keyword so that all regs are saved and restored by the compiler. I don't know the ins and outs of Turbo C++ 3.0 (my compiler), so figuring out the linkage was a nightmare. Your linkage is different - how did you come up with all of that?

Mike

wiwa64
December 30th, 2009, 05:33 AM
I take it that the cursing occurred while you were trying to figure out exactly what regs to push and pop to make the linkage work with the C code?That's correct! ;)


... it is conceptually simpler to use the 'interrupt' keyword so that all regs are saved and restored by the compiler. ...This is not a clever idea!

The interrupt keyword is intended for subroutines that are either invoked by a true hardware interrupt or by the machine instruction int nn (opcode 0xCD 0xnn). On entry such a routine expects three words on the stack: the processor status word, the current code segment register and the instruction pointer, in that order. Such a routine will be left by an IRET (interrupt return, opcode 0xCF) instruction, which pulls theese three words off the stack and restores them into the respective registers.

An ordinary subroutine, in contrast, is called by a near or far CALL instruction (various opcodes) and left by a near or far RET (return from subroutine, also various opcodes) instruction. These instructions only push and pull one (or two for the far variants) words onto (and off) the stack.

The packet driver calls his receiver routine through a far CALL to the address you passed it in registers ES:DI of the access_type() function and hence pushes two words onto the stack. Declaring such a routine as "INTERRUPT" to the compiler, will cause it to use the IRET instruction to leave the routine and hence pull three words off the stack. Trying it this way, you may almost certainly expect trouble and the misalignment of the stack will probably be the least one of it.


... how did you come up with all of that?The BorlandC compiler has a feature which causes it to produce assembler code (upon request). This way one can easily inspect how the complier translates the various C-instructions into mashine language. Of course it is helpfull to understand assembler code to do this, at least to some extend. Other compilers may or may not have a similar feature.

mbbrutman
December 30th, 2009, 06:30 AM
That's correct! ;)

This is not a clever idea!

The interrupt keyword is intended for subroutines that are either invoked by a true hardware interrupt or by the machine instruction int nn (opcode 0xCD 0xnn). On entry such a routine expects three words on the stack: the processor status word, the current code segment register and the instruction pointer, in that order. Such a routine will be left by an IRET (interrupt return, opcode 0xCF) instruction, which pulls theese three words off the stack and restores them into the respective registers.

An ordinary subroutine, in contrast, is called by a near or far CALL instruction (various opcodes) and left by a near or far RET (return from subroutine, also various opcodes) instruction. These instructions only push and pull one (or two for the far variants) words onto (and off) the stack.

The packet driver calls his receiver routine through a far CALL to the address you passed it in registers ES:DI of the access_type() function and hence pushes two words onto the stack. Declaring such a routine as "INTERRUPT" to the compiler, will cause it to use the IRET instruction to leave the routine and hence pull three words off the stack. Trying it this way, you may almost certainly expect trouble and the misalignment of the stack will probably be the least one of it.

The BorlandC compiler has a feature which causes it to produce assembler code (upon request). This way one can easily inspect how the complier translates the various C-instructions into mashine language. Of course it is helpfull to understand assembler code to do this, at least to some extend. Other compilers may or may not have a similar feature.

My code actually works very well - it runs for over a week at a time under a decent load under a variety of packet drivers, so I am sure that I am not having stack alignment issues.

I used the interrupt keyword because it has the advantage of forcing the compiler to push all of the registers onto the stack for me, and restore them on exit. The packet driver is an unknown environment to me, so rather than try to guess which registers were safe to use I just save and restore them all. This isolates me from implementation differences in the packet drivers. And I didn't have to figure out how to get addressability to my data segment - the compiler did that for me.

The only problem I had with linkage was specific to a particular packet driver - the Xircom PE3-10BT. That particular packet driver was poorly implemented, and had problems with the iret. Most packet drivers can figure out if you used a retf or an iret, and adjust accordingly because the packet specification is not clear on how the call should be made. (Most programmers use the interrupt keyword when getting a callback from a packet driver.) Russell Nelson (from Crynwr, the originator of the packet specification) helped me get through the specific problem with the Xircom packet driver - the solution was to continue use the interrupt keyword, but to write my own epilog code that does everything except the iret.

Before talking to Russell my code worked perfectly with the Davicom NE2000 clone, but not the Xircom .. this was my clue that there were implementation differences in the packet drivers that were beyond my control. My usage of the interrupt keyword is correct and isolates me from any possible difference, including inadvertently stepping on registers that might be in use.

wiwa64
December 30th, 2009, 02:52 PM
Well i just had a short look on your code, actually just at your reciever routine and of course you wont have stack problems, despite of using the INTERRUPT keyword. Simply because you circumvented the problem with the IRET instruction by using your own epilogue. Effectively you declared the routine as interrupt to the compiler and behind its back turned it into an ordinary subroutine. Ok, that's a trick. I used a different trick and i consider my solution a bit cleaner, but that may be a matter of taste.

Actually it isn't necessary to save all registers on the stack. Have a look on what the compiler made of the above C-code:
_pkt_callback proc far
push bp
mov bp,sp
push di
pop di
push ds
mov di,DGROUP
mov ds,di
cli
or ax,ax
je short @1@198
or word ptr DGROUP:_pkt_len,-32768
jmp short @1@310
@1@198:
cmp word ptr DGROUP:_pkt_len,0
jne short @1@254
cmp cx,2100
jb short @1@282
@1@254:
xor ax,ax
mov es,ax
xor di,di
jmp short @1@310
@1@282:
mov word ptr DGROUP:_pkt_len,cx
mov ax,ds
mov es,ax
mov di,offset DGROUP:_pkt_data
@1@310:
sti
pop ds
push di
pop di
pop bp
ret
_pkt_callback endp
You see, the only registers that are modified are AX, DI and ES. Now, ES and DI take the buffer address as return values, so the packet driver expects them to be changed. Remains AX which turned out to be not critical as well and if it were, it would have been easy to save this single register on the stack and restore it on return.

I didn't even know, that there are packet drivers around that tolerate a misaligned stack or compensate for it by themselves, so my code doesn't rely on that feature. However i faced two other problems:

First, the packet driver doesn't know about the C-programs data space. Therefore the data segment register has to be set to the C-programs data segment, which is done by the pair of intructions "mov di,DGROUP / mov ds,di". On exit, of course, the packet drivers data segment has to be restored to its original value which was saved on the stack. This is the reason, why my code has to be compiled with the "compile through assembler" option because the C-compiler itself doesn't know about the symbol DGROUP. But this is no big deal, you just check a box in the options/compiler menu and that's it. Everything else works as usual. Nevertheless, your approach with the interrupt keyword avoids this little inconvenience.

Second, the complier wants to be clever and saves register DI on the stack and restores it on return. This may be a good idea in ordinary C-routines. In our case however, we want to use register DI as a return path therefore it is absolutely counter productive if the compiler restores DI to the value it had on entry. A pair of pop di / push di instructions compensates for that.

But ooops, i just recognize, that your code does the same thing. It also restores the ES and DI registers in the epilogue, thus overwriting the values written into theese registers in the body of the routine. Now i really wonder how your buffer address ever arrives at the packet driver. I might have to explore this by passing your code, at least the relevant fragment, through the compiler to see how it ist actually translated. But to do this, it ist definitely to late now. (It's one 'clock AM, local time) So maybe tomorrow, or next year . . .

mbbrutman
December 30th, 2009, 04:09 PM
I had to go back and dig through my email .. and some of it is missing. But here is the background on the Xircom driver:

The Xircom driver expects an RETF in the receiver routine, not an IRET. My original code used the interrupt keyword without the custom epilog, so it crashed all of the time. The Davicom packet driver worked perfectly, so I suspected a problem with the Xircom packet driver.

The original packet spec expects a RETF. The current packet spec and reference code is flexible, and allows for either a RETF or an IRET - it can tell what the upcall routine did by checking the flags. The Xircom code, being older, was never updated with this ability. So technically it is correct, but just not as nice as the newer packet drivers. Too bad it absolutely does not work with the default code generated by the interrupt keyword. My custom epilog is essentially the same as the compiler generated one, except for the RETF instruction.

At the time I wrote my code I was an assembler neophyte so it was easier for me to use the interrupt keyword than to try to re-establish addressability to the data segment used by the compiler. My compiler (Turbo C++ 3.0, not Borland 3.1) doesn't have a "compile through assembler" option either, so it's not an option.

One important part of the semantics of the interrupt keyword and my function that you missed are that all of the registers that I used treated as parameters pushed on the stack, not live registers. The interrupt keyword generates the pushes in the same order that I have the parameters specified. In the body of the routine those register variables look like parameters passed on the stack, and if the values are changed they are changed on the stack. The epilog code will pop the correct values into the proper registers.

Interesting conversation ...

wiwa64
January 1st, 2010, 09:19 AM
In the meantime i found some time, to pass the code through the compiler and look, what code it generates. Here is the result:


; static void far interrupt receiver( unsigned bp, unsigned di, unsigned si,
; unsigned ds, unsigned es, unsigned dx,
; unsigned cx, unsigned bx, unsigned ax ) {
assume cs:_TEXT
@receiver$quiuiuiuiuiuiuiuiui proc far
push ax
push bx
push cx
push dx
push es
push ds
push si
push di
push bp
mov bp,DGROUP
mov ds,bp
mov bp,sp
; if ( ax == 0 ) {
cmp word ptr [bp+16],0
jne short @8@198
; if ( (cx>1514) || (Buffer_fs_index == 0) ) {
cmp word ptr [bp+12],1514
ja short @8@114
cmp byte ptr DGROUP:_Buffer_fs_index,0
jne short @8@142
@8@114:
; es = di = 0;
xor ax,ax ; this is the actual trick: writing to the
mov word ptr [bp+2],ax ; saved register values on the stack instead
mov word ptr [bp+8],ax ; of writing to the actual registers.
; Packets_dropped++;
add word ptr DGROUP:_Packets_dropped,1
adc word ptr DGROUP:_Packets_dropped+2,0
; }
jmp short @8@170
@8@142:
; else {
; Buffer_fs_index--;
dec byte ptr DGROUP:_Buffer_fs_index
; Buffer_packet_tmp = Buffer_fs[ Buffer_fs_index ];
mov al,byte ptr DGROUP:_Buffer_fs_index
mov ah,0
mov cl,2
shl ax,cl
mov bx,ax
mov ax,word ptr DGROUP:_Buffer_fs[bx+2]
mov dx,word ptr DGROUP:_Buffer_fs[bx]
mov word ptr DGROUP:_Buffer_packet_tmp+2,ax
mov word ptr DGROUP:_Buffer_packet_tmp,dx
; es = FP_SEG( Buffer_fs[ Buffer_fs_index ] );
mov al,byte ptr DGROUP:_Buffer_fs_index
mov ah,0
mov cl,2
shl ax,cl
mov bx,ax
mov ax,word ptr DGROUP:_Buffer_fs[bx+2]
mov word ptr [bp+8],ax
; di = FP_OFF( Buffer_fs[ Buffer_fs_index ] );
mov al,byte ptr DGROUP:_Buffer_fs_index
mov ah,0
mov cl,2
shl ax,cl
mov bx,ax
mov ax,word ptr DGROUP:_Buffer_fs[bx]
mov word ptr [bp+2],ax
@8@170:
; }
; }
jmp short @8@254
@8@198:
; else {
; Packets_received++;
add word ptr DGROUP:_Packets_received,1
adc word ptr DGROUP:_Packets_received+2,0
; Buffer[ Buffer_next ] = Buffer_packet_tmp;
mov al,byte ptr DGROUP:_Buffer_next
mov ah,0
mov cl,2
shl ax,cl
mov dx,word ptr DGROUP:_Buffer_packet_tmp+2
mov bx,word ptr DGROUP:_Buffer_packet_tmp
mov si,ax
mov word ptr DGROUP:_Buffer[si+2],dx
mov word ptr DGROUP:_Buffer[si],bx
; Buffer_len[Buffer_next] = cx;
mov al,byte ptr DGROUP:_Buffer_next
mov ah,0
shl ax,1
mov dx,word ptr [bp+12]
mov bx,ax
mov word ptr DGROUP:_Buffer_len[bx],dx
; Buffer_next = (Buffer_next + 1) % (PACKET_RB_SIZE);
mov al,byte ptr DGROUP:_Buffer_next
mov ah,0
inc ax
mov bx,21
cwd
idiv bx
mov byte ptr DGROUP:_Buffer_next,dl
; if (Buffer_lowFreeCount > Buffer_fs_index ) {
mov al,byte ptr DGROUP:_Buffer_lowFreeCount
cmp al,byte ptr DGROUP:_Buffer_fs_index
jbe short @8@254
; Buffer_lowFreeCount = Buffer_fs_index;
mov al,byte ptr DGROUP:_Buffer_fs_index
mov byte ptr DGROUP:_Buffer_lowFreeCount,al
@8@254:
; }
; }
; asm {
; pop bp
pop bp
; pop di
pop di
; pop si
pop si
; pop ds
pop ds
; pop es
pop es
; pop dx
pop dx
; pop cx
pop cx
; pop bx
pop bx
; pop ax
pop ax
; retf
retf
; }
; }
pop bp
pop di
pop si
pop ds
pop es
pop dx
pop cx
pop bx
pop ax
iret
@receiver$quiuiuiuiuiuiuiuiui endp


Now i understand, how it works and how the results find thier way back to the driver in this version. :)
Certainly this should work with my programs as well and it shouldn't even be to difficult to change.

On the other hand, i remember how often i already thought this . . .

So i will probably stay with my version, as it works fine too, but keep this approach in mind as an option for the future