PDA

View Full Version : New Project: LAN supervisor program



wiwa64
August 25th, 2010, 07:46 AM
I don't know if the world really needed that program, but i simply was curious to see whether (and how) it could be done. ;)


I'm writing about my latest project: LANSTAT a supervising program for LANs. It's very compact (<28 kB) yet also very flexible, therefore it should be usable under virtually any type or brand of networking software that is available for DOS. I expect it to co-operate with mbrutman's mTCP as well, though i haven't tried it yet. :???:

Hoping to have made you all curious enough, i all invite you to visit http://www.bttr-software.de/products/jhoffmann/, download it from there to explore it and to report your experiences.

mbbrutman
August 25th, 2010, 08:32 AM
Hello Jürgen,

If there is something specific you'd like added to my ping command to make this easier to integrate, let me know. I'll give it a test as soon as I'm allowed near the old computers again. (I need more vacation time!!!)


Mike

wiwa64
August 25th, 2010, 12:00 PM
I eventually downloaded your mTCP package (i should have done that before, shame on me) and tried its ping utility. As far as i can see after some brief and somewhat superficial tests, i find that it is quite ok with respect to LANSTAT's needs. I suggest the follwing test definition: PTEST "ping -count 1 \a" reply:

If one goes into it a litle bit deeper, than one can find minor problems.

To understand them, one first has to understand how LANTEST works. It uses a "system" call with a line composed of the defined string (in this case "ping -count 1 \a") concatenated with a " > " and the name and path of the temporary working file as argument, where the \a is automatically replaced by the argument of the HOST definition (in this case the ip-address). The system call in-turn starts a command shell and passes it the line it got as argument. As a result, the defined command (in this case ping) will be called with its stdout re-directed into LANSTAT's temporary working file. Upon return from the system call LANSTAT searches that file for the occurence of the string definded as second argument of the PTEST definition (in this case "reply:"). If it finds that string, then the test is successful, otherwise it isn't.

This approach requires, that the respective program writes its output to stdout (and not to the screen directly) and nothing to stderr either. Your ping does not completely fulfill this last requirement with the consequence that under certain conditions LANSTAT's own screen could get messed-up with messages that were intended to go to stderr.

On the other hand i understand that it is often desirable to clearly distinguish between "normal" messages which go to stdout and "error" messages going to stderr, therefore i dont want to criticize that. Furthermore things are not quite as bad as it might appear so far. As far as i understood, your ping writes "normal" results to stdout and only very foundamental error messages like config file not found or packet driver not found to stderr.

One possible suggestion could be a switch that (only when set) causes all messages, including error messages, to go to stdout. But this is perhaps not yet the best idea. As your ping is not the only program writing some mesages to stderr, WATTCP programs do it the same way, LANSTAT does its best to cope with this effect.

As LANSTAT heavily depends on the external programs it provides a means to test their accessability and proper operation at program start-up and aborts if this is not the case. In the case of ping this would typically be a ping to itself. Under a full-blown TCP/IP stack this should always succeed, unless something very basic goes wrong. So this would be a good criterion for LANSTAT to decide, whether it makes sense to start normal operation or whether it should rather abort.

Unfortunately your ping seems to be unable to ping itself. This is probably due to the fact, that mTCP is no TCP/IP stack as WATTCP shows the same problem. So one possible improvement could be, that your ping produces a fake reply message if it recognizes a ping request to its own address. This however has a bit of a "dirty trick" so it might still be just the second best idea.

The probably cleanest way would be an additional switch "-test". If set, then ping should operate almost as usual, especially it should perform all the tests for configuration file, packet driver and so on. Just immediately before the actual ping is carried out, it should terminate with a message like "Test OK" or so (Provided that everything was OK). This message could be the signal for LANSTAT to rely on ping and to assume that subsequent regular pings will only fail if the remote host is un-accessable but not due to internal reasons.

mbbrutman
August 25th, 2010, 12:54 PM
Jürgen,

There are good things to know.

I have been kind of torn about the proper use STDERR and STDOUT in DOS since day one. On Unix it makes a difference, and you can redirect each of those separately. DOS allows redirection of STDOUT, but not STDERR. As you noted I generally use STDERR for warnings or fatal messages, and STDOUT for normal output. (Telnet and IRCjr are two notable exceptions - they do direct screen writes.)

None of the mTCP code supports a loopback interface or address. Conceptually loopback is great for testing, but most apps are not going to choose to use TCP/IP for sending messages when they can just use a pointer. It is not that hard to add, but it does add extra code that *every* packet being sent would have to traverse. And the only use I can see for it is to verify that the TCP/IP stack is loaded.

The system call in Open Watcom has a fairly useless return value - it basically tells you if the command processor was able to load. Have you considered using the spawn call? It allows you to invoke the target program directly (without the command processor) and get a return code directly from the program. With a well architected set of return codes from the target program you could do everything you need without the overhead of loading another command processor and manipulating temporary files.

I'd be willing to tweak PING which ever way you need to make this integrate better. I'd really rather just give you the PING app to integrate into your program; that would solve a lot of problems. ;-0


Mike

wiwa64
August 26th, 2010, 11:58 AM
Well, i might have been a bit unclear in my previous lenghty posting. Hoping to make things clearer:

It is not absolutely necessary to do any changes to your ping. It should co-operate with LANSTAT fairly well the way it is.
Modifications are only necessary if you want to make the step from "fairly well" to "perfect"
I don't think that using the spawn call is an option for me. It makes things much more complicated. Not only would i have to deal with the output re-direction myself but the whole concept of the program and it's set-up file is based on the fact, that external programs can be invoked (almost) the same way they would be called from the command line. Having to emulate all this command interpretation myself instead of letting command.com do that job would add an extra level of complication. And i don't use the return value anyway, rather i scan the textual output to stdout for the ocurence of some (definable) string.
Of course i understand that you don't want to "add extra code that *every* packet being sent would have to traverse" and you are perfectly right in that. I just thought you could add something to your ping program, at a point before it calls the TCP/IP stack. But this idea was born without knowing the precise interaction (and API) between your mTCP library and the application program (ping). Therefore such a point might perhaps not exist.
On the other hand: How does your mTCP library handle "extreme" addresses, like e.g. "0.0.0.0" or "255.255.255.255"? Does it not check for these? Adding the extra check(s) into such a branch would avoid having to traverse it for *every* packet yet it could handle the special cases.

mbbrutman
August 26th, 2010, 12:59 PM
Your previous post was clear - no problems there.

I think about the constant loading the command processor, having it invoke another program, redirecting the output, and parsing the output and I cringe. That's going to limit how many times per second you can call out to ping. On the other hand, it does allow you to use any ping program.

In the ping program I can add a flag like you suggested to test for the presence of a correctly configured network stack, or I can 'trap' the loopback address at a high level, presume that it is meant to test for the presence of a correctly operating stack, and fake a result. I actually prefer the separate command line option, because it is explicit. In my past post I was speaking in more general terms about proper loopback support for the stack, which would require the stack to detect the loopback address ranges and do a memcpy to an input buffer that is normally used by the receiving code. I might make this a configurable option using #defines so that the applications that don't need it don't have to pay for it. It might be useful.

Extreme addresses .. split your IP addresses into two fields, network and host

Network = 0, Host = 0: This network, this host (An alias for loopback)
Network = 0, Host = x: A host on this network, but the network is not specified
Network = all ones, Host = all ones: Broadcast to the local network
Network = x, Host = all ones: Broadcast to network x
Network = 127, Host = x: Loopback to this machine

The first two are generally considered invalid. I support broadcast to both the local network and to a different network. I don't support loopback because none of the apps use it yet.

I'm not sure what happens if you do a broadcast ping. My code might not parse the incoming responses correctly. But, that might be a more efficient alternative than calling out to command.com and ping for each machine in the list.

Lastly, I'd look into spawn again. Watcom doesn't say what happens to stdout, but it might go to whatever file descriptor 1 is pointing at right before the spawn. Which would give you a way to catch output on stdout without having to go through command.com. And you get a return code, which is missing from the 'system' method of doing it.


Mike

MikeS
August 26th, 2010, 01:13 PM
... And i don't use the return value anyway, rather i scan the textual output to stdout for the ocurence of some (definable) string.You probably already know this, but JIC: If you want to include stderr there are utilities that will do that, especially if you're starting a new command processor anyway:
e.g.:

http://www.teaser.fr/~amajorel/stderr/

Not as flexible as 'nix or WIN CMD, but maybe useful somewhere nevertheless.

(the other) mike

wiwa64
August 27th, 2010, 12:11 PM
Well, the good news first: I experimented a bit more with your mTCP and especially the ping utility and my conclusion is: Just leave it alone. With a little bit trickier configuration, LANSTAT can perfectly interact with mTCP's ping. Here's the configuration:
GTEST "c:\mtcp\ping -count 1 0.0.0.0" imeout
PTEST "c:\mtcp\ping -count 1 \a" replyThe first statement GTEST defines something i call global test. LANSTAT performs this test only once at program start-up. In this case it issues a ping to address 0.0.0.0. If this ping times out, then everything is OK because it means that no error of the type "You must set the MTCPCFG environment variable" or "packet driver not found" ocured. If it did, than further processing is pointless and LANSTAT would terminate.

The only minor annoyance is, that the ARP timeout spells the word as "Timeout" while the actual ping time-out is spelled "timeout". To cope with that, the statement just searches for "imeout". So if you absolutely want to improve something, then take care of a uniform spelling.

The second statement is the actual ping that regularly polls the different hosts. The "\a" wil be replaced by the actual ip-address and the result is positive, if the word "reply" can be found in the answer.


I think about the constant loading the command processor, having it invoke another program, redirecting the output, and parsing the output and I cringe. That's going to limit how many times per second you can call out to ping.I think, the time for loading the command processor and have it invoke another program, is still neglectable compared to the time ping takes for itself, especially when running into a time-out. (Well, running it on a floppy based system might not be such a good idea) Furthermore, it is not desirable to "flood" the net with ping requests, rather the extra network load caused by LANSTAT should be kept as low as possible. Therefore it even provides for the ability to define additional delays.


In the ping program I can add a flag like you suggested to test for the presence of a correctly configured network stack, or I can 'trap' the loopback address at a high level, presume that it is meant to test for the presence of a correctly operating stack, and fake a result. I actually prefer the separate command line option, because it is explicit. In my past post I was speaking in more general terms about proper loopback support for the stack, which would require the stack to detect the loopback address ranges and do a memcpy to an input buffer that is normally used by the receiving code. I might make this a configurable option using #defines so that the applications that don't need it don't have to pay for it. It might be useful.Well, as already written above, it is not really needed. On the other hand, it might be a good idea anyway, to have some kind of self test, some software, be it a special switch to ping, be it a separate utility, that clearly and relyably tells "configuration OK, network operational" (or not) and that other software can depend on.


Extreme addresses .. split your IP addresses into two fields, network and host

Network = 0, Host = 0: This network, this host (An alias for loopback)
Network = 0, Host = x: A host on this network, but the network is not specified
Network = all ones, Host = all ones: Broadcast to the local network
Network = x, Host = all ones: Broadcast to network x
Network = 127, Host = x: Loopback to this machine

The first two are generally considered invalid. I support broadcast to both the local network and to a different network. I don't support loopback because none of the apps use it yet.

I'm not sure what happens if you do a broadcast ping. My code might not parse the incoming responses correctly. But, that might be a more efficient alternative than calling out to command.com and ping for each machine in the list.As i just wrote, i experimented a bit. I didn't use a network sniffer though (even though i once wrote one myself) i only observed the external consequences. The address 0.0.0.0 is not rejected, it just runs into a time-out. I don't understand though, why it's sometimes already the ARP that times-out while in other cases it seems to be the actual echo-request that timed-out. I wonder who could have answered an ARP rquest for 0.0.0.0. The host's own address (not the loopback address 127.0.0.1) also runs into a time-out.

On the other hand, a ping to the braodcast address 255.255.255.255 was anwered. As already stated, i did not explore by whom, but there was only one more active host on the net at that time. I wonder what might have happened if 10 or maybe 100 more hosts would have been active and i could imagine that you will have a hard time if a network administrator finds out, that your program is the originator of such a broadcasted ping.


Lastly, I'd look into spawn again. Watcom doesn't say what happens to stdout, but it might go to whatever file descriptor 1 is pointing at right before the spawn. Which would give you a way to catch output on stdout without having to go through command.com. And you get a return code, which is missing from the 'system' method of doing it.Yes, i also looked into some code using spawn, i once wrote. It doesn't seem to be as hard to do, as i originally was afraid of. But it wouldn't have solved the problem anyway as i'm not just interested in the fact that something went wrong, i rather want to know, did it go wrong due to some external cause like e.g. host currently not responding, which might go away all by itself, or did it go wrong due to some fatal internal error, like unaccessable packet driver, which makes any further operation pointless.

mbbrutman
August 27th, 2010, 12:41 PM
The only minor annoyance is, that the ARP timeout spells the word as "Timeout" while the actual ping time-out is spelled "timeout". To cope with that, the statement just searches for "imeout". So if you absolutely want to improve something, then take care of a uniform spelling.

I have one word for you: stricmp ;-0


I think, the time for loading the command processor and have it invoke another program, is still neglectable compared to the time ping takes for itself, especially when running into a time-out. (Well, running it on a floppy based system might not be such a good idea) Furthermore, it is not desirable to "flood" the net with ping requests, rather the extra network load caused by LANSTAT should be kept as low as possible. Therefore it even provides for the ability to define additional delays.

I think you'd be surprised by this. My 386-40 can ping a machine and get a response back in less than 1ms. Unless things are perfect on the hard drive, you can't load the command processor and then ping (which is less than 40kb) in less than 1ms. It takes just one head movement to ruin things and make loading ping take longer than actually running it. Which is why network drives with fast machines are so appealing, but that's a different thread.



On the other hand, a ping to the braodcast address 255.255.255.255 was anwered. As already stated, i did not explore by whom, but there was only one more active host on the net at that time. I wonder what might have happened if 10 or maybe 100 more hosts would have been active and i could imagine that you will have a hard time if a network administrator finds out, that your program is the originator of such a broadcasted ping.

It's a perfectly valid test to run, and on a modern network using switches not a big problem. On an old collision prone network with hubs and slow machines it would have been anti-social, but not impossible either.

Some Unixes restrict the ability to do a broadcast ping. As always, we give the users enough rope to hang themselves with.

I think my ping code won't recognize the incoming responses because the replying machines will transmit using their IP addresses in the source field while Ping is expecting to see the same address (the broadcast address) it sent out. It would be a useful discovery tool if I fixed that glitch - broadcast ping could be used to map the network fairly efficiently, as opposed to just waiting for machines to generate traffic.



Yes, i also looked into some code using spawn, i once wrote. It doesn't seem to be as hard to do, as i originally was afraid of. But it wouldn't have solved the problem anyway as i'm not just interested in the fact that something went wrong, i rather want to know, did it go wrong due to some external cause like e.g. host currently not responding, which might go away all by itself, or did it go wrong due to some fatal internal error, like unaccessable packet driver, which makes any further operation pointless.

That's why ping uses different return codes - so one can differentiate between a local failure, an ARP timeout, or no response from the server. Text is definitive, but that means you have to parse and be very flexible.

I really like return codes, but it requires tighter integration than what you were looking for.

wiwa64
August 28th, 2010, 12:06 PM
I have one word for you: stricmp ;-0That would be an alternative :) but just scanning for "imeout" does the job equally well.


I think you'd be surprised by this. My 386-40 can ping a machine and get a response back in less than 1ms. Provided the other mashine responds that fast (and responds at all) The purpose of a program like LANSTAT is to see which ones are reponding and which ones not. So there will always be a certain percentage of relatively large delays. Furthermore, as already written, LANSTAT's objective is not to win the price for the "fastest ping on the globe", rather it is intended to constantly monitor the network with as little extra network load as possible.


I think my ping code won't recognize the incoming responses because the replying machines will transmit using their IP addresses in the source field while Ping is expecting to see the same address (the broadcast address) it sent out.Hmmm . . . :sigh:

It would be a useful discovery tool if I fixed that glitch - broadcast ping could be used to map the network fairly efficiently, as opposed to just waiting for machines to generate traffic.Some would perhaps rather call it (an "evil") hacking tool, but nevertheless it would be interesting.


That's why ping uses different return codes - so one can differentiate between a local failure, an ARP timeout, or no response from the server. Text is definitive, but that means you have to parse and be very flexible.

I really like return codes, but it requires tighter integration than what you were looking for.Well, LANSTAT is indeed intended to be very flexible. The downside of this is, that it has to be configured and i didn't want to make this configuration process more complicated that necessary.

Given that, i found it more intuitive if you could just try the programs you want to use as "test-agent" on the command line, see what happens and you know what you have to define in the configuration file.

Return codes are not immediately visible (there is no "$?" like in UNIX). So you have to have a very well documented program or a special tool that tells you the return codes. Ok, both ways are not impossible to go but i found it more convenient for a user, the way i designed it. In a different context however, return codes can make much sense.

wiwa64
August 31st, 2010, 12:22 PM
I just uploaded an improved version (1.3) of LANSTAT and i would really appreciate some more feed-back. As the program is very versatile and configurable in very many ways, it is almost impossible for a single developer to test every thinkable configuration.

wiwa64
October 26th, 2010, 04:30 AM
A new version of my network monitoring program LANSTAT is available and can be downloaded from here: http://www.bttr-software.de/products/jhoffmann/. It now also provides for logging, offers a considerably greater flexibility in the display layout and provides many minor improvements in various details.

But not only the program was improved, the documentation file as well. It is now actually larger then the program itself, offering to those, who are not afraid of reading a more elaborate text, some (hopefully) useful ideas about the program's flexibility, the many ways to configure it and the various purposes it can be used for.

As little "appetizer" here some screenshots:

http://www.bttr-software.de/products/jhoffmann/images/lanstat1a.jpg

http://www.bttr-software.de/products/jhoffmann/images/lanstat1b.jpg

http://www.bttr-software.de/products/jhoffmann/images/lanstat2.jpg