Announcement

Collapse

Forum Rules and Etiquette

Our mission ...

This forum is part of our mission to promote the preservation of vintage computers through education and outreach. (In real life we also run events and have a museum.) We encourage you to join us, participate, share your knowledge, and enjoy.

This forum has been around in this format for over 15 years. These rules and guidelines help us maintain a healthy and active community, and we moderate the forum to keep things on track. Please familiarize yourself with these rules and guidelines.


Rule 1: Remain civil and respectful

There are several hundred people who actively participate here. People come from all different backgrounds and will have different ways of seeing things. You will not agree with everything you read here. Back-and-forth discussions are fine but do not cross the line into rude or disrespectful behavior.

Conduct yourself as you would at any other place where people come together in person to discuss their hobby. If you wouldn't say something to somebody in person, then you probably should not be writing it here.

This should be obvious but, just in case: profanity, threats, slurs against any group (sexual, racial, gender, etc.) will not be tolerated.


Rule 2: Stay close to the original topic being discussed
  • If you are starting a new thread choose a reasonable sub-forum to start your thread. (If you choose incorrectly don't worry, we can fix that.)
  • If you are responding to a thread, stay on topic - the original poster was trying to achieve something. You can always start a new thread instead of potentially "hijacking" an existing thread.



Rule 3: Contribute something meaningful

To put things in engineering terms, we value a high signal to noise ratio. Coming here should not be a waste of time.
  • This is not a chat room. If you are taking less than 30 seconds to make a post then you are probably doing something wrong. A post should be on topic, clear, and contribute something meaningful to the discussion. If people read your posts and feel that their time as been wasted, they will stop reading your posts. Worse yet, they will stop visiting and we'll lose their experience and contributions.
  • Do not bump threads.
  • Do not "necro-post" unless you are following up to a specific person on a specific thread. And even then, that person may have moved on. Just start a new thread for your related topic.
  • Use the Private Message system for posts that are targeted at a specific person.


Rule 4: "PM Sent!" messages (or, how to use the Private Message system)

This forum has a private message feature that we want people to use for messages that are not of general interest to other members.

In short, if you are going to reply to a thread and that reply is targeted to a specific individual and not of interest to anybody else (either now or in the future) then send a private message instead.

Here are some obvious examples of when you should not reply to a thread and use the PM system instead:
  • "PM Sent!": Do not tell the rest of us that you sent a PM ... the forum software will tell the other person that they have a PM waiting.
  • "How much is shipping to ....": This is a very specific and directed question that is not of interest to anybody else.


Why do we have this policy? Sending a "PM Sent!" type message basically wastes everybody else's time by making them having to scroll past a post in a thread that looks to be updated, when the update is not meaningful. And the person you are sending the PM to will be notified by the forum software that they have a message waiting for them. Look up at the top near the right edge where it says 'Notifications' ... if you have a PM waiting, it will tell you there.

Rule 5: Copyright and other legal issues

We are here to discuss vintage computing, so discussing software, books, and other intellectual property that is on-topic is fine. We don't want people using these forums to discuss or enable copyright violations or other things that are against the law; whether you agree with the law or not is irrelevant. Do not use our resources for something that is legally or morally questionable.

Our discussions here generally fall under "fair use." Telling people how to pirate a software title is an example of something that is not allowable here.


Reporting problematic posts

If you see spam, a wildly off-topic post, or something abusive or illegal please report the thread by clicking on the "Report Post" icon. (It looks like an exclamation point in a triangle and it is available under every post.) This send a notification to all of the moderators, so somebody will see it and deal with it.

If you are unsure you may consider sending a private message to a moderator instead.


New user moderation

New users are directly moderated so that we can weed spammers out early. This means that for your first 10 posts you will have some delay before they are seen. We understand this can be disruptive to the flow of conversation and we try to keep up with our new user moderation duties to avoid undue inconvenience. Please do not make duplicate posts, extra posts to bump your post count, or ask the moderators to expedite this process; 10 moderated posts will go by quickly.

New users also have a smaller personal message inbox limit and are rate limited when sending PMs to other users.


Other suggestions
  • Use Google, books, or other definitive sources. There is a lot of information out there.
  • Don't make people guess at what you are trying to say; we are not mind readers. Be clear and concise.
  • Spelling and grammar are not rated, but they do make a post easier to read.
See more
See less

8 bit IDE (XTA) Replacement Project

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Glad this has been of some use, I guess.

    Originally posted by JayesonLS View Post
    I am thinking the boot menu would be like a terminal connection to the drive so all the logic can be implemented in the drive firmware in modern C++. The code running on the PC would be limited to displaying text, capturing input and a few other actions like "install floppy drive redirect code and run bootstrap.". This opens up all sorts of options. Things like implementing SD card initialization on the drive would become easier to implement.
    That sounds like a fine idea. I suppose the only thing I would caution is, obviously, don't get carried away with building a giant list of deliverables/user stories in the queue before you've proved the basic idea.

    As a proof of concept/development milestone the idea I suggested doesn't require the implementation of any special "API" or other back channel communication to the drive; on the drive; it only requires the firmware to know it hasn't been configured with a geometry translation yet (and if that's true it should offer up that fake MBR boot sector to a read command against sector 0/0/0). Meanwhile the PC side of the code probably needs only a dozen or two assembly instructions to implement. IE, in the simplest iteration all it needs to do after the code is loaded by the BIOS and jumped to is:

    A: run an INT13h AH=08 call to get the highest sector address BIOS thinks the disk has, parse that value out

    B: make an INT13h read or write (ah=03 or ah=04) with the maximum CHS value. (Which the drive accepts via the normal controller command syntax and uses to set the LBA translation value.)

    C: Reboot the PC. Or slap a message on the screen saying "drive initialized, hit ctrl-alt-delete", if you think an instantaneous spontaneous reboot is too scary.

    The coding on the PC side is trivial (I'm a pretty raw newb with x86 assembly and I think even I could do it in an afternoon), and in the C++ drive firmware it's going to be even easier.

    (Obviously once you decide what your API is you could evolve this in baby steps, like initially instead of just doing an INT13h read you could have it send the info from the AH=08 call directly to the new communication channel and demonstrate receiving an acknowledgement back, etc.)

    FWIW, since you seem pretty concerned about making sure that all possible avenues are covered here I will point out that there were a few *really* off-spec cases for the use of these drives; for instance, I think they were used in hard disk expansions for some Amstrad PCW machines, which don't run DOS, presumably don't have conventional MBRs, and don't even have x86 cpus able to run this smart menu. For this use case specifically allowing the drive to be manually configured to emulate some "real" drive based on public specifications would still be a must, either by providing, I dunno, a serial port header, or the ability to manipulate a config written to the storage medium inserted into it. As much as you might try you simply can't bubble wrap every possible thing for every user, sometimes they just have to be able to read a manual.
    My Retro-computing YouTube Channel (updates... eventually?): Paleozoic PCs Also: Blogspot

    Comment


      Originally posted by Svenska View Post

      Did I miss anything important?
      Thanks, I think we are thinking the same things.

      And standard BIOS calls only for sure, along with any drive interface register level access where needed to access extra features.

      Comment


        Originally posted by Eudimorphodon View Post
        That sounds like a fine idea. I suppose the only thing I would caution is, obviously, don't get carried away with building a giant list of deliverables/user stories in the queue before you've proved the basic idea.
        Indeed. The first pass will be a basic drive. It will not have any MBR injection or support for the second SD card. Maybe it will sniff changes to the first partition table entry, and if everything look valid, save the inferred heads and sectors value. That seems easy to implement will at least get the heads/sectors right for folks who set up the drive with fdisk.

        Originally posted by Eudimorphodon View Post
        FWIW, since you seem pretty concerned about making sure that all possible avenues are covered here I will point out that there were a few *really* off-spec cases for the use of these drives; for instance, I think they were used in hard disk expansions for some Amstrad PCW machines, which don't run DOS, presumably don't have conventional MBRs, and don't even have x86 cpus able to run this smart menu. For this use case specifically allowing the drive to be manually configured to emulate some "real" drive based on public specifications would still be a must, either by providing, I dunno, a serial port header, or the ability to manipulate a config written to the storage medium inserted into it. As much as you might try you simply can't bubble wrap every possible thing for every user, sometimes they just have to be able to read a manual.
        That is a good point. So far, the drive replacement should always work as well as a real drive does. If the SD card is readable in another computer, that is a bonus. However I should definitely keep non-DOS usage iin the back of my mind. My understanding is that early DOS versions don't have a MBR either - the first sector is the boot sector of the FAT filesystem. I assume partitions were motivated by drives getting larger than the max filesystem size.

        On serial, the RPi Pico has a USB port on it. I intend it to be the initial mechanism for updating the firmware and have made sure the port is accessible on the board layout. Firmware updating is built into the silicon: hold the button on the Pico while inserting the usb cable. It shows up as a USB drive and you just drag on the firmware file. This bootloader is built into ROM so it is not even possible to brick the Pico. This may be the only method for firmware update unless somebody else wants to write an alternative. I am not very keen on doing it.

        My current WIP firmware already configures the USB port as a serial device. It would probably not be very hard to expose a subset of the boot menu over USB serial. Something to be able to manually configure the drive heads, cylinders and perhaps the size bits register would go a long way.

        Comment


          Originally posted by JayesonLS View Post
          My understanding is that early DOS versions don't have a MBR either - the first sector is the boot sector of the FAT filesystem.
          To my knowledge, any version of DOS with support for hard disks also supports MBR partition tables.

          Originally posted by JayesonLS View Post
          My current WIP firmware already configures the USB port as a serial device. It would probably not be very hard to expose a subset of the boot menu over USB serial. Something to be able to manually configure the drive heads, cylinders and perhaps the size bits register would go a long way.
          Possibly, although a configuration file next to the disk image should be easier to use for most users. The common case is using a known geometry, which can be inferred from the image file size even if no configuration file is present. This is how FlashFloppy works, and I like both its simplicity and configurability.

          Comment


            Originally posted by Svenska View Post
            To my knowledge, any version of DOS with support for hard disks also supports MBR partition tables.
            You are correct from what I can tell. The original PCDOS 2.0 for the 5160 does come with fdisk and creates an MBR. It only allows creation of one partition - apparently you were supposed to create more partitions for other operating systems. Although how exactly is not clear to me.

            Complicating MBR sniffing slightly, PCDOS 2.0 fdisk creates the DOS partition in the fourth partition slot, leaving the first three slots blank. The DOS partition also starts on the second sector. DOS 3.3 or 6.22 (I can't remember which) on the other hand started the first partition on an even sector boundary (first track, second head, first sector).

            Comment


              Will it be configureable? I mean, that I can set up a specific XTA harddisk type from specific manufacturer like 20 MB Conner, or can I tell the CHS data for the drive image on the SD-Card? Another sexy thing would be that I can swap between different diskimages with a button (or by software) ? I know during that there should not be a disk access and then the system needs a reboot. And one more neat function for at least FAT partitions would be if the device can extract the files inside the image to a separate folder on the SD card so that it is easy for file transfer. Maybe also the other way arround to get data into the disk image...
              <album>

              Comment


                Originally posted by JayesonLS View Post
                You are correct from what I can tell. The original PCDOS 2.0 for the 5160 does come with fdisk and creates an MBR. It only allows creation of one partition - apparently you were supposed to create more partitions for other operating systems. Although how exactly is not clear to me.
                Microsoft just kicked that ball over the fence, assuming that any alternative OS would come with an FDISK/installer smarter than theirs and just giving you the option to leave empty space for that other, smarter FDISK was where their responsibility ended.

                Complicating MBR sniffing slightly, PCDOS 2.0 fdisk creates the DOS partition in the fourth partition slot, leaving the first three slots blank. The DOS partition also starts on the second sector. DOS 3.3 or 6.22 (I can't remember which) on the other hand started the first partition on an even sector boundary (first track, second head, first sector).
                It's only the end positions you care about, so if you go with the MBR analysis strategy just assume you should check all slots. Since the "must end on cylinder boundaries" thing seems like a pretty safe assumption reading *any* populated partition entry should give you the highest head and sector numbers, which is strictly all you care about. (As noted, doesn't matter if the drive thinks it has 1024 cylinders but the BIOS thinks it's only 900 or whatever, doesn't affect the CHS to LBA translation.

                I guess it is also worth noting that strictly speaking the physical order of partitions in the MBR doesn't need to correspond with their position on the disk. It is technically valid to have the partition in the lowest cylinders in the highest MBR slot and vice-versa. It's probably bad practice, sure, but it is *allowed*.)

                This is, again, strictly speaking, the advantage of my suggestion of running some kind of program that directly just makes an INT13h call and instructs the drive to read the last sector, whether you do it the fancy fake MBR way or, for Rev 0, just run this binary off a floppy. It eliminates all possible ambiguity. (If you look up some of the ugly details about, say, partition type labels, there does exist the possibility that, say, if you expect a DOS partition entry to be a particular type there may be some OEM version of DOS that used something else.)

                FWIW, I would say that DOS 2.x is going to be a pretty rare edge case. It was *long* obsolete before XTA hard drives came out. The oldest machines I can think of with XTA ports are, what, 1988-ish vintage? About the only computer still on the market still shipping with DOS 2.x that late was the little stub inside of a Tandy 1000HX's ROM, and that version of DOS didn't even come with FDISK on the utility disk.
                Last edited by Eudimorphodon; October 25, 2021, 10:57 AM.
                My Retro-computing YouTube Channel (updates... eventually?): Paleozoic PCs Also: Blogspot

                Comment


                  Originally posted by 1ST1 View Post
                  Will it be configureable? I mean, that I can set up a specific XTA harddisk type from specific manufacturer like 20 MB Conner, or can I tell the CHS data for the drive image on the SD-Card? Another sexy thing would be that I can swap between different diskimages with a button (or by software) ? I know during that there should not be a disk access and then the system needs a reboot. And one more neat function for at least FAT partitions would be if the device can extract the files inside the image to a separate folder on the SD card so that it is easy for file transfer. Maybe also the other way arround to get data into the disk image...
                  CHS Parameters

                  The drive, will for sure, always work with any CHS parameters the system BIOS is using. It can do this because I will make the minimum primary SD card size 128MB. [EDIT: I am not sure it is even possible to buy MicroSD cards under 128MB]. Any CHS parameter mismatch between the BIOS and drive just means sectors will not be written in contiguous order. Or in other words, there will be unused sectors sprinkled across the SD card. Sectors not being contiguous means the file system on the SD card would not be accessible if used in a modern PC.

                  Most of the recent thread discussion has been how to configure the drive with CHS parameters so the sectors end up on the SD card in contiguous order. Eudimorphodon came up with a couple of great approaches on how to do this automatically. Assuming sufficient time and motivation, I am open to implementing versions of those and also a fully explicit CHS configuration. In the end, I am not overly concerned about the primary SD card being usable in another PC. I think it is most useful for getting files back and forward, especially when setting things up the first time. The plans for the secondary SD card will offer a couple of alternate ways of doing that. I am most keen on the ability to mount a floppy disk image as the A: drive. Making 360K or 720K floppies tends to be quite involved and error prone. A lot of the target machines seem to have unreliable floppy drives also.

                  There are challenges with trying to match specific drive parameters. All drives I have looked at have jumpers and the Seagate ST351 A/X for example has more configurations than I think I will ever figure out. Even if the drive is configured to match a real drive, there is still no guarantee that the BIOS CHS values match. They don't in the case of an early Commodore CPx0 BIOS I read about yesterday. It interprets 40MB drives as 10MB. I also suspect my Tandy TL/2 was wasting a head (entire platter surface) on the WD drive it came with. The drive seems to longer be working so I can't confirm that now. The better approach seems to be tell the drive what the BIOS is using. Or just not bother and skip being able to read the primary SD card in another PC.

                  All that said, it does makes sense for a 2.5" version of the drive to default to the CHS parameters for the little Connor 20MB.

                  Where I use the term CHS above, I really only care about heads and cylinders. The drive will always handle cylinder values up to the max possible (1023 or 1024, I can't recall what the max is).

                  Hard Drive Images

                  I think if I support booting hard drive images, it will be a bonus, advanced feature and the images will live on the secondary SD card. I know it would be quite powerful and seems especially useful because you could create the drive image in an emulator. Things can get very fussy about CHS values matching and the failures are hard to diagnose - usually the system crashes trying to boot DOS. I think even the cylinder count being a little off can cause problems and the vintage PC BIOSes sometimes have slightly lower cylinder counts than the physical drives.

                  Comment


                    Re: your suspicions that the Tandy TL/2 was “wasting a platter” on a drive, that’s highly unlikely.

                    One thing you’ll notice when you look at the specs for these XTA drives is the very same drive model will list multiple possibilities for the geometry. This is *not* because they made a bunch of secretly different versions of the hardware, it’s because many (most?) of these drives used translation schemes to camouflage their true hardware configuration. Some of these drives that claimed to have up to six heads in the BIOS had as little as two (a single platter) in real life. (ST351/AX) This Statson reference for some Western Digital XTA drives has a couple tables in it showing different virtual geometries for the same 40 MB drive:

                    https://stason.org/TULARC/pc/hard-dr...HH-IDE-XT.html

                    If set to fake 17 sector tracks it had “five heads”, if set for 27 sectors it had four.

                    I know it sounds stupid today, but in the late 80’s there was all kinds of ridiculousness going on with hard disks. Some low level optimization/data recovery tools made catastrophically stupid assumptions, like that all hard disks had 17 sectors per track, and it was enough of a problem that some manufacturers thought they needed translation to hide that. And hard disks also started being made with more than 1024 tracks, which the standard INT13h/DOS CHS format didn’t support, so you needed translation for that. And really high capacity hard disks were starting to use zone density recording to fit more data on the longer outer tracks of the disk, which completely blows up CHS entirely, so you need translation for *that*…

                    Long story short, unless you’ve cracked open the actual physical drive and counted platters assume all BIOS geometries listed for a given XTA hard disk technically are lies. Pure unadulterated BS.
                    My Retro-computing YouTube Channel (updates... eventually?): Paleozoic PCs Also: Blogspot

                    Comment


                      … FWIW, though, some real hard drives had an odd number of data heads. Early voice coil positioner drives often used a “dedicated servo” surface on one platter, meaning one head was used solely to read information written when the drive was manufactured to provide positioning feedback, Most modern drives use “embedded servo” info instead, where data shares the same surface.

                      This is why you can destroy modern hard drives irreparably by running them through a bulk eraser. Erase the servo info and the drive is boned, you can’t write it back with the voice coil.
                      My Retro-computing YouTube Channel (updates... eventually?): Paleozoic PCs Also: Blogspot

                      Comment


                        Originally posted by Eudimorphodon View Post

                        Long story short, unless you’ve cracked open the actual physical drive and counted platters assume all BIOS geometries listed for a given XTA hard disk technically are lies. Pure unadulterated BS.
                        I didn't mean to refer to physical heads. I should not have confused things by referring to platters. All I know is that it had a 20-something meg partition on a 40 meg drive and fdisk seemed to think it was full. Maybe it just got imaged that way at the factory. I am hoping that one day it will come alive again. It had a few bad sectors which I was quite happy about so I could see how the drive behaved on read errors.

                        Comment


                          None of the 40 MB IDE-XT drives are "wasting a head". They're simply using the 980/5/17 parameters because those were the most common, based on Seagate's ST-251.

                          The 20 MB drives have one platter (two heads) and the 40 MB drives have two (four heads). But I dunno about the WD 30 MB drives -- they're listed as having 3 heads. So maybe they're actually 40 MB drives with one faulty platter surface? An easy way to make use of parts that otherwise would be discarded.

                          Comment


                            Originally posted by vwestlife View Post
                            None of the 40 MB IDE-XT drives are "wasting a head". They're simply using the 980/5/17 parameters because those were the most common, based on Seagate's ST-251.
                            ... Yeah, I forgot about that reason for translation. Many early IDE drives used translation specifically to emulate drive geometries that were common in AT BIOS drive type tables. Having a user-specified type wasn't really common until 1990 or so.

                            I guess I will note that I have discovered it may be possible to confuse FDISK if you image a disk to the "wrong" size so maybe we can't entirely rule that out. I was experimenting at one point with using an emulator to get PC-DOS 7 installed directly off a CD-ROM distribution of it to use on my Tandy with XTIDE, and while that "worked" to generate a 2GB image and DD-it onto the flash device subsequently I discovered that I couldn't use FDISK on the Tandy to make any additional partitions. Normally you should be able to use the first 8GB of a drive (max supported by DOS CHS translation), but for some reason the 2GB total size from the DD'ed emulator image was "honored". That doesn't quite make sense to me given I *thought* the only place geometry is supposedly mentioned in the MBR is in the individual partition entries, but... shrug. I didn't dissect the MBR to see if DOS had created some dummy "free space" entry in another slot or anything.
                            My Retro-computing YouTube Channel (updates... eventually?): Paleozoic PCs Also: Blogspot

                            Comment


                              Originally posted by JayesonLS View Post
                              The original PCDOS 2.0 for the 5160 does come with fdisk and creates an MBR. It only allows creation of one partition - apparently you were supposed to create more partitions for other operating systems. Although how exactly is not clear to me.
                              Microsoft XENIX comes with its own FDISK utility, and so does OS/2. DOS simply ignores foreign partitions, and FDISK cannot not create such partitions because it does not know if other restrictions apply. (Although DOS FDISK shows XENIX partitions by name.)

                              While partitions should generally start and end on a cylinder boundary, it is not a strict requirement. But I think some versions of DOS (or other operating systems) require it for their partitions, at least. Some boot managers also require the remainder of the first cylinder to hide (e.g. GRUB), but others use their own partition (e.g. OS/2) or are made small enough (XFDisk).

                              Originally posted by vwestlife View Post
                              The 20 MB drives have one platter (two heads) and the 40 MB drives have two (four heads). But I dunno about the WD 30 MB drives -- they're listed as having 3 heads. So maybe they're actually 40 MB drives with one faulty platter surface? An easy way to make use of parts that otherwise would be discarded
                              Either the last head contains diagnostic (or factory) data, or they use a more efficient encoding internally (RLL instead of MFM) and allocate the additional space as an additional head to avoid changing the geometry.

                              Originally posted by Eudimorphodon View Post
                              That doesn't quite make sense to me given I *thought* the only place geometry is supposedly mentioned in the MBR is in the individual partition entries, but... shrug. I didn't dissect the MBR to see if DOS had created some dummy "free space" entry in another slot or anything.
                              Unfortunately, some parts of the drive geometry are part of the FAT BPB, and some versions of DOS will corrupt other partitions if they don't match the drive. The 8 GB limit only applies to the maximum 1024/255/63 CHS values; if the BIOS assumes 17 sectors/track instead of 63, you end up with a 2.1 GB limit instead. The ECHS translation does not change the sector count (it keeps doubling heads/halving cylinders).

                              Comment


                                Originally posted by Svenska View Post
                                Either the last head contains diagnostic (or factory) data, or they use a more efficient encoding internally (RLL instead of MFM) and allocate the additional space as an additional head to avoid changing the geometry.
                                The Western Digital 20, 30, and 40 MB IDE-XT drives all have physical parameters of 782 cylinders and 27 sectors per track. The only difference is 2, 3, or 4 heads.

                                Comment

                                Working...
                                X