I am in two minds about this.
Something 'like SDL or Allegro' is not what I'm interested in personally, since they are relatively high-level, and mainly allow you to manipulate (linear) framebuffers.
To get things fast on 8088/CGA, you generally need to do special trickery, which would probably break generic routines.
So I wouldn't want to go as far as having an entire 'framework' which sets up the videomode for you and manages the framebuffer (and even keyboard and other input).
Having said that, I am working on my own asm and C headers and helper functions, aimed more at doing this special trickery. This is more in the form of simple macros and such to perform low-level things, like inserting an unrolled loop to poll for hsync or such. Or to put the PIC in auto-EOI mode, or reprogramming the PIT etc.
I also think that preprocessing tools may be useful. Eg, for 8088 MPH I wrote some some tools to convert any image (JPG, PNG, GIF, whatever) to an assembly listing in CGA, EGA or VGA memory layout. And of course the sprite compiler, which outputs an assembly listing with routines to draw and erase sprites.
Having such a tool, in source form, would be useful, because you can easily expand it for custom tricks.
Likewise, having line drawing routines, blitters etc in source/macro form would also be useful. They give you a basis you can adapt for any custom videomode or other tricks you can think of. But I think having a fixed library/framework is an abstraction level too high for efficient 8088 code.