You are almost describing the MAME CHD format. As they have the problem that the object (hard drive, cd, dvd, etc) must be in one file. Have the ability to do differences (writable in some cases). But also compressed (compressed hunks of data). They also need that sub track data too as some systems do interesting things with that sub data. As some even hide their encryption format in the SBI fields. The CHD format is more like a container that acts like whatever media it was. Depending on what system they hook it up to. The downside is there is no concept of 'metadata' to find different things in CHD. It is up to the system it is hooked up to to interpret what that data stream is.
This could be a good avenue. It might be possible the CHD format could be extended and be backward-compatible, or even just as simple as bodging all the extra data onto the end of the file and hope it is ignored by other readers. This is an avenue worth exploring, thank you.
There is a way to extend the format. As it has version number. It does have some metadata fields (drive geometry, compression formats, version of mame created with, etc). The trick would be getting the MAME team to accept the changes. Just dumping it on the end of the file I would guess they would not be too happy with.
There are a number of requests out there to extend the chd format and fix a few things. They are currently tracking some of this info in XML files (called hash files). They would be down with more accurate information though. So you would need a proposal that adds more accurate information and gives them something to work with. More like getting all the info that something like the redump project tracks in there would make them very happy.
There is a separate project that some other emus use libchdr which is a soft fork of MAME. I think they are trying to track closely to what the MAME group is doing but let other emus use it.
