Mods shouldn't work with the SW version of quake (just pak0.pak), yes, I remember that agreement at the time. But pak tools could open pak0.pak just fine. I've just fired up a windows vm again and pakexplr managed them both and id1.pak too. So the issue seems to be with the from-source builds of the amateur tools on a modern machine.
I think the most likely explanation for why the tools I tried failed is that they make implicit assumptions that don't hold true in my OS X environment and basically throw a wobbly, probably sizeof(int) etc. being larger than expected (not being a 32 bit system)