Heya! First off, woah! What a cool project; looks great!
Looking at the TDM format as shown in the Datasheet, it looks like this would be totally possible. Depending on the Peripheral architecture of the teensy, it may not be trivial to port the driver, but SAI2 can be configured for 8 slots of 32 bits (256 bits total per frame) for each channel. The limitation being that it can be a maximum of 256 bits in a frame.
Setting up that configuration shouldn’t be too hard, and then it looks like either SPI or I2C can be used to control the codec.