My guess (no more than that) is that the XACT convertor has come across something it doesn't like in the project. Try replacing it with a super-simple (or empty) project and making sure that works OK. If it does, something in your project must be doing something the converter doesn't expect.
It won't be size in itself at issue though - XACT has been used before with hundreds of banks and tens of thousands of waves. Sixteen banks and a couple of hundred sounds really isn't "big".
XapParse, a parser for XAP (XACT) files