11-01-2022, 03:05 PM
Thanks for sharing that. Its completely normal to have to configure the buffer size of the sending software to the size the MIDI device requires. Any patch management software like Patch Base must comprehend the correct size chunks for each device they support, and for each type of data they can transfer.
More boring details for those that are interested...
Having a "global" sysex size wouldn't work for most devices because most devices, especially of the vintage variety, have a specific maximum size chunk of sysex they can consume. That size can be different depending on what kind of data is being sent. As an example, a DSI Poly Evolver wants the software sending SysEx to send 288 byte messages for patches, 1024 byte chunks for their "combo patches" file, and 128 byte chunks for a "Main" firmware update. Unlike the CZ, the Evolver has no flow control or ack.
The DSI Evolver "All Programs" sysex file that contains all four banks of stock patches has a leading F0 (start of sysex message) and trailing F7 (end of sysex message) for each patch in the file, so you could say its pre-chunked up into 288 byte messages. While the Evolver's 151KB "Main" firmware file has an F0 at the beginning and F7 at the end. They don't chunk it up in the file, so the sending software has to send it in 128 byte chunks (according to DSI's instructions). Sending the firmware in larger chunks would probably over-run the Evolver's input buffer, or exceed the speed at which they can write the data to flash storage.
All this to say, how SysEx is formatted is left to the device maker, and as a MIDI router, all we know is there's an F0 at the beginning, a bunch of data then an F7 when its done. The reason that a MIDI router has to care about this at all is because its doing much more processing than a 1 input, 1 output USB MIDI interface that just shoves the bytes through; it must examine every incoming byte of MIDI data; for filtering, to mix SysEx with MIDI real time data or merged data (and not other data), and it must have some "policy" about what gets prioritized when large chunks of SysEx data is sent to a merged output.
That policy for large SysEx is documented in the MRCC 880 User Guide.. here's a excerpt.
"Care must be taken when sending SysEx messages larger than 128 bytes while merging MRCC 880 inputs. When merging inputs, small SysEx messages, up to 128 bytes long will merge nicely with Realtime and other MIDI data, minimizing delay of Note On and Off messages. However, MRCC 880 will give priority to SysEx data exceeding 128 bytes, potentially blocking MIDI data on the other merged inputs."
In practice, when we tested sending moderately larger than 128 byte chunks of SysEx while merging, we did not "hear" delays or block data. It works as seamlessly as a 1 in, 1 out MIDI interface. But if you send a 100K+ SysEx firmware update, it will block other MIDI messages at some point when all the buffers from merged inputs are full. I don't know why anyone would be jamming while sending a firmware update or patch dump, but our advice is, don't do that. For a critical SysEx transfer like a firmware update or full patch dump, turn off other MIDI controllers so they aren't sending clock or active sense messages, etc. Or temporarily un-route inputs other than the one input receiving SysEx. Though it would probably be ok to let the MIDI router block this excess data, its probably a good idea to be cautious in this scenario.
More boring details for those that are interested...
Having a "global" sysex size wouldn't work for most devices because most devices, especially of the vintage variety, have a specific maximum size chunk of sysex they can consume. That size can be different depending on what kind of data is being sent. As an example, a DSI Poly Evolver wants the software sending SysEx to send 288 byte messages for patches, 1024 byte chunks for their "combo patches" file, and 128 byte chunks for a "Main" firmware update. Unlike the CZ, the Evolver has no flow control or ack.
The DSI Evolver "All Programs" sysex file that contains all four banks of stock patches has a leading F0 (start of sysex message) and trailing F7 (end of sysex message) for each patch in the file, so you could say its pre-chunked up into 288 byte messages. While the Evolver's 151KB "Main" firmware file has an F0 at the beginning and F7 at the end. They don't chunk it up in the file, so the sending software has to send it in 128 byte chunks (according to DSI's instructions). Sending the firmware in larger chunks would probably over-run the Evolver's input buffer, or exceed the speed at which they can write the data to flash storage.
All this to say, how SysEx is formatted is left to the device maker, and as a MIDI router, all we know is there's an F0 at the beginning, a bunch of data then an F7 when its done. The reason that a MIDI router has to care about this at all is because its doing much more processing than a 1 input, 1 output USB MIDI interface that just shoves the bytes through; it must examine every incoming byte of MIDI data; for filtering, to mix SysEx with MIDI real time data or merged data (and not other data), and it must have some "policy" about what gets prioritized when large chunks of SysEx data is sent to a merged output.
That policy for large SysEx is documented in the MRCC 880 User Guide.. here's a excerpt.
"Care must be taken when sending SysEx messages larger than 128 bytes while merging MRCC 880 inputs. When merging inputs, small SysEx messages, up to 128 bytes long will merge nicely with Realtime and other MIDI data, minimizing delay of Note On and Off messages. However, MRCC 880 will give priority to SysEx data exceeding 128 bytes, potentially blocking MIDI data on the other merged inputs."
In practice, when we tested sending moderately larger than 128 byte chunks of SysEx while merging, we did not "hear" delays or block data. It works as seamlessly as a 1 in, 1 out MIDI interface. But if you send a 100K+ SysEx firmware update, it will block other MIDI messages at some point when all the buffers from merged inputs are full. I don't know why anyone would be jamming while sending a firmware update or patch dump, but our advice is, don't do that. For a critical SysEx transfer like a firmware update or full patch dump, turn off other MIDI controllers so they aren't sending clock or active sense messages, etc. Or temporarily un-route inputs other than the one input receiving SysEx. Though it would probably be ok to let the MIDI router block this excess data, its probably a good idea to be cautious in this scenario.