Just in case someone else hits this issue I'm amending my post with the answer. I looked at the DSP program and saw that there is a threshold for toslink. Then I found the setting of that threshold in the source code. There is code in the toslink extension (beo-extensions/toslink/index.js) that maps the three available values for 'Sensitivity' to dBFS values. These are: low: -20, medium: -40, high: -60. For my purposes I updated the code to map High to -100. Not sure why I wasn't seeing this before but I must have altered some other default in my code to cause my tone generation levels to fall below that level of -60.
==================================================
I have a DAC+DSP setup that I've been working with for a few months. Until yesterday I was not having any problems with sending digital audio of any dBFS level to the optical input of the DAC+DSP and seeing the data at the optical and analog outputs. Then yesterday, inexplicably, I started to notice some behavior where the optical input would go into an 'inactive' state if the digital audio level was not at least -65.9 dBFS. As soon as the signal goes from -65.9 dBFS to -66 dBFS the sound stops and the optical input shows as inactive. The sound stops both at the optical output as well as the RCA analog outputs so it appears to be an issue on the optical input side. Once the problem cropped up I took another RPI4/DAC+DSP setup that I have and tested it and I see the same issue. I also did a test with my board after booting it from a backup that I have from February. Undoubtedly this issue is due to something in my environment but I have been unable to track it down. Besides trying different hardware I have tried the following:
1) Reset the dsp using 'dsptoolkit reset' and reinstall the default version 12 DSP profile.
2) Send the digital audio from my REW-based signal generator directly to my DAC via toslink optical, so the DAC+DSP is completely out of the picture. No problem.
3) Send the digital audio to the DAC+DSP via Airplay. No problem except the expected latency when the audio is started/stopped.
Any ideas about resolving this issue would be much appreciated.