In the past two months I attended audio only team meetings on a self-hosted Jitsi server (using a 2GB RAM server, the cheapest I could find). The experience and the audio quality was good despite participants having various connection problems (ranging from poor ADSL to poor 4G).
After discovering the difference between SFU and MCU, I was however highly motivated to try BigBlueButton and upgrade to having video in addition to audio. And dozens of participants instead of ten. So I went ahead, installed BBB and organized the team meeting on this server instead of Jitsi the next week. But it was not a good experience because:
- there were micro gaps (less than a second) in the sound
- cracks could be heard, sometime so frequently that the conversation was disturbed
- occasionally the sound was so distorted that the person had to repeat
Removing the video feeds did not improve the situation and there was less than ten of us.
I was not discouraged because people I know and trust reported using BBB successfully with video and a large number of participants (sometime over 100). I soon discovered that the machine I provisioned for the job was too small. But upgrading did not fix the problem. Looking a bit more it appeared that I was a victim of a semi-random bug that nobody seemed to fully understand.
It started in March 2020 and was still open at the time, with many reports similar to mine and also many reports of people saying they could not repeat the problem, developers included. A consensus developed around the idea that the root cause of the bug was the ancient version of Freeswitch included in BBB. It was upgraded and the bug was closed.
I was tracking this bug, patiently waiting for it to be resolved and was happy to resume using BBB. This time, instead of installing my own server, I had the opportunity to use a server known to successfully host meetings with over 100 people. And the next week, the team meeting happened on this server. Much to my surprise the problems were not as severe but they still persisted. Extracts of the recording of the meeting demonstrate:
Interestingly the micro gap is not that disturbing because it is so short. However, once you noticed it is difficult to forget about it. In the 2 second sequence above, it is audible in the avoir word and can be seen in a mixer. The light blue sound should be one word but is cut during a fraction of a second.
I have a theory that would explain why some people have a good experience and others keep having the problems described above. It could simply be because the BBB server does not mix audio properly when one of the speakers has a connection that either has high latency (>5ms) or poor bandwidth (occasionally dropping below 10Mb/s). In all the success stories I’ve read, the speakers (those broadcasting audio and video) presumably had an excellent connectivity. And the audience was muted by the moderators. It’s difficult to know for sure because none of the success story are specific about that particular constraint.
Although 150+ participants consume significant resources on the server (i.e. around 12 threads), I’ve been told that a 40+ participant meeting was held successfully on a virtual machine with 4 cores on an otherwise rather occupied server. And since the server on which the last team meeting was held had much more hardware resources than necessary, I reckon this is not the root of the problem.
It could simply be that since half of the participants have a poor connection, it creates the conditions for the BBB server to do poorly when mixing audio. When I get a chance I’ll try to conduct a team meeting with all participants having excellent connectivity and report my findings here.
To be continued!