Realtime Music Solutions May Have Your Distance Rehearsal Solution Today

For over two decades, New York City-based Realtime Music Solutions (RMS) has been delivering solutions for rehearsing music to the musical theater market. Through software products, RMS has helped countless national, regional, and local theater productions, in addition to serving the school musical production community. And like everyone, when COVID-19 hit, their business was hit hard. No productions means no rehearsals.

But as SBO covered last month (and will write about again next month), companies needed to reinvent themselves to try to solve an urgent need to bring together educators struggling to find a way to continue to teach instrumental and vocal music. This gave birth to a new product, Rehearsal Live Share (RLS), in constant development since release May 2020.

RLS is a distance rehearsal system using audio and video, but one that does not focus on the monstrous latency challenge, instead putting tools in the hands educators to get everyone on video, start their performance track playback parts for the group, have the students perform their part, which is recorded and sent back to the director in sync. The director can take all of the performed parts, mix them as needed and send it back to the musicians so they all then hear each other’s performed parts, and allows the director to minus any part that musician needs removed. It is a fascinating approach to the distance rehearsal solution. I was able to watch a screen captured rehearsal of a choral group, courtesy of their director, Dr. Richard Zielinski, professor, director of choral activities, University of Oklahoma, School of Music. It works.

SBO spoke with Jeff Lazarus, CEO, and David Smith, director of research and development, both founding partners, to learn more.

Please share with our readers the background of RMS.

Jeff: Our focus was the theater industry providing various technological resources. We also work with community theaters, colleges, high schools, middle schools putting on their school musical.

RLS is currently an offshoot feature of existing products, RMS Mix and RMS Coach.

Jeff: At the moment, the feature is embedded into those apps: RMS Coach, which lives in the MIDI world, and RMS Mix, which lives in the audio world. We are building a standalone RLS app. Prior to COVID, we would develop a show, say, “Oklahoma!” and the rendering would be based on the piano vocal score sent out by the licensing company. When you are producing at your high school, this [MIDI] product, RMS Coach, could be distributed across cast members. They use it to run rehearsals independently or drill at home. Each part is broken out by the vocals. If you have a choral number, and it’s split out amongst the lead singers in that particular song plus to soprano, alto, tenor, bass chorus. Each of those parts have an individual track, which could be manipulated according to user needs. If you are an alto, you need to learn your parts, you start out by muting out all the other parts, buff the alto and the piano, so it serves as a very standout guide track. You might slow it down to get some tricky rhythms you need to master. As you get more comfortable, add the other parts back in. When you get even more comfortable, mute your part altogether so you can hear how you hold your own with the harmonies. On RMS Mix, it is standard audio. That could be a full orchestra background track, or an acoustic piano quickly laid down by the music director.

David: The overall goal [of RLS] is that the director does not have to think any differently about the rehearsal process. They can handle the rehearsal in the traditional way and there’s someone that is controlling the other system that performs that particular interface.

To keep yourself a viable company in this trying era, is it fair to say this is almost the triage approach to programming to meet this urgently needed solution?

Jeff: That’s absolutely right. I mean, since our first released RLS in May, we have pretty much had a major new feature come out in addition to just bug fixes but adding to the scope of what the product does every week. I don’t see that stopping anytime soon. We’re going about it hard charging, which means sometimes… we are a little bit more aggressive than we would be in the pre-COVID days.

Many these days are, but certainly not all of our music educators are tech-savvy. How do you create a product that is easy enough for the five-year-away-from-retirement music teacher to quickly learn how to use?

Jeff: We are a very tech support-centric company. We have spent a ton of time working with that exact constituency, the five years from retirement, where we will get on a session with them maybe with 10 or so students that are gonna be part of the rehearsal. We want to make it so that it is intuitive and don’t have to think about, “How do I this?” I’ll give you an example that we’re grappling with at this very moment. Recently, we just released the first multichannel recording version of RLS with the idea, “here is your auto session,” let’s say with 10 people. Each of the participants, in this case, one director plus nine participants. The nine participants, as they are running through a song, they’re gonna capture an audio file of that take. And that can then be assembled as a full mix by the director for any number of users, whether it’s to edit in Pro Tools or Logic or return to the ensemble as a mix. It then becomes the basis for subsequent rehearsals. Now, that automation is yet to be done, you know. We’ve got a lot of cool ideas about how we’re gonna do it. Where, basically, as soon as you finish the recording, it automatically gets shared over the cloud to all participants or at the director’s discretion. Maybe the director will want to pick and choose between takes and then automatically send it back to everybody and hit play and it’s just all done for you under this, under the hood. But that’s gonna take a little time to get into place. Right now, the basic recording feature is there, but it’s a little buried, and we have to talk with the teachers about where to go ahead and find those recordings and how to go about collecting it for further use. It is admittedly not as user-friendly as we want it to be. But the choice was, do we wait, and only release it when all those other aspects are in place? And, I suppose there’s an argument for that, but we came to the conclusion that it was better to get it out sooner, even if it’s a little more difficult to work with right now.

David: I think also that our early adopters that have been working with us appreciate the fact that we’re in very close touch with them if there’s a rough edge. They know that we’re there for them, and often, I’m sitting in with the rehearsals to troubleshoot. That also gives us the feedback because we’re seeing where the rough edges are and seeing where they struggle.

There was no video involved in the previous products, correct? And no return of MIDI or audio to the director?

Jeff: Correct. There was no video before [and no parts recorded and returned to the conductor]. The idea is anybody could use it, but they were working in isolation with the contents by themselves.

What is the minimum standard for optimum performance experience by the director in a Mac or Windows hardware setup to run this?

Jeff: There is the director role and the participants. The director is the one in charge [and] running the session. The participants are everyone else. There are different requirements depending on your role. The director carries the heaviest load and will need the best computer. The most important thing is to have at minimum a four-core computer. The number of cores becomes an important factor. If you have more than four, then that just means you have that much better chance of running an even larger session. On the Mac side, the minimum OS is High Sierra 10.13.

David: Or Windows 8.1 or higher.

Jeff: I’ve been primarily testing with a relatively old four-core iMac and I’ve run 20-person sessions on it.

David: Participants can get by perfectly well on a two-core machine. A director might be able to run a two-core computer with only a couple of participants. If you’re dealing with two, maybe three, you can probably get away with a two-core. We highly recommend the four-core or higher for the director.

What are the practical limits today of participants including the conductor?

David: We’re up to 20 now and we feel that we could go more. The only reason that we haven’t raised the level above 20 right now is because we haven’t put together more than 20 people to test ourselves. So, I have choral directors with courses of 50 and 80. We don’t think at this moment that we’ll be able to get up to 80 on a regular four-core machine, but we feel comfortable that we’ll be able to get up to 25 or 30. And then on top of that, we have some ideas that are not far behind the completion of the recording-sharing feature that would allow us to serialize the playback. And at that point, you can take a group of 30, munch them into one, and then you can have 30 groups of 30.

One of the things that I think differentiates us from the other solutions is that every other solution that we’re aware of, and of course you know there may be other people working at it with a similar model as we are, is basing their solution on lowering the latency, trying to get the latency below 30 milliseconds. We didn’t want to get into that particular area. We recognize that with 20 years of this sophisticated technology that is really rock solid, under the hood, that this gave us a starting place that we have some technology capabilities that other people don’t have.

It is a Wi-Fi world. Many think it’s their bandwidth or computer causing problems with audio/video over the Internet when what they like need is a cable sending consistent data packets. Is WiFi usable with this solution?

Jeff: We prioritize sync over latency. We don’t care about the latency, as long as all the signals come together in sync. If the director ends up getting the whole signal back a second late, it’s not bad thing, but let’s just take that as an extreme example. If they get it back a second late, as long as everything together is at the exact same second late, they hear a complete ensemble sound. Our approach is we have an underlying accompaniment, whether that’s MIDI in RMS Coach or standard audio in RMS Mix. There is a set accompaniment that lives locally for everyone. The director will hit play, and that play signal gets broadcast to all the participants and hits play on the local machine. It doesn’t matter if all those participants are slightly out of sync from each other, when that signal then goes back to the director, that information gets encoded with the proper metric information so that packets are aligned for the director, and the director receives a synchronized feed. We run a lot of our stuff on WiFi. WiFi is not a problem for us, particularly for the participants. But we’ve run large sessions, eight, 10,12 with the director on WiFi as well.

David: If you’re a participant, you need three megabits per second. Most WiFi gets to you up to about 20, 30 is a normal thing. So even in a degraded environment, you have plenty of bandwidth in order to deal with what we need. The director side, it becomes additive. If you have a good, robust WiFi connection with high-speed WiFi, you can probably easily get eight, nine, 10 people onto that WiFi connection. The important thing to remember is that no one can hear each other in the present, for that present layer, whatever it is, but you can hear anything from the past.

What we’re looking is serialization. What we want to do first is to be able to be as broad as possible with a single-parallel version of this. How many people can we get up to simultaneously? And then, you start to layer additional serial on top of that. And then, we have this other advantage because if we deal with the recording sharing, even though you may be singing by yourself with guide tracks that provide you with your pitch and your rhythm and everything that is important for your performance, the very next time, you can be performing with the last performance [of your fellow musicians]. You can get all the stems of every single person that was playing the last time. If you’re gonna say, “Okay, let’s do it again,” these are now shared. And then, the next time you’re playing, you’re playing with the previous performance.

We want to be in the same room. We want to hear everybody. We want the conductor to be able to perform the beats and then adjust the tempo and things like that. With latency, there are things that you cannot do there. And so we recognize that this approach, which is quite different from what the other solutions are trying to impose, gives us sort of an intellectual space that we can really push a product that is not only gonna be useful for the rehearsal within the age of COVID, but will become a quite powerful support rehearsal and also performance-in-production tool after COVID.

What you’re offering seems extremely practical in a structured music instrumental and voice educational environment. What does this cost?

Jeff: We currently charge the director, not the students. The director gets charged a subscription fee of either $9.99 a month or $99.99 a year.

The big question — What about Chromebooks? iOS devices?

Jeff: A lot of schools have asked us. The Chromebook version of RMS Coach is there. There is value in rehearsing on your own with the RMS Coach content. We just don’t have the Rehearsal Live Share components integrated into Chromebook yet. It is on our list. It is something that is definitely going to happen. I would say, in 2021, we’ll get to it. I think we have reasonable odds of having the first iOS version within a month.

David: Well, we’re already in the Apple world you know with school’s macOS. So, what that means is that Apple has a much better way of integrating these materials together. There are things, because so many people want to have both out of the iOS as well as the macOS version so that you can have onto your phone. We’ve found with our internal initial polling, that about 75 percent of people would rather have it on the iPhone before having it on a Chromebook, even though we get a lot of Chromebook questions. And Android as well.

• Basic RLS session with MIDI accompaniment in RMS Coach

• A cappella RLS rehearsal session example in RMS Coach

• Standard audio RLS session with multichannel recording in RMS Mix.

Learn more at rms.biz