Whitepaper - HELLO! MOVIE

IMPROVED ACCESSIBILITY IN MOVIE THEATERS AND THEATERS WITH ACOUSTIC COMMUNICATION: HELLO! MOVIE

1. INTRODUCTION (PREFACE)

Evixar Inc. (“Evixar”) creates novel technologies to provide barrier-free environments in public entertainment spaces such as movie theaters and theaters, which have been commercially rolled out across Japan since around 2020. This paper reports on the positive impacts of the barrier-free film screening and performing arts exhibition method (“Method”) that was designed by Evixar for individuals with visual or auditory impairment and offers them an audio guide that can be heard through a smartphone along with a caption guide that can be viewed through glasses. These technologies have already been widely integrated into the industry’s infrastructure. The paper also explains how the acoustic communication technology underpinning the Method works, describes the technological background of the related patents, and introduces the key features of Evixar’s new product, HELLO! MOVIE.
Since the Act for Eliminating Discrimination against Persons with Disabilities came into effect in 2016 followed by the Act on the Promotion of Measures for Improving Information Accessibility, Use, and Communication for Persons with Disabilities that became effective in 2022, public and commercial facilities have been facing a growing need to make their information environments more barrier-free for persons with disabilities who use them. In particular, in light of the amended Act for Eliminating Discrimination against Persons with Disabilities that is scheduled to come into effect in April 2024, the industry needs to reach a consensus on a comprehensive service solution that can be adopted nationwide across all films, theatrical productions, etc. and implement the solution.
The Method had to meet the two distinct requirements of reasonable accommodation and entertainment value. According to the Convention on the Rights of Persons with Disabilities, reasonable accommodation means “necessary and appropriate modification and adjustments not imposing a disproportionate or undue burden, where needed in a particular case, to ensure to persons with disabilities the enjoyment or exercise on an equal basis with others of all human rights and fundamental freedoms.”

2. BACKGROUND: BUSINESS REQUIREMENTS AND TECHNOLOGICAL REQUIREMENTS

Reasonable accommodation must be considered not only at actual service sites catering to persons with disabilities but also during the production, development, and preparation phases in order to ensure sustainable operation and effective cost control. Therefore, the requirements have been jointly examined by persons with disabilities as end users, film producers, theater companies, event promoters, experts on barrier-free design, etc.
To implement highly-scalable barrier-free design for the viewing of films, theatrical productions, etc., the experience must be delivered using a post-production, supplementary service method that is entirely separate from the films and performing arts programs. The method must also be deployed with the prior consent of the producers so that their artistic intentions and envisioned entertainment value are not compromised to the extent possible. In the film and performing arts industry, there has been a viewing option that provides audio and caption guides synced with the visual and auditory content as it unfolds. The realistic approach has been to devise a method that can widely distribute and provide a guide function without hindering the production process. However, the conventional guide service method can only be deployed using special equipment that ensures information security, which means that few facilities can adopt the method. In addition, for commercial reasons the conventional method is only available for limited periods after the films are released.
Since movie theaters and theaters are where the service is offered, the ideal method would allow both able-bodied and disabled people to simultaneously enjoy the same content in the same space once the programs are released to the public, regardless of time or facility. However, it is difficult to accurately predict the number of people with disabilities who will come to each showing and require the guide service. As the producers decide at their discretion whether to create and offer a guide service to accompany their films and performing arts programs, the guide method must allow the scale of the service to be adjusted after its actual introduction and operation.
In view of the business requirements described above, Evixar set out to identify the technological requirements of an app that would be used by persons with disabilities on the smart devices (smartphones and smart glasses) that are widely used by them, assuming that they will bring their own devices to the event venues and operate the guide app themselves. Evixar identified the following basic requirements:
– can work on widely used smart devices;
– can function with existing equipment and in the existing environment available at any movie theaters and theaters nationwide;
– is ease to use;
– can function on devices in airplane mode with all telecommunication functions disabled; and
– the guide function can only be used in movie theaters and theaters.
As for the caption guide, Evixar decided to limit the use on smart glasses so that light leakage from the devices would not affect other audience members sitting nearby.

3. EVIXAR’S PROPRIETARY ELEMENTAL TECHNOLOGIES OF AUDIO FINGERPRINT, AUDIO WATERMARK, AND ACOUSTIC COMMUNICATION

Evixar’s proprietary acoustic communication technology combines the elemental technologies developed by Evixar, which are called audio fingerprints and audio watermarks, that are related to acoustic signal processing. The Method controls the users’ and audience-space devices in sync with the program as it progresses, which is determined by the auditory data as the content unfolds. This syncing function is achieved by collating the time data of the unfolding scene with the time data of the audio and visual guide program without communicating with any external server, etc.
Audio fingerprinting is a technology that operates on teacher data comprised of the signal-processed and encoded features of the predetermined target sounds, and matches them to the encoded external sounds that are acquired through the devices’ microphones, etc., to determine their identicality. One of the advantages of this technology is that it cannot be misused in violation of privacy or copyright, as it does not track the film or performing arts contents, and the data, once encoded, are no longer decodable, making it easier to obtain permission from film producers and production companies when negotiating the terms of their business use. Evixar’s proprietary algorithm has been developed mainly for delivering the guide function through smartphones and smart glasses and is highly robust. Its key features include the ability to detect audio signals that are emitted from loudspeakers, transmitted through the air, and captured by the devices’ microphones.

Audio watermarking is Evixar’s technology of embedding text data, etc. that have been converted from encrypted audio signals. Evixar’s proprietary algorithm provides superb media durability, secrecy protection, reverberation, and noise resistance, and delivers proven resistance to deterioration in sound quality. Hence, it can successfully decode the audio signals emitted by the loudspeakers, transmitted through the air, and captured by the devices’ microphones. This, for example, enables broadcast-type communication between the loudspeakers that already exist at entertainment facilities and the audience-space microphones, even in environments where there is no radio wave or wireless communication technology.

Evixar delivers highly distinct communication solutions by optimally combining the two elemental technologies described above to meet the business and technological requirements of each project.

4. TECHNOLOGY IMPLEMENTATION IN MOVIE THEATERS

The Method can be deployed merely by using a special app called HELLO! MOVIE (https://hellomovie.info/ – in Japanese) installed on the users’ devices, to which audio and caption guide data created in advance is securely distributed over the internet. HELLO! MOVIE is available to the public and can be downloaded at any time for free, and used on iOS, Android, and smart glasses. The app also has an operability check function that can be used on the way to a movie theater, along with a search function to find out which films are currently screening barrier-free (22 out of the 26 Japanese films that were released in 2022 and achieved at least 1 billion yen in ticket sales were HELLO! MOVIE-adapted). If the app detects the auditory data of a film being screened in a movie theater, which is unavailable elsewhere, the guide function is activated only for those users who have purchased tickets. This unlocking process is achieved with the audio fingerprint technology explained earlier.

For persons with disabilities, being able to watch the latest box-office hits alongside able-bodied fellow movie-goers is itself a novel value proposition for their entertainment experience, which also produces positive knock-on effects on their families and friends. The Method enables the barrier-free screening of films from the first day of their release in any movie theater and in any seating position, through the guide data playback function of the app installed on smart devices that are also commonly used by persons with disabilities. As for smart glasses, which are not as widely used as smartphones, Evixar has been making them available for rent with a growing number of movie theaters, so users can go to their service counters to rent them. (This rental service is offered by over 80 movie theaters across Japan as of February 28, 2023.)

Furthermore, to meet the production and operational requirements, audio watermarking is used to sync with the video content that is played before each film starts, while audio fingerprints are used for syncing with the main film content. These two distinct sync techniques have two objectives: to start the syncing immediately after the main film starts, and to completely separate the main film production from the audio and caption guide production employing the Method.

5. TECHNOLOGY IMPLEMENTATION IN THEATERS

Theaters where musicals, plays, and other theatrical productions are performed, mainly use the caption guide function integral to the Method. The guide text is produced in advance by observing the scripts and practice sessions and is transmitted to the special app installed on the users’ devices, which is similar to how the app is implemented in movie theaters. However, unlike films, which are commonly referred to as complete packages, theatrical productions unfold on a time schedule that varies slightly from one showing to the next, which requires time adjustments on the guide data playback function in real time. Theaters’ business and technological requirements for achieving the same barrier-free viewing experience across all showings also mean that making such time adjustments for each showing must not require extra labor. In addition, theaters use radio communication to operate certain stage devices and interference is not permitted, so the use of wireless communication on the audience-space devices must be avoided.
To solve this issue, Evixar adjusted how the technology is implemented on films, and embedded audio watermarks in the sound effects and BGM of theatrical productions. This approach allowed Evixar to embed in the theatrical production a function to correct in real time any temporal asymmetry that had occurred between the on-stage performance and the caption guide playback. This effort paid off, as the Method could then be implemented on long-running productions. One example is the Disney musicals produced by Shiki Theatre Company, which uses the Method to provide a caption guide service to the hearing impaired and also a foreign language caption service to those not proficient in Japanese.

6. CONCLUSION

The key advantage of the Method and Evixar’s new HELLO! MOVIE product that implements it is the ability to deliver the guide function as a post-production, supplementary service, by harnessing acoustic communication technology, without imposing any new burden on the conventional production flow or requiring any additional investment in facilities and equipment. As of February 28, 2023, the HELLO! MOVIE app had been downloaded over 300,000 times, while the caption glasses had been rented over 13,000 times, establishing the Method as a crucial component integral to the industry’s infrastructure.
As for Evixar’s audio and caption guide service, it could be implemented for wider applications to broaden the user base, by catering to the needs of multilingual services and with better contents planning, featuring commentaries on the backstories of films and theatrical productions, how the world would be different in their eras, etc. Furthermore, assuming that the technologies found in users’ devices will continue to evolve, Evixar’s innovative solutions will likely be applicable to immersive theaters (where audiences experience theatrical performances while being immersed in them) and other next-gen entertainment experiences that will be rolled out in the years to come.
Moreover, according to the 2022 Annual Report on the Aging Society issued by the Government of Japan, it is estimated that population aging will sharply increase over the next forty years, as the percentage of individuals aged 65 years and older among the country’s entire population was only 5.1% in 1950 but rose to 9.3% in 2020 and is projected to jump to 17.8% in 2060. Needless to say, as the country’s population ages, the number of persons with visual and auditory disabilities will also likely rise.
Evixar intends to step up its business development efforts by proposing solutions to Hollywood-based film studios in the U.S. and other entertainment event promoters around the world, building on its successes as described in this report and leveraging its registered international patents on the acoustic-communication-based synchronization technologies.

Sample media coverage of Evixar’s technologies

Related commendations