The development of automatic speech recognition able to perform well across a variety of acoustic environments and recording scenarios on natural conversational speech represents one of the biggest challenges in speech recognition research and development. Previous work in the literature has shown that automatic speech recognition (ASR) performance degrades in microphone recordings especially when data used for training is mismatched with data used in testing. The Intelligence Advanced Research Projects Activity (IARPA) is seeking to identify approaches to mitigate the effects of this mismatch by running this Automatic Speech recognition in Reverberant Environments (ASpIRE) Challenge.
This is a Reduction-to-Practice Challenge that requires written documentation and delivery of output from the Solver’s automatic speech recognition system applied to supplied evaluation data. The Seeker does not wish to obtain IP transfer or licensing of solutions and seeks only to identify the leading systems and Solvers in this field. Additionally, as a Prodigy Challenge a real-time online scoring utility and leaderboard will be available to track Solver performance for this Challenge.
Evaluation will be performed with both single microphone and multiple microphone data. There will be separate monetary awards given to the best system in the single-microphone ($30,000) and the multi-microphone ($20,000) conditions. The winner in each condition must achieve a word error rate (WER) that is at least 1% lower than the performance levels attained by the second best system to win.
The IARPA ASpIRE (Automatic Speech recognition in Reverberant Environments) Challenge, a spin-off of the IARPA Babel program, seeks to identify approaches and technology for automatic speech recognition (ASR) under less than ideal conditions. Previous work has shown that ASR performance degrades in microphone recordings, especially when data used for training is mismatched with data used in testing.
The ASpIRE challenge asks solvers to develop innovative speech recognition systems that can be trained on conversational telephone speech, and yet work well on far-field microphone data from noisy, reverberant rooms. Participants will have the opportunity to evaluate their techniques on a common set of challenging data that includes significant room noise and reverberation. Whereas the Babel program seeks to develop agile and robust technology that can be rapidly applied to any human language, this Challenge focuses on English language speech recognition.
The Seeker (IARPA) asks Challenge Solvers to create innovative automatic speech recognition software that works in a variety of acoustic environments and recording scenarios without having access to matched training and development data. There are two evaluation conditions:
In both conditions, word error rate (WER) will be used as the objective measure of performance. Solvers can participate in either or both conditions. There will be separate monetary awards given for the best system in the single microphone ($30,000) and the multiple microphone ($20,000) conditions. The winner in each condition must achieve a WER that is at least 1% lower than the performance levels attained by the second best system to win.
Separate submissions are required for the single microphone and multiple microphone conditions and each must include the following:
If you are selected for validation in either condition, you would also be required to either:
The award is contingent upon validation of the submitted Solutions by the Seeker or their representative. To receive an award, Solvers will not have to transfer their IP rights or grant a license to the Seeker – the purpose of the Challenge is to gauge how far recent advances in speech recognition have come in solving this important problem. With broad participation, this Challenge has the potential to provide IARPA with insights on the best next steps to stimulate research for solving this challenging problem.
By making a submission to this Challenge, Solver(s) are providing written consent to the Seeker to publicly disclose their identity if their submission is chosen for an award and they choose to accept the award. Disclosure will not occur until the Challenge has been fully completed, that is, all submissions have been evaluated, rejected or accepted, and any awards have been transferred to the Solver(s).
Federal entities or Federal employees acting within the scope of their employment are NOT eligible to participate in this Challenge. Please note that winners will have to submit an Academic Institution Acknowledgement Letter acknowledging the role of IARPA in this Challenge if you are: (i) a U.S. Academic Institution, (ii) an employee of such institution who is participating on behalf of that institution, or (iii) an employee of such institution who is participating in their personal capacity if they are using the resources of such institution to respond to this Challenge. Entities or employees of entities from the following countries subject to U.S. economic sanctions are not eligible to participate in this Challenge: Iran, Syria, Sudan, Cuba, and North Korea. In addition, individuals and entities listed on the U.S. Government’s Consolidated Screening List (available at http://export.gov/ecr/eg_main_023148.asp) are not eligible to participate in this Challenge.
About the Seeker
This Challenge is sponsored by the Intelligence Advanced Research Projects Activity (IARPA). IARPA invests in high-risk, high-payoff research programs that have the potential to provide the United States with an overwhelming intelligence advantage over future adversaries. Opinions, interpretations, recommendations and conclusions are those of the authors and are not necessarily endorsed by the United States Government.
What is an RTP Challenge?
An InnoCentive RTP (Reduction to Practice) Challenge is a prototype that proves an idea, and is similar to an InnoCentive Theoretical Challenge in its high level of detail. However, an RTP requires the Solver to submit a validated solution, either in the form of original data or a physical sample. Also the Seeker is allowed to test the proposed solution. For details about treatment of IP rights, please see the Challenge-Specific Agreement.