At dusk, a car stops at a checkpoint in Afghanistan. It is a
tense moment for all. Because an interpreter is not available, U.S.
Marines use hand gestures to ask the driver to step out of the car
and open the trunk and hood for inspection. There’s a lot of room
for error.
This scene was re-enacted recently during an evaluation at
the National Institute of Standards and Technology
(NIST)—but, this time, the Marine had a new smart phone-based
device that translates his English into the driver’s native Pashto
and the Pashto back into English.
For the past four years, scientists at NIST have been
conducting detailed performance evaluations of speech translation
systems for the Defense Advanced Research Projects Agency (DARPA).
Previous systems used microphones and portable computers. In the
most recent tests, the NIST team evaluated three two-way,
real-time, voice-translation devices designed to improve
communications between the U.S. military and non-English speakers
in foreign countries.
Traditionally, the military has relied on human translators
for communicating with non-English speakers in foreign countries,
but the job is dangerous and skilled translators often are in short
supply. And, sometimes, translators may have ulterior motives,
according to NIST’s Brian Weiss. The DARPA project, called TRANSTAC
(spoken language communication and TRANSlation system for TACtical
use), aims to provide a technology-based solution. Currently, the
focus is on Pashto, a native Afghani tongue, but NIST has also
assessed machine translation systems for Dari—also spoken in
Afghanistan—and Iraqi Arabic.
All new TRANSTAC systems all work much the same way, says
project manager Craig Schlenoff. An English speaker talks into the
phone. Automatic speech recognition distinguishes what is said and
generates a text file that software translates to the target
language. Text-to-speech technology converts the resulting text
file into an oral response in the foreign language. This process is
reversed for the foreign language speaker.
NIST researchers held focus groups with U.S. military
personnel who have served overseas to determine critical
communication interactions to simulate and evaluate in tests. The
research team then devised 25 scenarios for evaluating the
performance of translation devices. These included vehicle
checkpoints; communication of key information, such as how long
electricity will be available each day; facility inspections;
medical assessments; and Afghani-U.S. military training exercises.
Marines experienced in these tasks and native Kandahari-dialect
Pashto speakers acted out the scenarios without a script. Each
scenario was performed using the three industry-developed
translation devices.
 |
||||
For each test, on-site judges observed the scenarios, and the
participating Marines and Pashto speakers were surveyed about the
ease of interaction with the systems. Later, a separate panel of
judges fluent in English and Pashto viewed videos of the exercise
and evaluated each of the three systems in terms how accurately
concepts were communicated in both languages, Schlenoff
says.
“We are writing a detailed assessment of the evaluation for
DARPA so they can make an informed decision to determine where to
direct funds and efforts in the TRANSTAC project,”says
Schlenoff.
