News

Oxford Develops AI System for Significantly Improved Lip-reading

Using thousands of hours of BBC News programmes including Breakfast, Newsnight, Question Time and more, scientists at Oxford have developed an artificial intelligence system that can lip-read better than humans. The project has been developed in collaboration with Google’s DeepMind AI division, dubbed the “Watch, Attend and Spell” system, now boasts a 50% lip-reading hit-rate.  In comparison, the same lip-reading tests shown to professional lip-readers only have an accuracy of 12%.

The neural network using speech recognition and image recognition algorithms is able to gather 17,500 words for its vocabulary from examining 118,000 sentences in the clips. Since it is fed mostly news programmes, it has grown to associate the likelyhood of certain words following another in a particular topic such as “minister” after “prime”, but this also means that it is limited and cannot recognize many words that are not spoken by newsreaders.

As good as the system is now, the researchers still expect a lot of work needed before it can be put to practical use. Many groups, such as those who are advocates of the hearing impaired, are very excited about the development however.

“AI lip-reading technology would be able to enhance the accuracy and speed of speech to text,” says Jesal Vishnuram, Action on Hearing Loss technology research manager. “This would help people with subtitles on TV, and with hearing in noisy surroundings.”

The next objective for the Oxford researchers, is to make the system work in real time. For now, it is only able to operate on full sentences from recorded video. According to Joon Son Chung, a doctoral student at Oxford University’s Department of Engineering, this is actually a simpler task than refining the accuracy of the AI system, so it is not as challenging as it sounds.

Ron Perillo

Disqus Comments Loading...

Recent Posts

BenQ MOBIUZ EX2710Q 27″ QHD 165Hz 1ms, HDRi IPS Gaming Monitor

SpeakersSpeakersYesSpeaker amount and power output2x 2 WattDimensionsLength / Depth252.5 mmWidth614 mmHeight525.8 mmWeight7.4 kgStandards / SpecificationsAdaptive…

3 hours ago

Intel Core i7-12700KF 3.60GHz Socket LGA1700 Processor

Thermal SpecificationsMax. TDP125 WCPUCPU ManufacturerIntelCPU SeriesIntel Core i7CPU Socket1700CPU ArchitectureIntel Alder Lake-SCPU Cores12CPU Threads20Performance Cores8Efficiency…

3 hours ago

AOC 24B3HA2 24″ 1920×1080 VA 100Hz 1m Widescreen LED Multimedia Monitor 

AOC 24B3HA2 23.6 1920x1080 VA 100Hz 1m Widescreen LED Multimedia Monitor - Black High-performance clarity…

3 hours ago

Corsair Hydro Series iCUE Link H115i RGB Performance Liquid CPU Cooler

Fan SpecificationsFan Size140 mmColourPrimary ColourBlackSecondary ColourWhiteMaterialsMaterialsAluminium, Copper, RubberLightingLightingYesLighting ColourRGBLighting CompatibilityCorsair iCUEAdditional ContentsIncluded fans2x 140 mmTypeCPU…

3 hours ago

Philips Evnia 34″ 34M2C6500/00 3440×1440 QD-OLED 175Hz 1ms FreeSync Curved Ultrawide Gaming Monitor

This monitor is built with features that make incredible visuals. With VESA ClearMR 9000 and…

3 hours ago

Asus Radeon RX 7900 XTX TUF OC 24GB GDDR6 PCI-Express Graphics Card

The AMD RDNA™ 3 Architecture elevated by buffed cooling and power delivery to effortlessly churn…

3 hours ago