Voice authentication has emerged as a convenient biometric security mechanism, deployed across smartphones, voice assistants, and financial applications. But what if an attacker could bypass these systems using nothing more than a photo from your social media profile? This article examines "Can I Hear Your Face? Pervasive Attack on Voice Authentication Systems with a Single Face Image" by Jiang et al., published at USENIX Security 2024, which introduces Foice—a framework that synthesizes voice recordings from facial images to attack speaker verification systems.
The Voice Authentication Landscape
Voice authentication systems operate in two phases:
- Enrollment Phase: User speaks a prompted phrase; system extracts voice signature
- Authentication Phase: User speaks again; system compares voice characteristics against stored signature
These systems power security features across:
- WeChat: Voiceprint login for 1.3 billion users
- Apple Siri: "Hey Siri" voice recognition
- Google Assistant: Personalized voice matching
- Samsung Bixby: Voice-based device control
- Banking Apps: Voice verification for transactions
The Audio Deepfake Threat
Traditional voice cloning attacks require audio samples of the target speaker. Systems like SV2TTS can synthesize speech from a few seconds of reference audio. However, obtaining voice samples requires either:
- Physical proximity to record the victim
- Access to videos/calls featuring their voice
- Social engineering to elicit voice responses
The key question: What if the attacker has no voice samples at all—only a photograph?
The Face-Voice Connection
The Foice attack exploits a fundamental physiological insight: facial structure correlates with vocal characteristics.
Why Faces Reveal Voice Information
Voice production depends on physical structures that are partially visible in facial appearance:
| Structure | Voice Impact | Facial Correlation |
|---|---|---|
| Nasal Cavity | Resonance, nasal quality | Nose shape, bridge width |
| Oral Cavity | Articulation, vowel sounds | Mouth shape, lip thickness |
| Larynx | Pitch, vocal range | Neck structure (indirect) |
| Vocal Cords | Fundamental frequency | Gender, age indicators |
Additionally, demographic factors visible in faces—gender, age, ethnicity—strongly influence voice characteristics like pitch range, speaking rate, and timbre.
Voice Feature Decomposition
Foice models the voice feature vector as two components:
- Face-dependent features: Characteristics predictable from facial structure (gender, age, resonance patterns)
- Face-independent features: Characteristics not correlated with face (accent, speaking habits, emotional state)
The attack generates the face-dependent portion from photos, then samples the face-independent portion from a learned distribution.
Foice Attack Overview
The attack proceeds in four steps:
Step 1: Information Gathering
Attacker obtains:
- Victim's face image (social media profile photo)
- Account identifier (phone number, email)
Step 2: Voice Synthesis
Foice generates N synthetic voice recordings from the face image, each with different sampled face-independent features.
Step 3: Authentication Attempts
System presents random digit sequence (e.g., "72608594"). Attacker plays pre-generated recordings through speaker.
Step 4: Iterative Bypass
Since many systems don't limit authentication attempts, attacker iterates through N recordings until one passes verification.
System Architecture
Foice consists of three main components trained on paired face-voice datasets:
Data Processing
Face Processing:
- Face detection and cropping
- Normalization to standard dimensions
- Blurriness assessment (quality score > 100 to pass)
Voice Processing:
- Speaker encoder extracts voice feature vector
- Features split into face-dependent and face-independent portions
Face-Dependent Voice Feature Extractor
Learns to predict voice characteristics from facial images:
Training objective:
The model learns correlations between:
- Jawline shape → Vocal tract resonance
- Mouth/lip structure → Articulation patterns
- Age indicators (wrinkles) → Pitch characteristics
- Gender features → Fundamental frequency
Face-Independent Voice Feature Generator
Models the distribution of non-face-correlated voice features:
Training:
Key design choices:
- Bottleneck (B): Projects to lower-dimensional space following Gaussian distribution
- Reconstructor (R): Combines face-dependent and face-independent features
- KL Regularization: Ensures face-independent features can be sampled from standard normal
Attack Phase Sampling
During attack:
- Extract F_dep from victim's face image
- Sample N vectors from N(0, I) for face-independent features
- Generate N complete voice feature vectors
- Synthesize N voice recordings using TTS
Experimental Setup
Target Systems
| Category | System | Type | Commercial/Academic |
|---|---|---|---|
| On-Device | Authentication | Commercial | |
| Siri | Voice Assistant | Commercial | |
| Google Assistant | Voice Assistant | Commercial | |
| Bixby | Voice Assistant | Commercial | |
| Cloud | Microsoft API | Authentication | Commercial |
| iFlytek API | Authentication | Commercial | |
| VGGVox | Authentication | Academic | |
| DeepSpeaker | Authentication | Academic |
Datasets
- VoxCeleb1: 100K videos, 1,251 celebrities (evaluation)
- VoxCeleb2: 1M videos, 6,112 celebrities (training)
- Custom: 10 participants with controlled recordings
Metrics
- Overall Success Rate: Percentage of speakers with at least one successful attack
- Individual Success Rate: Average percentage of successful attempts per speaker
- Foice vs. SV2TTS: Comparison with voice-sample-based deepfake
Results
On-Device Systems
| System | SV2TTS Success | Foice Success | Avg Individual SR |
|---|---|---|---|
| 50.0% | 100% | 29.7% | |
| Siri | 50.0% | 70.0% | 40.9% |
| Google Assistant | 50.0% | 60.0% | 10.3% |
| Bixby | 30.0% | 50.0% | 3.6% |
Key finding: Foice achieves 100% overall success rate on WeChat—every target account was compromised.
Cloud Services
At default/optimal thresholds:
| System | Threshold | SV2TTS | Foice | Augmented (Foice+SV2TTS) |
|---|---|---|---|---|
| Microsoft | 0.5 | 5.5% | 0.5% | - |
| iFlytek | 0.6 | 2.0% | 5.7% | - |
| VGGVox | 0.6 | 39.6% | 67.6% | - |
| DeepSpeaker | 0.5 | 51% | 87.7% | - |
On academic systems with known thresholds, Foice significantly outperforms traditional voice cloning—achieving 87.7% success rate on DeepSpeaker.
Why Foice Outperforms SV2TTS
- No voice sample required: SV2TTS needs clean audio; Foice needs only a photo
- Diversity of attempts: Foice generates 100 diverse voice variants; SV2TTS produces one
- Robustness to noise: SV2TTS quality degrades with noisy reference audio
Foice vs. Descriptive Attack
Can simple demographic attributes (age + gender) achieve similar results?
Descriptive Attack: Use age and gender to select voices from a pool of candidates.
| Participant | Foice SR | Descriptive SR |
|---|---|---|
| P1 | 65% | 21.1% |
| P2 | 61% | 20.6% |
| P3 | 60% | 21.3% |
| P7 | 11% | 3.3% |
| P10 | 2% | 0% |
Foice consistently outperforms demographic-based selection, demonstrating that facial features encode voice information beyond simple age/gender.
Augmentation Attack
Combining Foice with traditional voice cloning when both face and voice samples are available:
Method: Average voice feature vectors from Foice and SV2TTS, then generate 100 synthetic recordings.
| System | SV2TTS Only | Foice Only | Augmented |
|---|---|---|---|
| VGGVox | 39.6% | 67.6% | - |
| DeepSpeaker | 51% | 87.7% | - |
| Microsoft | 5.5% | 0.5% | 16.7% |
| iFlytek | 2.0% | 5.7% | 6.5% |
The augmented attack achieves the best results across most systems, suggesting face and voice information are complementary.
Robustness Analysis
Number of Attempts
Success rate increases with more authentication attempts:
| Attempts | VGGVox (Foice) | DeepSpeaker (Foice) |
|---|---|---|
| 1 | 10.9% | 29.1% |
| 2 | 17.2% | 41.5% |
| 3 | 47% | 79.8% |
| 4 | 54.1% | 84.4% |
| 5 | 61.3% | 89.6% |
Image Occlusion
Foice remains effective even with partial face occlusion:
| Condition | VGGVox SR | DeepSpeaker SR |
|---|---|---|
| Complete face | 28.4% | 57.9% |
| Covered eyes | 25.2% | 55.6% |
| Covered eyebrows | 28.3% | 55.6% |
| Covered nose | 30.0% | 54.7% |
| Covered mouth | 28.4% | 58.9% |
Surprisingly, covering facial features has minimal impact—the model learns holistic face-voice correlations.
Image Resolution
| Quality | VGGVox SR | DeepSpeaker SR |
|---|---|---|
| High (>100) | 28.4% | 57.9% |
| Borderline (~100) | 30.3% | 57.1% |
| Low (<100) | 26.8% | 54.4% |
| Extremely low (=0) | 26.9% | 52.2% |
Even very low-quality images enable successful attacks.
System Vulnerabilities Exploited
Low Verification Thresholds
Voice authentication systems set low thresholds to:
- Accommodate noisy environments
- Reduce user friction from false rejections
- Handle natural voice variations (illness, fatigue)
This creates a large acceptance region that synthetic voices can exploit.
Unlimited Authentication Attempts
Many systems don't limit failed attempts for voice authentication, unlike:
- PIN codes (typically 3-5 attempts)
- Passwords (lockout after failures)
- Fingerprints (limited attempts)
This allows attackers to iterate through many synthetic recordings.
Single-Factor Trust
Voice authentication often serves as sole authentication factor, without:
- Liveness detection
- Multi-modal verification
- Behavioral analysis
Defense Recommendations
For System Designers
- Limit authentication attempts: Implement lockout after N failures
- Liveness detection: Verify human speech characteristics
- Deepfake detection: Deploy audio synthesis detection models
- Multi-factor authentication: Combine voice with other factors
- Higher thresholds: Accept usability tradeoff for security
For Users
- Avoid voice-only authentication for sensitive accounts
- Use unique passphrases rather than common phrases
- Limit public face photos that could enable attacks
- Enable additional security factors when available
Future Directions
Enhanced Attacks
- 3D Face Input: iPhone Face ID captures detailed facial geometry that could improve voice prediction
- Video Input: Lip movements and speaking patterns from video could augment face-based features
- Multiple Photos: Ensemble predictions from different angles
Improved Defenses
- Audio forensics: Detect synthesis artifacts in generated speech
- Behavioral biometrics: Analyze speaking patterns beyond voice characteristics
- Continuous authentication: Monitor voice throughout session, not just at login
Conclusion
Foice demonstrates a concerning vulnerability in voice authentication: a single face photo, publicly available on social media, can enable voice-based account compromise. The attack achieves 100% success rate on WeChat and over 87% on academic speaker verification systems.
The implications extend beyond voice authentication to multimodal biometrics generally. As AI capabilities advance, the boundaries between modalities—face, voice, fingerprint, gait—may become increasingly porous. Information leaked by one biometric can potentially be used to forge another.
For now, the message is clear: voice authentication alone is insufficient for high-security applications. Systems must implement attempt limiting, liveness detection, and multi-factor verification to resist the coming wave of cross-modal biometric attacks.
Reference: Jiang et al., "Can I Hear Your Face? Pervasive Attack on Voice Authentication Systems with a Single Face Image," USENIX Security Symposium, 2024.
- Paper: USENIX Security '24
- Code: GitHub
- Slide: 0912_CanIHearYourFace.pdf