Quantitative Results

To rigorously evaluate the performance of our proposed TransFBP, we conduct experiments on the widely-used SCUT-FBP5500 dataset. We compare our results against a comprehensive range of classical machine learning methods, foundational deep learning architectures, and recent state-of-the-art models for Facial Beauty Prediction (FBP). The evaluation is performed using two standard protocols from the literature: a fixed 60%–40% train-test split and a more robust 5-fold cross-validation scheme.

First, we report results on the fixed 60-40 split in Table 1. Our model achieves a Pearson Correlation (PCC) of 0.9218, a Mean Absolute Error (MAE) of 0.2028, and a Root Mean Squared Error (RMSE) of 0.2643. These results demonstrate that our method is highly competitive, slightly outperforming the previous state-of-the-art method, CNN-ER [36], across all metrics on this specific split.

Next, to ensure a more thorough and generalizable comparison, we present the results of 5-fold cross-validation in Tables 2, 3, and 4. This protocol provides a more reliable measure of a model’s performance by testing its ability to generalize across different subsets of the data.

As shown in Table 2, our TransFBP achieves a mean PCC of 0.9291, establishing a new state-of-the-art and surpassing the strong CNN-ER ensemble method by a notable margin. The performance is consistently high across all five folds, indicating the stability and robustness of our approach.

The superiority of our model is further confirmed by the error metrics. In Table 3, our model obtains a mean MAE of 0.1923, representing a reduction in error compared to the next best result of 0.2009. Similarly, in Table 4, we achieve a mean RMSE of 0.2547, again demonstrating a clear improvement over prior work.

A particularly noteworthy finding is that our single model consistently outperforms heavily-engineered ensemble methods such as CNN-ER (which uses 6 models) [36]. This highlights the architectural efficiency and superior feature representation capabilities of our proposed cross-attention mechanism combined with the Vision Transformer backbone. By effectively aggregating salient patch features, our model captures the nuances of facial aesthetics more effectively than previous approaches, leading to a new benchmark in performance for this challenging task.

Table 1 Performance comparison of different methods under 60–40% train-test split on SCUT-FBP5500Table 2 Pearson correlation (PCC\(\uparrow \)) across 5 folds on SCUT-FBP5500Table 3 Mean absolute error (MAE\(\downarrow \)) across 5 folds on SCUT-FBP5500Table 4 Root Mean Squared Error (RMSE\(\downarrow \)) across 5 folds on SCUT-FBP5500Table 5 Performance breakdown by demographic subgroup (Standard Model)Cross-Demographic Generalization Study

To rigorously address concerns regarding dataset bias and evaluate the model’s fairness and robustness, we conducted a two-stage evaluation comparing our approach against a standard CNN baseline.

1. Subgroup Performance

First, we analyzed the performance of our standard model (trained on the mixed dataset via 5-fold CV) on the four specific demographic subsets. As shown in Table 5, while the model performs best on Asian Females (the majority class), it maintains competitive performance on Caucasian faces (PCC \(> 0.91\)). This confirms that when provided with representative data, the model does not suffer from severe algorithmic bias against the minority group.

2. Cross-Ethnicity Generalization (OOD Test)

To test if the model learns universal aesthetic features, we conducted an out-of-distribution experiment. We derived two subsets: Asian (4000 images) and Caucasian (1500 images). We trained exclusively on one group and tested on the unseen group.

To demonstrate that the performance drop is a natural consequence of domain shift rather than a model defect, we compared TransFBP against a standard ResNet-18 baseline. The results in Table 6 show that while both models suffer when transferring between ethnicities, TransFBP generalizes significantly better. In the Asian \(\rightarrow \) Caucasian setting, our model maintains a PCC of 0.8841, whereas ResNet-18 drops to 0.8615. This suggests that the Cross-Attention mechanism is more effective at capturing structural features (geometry, symmetry) that remain valid across ethnicities, while standard CNNs may overfit to ethnic-specific textures seen during training.

Ablation StudyTable 6 Cross-Demographic Evaluation (Cross-Ethnicity). Comparison of our method against a standard CNN baseline (ResNet-18). The models are trained on one ethnicity and tested on a completely unseen ethnicity

To deconstruct the sources of our model’s performance and rigorously validate the specific contributions of our proposed Attention-Guided TransMix augmentation, we conducted a comprehensive ablation study. We trained and evaluated three distinct model variations under the identical 5-fold cross-validation protocol, ensuring that all hyperparameters and the base TransFBP architecture remained constant. The results, summarized in Table 7, allow for a direct comparison of the following configurations:

1.

No Augmentation (Baseline): The TransFBP architecture was trained using only standard transformations (random horizontal flips). This experiment establishes the baseline performance of the Vision Transformer with our cross-attention head.

2.

Conditional CutMix (Geometric GT): The model was trained with a strong content-aware mixing strategy. Here, a patch from an "opposite" image is pasted onto the source, and the ground-truth label is interpolated using a geometric ratio, \(\lambda _\), based on the patch’s area. This isolates the effect of mixing images with disparate scores.

3.

Attention-Guided TransMix (Ours): The full proposed method was used, where the ground-truth label for a mixed image is interpolated dynamically using the attention-guided ratio ((7)) , derived from the model’s own saliency map for that mixed image.

Table 7 Ablation study results (mean over 5 folds) on the SCUT-FBP5500 dataset. Our proposed Attention-Guided TransMix shows significant improvements over baselines, highlighting the effectiveness of the dynamic, attention-based label generation

The results in Table 7 provide clear evidence for the efficacy of our approach. First, comparing row (2) to row (1), the introduction of Conditional CutMix yields a substantial performance boost across all metrics (e.g., PCC increases from 0.9085 to 0.9203). This confirms that a content-aware mixing strategy is a powerful regularizer for this task.

More importantly, the comparison between row (3) and row (2) directly measures the impact of our attention-based augmentation scheme. By switching from a simple geometric ground-truth interpolation to our dynamic, attention-guided method, the Pearson Correlation improves from 0.9203 to 0.9291, and the MAE and RMSE are further reduced significantly. This demonstrates that forcing the model to generate its own ground truth based on its perception of the mixed image provides a superior and more nuanced training signal than a static, area-based calculation. This self-referential mechanism is the key driver behind our state-of-the-art performance.

Training Dynamics

Figure 5 plots the training and validation loss, as well as validation metrics over 50 epochs. The training loss decreases steadily, while the validation loss platens after around 35 epochs, indicating that the model has converged well. The gap between training and validation loss is minimal, which we attribute to the strong regularization effects of our augmentation strategies and dropout, successfully mitigating overfitting.

Fig. 5The alternative text for this image may have been generated using AI.

Training and validation curves over 50 epochs. (Left) MSE loss. (Right) Validation PCC and MAE

Fig. 6The alternative text for this image may have been generated using AI.

Attention map visualizations from our TransFBP model on unseen test images. The top row shows original images with their ground-truth Label and model Prediction. The bottom row displays the corresponding heatmaps, where warmer colors (red/yellow) indicate regions of higher importance to the model’s decision. The model consistently focuses on salient facial features such as the eyes, nose, and mouth, demonstrating its interpretability and strong alignment with human perceptual cues for aesthetic judgment

Qualitative Analysis: Attention Visualization

Beyond achieving strong quantitative metrics, a key advantage of our TransFBP model is its inherent interpretability. We generate saliency heatmaps directly from the model’s cross-attention module to provide a direct window into its decision-making process. This visualization is achieved by (1) extracting the attention weights from all heads for a given input image, (2) averaging these weights to create a unified \(14 \times 14\) grid corresponding to the image patches, and (3) upsampling this grid using bilinear interpolation to the original image resolution. The complete procedure for computing the heat map \(\text\) is thoroughly detailed in Section 4.2.

These visualizations, shown in Fig. 6, allow for a qualitative assessment of the model’s focus. From them, several crucial insights emerge:

1.

Focus on Salient Facial Features: As a fundamental sanity check, the model has correctly learned to concentrate its attention almost exclusively on the face, largely ignoring irrelevant background details such as hair, clothing, and backdrops. This confirms that the model is learning features pertinent to the subject rather than spurious correlations in the environment.

2.

Alignment with Human Perceptual Cues: The high-attention regions align remarkably well with features known from psychological studies to be critical for human aesthetic judgment. Across nearly all examples, the model places strong emphasis on the eyes, nose, and mouth—the core components of facial recognition and expression [45]. For instance, in images with high scores (e.g., the second image with label 3.13), the model attends strongly and broadly to both the clear eyes and the pleasant smile.

3.

Implicit Learning of Holistic Properties: The attention patterns are not merely localized to specific landmarks but also capture broader, configural properties. The often-symmetrical heatmaps on the cheeks and forehead (e.g., in images 1, 5, and 8) suggest that the model has implicitly learned the importance of facial structure, proportion, and symmetry in its assessment, without being explicitly trained to do so.

4.

Pinpointing Differentiating Features: The model appears to use its attention mechanism to identify both positive and negative differentiating features. It focuses on attributes like clear skin and a sharp jawline in highly-rated faces, while in others with lower predicted scores, attention may be drawn to areas of less skin clarity or asymmetrical features, which likely contribute to the final score calculation.

This qualitative analysis provides compelling evidence that our TransFBP learns a human-aligned, semantically meaningful representation of facial beauty. The ability to produce these clear, interpretable heatmaps is a significant advantage over opaque models, increasing trust in the model’s predictions and offering a powerful tool for investigating the computational basis of aesthetic preference.

Table 8 Comparison of model complexity and computational efficiencyDiscussion

The empirical results presented in this work convincingly demonstrate that our proposed TransFBP sets a new state-of-the-art for Facial Beauty Prediction on the SCUT-FBP5500 benchmark. Our discussion focuses on interpreting why our approach is so effective, its broader implications, its inherent limitations, and promising avenues for future research.

On the Efficacy of Cross-Attention Aggregation

Our primary architectural innovation, the cross-attention aggregation head, is central to the model’s success. Traditional methods, such as the global average pooling in CNNs or the standard MLP head on a ViT’s "[CLS]" token, treat all spatial or patch features with largely uniform importance during the final aggregation stage. In contrast, our cross-attention mechanism forces an explicit, dynamic re-weighting of patch features. By using the global "[CLS]" token as a query, the model learns to "ask" which patches contain the most discriminative information for the regression task. As corroborated by the heatmaps in Figure 6, this enables the model to selectively amplify signals from salient regions (e.g., eyes, mouth, facial contour) while suppressing noise from irrelevant patches (e.g., background, hair, ears). This sophisticated feature-filtering and aggregation process is arguably a more effective strategy for modeling a concept as nuanced as facial aesthetics, explaining our model’s superior performance over even strong ensemble-based methods.

Computational Complexity vs. Performance

A potential concern for deploying Vision Transformers is their computational cost compared to lightweight CNNs. To address this, Table 8 provides a comparison of Parameters (Params) and Floating Point Operations (FLOPs). While our ViT-based architecture (86M params) is heavier than a ResNet-18 (11M params), it is comparable to a ResNeXt-101 while achieving significantly higher accuracy (PCC 0.9291 vs 0.9142). We argue that for aesthetic assessment applications like surgical planning or photo-editing software—where accuracy and interpretability are paramount—this trade-off is well-justified.

The Power of Interpretability and Human-AI Alignment

While achieving top-tier performance is a primary goal, the "black box" nature of many deep learning models has hindered their trustworthiness and scientific utility. Our work directly confronts this issue. The clear, intuitive heatmaps generated from the cross-attention weights are not a post-hoc rationalization but a direct window into the model’s reasoning. The strong alignment between the model’s high-attention regions and the facial features known to be critical in human aesthetic perception is a profound result. It suggests our model is not merely exploiting statistical quirks in the dataset but is learning a representation that is semantically aligned with human cognition. This interpretability moves the field from simple prediction to potential explanation, offering a computational tool that could even provide insights back into the field of psychology.

Quantitative Assessment of Saliency.

To move beyond qualitative observation, we performed a quantitative audit of the attention maps using 68 facial landmarks. We calculated the Attention Ratio (AR), defined as the percentage of total attention probability mass falling within the convex hulls of critical features (eyes, nose, mouth). On average, \(\mathbf \) of the model’s attention is concentrated within these landmark regions, despite them occupying less than \(30\%\) of the image area. This confirms quantitatively that TransFBP is not relying on background or spurious skin-texture shortcuts, but is actively "looking" at the facial geometry that human raters prioritize.

Limitations and Considerations

Despite its strong performance, our work is subject to important limitations.

First, we must address the dataset bias. The SCUT-FBP5500 dataset, while standard, heavily over-represents East Asian faces (approximately 73% of the total) compared to Caucasian faces, with no representation of other ethnic groups (e.g., African, Latino). Consequently, our model effectively learns a culturally specific standard of beauty primarily aligned with Asian and Western preferences. Users must be cautioned that the model is likely to underperform or exhibit bias if applied to underrepresented populations.

Second, there is a risk of ethical reinforcement. By training an AI to mimic average human ratings, we inherently risk codifying subjective beauty standards into objective "scores." Deployment of such models must be carefully controlled; they should be used for feedback or analysis (e.g., animation, reconstruction) rather than as arbiters of human worth. The high fidelity of our attention maps serves as a tool to audit these biases—if a model consistently attends to features like skin color rather than facial geometry, it can be identified and corrected.

Practical Applications and Deployment Challenges

The unique capabilities of TransFBP extend beyond academic benchmarking to tangible real-world scenarios, particularly where interpretability is paramount.

Computer-Aided Aesthetic Medicine: In plastic and reconstructive surgery, the model offers a quantitative tool for preoperative planning and postoperative evaluation. Unlike opaque CNN scores, our attention heatmaps provide surgeons and patients with a visual rationale, highlighting specifically how changes in facial symmetry or feature proportions influence the perceived aesthetic score.

Digital Content Creation: For the animation and gaming industries, TransFBP can serve as an automated critic for character design, ensuring that digital avatars meet intended aesthetic criteria without requiring expensive continuous human focus groups.

However, deploying Vision Transformers in these scenarios presents specific challenges. The quadratic computational complexity of the self-attention mechanism (\(O(N^2)\)) imposes higher latency compared to lightweight CNNs (e.g., MobileNet). While TransFBP delivers state-of-the-art accuracy suitable for server-side processing or high-precision offline software (e.g., medical imaging suites), it is currently less optimized for real-time video processing on resource-constrained edge devices (such as mobile phones). Future deployment in real-time apps would likely require model distillation or efficient attention approximation techniques.

Future Research Directions

The limitations of our study naturally illuminate promising paths for future work.

Cross-Cultural and Fairness-Aware FBP: A crucial next step is to train and evaluate FBP models on more diverse, multi-cultural, and multi-ethnic datasets. This would involve not only building fairer models but also studying how aesthetic criteria differ and manifest across populations. Incorporating techniques from the field of algorithmic fairness to audit and mitigate biases will be essential.

Architectural Refinements: The success of our cross-attention head invites further exploration. Investigating different attention mechanisms, hierarchical transformers (like the Swin Transformer), or fusing our attention-based approach with models that explicitly encode geometric facial features could yield further performance gains.

Beyond Static Images: Real-world aesthetic perception is dynamic. Extending this work to video-based FBP, where models must assess beauty from facial expressions, speech, and motion, presents a challenging and exciting frontier.