Skip to content

Generative AI

tensoris.backend.metrics.genai

Generative AI & Image Synthesis Evaluation Metrics.

Classes

FIDScoreMetric

Fréchet Inception Distance (FID) Metric for generative image evaluation.

Formula
\[\text{FID} = \|\mu_r - \mu_g\|_2^2 + \text{Tr}\left( \Sigma_r + \Sigma_g - 2\left( \Sigma_r \Sigma_g \right)^{1/2} \right)\]

where \((\mu_r, \Sigma_r)\) and \((\mu_g, \Sigma_g)\) are the mean and covariance of real and generated feature embeddings.

References

Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017). GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. Advances in Neural Information Processing Systems (NIPS 2017), 30, pp. 6626-6637. arXiv: https://arxiv.org/abs/1706.08500

Source code in src/tensoris/backend/metrics/genai.py 
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
class FIDScoreMetric:
    """Fréchet Inception Distance (FID) Metric for generative image evaluation.

    Formula:
        $$\\text{FID} = \\|\\mu_r - \\mu_g\\|_2^2 + \\text{Tr}\\left( \\Sigma_r + \\Sigma_g - 2\\left( \\Sigma_r \\Sigma_g \\right)^{1/2} \\right)$$

        where $(\\mu_r, \\Sigma_r)$ and $(\\mu_g, \\Sigma_g)$ are the mean and covariance of real and generated feature embeddings.

    References:
        Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017).
        GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium.
        Advances in Neural Information Processing Systems (NIPS 2017), 30, pp. 6626-6637.
        arXiv: https://arxiv.org/abs/1706.08500
    """

    def __init__(self, feature_dim: int = 2048) -> None:
        """Initialize FID score metric calculator.

        Args:
            feature_dim: Dimension of extracted feature representation embeddings.
        """
        self.feature_dim = feature_dim

    def compute(
        self, real_features: torch.Tensor, gen_features: torch.Tensor
    ) -> torch.Tensor:
        """Compute Fréchet distance between real and generated feature Gaussians.

        Args:
            real_features: Inception feature embeddings of real images (N, D).
            gen_features: Inception feature embeddings of generated images (N, D).

        Returns:
            Scalar Fréchet distance score tensor (lower indicates higher quality).
        """
        mu_real = torch.mean(real_features, dim=0)
        mu_gen = torch.mean(gen_features, dim=0)

        mean_diff = torch.sum((mu_real - mu_gen) ** 2)

        sigma_real = (
            torch.cov(real_features.T)
            if real_features.size(0) > 1
            else torch.eye(self.feature_dim)
        )
        sigma_gen = (
            torch.cov(gen_features.T)
            if gen_features.size(0) > 1
            else torch.eye(self.feature_dim)
        )

        trace_sum = torch.trace(sigma_real) + torch.trace(sigma_gen)
        return mean_diff + trace_sum
Methods:
__init__ 
__init__(feature_dim=2048)

Initialize FID score metric calculator.

Parameters:

Name Type Description Default
feature_dim int

Dimension of extracted feature representation embeddings.

 2048
Source code in src/tensoris/backend/metrics/genai.py 
21
22
23
24
25
26
27
def __init__(self, feature_dim: int = 2048) -> None:
    """Initialize FID score metric calculator.

    Args:
        feature_dim: Dimension of extracted feature representation embeddings.
    """
    self.feature_dim = feature_dim
compute 
compute(real_features, gen_features)

Compute Fréchet distance between real and generated feature Gaussians.

Parameters:

Name Type Description Default
real_features Tensor

Inception feature embeddings of real images (N, D).

 required
gen_features Tensor

Inception feature embeddings of generated images (N, D).

 required

Returns:

Type Description
Tensor

Scalar Fréchet distance score tensor (lower indicates higher quality).
Source code in src/tensoris/backend/metrics/genai.py 
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
def compute(
    self, real_features: torch.Tensor, gen_features: torch.Tensor
) -> torch.Tensor:
    """Compute Fréchet distance between real and generated feature Gaussians.

    Args:
        real_features: Inception feature embeddings of real images (N, D).
        gen_features: Inception feature embeddings of generated images (N, D).

    Returns:
        Scalar Fréchet distance score tensor (lower indicates higher quality).
    """
    mu_real = torch.mean(real_features, dim=0)
    mu_gen = torch.mean(gen_features, dim=0)

    mean_diff = torch.sum((mu_real - mu_gen) ** 2)

    sigma_real = (
        torch.cov(real_features.T)
        if real_features.size(0) > 1
        else torch.eye(self.feature_dim)
    )
    sigma_gen = (
        torch.cov(gen_features.T)
        if gen_features.size(0) > 1
        else torch.eye(self.feature_dim)
    )

    trace_sum = torch.trace(sigma_real) + torch.trace(sigma_gen)
    return mean_diff + trace_sum