As we will see, it is nontrivial to properly capture the richness of these images (even in this limited Snacks domain). Our goal here is to generate a synthetic dataset that can be used either: in place of the real Snacks dataset, or to augment the original data with additional synthetic images. This dataset depicts a broad range of snack types (ranging from fruits to beverages) and has rich depth (each class of food is depicted in several ways with varied appearances/contexts). Random photographs from the Snacks dataset. Mathematically, this score is computed as 1 minus the proportion of the synthetic samples that are near-duplicates of examples from the real dataset. the synthetic data generator may be memorizing the real data too closely and failing to generalize. High values indicate many synthetic samples look like copies of things found in the real dataset, i.e. Unoriginal: This score measures the novelty of the synthetic data. Mathematically, this score is computed as 1 minus the proportion of the synthetic samples that are near-duplicates of other synthetic samples. High values indicate an overly repetitive synthetic data generator that produced many samples which all look similar to one another. Unvaried: This score measures how much variety there is among synthetic samples. Mathematically, this score is computed as 1 minus the mean Cleanlab label issue score of all real images in a joint dataset with binary labels real or synthetic. High values indicate there may exist tails of the real data distribution (or rare events) that the distribution of synthetic samples fails to capture. Unrepresentative: This score measures how well represented the real data is amongst the synthetic data samples. Mathematically, this score is computed as 1 minus the mean Cleanlab label issue score of all synthetic images in a joint dataset with binary labels real or synthetic. High values indicate there are many unrealistic-looking synthetic samples which are obviously fake. Unrealistic: This score measures how indistinguishable the synthetic data appears from real data. When you provide both real data and synthetic data that is supposed to augment it, this tool computes four scores that contrast your synthetic vs. While you can get an idea by simply looking through the generated samples one by one, this is laborious and not systematic.Ĭleanlab Studio offers an automated way to quantitatively assess the quality of your synthetic dataset. Evaluating the Quality of Synthetic DatasetsĪfter generating synthetic data from any prompt, you’ll want to know the strengths/weaknesses of your synthetic dataset.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |