Automating the Fitzpatrick Skin Type Test with Image-Based Analysis

What is Fitzpatrick Skin Type Test? 

Human skin tone is basically a product of the density and distribution of melanocytes in the basal epidermis. It is a very complex trait or characteristic of the human body, which is directly affected by genetic, cultural, and environmental factors and presents issues for dermatological assessment and AI skin analysis as it manifests along a continuum. It is necessary to assess this classification for timely diagnosis, evaluation in clinical practices, and further epidemiological research. The Fitzpatrick test classifies skin color as per the specific reaction of the skin towards ultraviolet (UV) radiation due to exposure to sun. It was developed by Dr. Thomas Fitzpatrick in 1975 and determines the level of tan or burn a person might get when exposed to the sun. It is used as one of the prominent FMCG beauty technology solutions in dermatological procedures and cosmetic practices such as laser therapy and chemical peels for devising strategies to protect against the sun and guided treatment plans against tans and burns. 

For example, darker skin types need meticulous energy calibration during laser treatments to avoid burns or hyperpigmentation, whereas fairer skin types, being more prone to sunburn and skin cancer, require cautious sunscreen usage. Cosmetic procedures performed on the basis of this scale can effectively reduce the risk of changes in pigmentation or scarring. The Fitzpatrick scale test involves a questionnaire that includes specific questions such as skin color prior to exposure, response to exposure to the sun such as level of burning or tanning, tanning-related habits, and other information such as genetic background. By answering all these questions and scoring them, one’s skin type, issues, treatment plan, and more can be determined. It can be rendered more efficient when automated with the infusion of artificial intelligence (AI) in image-based analysis. As of 2024, the global AI-powered skin analysis market was valued at US $1.54 billion and may reach an approximate value of US $1.79 billion by 2034, rising at a CAGR of 16.53%. The complexion analysis system in particular was valued at US $392.6 million in 2023 and may increase to reach US $1,692.6 million in 2033, surging at a CAGR of about 15.7% during this forecast period. The growth in these markets is potentially due to an increase in customized skincare solutions, treatment, and products; recent advancements in AI; health awareness; intelligent and timely diagnostics. 

Source: Precedence Research 

Growing market size of AI-based Fitzpatrick scale skin tone analysis during 2025 to 2034 forecast period 

History of Fitzpatrick Skin Type Test 

Given below is a brief timeline of the original development and advancements in this test. 

1950s: The von Luschan’s chromatic scale was used to classify skin color by utilizing 36 opaque glass tiles. These tiles were compared to the subject’s skin color in parts that had minimal or no exposure to the sun in comparison to the Fitzpatrick test. 

1975: The software Dr. Thomas B. Fitzpatrick developed the scale to predict how exposure to ultraviolet rays could lead to effective skin response during Psoralen Ultraviolet A (PUVA) therapy against skin diseases like vitiligo, psoriasis, large parapsoriasis, eczema, cutaneous T-cell lymphoma, graft-versus-host diseases, and mycosis fungoides. Only four skin types (I – IV) were considered initially as per their tendency to get tanned or burnt. 

1980s: This test eventually became the standard practice for skin complexion scale and tone determination in dermatology. It assisted in recommending exposure to sun, melanoma risk assessment, and phototherapy planning in cosmetic and laser treatment. 

1988: The greater diversity in skin pigmentation was acknowledged to include brown skin type V and dark brown to black skin type VI. This included a few more ethnicities, such as people of African, Asian, Middle Eastern, and Latin American descent, and rendered the test as globally inclusive and clinically relevant for treating a greater number of patients. 

1990s: From the late 1990s to the 2000s, this complexion classification test was integrated into aesthetic medicine and was routinely used in laser hair removal, cosmetic dermatology, and chemical peels. This helped in reducing side effects of sun exposure in darker skin types. The test became a standardized part of skin consultation protocols in medical Sanus per Aquam or spas, plastic surgery centers, and dermatology clinics. Individual Typology Angle (IPA) was introduced by Chardon et. all in 1991 as an objective skin tone classification system that could quantify pigmentation. This method derives measurements from a tristimulus colorimeter that observes the color space defined by the International Commission on Illumination (CIE). This includes perceptual lightness (L) and the four colors of human vision, which are red/green (a*) and blue/yellow (b*) which are put into the formula ITA = [arctan (L*-50)/b*] *180/π. 

CIELAB is a uniform space with a particular numerical change corresponding to the perceived change in color. Measurements are taken against the CIE standard observer value, which equals the average of the results of color-matching experiments in vitro. This period also saw the rise of alternative methods of the Fitzpatrick skin test, such as the Goldman world scale in 2002, the Taylor hyperpigmentation scale in 2006, and the Roberts skin type classification in 2008. 

2010s: AI-powered skin analysis tests, applications, and digital assessment tools were introduced during this period. The test was used by organizations and R&D centers for developing machine learning models and skin cancer detection tools. 

2020s: Given the scale’s strengths, validations, oversimplification of skin tone diversity, non-ethnic accountability, non-effectivity in precision medicine, limited granularity, and technological advancements, cultural critique and reassessment occurred, and many alternate methods were devised. This includes advancement in the aforementioned alternative methods, measurement of eumelanin index or melanin index and ITA powered by AI, the Monk Skin Tone Scale in 2022, and other AI-based imaging and analysis techniques, as these present more clinical utility for skincare, injectables, and lasers. Modern methods feature higher accuracy in assessment of UV sensitivity, inclusivity, pigmentation, and ease of use. 

Various types of skin complexion scale 

Automated Fitzpatrick Scale Skin Tone Classification 

In contemporary times, these methods have been enhanced to develop AI-powered, data-driven, objective, and automatic measurement of skin tones on the basis of characteristics such as Fitzpatrick type, wrinkles, redness, pigmentation, and oiliness. 

Image Acquisition 

High-resolution facial and skin images are captured in real-time using Digital Single-Lens Reflex (DSLR) cameras or smartphones for face mapping skin analysis. It is made sure that the process takes place in a controlled or normalized environment using a ring light or other sources. It is optimal to capture images from multiple angles and regions, such as the forehead, chin, cheeks, tanned areas like the upper arm, etc. The dataset typically contains around 1,000 to 1,500 Fitzpatrick skin test-annotated skin condition images. It must include a wide range of ethnicities and age groups taken under various lighting conditions. Common labels indicate skin type, hyperpigmentation, acne, redness, wrinkles, etc., and some commonly used public sources include DermNet, Fitzpatrick 17k, SD-198, UFC Skin, etc. 

Preprocessing 

The captured images are converted from Red, Green, Blue (RGB) to CIE L*a*b or Hue, Saturation, Value (HSV) color space using preprocessing tools like OpenCV, Python Imaging Library (PIL), Scikit-image, and Dlib for face detection and alignment. The HSV color space defines the color’s type, intensity, and brightness as respective letters. The brightness, lighting variations, and contrast of the images are normalized, and they are resized to match the AI model’s input size (minimum 224 x 224 pixels) using histogram equalization. The images are subjected to facial or region-of-interest detection for further analysis. Training of the model requires a graphics processing unit (GPU) such as NVIDIA RTX 3080 for EfficientNet or Transformers, a central processing unit for MobileNet, and a minimum of 16 to 32 GB of random-access memory, where `0 to 20 GB may be consumed per dataset, including images and labels. 

Detection & Segmentation 

Techniques such as color-based thresholding are used for detection of skin in input images post usage of face shape analyzer. U-Net (U-shaped architecture) or Mask Region-based Convolutional Neural Network (R-CNN) are utilized for deep segmentation. Non-skin areas such as eyes, hair, and lips are removed during background removal for accurate results of the Fitzpatrick skin type test. 

Feature Extraction 

The relevant features are extracted from the images, such as color metrics, including melanin index, erythema index, brightness, and morphology, including spot shape, shadow, uneven tone, etc. Also, other features like texture are extracted using filters or CNN layers, which may include wrinkle depth, pore size, etc. 

Model Inference & Output 

The processed images are fed into the AI model, such as the ones mentioned below. Average inference speed per image may take 100 to 300 milliseconds (MobileNet-V2, quantization, and optimization) in the case of mid-range phones, 0.5 to 2 seconds in the case of GPU (EfficientNet, Inception), and 1.5 to 3 seconds in the case of Transformers (Swim). 

• EfficientNet-V2M: It simultaneously performs predictions related to redness, hyperpigmentation, wrinkle severity, and the Fitzpatrick skin type test with high mean accuracy and area under the receiver operating characteristics curve (AUROC) of the classifier system. 

• Inception-v3: This AI model is used for classifying sensitive versus resistant skin types accurately. 

• Transformer-based Architecture: These models provide higher AUC but demand high computing power. A few examples may include Swim Transformer (high-resolution facial skin classification), GoogLeNet-TL, or SkinNet fusion models (combines deep features and color metrics), etc. 

• Others: Models such as Efficient-V2, Inception-V2, and MobileNet-V2 (lightweight and mobile-deployable) can perform general classification of normal, oily, and dry skin. 

The models are deployed via Flask, FastAPI, REST API, docker container with PyTorch or TensorFlow (TensorFlow Lite converted to ONNX Runtime Mobile for edge devices) in the backend, and HTML or JavaScript in the frontend web application (Android Studio or iOS Swift for building lightweight UI in mobile apps), integrated with existing patient electronic health records, imaging booths, or dermatology tools like dermatoscope. 

The automatic classification output report includes pigmentation level, redness, inflammation level, wrinkle score, aging markers, and Fitzpatrick scale skin tone (I-VI). Here, the skin type results are referred to as mentioned below. 

• Type I: Generally classified as ‘Ivory’ skin type that has characteristic freckles, bumps, peels and showcases no changes on exposure to the sun. 

• Type II: This skin type is generally classified as ‘Pale or Fair’, which may feature freckles, bumps, peels and may showcase changes on exposure to the sun in rare cases. 

• Type III-: This is the ‘Fair to Beige’ skin type that might include burns on occasion and tans from exposure to the sun at times. 

• Type IV: The ‘Olive or Light Brown’ skin type doesn’t commonly feature freckles, rarely burns, but tans easily from exposure to the sun. 

• Type V-: ‘Dark Brown’ skin type may feature freckles and experience skin burns on a rare basis but easily gets tanned from exposure to the sun. 

• Type VI: The ‘Dark Brown to Darkest Brown’ skin type is generally deeply pigmented and always tans from exposure to the sun. It might never feature freckles or burn during exposure. 

Results can be displayed in a visual or numerical format, such as skin type, skin chart, or risk profile, with confidence scores for each prediction. The models with human-in-the-loop, such as dermatologists and surgeons, can also suggest skincare and treatment plans accordingly.  

Baumann skin types, sample dataset, and regimen set as per complexion classification 

Explore KritiKal’s Accurate Complexion Classification 

KritiKal Solutions offers automatic, computer vision-based skin type measurement, which is known to be a key baseline tool. It is powered by artificial intelligence and facial landmark-based model extraction for enhanced precision and data-driven skin tone assessment across regions such as the forehead, cheeks, etc. It is necessary in sun safety education, research, development, and treatment planning across the cosmetics industry, dermatological and healthcare sectors, as well as in photography field. Our team has developed robust AI models that feature objective, quick, and multi-dimensional analysis for various skin-related parameters such as tone, wrinkles, pigmentation, and redness. The solutions can be easily developed, ported, and used across mobile applications, clinics, and devices. There is a plethora of benefits rendered by the solutions, including adaptability, swift and real-time results, high data volume, and scalability. 

Apart from offering Fitzpatrick scale test-related outputs, the clustering algorithms-based solutions can overcome contemporary developmental shortcomings cum limitations, including biased training data, which may affect accuracy in results for darker skin tone, variations in lighting that may result in misclassification; and other challenges such as mid-tone ambiguity. We also make sure that our solutions clear all ethical, regulatory, privacy, and security standards. The models are trained to include further diversity in tones, lighting control and conditions, and security measures, which are necessary for a broader global outreach and application. Please get in touch with us at sales@kritikalsolutions.com to know more about our products, solutions, services and realize your beauty tech requirements. 

Raghvendra Singh

Raghavendra Pratap Singh currently works as a Senior Software Engineer at KritiKal Solutions. He is proficiently skilled in DevOps, Stack Development, React, JavaScript, Bootstrap, MySQL, Git and more. With his ability to work efficiently in teams and extensive experience of working with software development, he has assisted KritiKal in delivering various projects to some major clients.