Introduction

Dementia is a syndrome associated with neurodegenerative diseases, characterized by a general decline in cognitive abilities, which affects daily activities. It ultimately has a significant impact on patients and their careers and social relationships. The most common symptoms of dementia include problems with memory, thinking, behavior, emotions, language, and reduced motivation [1]. The population of Japan is aging more rapidly than that of any other country. As of 2020, the number of Japanese people older than 65 years of age has reached a record high of 36 million, or 29% of the total population of 125 million. The number of people with dementia in Japan is projected to reach 7 million by 2025. To address this issue, the Basic Act for Dementia to Promote an Inclusive Society was approved on June 14, 2023, and took effect on January 1, 2024. This law requires national and local governments to ensure the social participation of people with dementia, develop a consultation system, and promote research for the development of a symbiotic society so that people with dementia can live with peace of mind and protect their human rights [2].

The Basic Act for Dementia to Promote an Inclusive Society includes measures to promote early detection, diagnosis, and response to dementia regarding prevention strategies. Early identification through screening would also allow patients and their families to receive care at an earlier stage in the disease process, potentially facilitating discussions regarding decision-making (eg, health care, financial, and legal), while the patient still has decision-making capacity [3]. Several progressive local governments have already initiated screening programs for the early detection of dementia risk. However, the cognitive function tests used as screening methods are Mini-Mental State Examination (MMSE) or similar conventional paper-and-pencil tests. The delay in digitizing cognitive tests might limit the number of older adults who are eligible for primary screening. A meta-analysis reported a high rate of undetected dementia, particularly in populations with low socioeconomic status [4]. This is likely due to the lack of widespread implementation of appropriate cognitive screening tests. Furthermore, the meta-analysis results suggested that studies that used the Diagnostic and Statistical Manual of Mental Disorders, Third andFourth Editions criteria to identify dementia had a higher rate of underdetection than those that used the MMSE [4].

Lecanemab (Leqembi), a treatment agent for early Alzheimer disease (AD), can significantly alter the disease course for patients in the early stages [5]. Among participants with early symptomatic AD and amyloid and tau pathology, donanemab significantly slowed clinical progression in those with low or medium tau and in the combined low or medium and high tau pathology population [6]. The World Health Organization guidelines on risk reduction for cognitive decline and dementia provide evidence-based recommendations regarding lifestyle behaviors and interventions to delay or prevent cognitive decline and dementia [7]. These preventive paradigms create the need to identify patients with mild symptoms, typically in primary care settings, and refer them to AD specialists for formal diagnosis and to determine treatment or prevention eligibility.

Primary screening of cognitive function in the community is required to facilitate early hospital visits for early diagnosis of AD. The involvement of psychologists and other professionals in administering specialized neuropsychological assessments in the community is limited, and a screening test for cognitive function that can be administered by nonprofessionals would improve primary screening. To address these issues, cognitive testing as a digital biomarker that is reliable and self-administered and can detect cognitive dysfunction at an early stage is required. A review and recommendation on routine screening for cognitive impairment in adults aged 65 years and older by the US Preventive Services Task Force asks that clinicians remain alert to early signs of cognitive impairment for individual evaluation [3].

A systematic review on digital cognitive tests reported that most of these tests showed diagnostic performance comparable with that of paper-and-pencil tests for mild cognitive impairment (MCI) and dementia [8]. Furthermore, digital biomarkers related to memory and executive function might be more sensitive than those related to other cognitive domains [9]. These findings suggest that digital cognitive battery tests are promising sensitive clinical tools for detecting MCI and early dementia. However, most studies have small sample sizes, with fewer than 100 individuals, and are based on case-control or cross-sectional designs [9]. Further studies may require the use of digital cognitive biomarkers in long-term follow-up to monitor the conversion from normal cognition to dementia.

We developed a digital cognitive test, the Japanese Cognitive Function Test (J-Cog), which assesses orientation, verbal ability, attention, visuospatial performance, reasoning, and memory, to identify older adults with cognitive impairment in primary settings. The purpose of this study was to evaluate the effectiveness of the J-Cog in predicting the development of dementia and compare the accuracy of the J-Cog with that of the MMSE [10], one of the most widely used tests of cognitive function. If the detective validity of the J-Cog is confirmed, it can be widely used in the community as a cognitive screening tool.

Methods

Participants

In total, 4167 community-dwelling older adults who were recruited from Takahama City, Japan, for the National Center for Geriatric and Gerontology-Study of Geriatric Syndromes (NCGG-SGS) [11], which is a Japanese national cohort study, were enrolled in this study. The inclusion criteria were residence in Takahama and age ≥60 years at the time of the examination (September 2015 to February 2017). The exclusion criteria were (1) availing the Japanese certified public long-term care insurance system, which requires support or care (n=76); (2) not availing the certified public long-term care insurance system but having the inability to perform basic activities of daily living (such as eating, dressing, bathing, climbing stairs) and maintain functional mobility, personal hygiene, grooming, and toilet hygiene (n=9); (3) history of dementia (n=7) and the following conditions or symptoms suspected of causing dementia: Parkinson disease (n=16), stroke (n=219), or depression (n=97), and a score of <21 on the MMSE [12] (n=67); and (4) missing data for cognitive assessments and other measurements (n=1156). Of the initial 4167 participants, 1647 were excluded, and data from 2520 older adults were analyzed (1017 men and 1503 women). The mean age was 72.7 (SD 6.7; range 60-96) years. Analyses with dementia incidence as an outcome were conducted on 2292 participants who were available for follow-up.

Measurements of Cognitive Function

Overview

Cognitive tests included the J-Cog and MMSE, administered once on the same day by well-trained assessors. The MMSE is commonly used to assess cognitive problems and is often a component of dementia diagnosis (range 0-30, higher scores indicating better functioning) [10].

Japanese Cognitive Function Test

Overview

The J-Cog was developed by neuropsychological researchers at the National Center for Geriatrics and Gerontology as a cognitive test for measuring the domains of cognitive impairment that are generally recognized in MCI. The initial version comprised 15 tasks, which were evaluated in terms of their simplicity, importance, and feasibility. Based on these assessments, the final version, comprising 12 tasks, was developed.

The J-Cog was presented on an iPad (Apple Inc) with a 9.7-inch touch display. The task instructions and questions were presented on the display with a letter size of at least 1.0 cm². This battery consists of 12 tasks—orientation, designation, attention and calculation, mental rotation, verbal fluency, sentence completion, working memory, logical reasoning, attention, common knowledge, word memory recall, and episodic memory recall [13]—that assess cognitive functions that tend to decline with age. The participants were given 20-30 minutes to complete the battery (described further in this study). Higher J-Cog scores indicate better cognitive ability (range 0-72). An operator supported each participant in setting up the iPad, understanding the task protocols, and recording their data. The participants only needed to touch the display to complete the tasks using a digital pen. Because the J-Cog is a mixture of examinations in which the display of the screen is made invisible to the participant and examinations in which the display of the screen is shown to the participant, the examinations were conducted by well-trained operators.

Task 1: Orientation

The participants indicated the month, day, and weekday of the test date. The participants also indicated the prefecture and city of their current location. All 5 questions in each task were shown, and we calculated the total number of correct answers (0-5 points).

Task 2: Designation

The participants looked at pictures of eyeglasses, keys, wallets, watches, and cell phones and responded with the name of the object. One point was awarded for each item (0-5).

After naming the objects, they had 15 seconds to memorize the 5 objects they named. The participants then chose 1 of 5 items and answered where it would be stored, that is, under the television, on the dresser, on the shoebox, under the table, or in the dresser. These encoding tasks were scored during memory recall.

Task 3: Attention and Calculation

The participants responded to additions and subtractions of up to 2 digits by rote calculations. There were 6 questions, each presented only once (0-6 points). If there was obvious noise, the questions were presented again.

Task 4: Mental Rotation

The participants saw a completed figure with 3×3 and 4×4 squares showing kana (Japanese alphabet) and completed an incomplete figure with rotated kana missing (0-5 points). First, the participants were asked to answer example questions. If the answer to the example question was incorrect, the correct answer was taught, and the test was conducted after the participants understood the rules. The time limit was 20 seconds for the 3×3 task and 30 seconds for the 4×4 task.

Task 5: Verbal Fluency

The participants were asked to say as many words as possible to fit the category (eg, for vegetables, answer carrots, onions, and lettuce). They had 15 seconds to respond to as many categories as possible (0-20 points).

Task 6: Sentence Completion

The participants responded to the words that applied to the blanks in the sentence from 5 choices (0-2 points). They read a sentence with a missing word and selected the word that fits the blank based on the context of the sentence; 2 types of sentences were presented, each containing 5 choices.

The participants were asked to read the question silently and provide the correct answer. If they did not know the answer, they were asked to select “don’t know.” After each question, the participants were asked to press the next button to complete their full response.

Task 7: Working Memory

Participants were shown images of blue dots. After a while, some of the dots changed to white, then returned to blue. They were then asked to indicate which dots had changed to white. In addition, 2 questions were asked: one with 3×3 dots, and the other with 4×4 dots (0-7 points).

Task 8: Logical Reasoning

The participants responded to a logical reasoning text task and 4 fill-in-the-blank questions (0-4 points). Participants were asked to read the instructional text and question silently and select the correct response according to the instructions.

Task 9: Attention

The participants were shown 2 similar pictures and had 30 seconds to find the 1 shape present in only 1 of the pictures (0-1 points).

Task 10: Common Knowledge

The participants looked at pictures of national flags and were asked to choose the corresponding country from 8 options. There were 8 questions (0-8 points).

Task 11: Word Memory Recall

The participants responded to the 5 objects memorized in task 2 (0-5 points).

Task 12: Episodic Memory Recall

The participants responded with the object and location of the object that they stored away in task 2. If the participant were unable to answer, they listened to the options and selected the correct answer (0-4 points).

Measurement of Incidence of Dementia

Participants who did not have dementia at baseline and were diagnosed with dementia during the 60-month, follow-up period were considered to have new-onset dementia. Data were collected from the Japanese National Health Insurance System and the Japanese long-term care insurance system. In Japan, all adults aged ≥65 years have public health insurance, which includes one of the following: health insurance for employed individuals (Employees’ Health Insurance), national health insurance for unemployed and self-employed individuals aged <75 years (Japanese National Health Insurance), or health care for individuals aged ≥75 years (Later-Stage Medical Care). In this study, participants were followed up monthly for new-onset dementia (AD, vascular dementia, frontotemporal dementia, or other dementia subtypes), as recorded by the Japanese National Health Insurance System and Later-Stage Medical Care. Participants were considered to have dementia based on a diagnosis made by a physician according to the International Classification of Diseases, Tenth Revision. We identified cases of dementia from the NCGG-SGS data according to the corresponding International Classification of Diseases, Tenth Revision codes (Multimedia Appendix 1). The diagnosis of dementia from the UK Biobank inpatient data, which have criteria similar to those of the NCGG-SGS (Multimedia Appendix 1), was previously validated and showed a positive predictive value of 87.3% for any dementia compared with the clinical expert adjudication of full-text medical records [14].

The long-term care insurance system is a mandatory form of social insurance that supports the daily lives of older adults with disabilities [15,16]. In Japan, all individuals aged ≥65 years are eligible for institutional or community-based services depending on their disability level, and those aged ≥40 years with disabilities related to aging, such as cerebrovascular disease, are also eligible to access benefits. The disability certification process consists of two parts: (1) the degree of disability based on a questionnaire developed by the Ministry of Health, Labor, and Welfare of Japan, and (2) a physician’s written opinion prepared by the attending physician [17]. We identified dementia based on the level of independence in the daily living of older adults with dementia in the disability certification process. The level of independence was classified into 5 ranks, from mild to severe (I, II, III, IV, and M); individuals with moderate dementia-related behavioral disturbances and cognitive impairment with slight dependence were classified as rank II. We defined dementia as symptoms of rank II or higher, as derived from a primary care physician’s written opinion. These criteria were the same as those used in previous studies in Japan [18-21].

Analysis of Potential Confounding Factors

Dementia results from a combination of factors that occur over time, including demographics and chronic conditions. All multivariate-adjusted models reported included the following covariates unless otherwise noted: age at enrollment, sex, educational level, drinking, smoking, presence of chronic conditions, BMI, and regular exercise. The self-reported chronic conditions included in the model were hypertension, diabetes, cancer, and osteoarthritis [22,23].

Ethical Considerations

This study was approved by the ethics committee of the Institutional Review Board of the National Center for Geriatrics and Gerontology (approval 770). All the participants were informed of the study and provided their written consent before their participation in this study, agreeing to the anonymous use of their data for publication. Data were deidentified before analysis and stored securely in a password-protected file on a password-protected computer. No compensation was provided to the study participants.

Statistical Analysis

First, those who participated at baseline and could follow the development of dementia during the follow-up period were randomly divided into the derivation and validation groups (1:1). Data homogeneity analysis regarding the control variables and J-Cog scores was tested using independent sample 2-tailed t tests or chi-square tests for contingency tests, depending on the variable structure. Incident dementia diagnoses per 1000 person-years were calculated for all participants and for participants classified by the cutoff point. Chi-square tests were used to compare the groups for incident dementia.

In the derivation group, receiver operating characteristic curves were plotted to determine the MMSE and J-Cog cutoff points that best discriminated between groups with and without dementia (the cutoff points are denoted with the 2 points on either side of the cutoff separated by a “/”). The area under the receiver operating characteristic curve was calculated, and the cutoff points for maximizing the sensitivity and specificity of each test were determined using the Youden index [24]. Based on these cutoff points, participants were categorized as having either high or low cognitive ability. We compared the accuracy of predicting the development of dementia by classifying participants using the MMSE cutoff point 23/24, which is used to determine the risk of dementia [10].

In the validation group, 3 Cox proportional regression models were developed to predict the associations of the group (lower or higher cognitive ability) classified using the J-Cog or MMSE with dementia incidence. Specifically, the first and second models used MMSE score cutoffs of 23/24 and 28/29, respectively, and the third model used a J-Cog score cutoff of 43/44. All the models were adjusted for the potential confounding factors mentioned above. The adjusted hazard ratio (HR) and 95% CIs were obtained from the respective models. Harrell C-statistic was estimated to summarize how well a predicted risk score described an observed sequence of events. Generally, a C-statistic value greater than 0.7 indicates a good match [25,26]. The Akaike information criterion (AIC) was calculated for relative goodness of fit, where lower absolute values indicate a better model fit. Differences in Akaike information criterion (ΔAIC) of 3 or more were considered meaningful [27]. Harrell C-statistic and AIC were calculated using the dynpred (version 0.1.2) and survival (version 3.5.7) R package [28,29]. As sensitivity analyses, Cox proportional regression models were used, stratified by age group (65-74 years vs 75 years or more) and educational history (less than 9 years vs 10 years or more). These are variables that have a robust relationship with cognitive decline and dementia [30,31], and we felt that the stratified analyses would show the benefits of population-specific cognitive testing.

All statistical tests were considered significant at P<.05, and statistical analyses were performed using IBM SPSS Statistics (version 24.0) and R version 4.3.2 (R Core Team).

Results

Of the 2520 participants, 181 (7.2%) were diagnosed with dementia, 29 (1.2%) moved away from Takahama City, 138 (5.5%) died during the follow-up period, and 61 (2.4%) terminated the study for other reasons (Figure 1). Incident dementia diagnosis per 1000 person-years (95% CI) for total participants was 15.7 (13.6-18.1).

We examined whether the possible confounders of dementia incidence and cognitive tests differed at baseline between the derivation and validation groups (Table 1). No significant differences were observed in any of the measurements between the 2 groups (all P>.05). Table 2 presents the possible confounding factors for dementia incidence and cognitive test results for participants grouped according to the presence or absence of dementia. In the derivation group, significant differences were found for age (P<.001), educational level (P<.001), drinking (P=.02), diabetes (P=.002), cancer (P=.04), MMSE (P<.001), all subscores in the J-Cog (all P<.05), and total J-Cog score (P<.001) between participants with and without dementia (Table 2). In the validation group, significant differences were found for age (P<.001), sex (P=.02), education level (P<.001), hypertension (P=.05), osteoarthritis (P<.001), MMSE, and all subscores in the J-Cog (all P<.05) except for orientation and designation, and total J-Cog score (P<.001) between the participants with and without dementia incidence (Table 2).

The Youden index was used to determine the cutoff points for future dementia incidence during a 60-month period, which were 28/29 and 43/44 points on the MMSE and J-Cog, respectively. Incident dementia diagnoses per 1000 person-years with 95% CI for the low and high cognitive ability groups were 21.0 (17.9-24.6) and 7.1 (5.1-10.0) according to the MMSE 28/29 cutoff, 35.0 (29.5-41.4) and 6.0 (4.5-8.0) according to the J-Cog 43/44 cutoff, and 42.5 (31.1-58.0) and 13.2 (11.2-15.5) according to the MMSE 23/24 cutoff. The low cognitive ability groups, which were divided according to the 3 cutoff points, included significantly more participants with incident dementia than those with high cognitive ability (P<.001).

The predictive performances of all 3 risk cutoff points for dementia are presented in Table 3. In the total validation group participants, significant HRs for dementia incidence were found using the MMSE 23/24 and J-Cog 43/44 cutoffs (MMSE 23/24: HR 1.93, 95% CI 1.13-3.27; J-Cog 434/44: HR 2.42, 95% CI 1.50-3.93; Figure 2). The C-statistic was above 0.7 for all cutoff points. The ΔAIC with the MMSE 23/24 cutoff as a reference indicated that the MMSE 28/29 cutoff had poor discrimination, and the J-Cog 43/44 cutoff had good discrimination.

A subgroup analysis of participants aged 65-74 years showed no significant HRs (HR 1.31, 95% CI 0.53-3.22) at the MMSE 28/29 cutoff and a poor fit (ΔAIC=5.6). Participants aged 75 years and older showed a good fit at the J-Cog 43/44 cutoff (ΔAIC=–23.2), with significant HRs (HR 2.71, 95% CI 1.84-4.00) and good discrimination (C-statistics=0.714). In participants with less than 9 years of education, significant HRs for dementia incidence were found at the MMSE 23/24 and J-Cog 43/44 cutoff points (MMSE 23/24: HR 1.83, 95% CI 1.18-2.83; and J-Cog 43/44: HR 2.26, 95% CI 1.38-3.69). The ΔAIC using the MMSE 23/24 cutoff as a reference showed a poor fit at the MMSE 28/29 cutoff (ΔAIC=4.9) and a good fit at the J-Cog 43/44 cutoff (ΔAIC=–5.3). In participants with more than 10 years of education, significant HRs for dementia incidence were found at the MMSE 28/29 and J-Cog 43/44 cutoff points (MMSE 28/29: HR 1.84, 95% CI 1.06-3.22; and J-Cog 43/44: HR 3.53, 95% CI 2.10-5.93). The ΔAIC using the MMSE 23/24 cutoff as a reference showed good fit at the MMSE 28/29 (ΔAIC=–3.5) and J-Cog 43/44 cutoff points (ΔAIC=–22.7).

‎

‎

Discussion

Principal Findings

In this study, we developed and validated the J-Cog to establish a digital cognitive marker with a low economic and human burden to serve a large number of older adults. The J-Cog is a promising free tool for use in community centers and primary care clinics so that at-risk older adults can receive timely cognitive screening in the community. This study showed that the J-Cog has good criteria and predictive validity for identifying older adults who have developed dementia. Automated administration and scoring algorithms can reduce the workload of professionals and health care staff.

In this prospective study, dementia risk was significantly associated with the MMSE 23/24 and J-Cog 43/44 cutoff points, and a good fit at the J-Cog 43/44 cutoff point. The advantages of the MMSE include rapid administration and high acceptance as a diagnostic tool among health professionals and researchers [32]. Traditionally, a 23/24 cutoff has been used to select patients with suspected cognitive impairment or dementia [33]. However, sociocultural variables, such as age and education, among others, may affect individual scores [34,35]. A review to determine the accuracy of MMSE for the early detection of dementia reported no evidence supporting the substantial role of the MMSE as a stand-alone, single-administration test in the identification of participants who could develop dementia [36].

The optimal cutoff point for the development of dementia in this study was a high MMSE score of 28/29. The mean MMSE score was 27 (SD 2), and the median score was 28 (IQR 25-29). Only 211 (8.4%) of 2520 participants had MMSE scores below 23, which is traditionally considered a high-risk factor for dementia [10]. The reason for the higher cutoff point in this study may be the high distribution of MMSE scores.

The MMSE cutoff point of 23/24 showed better accuracy than 28/29 for participants in the total validation group, those aged 65-74 years, and those with less than 9 years of education. These results suggest that the MMSE 23/24 cutoff is a robust indicator of the development of dementia. However, the MMSE 28/29 cutoff point of showed a better fit than that of 23/24 for participants with more than 10 years of education. This result is in line with that of a previous study that reported the need for a higher MMSE cutoff for diagnosing dementia for patients with high educational levels [37].

The J-Cog showed a better fit for detecting the onset of dementia than the MMSE 23/24 cutoff points. The findings were similar in analyses in which participants were stratified by age and educational level, indicating the superiority of the J-Cog over MMSE in predicting incident dementia. In addition to predictive validity, the J-Cog has certain advantages over pen-and-paper cognitive function tests, such as the fact that (1) it does not require a professional to administer the test, (2) it can be administered free of charge, and (3) scoring and judgment are automated. These benefits are important for covering a large number of participants in a community setting for cognitive testing.

Comparison With Previous Work

The emergence of disease-modifying treatment options for AD has led to a paradigm shift in treatment strategies. Amyloid-targeted therapies are ideally used in the early stages of the disease, at the stage of MCI, and no later than mild dementia, because they slow disease progression but cannot reverse decline [38]. This preventive paradigm creates the need to identify patients with mild symptoms, typically in primary care settings, and many digital biomarkers have been developed. Systematic reviews on digital cognitive tests reported that most showed diagnostic performance comparable with that of paper-and-pencil tests for MCI and dementia. However, most studies have small sample sizes of fewer than 100 individuals and are based on case-control or cross-sectional designs [8,9]. In this study, a large population was followed for 60 months after implementation of the J-Cog to determine the predictive validity of the development of dementia in community-dwelling older adults. A recent systematic review reported the evaluation of digital cognitive testing applications, with a particular focus on the older people [39]. The review included an analysis of 20 articles. The number of cognitive tasks included in the accepted digital cognitive tests ranged from 1 to 10, with only 2 (10%) tests including more than 8 tasks. The cognitive domains included executive functions, memory, visuospatial, attention, language, orientation, processing speed, learning, praxis or gnosis, abstractions, and calculations [39]. Many of these domains were included in the J-Cog. Previous studies have not found any validation of predictive validity with dementia onset as an outcome. This study provides the first evidence that digital cognitive tests are a tool that can predict the future onset of dementia.

Limitations

Although the J-Cog has some advantages over pen-and-paper cognitive function tests, there are some limitations to interpreting the results of this study. First, the participants were not randomly recruited, which may have resulted in an underrepresentation of cognitive impairment, because the participants were sufficiently healthy to undergo health examinations. Second, data on dementia subtypes were not collected (eg, AD, vascular dementia, dementia with Lewy bodies, and frontotemporal dementia). Thus, the data do not allow inferences regarding the correlation between the J-Cog and dementia pathology. Third, we were unable to contact an informant (eg, a family member) to verify the medical records, lifestyle information, or asymptomatic aberrant behavior. In addition, this study only identified dementia using specific International Classification of Diseases, Tenth Revision codes and information from the long-term care insurance system, so it was not possible to identify the onset of dementia in people who had dementia due to other factors or who had no record. These may have led to an underestimation of early-onset dementia in particular. Finally, information about the participants’ medical conditions and comorbidities was collected through self-reports, and because we did not have access to medical records, we were unable to confirm these reports.

Conclusions

In this study, the J-Cog, a computerized cognitive battery test, was shown to be valid for detecting dementia development. The J-Cog is a cognitive function test that can be widely used in the community, and its social implementation should be promoted in the future so that AD treatment can be implemented in a timely and appropriate manner. A future extension of this study would be to improve the usability of the J-Cog so that it could be administered remotely and to remove the need for an operator to improve efficiency compared with paper-and-pencil tasks.