Haftenberger et al.

Nutrition Journal 2010, 9:36

http://www.nutritionj.com/content/9/1/36

RESEARCH Open Access

Relative validation of a food frequency

questionnaire for national health and nutrition
monitoring
Marjolein Haftenberger1, Thorsten Heuer2, Christin Heidemann1, Friederike Kube1, Carolin Krems2,
Gert BM Mensink1*

Abstract
Background: Validation of a food frequency questionnaire (FFQ) is important as incorrect information may lead to
biased associations. Therefore the relative validity of an FFQ developed for use in the German Health Examination
Survey for Adults 2008-2011 (DEGS) was examined.
Methods: Cross-sectional comparisons of food consumption data from the FFQ and from two 24-hour recalls were
made in a sample of 161 participants (aged 18 to 80 years) of an ongoing nationwide survey, the German National
Nutrition Monitoring (NEMONIT). The data collection took place from November 2008 to April 2009.
Results: Spearman rank correlations between the FFQ and the 24-hour dietary recalls ranged from 0.15 for pizza to
0.80 for tea, with two third of the correlation coefficients exceeding 0.30. All correlation coefficients were
statistically significant except those for pizza and cooked vegetables. The proportion of participants classified into
the same or adjacent quartile of intake assessed by both methods varied between 68% for cooked vegetables and
94% for coffee. There were no statistically significant differences in food consumption estimates between both
methods for 38% of the food groups. For the other food groups, the estimates of food consumption by the FFQ
were not generally higher or lower than estimates from the 24-hour dietary recalls.
Conclusions: The FFQ appears to be reasonably valid in the assessment of food consumption of German adults.
For some food groups, such as raw and cooked vegetables, relative risks estimates should be interpreted with
caution because of the poor ranking agreement.

Background situation within a specified population, the dietary assess-

Many epidemiological studies investigate the effects of ment methods should be feasible to apply next to assess-
diet on health in large populations. For such studies, ments of other health relevant topics. FFQs measure
accurate methods to assess middle or long-term dietary usual intake over a middle or long-term period, which is
intake are needed. However, comprehensive dietary highly relevant for the survey objective of monitoring
methods are often expensive, time consuming and usual behaviour. In comparison to other dietary intake
request a high commitment of participants [1]. Self- assessment methods, FFQs are relatively inexpensive,
administered food frequency questionnaires (FFQ) ask easy and quick to administrate [2,3]. Their ability to com-
respondents about the frequency and often about the pare groups or rank persons according to their intake of
portion size of a limited number of usually consumed major food groups is often sufficient for health survey
foods. Within large health surveys, which primarily give a purposes and therefore FFQs are often the method of
broad representative overview of the actual health choice for such surveys [3]. However, only a limited
number of foods can be included in an FFQ, for feasibil-
ity reasons and to limit the burden for participants. This
* Correspondence: MensinkG@rki.de predefined food list may have to be adapted to the popu-
1
Robert Koch-Institute, Department of Epidemiology and Health Reporting,
Postbox 65 02 61, D-13302 Berlin, Germany lation of interest and to actual food habits [3]. Like all
Full list of author information is available at the end of the article

© 2010 Haftenberger et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative
Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and
reproduction in any medium, provided the original work is properly cited.
Haftenberger et al. Nutrition Journal 2010, 9:36 Page 2 of 9
http://www.nutritionj.com/content/9/1/36

dietary assessment methods, FFQs are prone to measure- the completion of the FFQ and the 24-hour recalls ran-
ment errors and it is highly recommended to validate ged from 24 to 125 days (mean 55 days) for the first
new FFQs [1-3]. recall and from 10 to 89 days (mean 35 days) for the
A self-administered, semi-quantitative FFQ was devel- second recall. For five participants, both recalls were
oped to assess usual food consumption within the within the reference period of the FFQ (28 days).
German Health Examination Survey for Adults Furthermore, 55 participants completed their second
2008-2011 (DEGS) [4]. The relative validity of this ques- recall within that reference period.
tionnaire was studied among participants of another The surveys were approved by the German federal
nationwide survey, the German National Nutrition data protection office. Respondents were informed in
Monitoring (NEMONIT). This sample was chosen, detail about the study objectives, interview and examina-
because they were already recruited and interviewed by tion procedures as well as the handling of data records
trained interviewers using the 24 h-recall method [5]. and analyses under pseudonymous conditions. It was
made clear that participation was on a voluntary base
Methods and could be terminated at any time. For the NEMO-
The German Health Examination Survey for Adults NIT and for the validation study all participants
2008-2011 (DEGS) aims to monitor overall health status provided written informed consent.
and its determinants in the German population aged
18 to 79 years in a cross-sectional component and aged The Food Frequency Questionnaire
18 years and over in a longitudinal component [4]. The The FFQ is a revision of the questionnaire used in the
FFQ is sent to the subjects by post prior to a local German Health Interview and Examination Survey for
examination visit. Children and Adolescents (KiGGS). The latter question-
To assess the relative validity of the DEGS question- naire was developed with consultation of experts
naire, food consumption derived from the FFQ was involved in large previous surveys about the food list
compared with intakes assessed by two 24-hour dietary and designed using elements from and considering cog-
recalls among participants of the German National nitive criteria of the National Cancer Institute Food
Nutrition Monitoring (NEMONIT). This is a longitudi- Frequency Questionnaire [6]. Design elements include
nal survey in a sample of about 2000 participants the application of consecutive questions instead of a
recruited from the German National Nutrition Survey grid format, the use of pictures to illustrate standard
II. NEMONIT aims to monitor food consumption in portions, standard use of 10 frequency categories (11 in
the German adult population aged 18 to 80 years, which DEGS), varying answer categories for portion size and
is the same population the FFQ was developed for. using additional questions on specific foods (like fat
From the participants, each year two 24-hour dietary content of milk). The questionnaire for children is
recalls are administered on randomly drawn days within described in detail elsewhere [7] and showed a good
a period of three months, in four consecutive waves [5]. compliance in KiGGS [8]. For DEGS, the children’s
Between the end of November 2008 and February 2009, questionnaire was primarily revised to adapt the food
487 participants completed both 24-hour recalls and item list for use in the adult population. To select the
209 of them completed at least one of those recalls food items, dietary intake data from previous surveys
between the 9th of January and the 16th of February (the German National Health and Examination Survey
2009. To ensure that both methods approximately 1998 [9] and the German National Nutrition Survey II
reflect the same reference period of four weeks, the FFQ [10] performed between 2005 and 2007) were analysed
was sent to those 209 participants at the end of Febru- to detect the most frequently consumed foods. Finally,
ary immediately after completion of both recalls. This the FFQ was cross-checked by several nutrition and
group included men and women aged 18 to 79 years. public health experts for completeness of relevant foods
The participants were asked to return the completed and usability. The FFQ was pre-tested and showed a
questionnaire preferably within the next week. To good usability. The feedback experience from the pretest
improve the response, they received a gift voucher of and the main study DEGS was positive and participants
value 7.50 € upon completion of the FFQ. In total, 164 had no problems to classify their consumed foods.
participants returned the completed FFQ. Three partici- The FFQ includes questions about the frequency and
pants were excluded from the analysis, because missing the amount of 53 food items, consumed during the past
values for food items exceeded 25%. In total, 161 partici- four weeks. The questionnaire was sent to the partici-
pants (77%) were included in the data analysis. For 58 pants by mail with the request to complete the ques-
participants (36%), at least one of the days of the recalls tionnaire at home and to return it. Frequency of
was a special day of food consumption due to feasts, tra- consumption of food items was asked according to spe-
vels, holidays, illness or shift work. The time between cified categories. The frequency categories were: never,
Haftenberger et al. Nutrition Journal 2010, 9:36 Page 3 of 9
http://www.nutritionj.com/content/9/1/36

once a month, two to three times a month, one to two Data and Statistical Analysis
times a week, three to four times a week, five to six Twenty-three participants had some missing values
times a week, one time per day, two times per day, (maximal four) for frequency of intake on food items in
three times per day, four to five times per day and more the FFQ. These participants were excluded from the
than five times per day. In addition, the respondents analysis of that particular food item. This explains the
had to indicate the portion sizes of the food items con- differences in number of participants for particular ana-
sumed in predefined answering categories. Pictures were lyses. For participants with reported frequency, missing
used to aid the estimation of portion size for 33 food values for portion size were replaced by the mean por-
items. Some questions about vegetarian and cooking tion size of the other participants for that food item
habits were included but not analysed here. Supplement (n = 8). If frequency of consumption of a food item was
use was assessed in a computer aided interview within reported as never, but portion size was given, the intake
DEGS and was therefore not part of the FFQ. For the of this food item was assumed to be zero. Participants,
analysis presented here, similar food items were cate- who reported both a high frequency of consumption
gorised in one group. For example the food items (≥ four or five times a day) and a high portion size (four
‘honey and marmalade’ and ‘chocolate spread’ were or more glasses/cups) for water, tea and non-alcoholic
grouped as ‘sweet spreads’, or the food items ‘fried and beverages on the FFQ, resulting in implausible high
curried sausages’ and ‘doner kebab and hamburger’ as amounts, were excluded from the analysis of the parti-
‘fast food’. In total, 29 food groups will be presented. cular item.
For most food groups, the food consumption was not
24-Hour Dietary Recalls normally distributed. Therefore, non-parametric meth-
For each 24-hour dietary recall, participants were asked ods were used to evaluate the validity of the FFQ rela-
by telephone on randomly drawn days about their food tive to the 24-hour recalls. Spearman rank order
consumption during the previous day in detail. The two correlation coefficients [14] were calculated for all parti-
interviews of each participant were at least one week cipants and stratified by sex and age group. In a sepa-
but no more than three months apart. The 24-hour rate analysis, the coefficients were calculated excluding
dietary recalls were equally distributed over all days of participants with special days of consumption in their
the week and the weekend. Recalls for Saturdays were recalls. In addition, participants were grouped into quar-
performed on the following Monday. Trained inter- tiles for each food group, to test the agreement in rank-
viewers of a marketing research institute performed the ing participants regarding their food consumption as
24-hour dietary recalls. For the interviews, the software estimated from both methods. The proportion of parti-
EPIC-SOFT was used. EPIC-SOFT was developed as a cipants classified into the same, adjacent or opposite
calibration instrument for the European Prospective quartile for both methods was calculated. The degree of
Investigation into Cancer and Nutrition Study by the agreement was evaluated by the weighted kappa coeffi-
International Agency for Research on Cancer (IARC) cient [14]. This analysis was not applied for food groups
[11,12]. The version applied was adapted to the German where more than 25% of the participants had a zero
dietary habits and modified for the use in the German consumption of these foods in their recalls which dis-
National Nutrition Survey II[13] and NEMONIT [5]. In abled the construction of quartiles. Mean intakes
EPIC-SOFT, personal characteristics, such as self-reports derived from both methods and differences of intakes of
of weight and height, and information on a special diet food groups between both methods are presented. The
(e.g. vegetarian) or a special day (e.g. travel), were regis- significance of differences in intake of food groups
tered. At the beginning of the interview, food items con- between the FFQ and the average of both 24-hour
sumed by the participant during the previous day were recalls was tested with the Wilcoxon sign rank test [14].
briefly listed using a quick list. Thereafter, each food All analyses were performed with SAS Version 9.2 (SAS
item was specified in detail (for example preparation Institute Inc., Cary, NC).
methods, recipes and brand names). To assist partici-
pants to indicate the consumed amount of a food, a pic- Results
ture booklet providing different photographed portion Sample Characteristics
sizes for various foods and dishes was used [11,13]. This Characteristics of the validation sample are shown in
was a short adapted version of the original EPIC-SOFT table 1. Fifty-one percent of the participants were men.
picture book. A trained assistant revised incomplete Mean age of the participants was 51 years. The majority
data from the recalls. Intake of each food item was cal- of the participants was 35 to 64 years old (70%). Only
culated as the average intake of both recall days. The 14% of the participants were younger than 35 years old
recalled food items were assigned to the food groups fit- and 16% were 65 years and older. Mean body mass
ting with the food groups defined by the FFQ. index (BMI), based on self-reports of weight and height,
Haftenberger et al. Nutrition Journal 2010, 9:36 Page 4 of 9
http://www.nutritionj.com/content/9/1/36

Table 1 Distribution of age and body mass index groups participants younger than 35 years, the correlations
in the validation sample (n = 161) were lower for several food groups as compared to the
Men Women higher age groups, especially for sweet spreads, milk,
(n = 82) (n = 79) alcoholic beverages, water, curd cheese/soured milk/
n % n % yoghurt, bread, sweets and meat, but stronger for snacks
Age group and fast food. Interestingly, the spearman correlation
18 - 34 years 11 13.4 11 13.9 coefficients for alcoholic beverages and sweets increased
35 - 50 years 28 34.2 29 37.2 with age.
51 - 64 years 29 35.4 27 34.6
65 - 79 years 14 17.1 12 15.4 Ranking Misclassification
The degree of potential misclassification associated with
BMIa categorised intakes assessed by the FFQ in comparison
Under-/normal weight (BMI < 25.0 kg/m2) 31 37.8 45 57.0 to the 24-hour dietary recalls was examined as the pro-
Overweight (BMI: 25.0 - 29.9 kg/m2) 42 51.2 23 29.1 portion of participants classified in the same, adjacent,
Obesity (BMI ≥ 30.0 kg/m2) 9 11.0 11 13.9 or opposite quartile (table 3). For tea, breakfast cereals,
a According to WHO classification [23] alcoholic beverages, cream cheese, cheese and curd
cheese/soured milk/yoghurt, eggs, fast food, fish, salty
snacks, pasta, rice, processed fruits, legumes and pizza
was 26.1 kg/m 2 in men and 25.3 kg/m 2 in women. ranking into quartiles was not possible, since more than
Sixty-two percent of the men and 43% of the women 25% of the subjects did not consume these foods on
were overweight or obese (BMI ≥ 25.0 kg/m2). each recall day. For the other 14 groups, the proportion
of participants classified within the same or the adjacent
Correlations quartile ranged from 68% for cooked vegetables to 94%
The correlations of the estimates of food consumption for coffee. Classification into the opposite quartile was
between both methods were moderate to high for most 10% or less for all food groups, with highest levels of
food groups (Table 2). High correlation coefficients opposite classification for meat (10%), raw vegetables
(≥ 0.70) were observed for tea, coffee and butter/mar- (10%), potatoes (9%) and cooked vegetables (7%). Except
garine. The correlation coefficients were moderate for cooked vegetables, the weighted kappa coefficients
(0.40 to 0.69) for sweet spreads, milk, breakfast cereals, were significant for all food groups. Exclusion of partici-
alcoholic and non-alcoholic beverages, meat products, pants with special days of consumption in their recalls
fresh fruits, water, bread, sweets, cream cheese, cheese did not improve the agreement for cooked vegetables
and curd cheese/soured milk/yoghurt. Only for the food (data not shown). A moderate to good agreement in
groups cooked vegetables and pizza, the correlation was ranking the participants according to their intake
not significant. between methods (weighted kappa > 0.40) was observed
After exclusion of participants, who recalled their food for coffee, butter and margarine, sweet spreads, non-
consumption of at least one day of special consumption, alcoholic beverages and milk. An acceptable agreement
the spearman correlation coefficients were similar or (kappa 0.20 to 0.39) was seen for meat products, water,
higher for most food groups. The correlation coefficient fresh fruits, bread and sweets.
for cooked vegetables improved remarkably and became
significant. However, the correlation coefficient for Comparison of mean intakes
breakfast cereals, pizza and pasta decreased with the Table 4 shows the mean food group intakes estimated
correlation for pasta being no longer significant (data by both methods. The mean intake of legumes, rice,
not shown). potatoes, raw vegetables, cheese, curd cheese/soured
For men and women, the correlation coefficients were milk/yoghurt, fresh fruits, non-alcoholic beverages and
similar. However, for some food groups, there were dis- milk estimated from the FFQ was significantly higher
crepancies between men and women. This especially than the intake assessed by the 24-hour recalls. The
concerned those food groups with lower correlation intake of coffee, meat products, sweets, butter and mar-
coefficients. The correlation coefficients for meat, garine, sweet spreads, fish, processed fruits, cream
legumes, eggs and pasta were only significant in men. cheese and pizza obtained by the FFQ were lower as
Whereas, the correlation coefficients for processed compared to the 24-hour dietary recalls. Food consump-
fruits, fast food and cooked vegetables were only signifi- tion estimated by the FFQ was not generally higher or
cant in women. lower than estimates from the 24-hour dietary recalls.
For most food groups, the age groups 35-50, 51-64 The mean intake of eleven food groups did not show
and ≥ 65 years showed similar correlations. Among the significant differences between both methods.
Haftenberger et al. Nutrition Journal 2010, 9:36 Page 5 of 9
http://www.nutritionj.com/content/9/1/36

Table 2 Spearman rank correlation coefficients of food group intake between the food frequency questionnaire and
two 24-hour dietary recalls
Sex Age group
All participants Men Women 18- 34 years 35 - 50 years 51 - 64 years 65 - 80 years
Food group n = 161 n = 82 n = 79 n = 22 n = 57 n = 56 n = 26
Tea 0.80‡ 0.74‡ 0.76‡ 0.74‡ 0.79‡ 0.81‡ 0.83‡
Coffee 0.78‡ 0.69‡ 0.83‡ 0.75‡ 0.81‡ 0.71‡ 0.76‡
Butter and margarine 0.70‡ 0.67‡ 0.67‡ 0.80‡ 0.61‡ 0.66‡ 0.71‡
Sweet spreads 0.65‡ 0.66‡ 0.63‡ 0.26 0.62‡ 0.52‡ 0.69‡
Milk 0.63‡ 0.61‡ 0.64‡ -0.03§ 0.67‡ 0.70‡ 0.72‡
Breakfast cereals 0.62‡ 0.64‡ 0.60‡ 0.58† 0.57‡ 0.63‡ 0.75‡
Non alcoholic beverages 0.61‡ 0.62‡ 0.59‡ 0.70‡ 0.60‡ 0.64‡ 0.29
Alcoholic beverages 0.60‡ 0.65‡ 0.53‡ -0.07§ 0.52‡ 0.73‡ 0.87‡
Meat products 0.58‡ 0.45‡ 0.49‡ 0.62† 0.58‡ 0.51‡ 0.55†
Water 0.53‡ 0.54‡ 0.52‡ 0.38 0.44‡ 0.52‡ 0.66‡
Fresh fruits 0.51‡ 0.55‡ 0.46‡ 0.51* 0.54‡ 0.53‡ 0.38
Curd cheese, soured milk, yoghurt 0.49‡ 0.41‡ 0.48‡ 0.24 0.49‡ 0.47‡ 0.40*
Bread 0.46‡ 0.41‡ 0.40‡ 0.31 0.46‡ 0.43‡ 0.56*
Sweets 0.45‡ 0.45‡ 0.45‡ 0.30 0.32* 0.53‡ 0.63‡
Cream cheese 0.40‡ 0.32† 0.48‡ 0.41 0.46‡ 0.34† 0.37
Cheese 0.40‡ 0.40‡ 0.39‡ 0.38 0.52‡ 0.25 0.30
Fish 0.36‡ 0.38‡ 0.32† 0.22 0.42‡ 0.38† 0.22
Salty snacks 0.33‡ 0.33† 0.33† 0.62† 0.17 0.34† 0.33
Meat 0.29‡ 0.32† 0.21 0.13 0.13 0.28* 0.62‡
Raw vegetables 0.29‡ 0.23* 0.27* 0.18 0.43† 0.24 0.18
Processed fruits 0.27‡ 0.16 0.38‡ 0.17 0.18 0.35† 0.41*
Potatoes 0.27‡ 0.26* 0.26* 0.36 0.17 0.12 0.47*
Fast food 0.25‡ 0.16 0.36* 0.43* 0.40* 0.11 0.07
Legumes 0.22‡ 0.28* 0.09 0.26 0.22 0.22 0.19
Eggs 0.20† 0.28* 0.10 -0.23§ 0.33* 0.15 0.44*
Pasta 0.19* 0.27* 0.09 0.16 0.20 0.14 0.26
Rice 0.19* 0.18 0.20 -0.15§ 0.15 0.35† 0.11
Cooked vegetables 0.16 0.08 0.23* -0.16§ 0.26 0.21 0.00§
Pizza 0.15 -0.10§ 0.19 0.10 -0.05§ 0.07 -§
Significance level: * P < 0.05, †P < 0.01, ‡ P < 0.001
§ The correlation coefficient was negative or could not be calculated due to a high number of participants who did not consume the food on both recall days

Discussion legumes, potatoes, soft drinks, milk, cheese and pro-

The DEGS-FFQ showed a reasonable to good agreement cessed meat. In our study higher correlations were seen
in ranking of participants towards their intake for most for cereals, desserts and fish and lower for vegetables,
food groups compared to two 24-hour dietary recalls. eggs, sweets and biscuits, alcoholic beverages and meat.
The spearman’s correlation coefficients for intakes mea- Despite a reasonable agreement for ranking the parti-
sured by the FFQ and the 24-hour dietary recalls ranged cipants according to their intake based on both meth-
between 0.15 and 0.80 with most values of 0.30 and ods, we observed considerable differences in absolute
higher. The observed correlation coefficients were in a intakes between the FFQ and the 24-hour dietary recalls
similar range as observed in other validation studies for particular food groups. However, there is no evi-
[15-20]. For example, a validation of the FFQ used in dence that the FFQ systematically over- or underesti-
the German EPIC cohort showed Spearman rank corre- mates food (group) intake in comparison to the 24-hour
lation coefficients between the FFQ and twelve 24-hour recall method.
dietary recalls between 0.14 and 0.90 [17]. Although a Some disagreement of food intake estimates was
direct comparison is difficult because of differences in expected because of the different reference time of both
food classification, similar correlations were seen for methods. Discrepancies were especially expected for
coffee and tea, bread, salty snacks, spreads, fruits, foods consumed rarely, because the probability of
Haftenberger et al. Nutrition Journal 2010, 9:36 Page 6 of 9
http://www.nutritionj.com/content/9/1/36

Table 3 Agreement of quartiles for food group intake assessed by the food frequency questionnaire and two 24-hour
dietary recalls*
Agreement of quartiles for food group intake (all participants)
Food group Same quartile% Adjacent quartile% Opposite quartile% Weighted Kappa 95%- Confidence Interval
Coffee 50.3 44.1 0.0 0.54 0.45 - 0.62
Butter and margarine 50.3 40.4 0.6 0.51 0.42 - 0.60
Sweet spreads 48.4 39.0 1.3 0.49 0.40 - 0.59
Milk 45.6 40.3 1.9 0.45 0.35 - 0.55
Non alcoholic beverages 49.0 33.6 3.4 0.45 0.34 - 0.56
Meat products 40.4 44.7 1.9 0.39 0.29 - 0.49
Water 38.4 44.2 2.2 0.36 0.25 - 0.47
Fresh fruits 39.7 37.9 4.9 0.34 0.23 - 0.44
Sweets 40.2 39.0 5.0 0.31 0.20 - 0.43
Bread 38.6 39.9 5.2 0.29 0.18 - 0.41
Raw vegetables 29.8 45.0 10.1 0.18 0.07 - 0.29
Meat 29.4 40.6 10.0 0.15 0.04 - 0.26
Potatoes 30.6 38.2 9.4 0.15 0.04 - 0.26
Cooked vegetables 31.0 37.3 7.0 0.09 -0.03 - 0.20
*Analysis was not performed for food groups with more than 25% of the participants who did not consume this food in the 24-hour dietary recalls.

assessing such foods on the two recall days is low. More completed both recalls within the reference period. In
recalls per person would probably lead to a higher an additional analysis, we compared spearman correla-
agreement with the FFQ [1]. tion coefficients and mean differences in four intervals
A multiple day weighed record was also considered as of time between completion of the 24-hour recall and
the comparative method since it has few correlated the FFQ (within 28 days, 29 to 42 days, 43 to 56 days,
errors with the FFQ [3], but it also would have a diver- more than 56 days), which showed only small differ-
gent reference period and is logistically less feasible than ences in relative validity. An exception was the intake of
the 24-hour recall. The dietary history method has simi- alcoholic beverages for which the underestimation by
lar sources of error as an FFQ and is therefore inap- the FFQ compared to the 24-hour recalls increased with
propriate. Biomarkers were not considered because they increasing time lag (data not shown). An explanation
are nutrient specific [3] and this FFQ was primarily may be that participants with a longer time interval
designed to measure food intake. The FFQ and the between both methods more often had a recall day in
24-hour dietary recall have some similar error sources, the Christmas period, in which drinking habits may dif-
like the reliance on memory and the perception of por- fer from usual intake. It was shown previously that
tion sizes [1,3]. However, the FFQ stresses long-term FFQs may give reasonably good estimates of average
memory and the 24-hour recall short-term memory. In amounts of usual alcohol intake, but they are not sensi-
addition, the 24-hour recall method was interviewer- tive for day to day variability in drinking habits [21].
based using open-ended questions, whereas the FFQ However, infrequent drinkers may report either no
was self-administered with close-ended questions. Such drinks within the 24-hour recalls or, as may be the case
differences let us assume that the errors are sufficiently in our study, report unusual high intake.
independent and that the 24-hour recall method is an A consequence of performing the validation study in
adequate comparison method. the ongoing survey seems to be some degree of selection
The validation study was performed in the ongoing of the participants. All ages, for which the FFQ was
survey NEMONIT which offered many advantages. Par- developed, were represented. However, the majority of
ticipants were allready available and motivated to parti- the participants were between 35 to 64 years old and
cipate. Accordingly, the response rate was high (77%). the proportions of participants younger than 35 years or
The validation study could be realized in a short time older than 64 years were rather small. The age-stratified
and resources could be saved. Nevertheless, there are analysis showed that the correlation coefficients for sev-
some limitations of this design. eral food groups were somewhat weaker and less often
A main limitation was the tight time schedule of the significant among the participants younger than
validation study. Ideally, the 24-hour recalls should be 35 years compared to the other age groups. On the
completed within the reference period of the FFQ [3]. other hand, for some foods, like fast food, salty snacks
In our validation study, only a few participants and non-alcoholic beverages, the correlations were
Haftenberger et al. Nutrition Journal 2010, 9:36 Page 7 of 9
http://www.nutritionj.com/content/9/1/36

Table 4 Food intake assessed by the food frequency questionnaire and two 24-hour dietary recalls and differences
between both methods: mean and standard deviation (SD)
Food intake (g/d)
Food frequency questionnaire 24-hour dietary recalls Differences FFQ - 24-hour dietary recalls
Food group N Mean SD Mean SD Mean SD P*
Coffee 159 403.5 343.2 509.4 406.3 -105.8 313.3 < 0.001
Alcoholic beverages 155 126.8 183.8 174.6 277.4 -47.8 180.6 0.059
Tea 159 324.6 479.2 350.9 493.4 -26.3 374.2 0.161
Meat products 161 29.7 28.0 44.3 45.7 -14.6 43.3 < 0.001
Sweets 159 50.9 52.0 65.4 62.9 -14.5 63.2 0.008
Pasta, noodles 161 23.8 21.6 34.1 61.8 -10.3 58.2 0.896
Butter and margarine 161 8.3 9.4 17.7 17.8 -9.4 14.7 < 0.001
Sweet spreads 159 9.8 10.2 18.8 22.4 -8.9 18.7 < 0.001
Fast food 159 10.5 15.7 18.3 41.2 -7.9 38.1 0.626
Fish 161 16.9 19.0 21.5 49.9 -4.7 49.2 0.026
Breakfast cereals 161 3.6 7.2 7.2 19.9 -3.6 16.6 0.857
Processed fruits 160 6.6 11.9 10.1 32.0 -3.5 31.0 0.029
Pizza 160 14.6 19.4 16.9 62.5 -2.2 63.1 < 0.001
Salty snacks 161 4.6 9.0 6.2 15.4 -1.6 14.0 0.099
Bread 153 143.6 98.5 144.7 76.6 -1.1 94.8 0.401
Eggs 161 12.5 10.9 13.5 23.3 -0.9 23.5 0.419
Cream cheese 161 4.3 8.0 4.7 14.8 -0.4 13.0 0.002
Legumes 158 14.3 21.9 13.8 36.9 0.5 40.5 0.003
Meat 160 52.7 46.4 52.1 58.6 0.6 66.0 0.501
Cooked vegetables 158 54.6 51.5 50.9 51.7 3.7 68.7 0.474
Cheese 161 26.2 25.9 16.2 18.2 9.8 25.5 < 0.001
Rice 161 19.2 21.6 9.1 27.4 10.1 30.3 < 0.001
Potatoes 160 85.8 57.1 70.6 82.7 15.2 81.8 < 0.001
Raw vegetables 158 72.9 81.8 39.4 48.8 33.5 77.3 < 0.001
Curd cheese, soured milk, yoghurt 160 95.1 107.3 57.6 86.7 37.5 111.4 < 0.001
Non alcoholic beverages 149 357.8 473.0 288.2 378.4 69.6 394.3 0.003
Fresh fruits 161 247.7 81.8 174.9 174.2 72.8 280.0 < 0.001
Water 138 842.3 608.7 743.3 514.1 99.0 594.6 0.292
Milk 158 220.2 244.7 111.5 170.3 108.7 249.6 < 0.001
* Sign rank test

highest in the youngest age group. This is probably the effects of vegetable consumption on various out-
because, fast food and non-alcoholic beverages are con- comes. The food group of vegetables includes a consid-
sumed more often on a regular base among younger erable number of various products, consumed either as
compared to older people. single foods or as part of mixed dishes. In the FFQ,
The weak associations for some food groups may in vegetable consumption is assessed by two global ques-
part be due to the within-subject variance in the tions. It is indicated, that global questions on vegetable
24-hour dietary recalls. The calculation of deattenuated consumption rather underestimate the vegetable con-
correlation coefficients to correct for intra-individual sumption, whereas a higher number of questions on dif-
variability is used in many studies [15,17]. Accordingly, ferently prepared vegetables rather result in agreeable
deattenuated spearman correlation coefficients are gen- levels of intake compared to six day dietary records and
erally higher than the attenuated correlation coefficients. four 24-hour dietary recalls [20]. For future development
However, due to the feasibility of only two recalls and of FFQs, it should be considered to inquire vegetable
the non-normal distribution of the intake data (even consumption by multiple detailed questions to increase
after log-transformation) in our study we did not pre- the level of precision. However, it was previously noted
sent deattenuated correlation coefficients. that there is a decreasing marginal gain in information
The weak relative validity for raw and cooked vegeta- when elongating the number of food items in an FFQ
bles should be discussed, since many studies focus on after a certain point [3]. The DEGS-FFQ is relatively
Haftenberger et al. Nutrition Journal 2010, 9:36 Page 8 of 9
http://www.nutritionj.com/content/9/1/36

short since it should not discourage to participate in the Conclusions

main DEGS examination. The DEGS-FFQ seems to be a valid instrument for both
There are some possible explanations for the consider- genders and the investigated age groups. It may be rea-
able differences of average food intake estimates of some sonably well used to assess the relative risk of food con-
food groups between both methods. One is the estima- sumption quantiles for most food groups. For some
tion of portion sizes. The FFQ used close-ended ques- food groups, such as raw and cooked vegetables, relative
tions with predefined portion sizes, while in the recalls risks estimates should be interpreted with caution
the consumed amounts are quantified as detailed as pos- because of the poor ranking agreement. However, we
sible. In addition, the broad food groups in the FFQ may observed considerable differences in absolute intakes
complicate it sometimes to average the portion size. For estimated by both methods with no evidence that either
instance, lettuce and cucumber both belong to the the FFQ or the 24-hour recalls better reflect real food
group of raw vegetables, but have very different specific consumption. Therefore, the use of absolute intake esti-
weights. Following, the application of a predefined aver- mates from the FFQ should also be considered with
age weight for such food group portions might result in care and the sensitivity to standard portion size use
measurement errors. Besides, many foods, such as vege- should be further investigated. In addition, food con-
tables, meat and potatoes, are often consumed as part of sumption information from the FFQ is suitable to be
mixed dishes. The FFQ inquired the frequency and used as confounding factors in exposure-outcome
amount consumed of single food items. Therefore, the analyses.
FFQ relies on the participant’s ability to quantify the
consumption of a food from single foods as well as from
Acknowledgements
mixed dishes. In contrast, the foods consumed as part of We thank Christiane Hanisch, Christina Kleiser, Simone Kohler, Martina
mixed dishes were quantified separately in the recalls. Rabenberg and Almut Richter for their help in the construction of the food
For milk, the relatively large difference in means (despite frequency questionnaire. The study was funded by the Robert Koch-Institute.
an acceptable agreement for ranking) may be caused by Author details
the use of different standard portion sizes. The question 1
Robert Koch-Institute, Department of Epidemiology and Health Reporting,
on milk in the FFQ includes both milk as a plain bever- Postbox 65 02 61, D-13302 Berlin, Germany. 2Max Rubner-Institute,
Department of Nutritional Behaviour, Haid-und-Neu-Straße 9, D-76131
age and milk added to coffee and breakfast cereals. This Karlsruhe, Germany.
may have complicated the estimation with predefined
portion sizes and may partly explain the observed differ- Authors’ contributions
GM and CH designed the study and wrote the study proposal. MH and TH
ence. In future, a separate question for milk in coffee coordinated the field work. TH, CK, MH and FK collected and managed the
and milk as a beverage would make sense. The differ- data. MH and FK performed the statistical analysis. MH wrote the
ences in the applied standard portion sizes are probably manuscript. All authors critically revised the manuscript, read and approved
the final manuscript.
a source of different findings and harmonizing portion
sizes is an important subject for future work. The FFQ Competing interests
could be used within NEMONIT for assessing infre- The authors declare that they have no competing interests.
quent and non-consumers of specific foods to improve Received: 18 February 2010 Accepted: 14 September 2010
the estimation of usual intake distribution from the Published: 14 September 2010
recalls [22]. Within DEGS the results will be useful to
improve the evaluation of risk analyses, for instance References
1. Willett W: Nutritional Epidemiology New York, Oxford: Oxford University
through sequentially calibrating the food frequency Press, Second 1998.
intakes with estimates of the recalls. 2. Subar AF: Developing dietary assessment tools. J Am Diet Assoc 2004,
The effect of diet on a health outcome is most fre- 104:769-770.
3. Cade J, Thompson R, Burley V, Warm D: Development, validation and
quently quantified as odds ratio or relative risk in epide- utilisation of food-frequency questionnaires - a review. Public Health Nutr
miological studies. Therefore, FFQs must be able to 2002, 5:567-587.
rank individuals along the distribution of intake to pro- 4. Robert-Koch-Institute: DEGS - Studie zur Gesundheit Erwachsener in
Deutschland. (DEGS - German Health and Examination Survey for Adults)
vide accurate risk estimates. The validation study of the Berlin: Robert Koch-Institute 2009.
DEGS-FFQ showed a moderate to good agreement for 5. Nationales Ernährungsmonitoring (NEMONIT, German National Nutrition
ranking participants towards their intake for the food Monitoring). 2009 [http://www.mri.bund.de/de/institute/
ernaehrungsverhalten/forschungsprojekte/nationales-
groups tea, coffee, butter and margarine, sweet spreads, ernaehrungsmonitoring.html], (accessed July 2010).
milk, breakfast cereals, alcoholic and non-alcoholic bev- 6. Subar AF, Thompson FE, Kipnis V, Midthune D, Hurwitz P, McNutt S,
erages, meat products, fresh fruits and water. In addi- McIntosh A, Rosenfeld S: Comparative validation of the Block, Willett, and
National Cancer Institute food frequency questionnaires: the Eating at
tion, rankings were reasonable for dairy products, bread America’s Table Study. Am J Epidemiol 2001, 154:1089-1099.
and sweets.
Haftenberger et al. Nutrition Journal 2010, 9:36 Page 9 of 9
http://www.nutritionj.com/content/9/1/36

7. Mensink GB, Burger M: Was isst du? Ein Verzehrshäufigkeitsfragebogen

für Kinder und Jugendliche. (What do you eat? Food frequency
questionnaire for children and adolescents). Bundesgesundheitsblatt
Gesundheitsforschung Gesundheitsschutz 2004, 47:219-226.
8. Mensink GB, Kleiser C, Richter A: Lebensmittelverzehr bei Kindern und
Jugendlichen in Deutschland. Ergebnisse des Kinder- und
Jugendgesundheitssurveys (KiGGS). (Food consumption of children and
adolescents in Germany. Results of the German Health Interview and
Examination Survey for Children and Adolescents (KiGGS)).
Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2007,
50:609-623.
9. Mensink GBM, Beitz R, Burger M, Bisson S: Lebensmittelkonsum in
Deutschland (Food consumption in Germany). Ernährungs-Umschau 2000,
47:328-332.
10. Krems C, Vasquez-Caicedo AL, Urban C: National reports - Germany. In
European Nutrition and Health Report 2009. Edited by: Elmadfa I. Basel:
Karger AG; 2009:299-304.
11. Slimani N, Deharveng G, Charrondiere RU, van Kappel AL, Ocke MC,
Welch A, Lagiou A, van Liere M, Agudo A, Pala V, et al: Structure of the
standardized computerized 24-h diet recall interview used as reference
method in the 22 centers participating in the EPIC project. European
Prospective Investigation into Cancer and Nutrition. Comput Methods
Programs Biomed 1999, 58:251-266.
12. Slimani N, Ferrari P, Ocke M, Welch A, Boeing H, Liere M, Pala V, Amiano P,
Lagiou A, Mattisson I, et al: Standardization of the 24-hour diet recall
calibration method used in the european prospective investigation into
cancer and nutrition (EPIC): general concepts and preliminary results. Eur
J Clin Nutr 2000, 54:900-917.
13. Krems C, Bauch A, Götz A, Heuer T, Hild A, Möseneder J: Methoden der
Nationalen Verzehrsstudie II (Methods of the National Nutrition Survey
II). Ernährungs-Umschau 2006, 53:44-50.
14. Gibbons J, Chakraborti S: Nonparametric statistical inference New York:
Marcel Dekker, 4 2003.
15. Deschamps V, de Lauzon-Guillain B, Lafay L, Borys JM, Charles MA,
Romon M: Reproducibility and relative validity of a food-frequency
questionnaire among French adults and adolescents. Eur J Clin Nutr 2009,
63:282-291.
16. Paalanen L, Mannisto S, Virtanen MJ, Knekt P, Rasanen L, Montonen J,
Pietinen P: Validity of a food frequency questionnaire varied by age and
body mass index. J Clin Epidemiol 2006, 59:994-1001.
17. Bohlscheid-Thomas S, Hoting I, Boeing H, Wahrendorf J: Reproducibility
and relative validity of food group intake in a food frequency
questionnaire developed for the German part of the EPIC project.
European Prospective Investigation into Cancer and Nutrition. Int J
Epidemiol 1997, 26(Suppl 1):S59-70.
18. Shu XO, Yang G, Jin F, Liu D, Kushi L, Wen W, Gao YT, Zheng W: Validity
and reproducibility of the food frequency questionnaire used in the
Shanghai Women’s Health Study. Eur J Clin Nutr 2004, 58:17-23.
19. Khani BR, Ye W, Terry P, Wolk A: Reproducibility and validity of major
dietary patterns among Swedish women assessed with a food-
frequency questionnaire. J Nutr 2004, 134:1541-1545.
20. Petkeviciene J, Simila M, Becker W, Kriaucioniene V, Valsta LM: Validity and
reproducibility of the NORBAGREEN food frequency questionnaire. Eur J
Clin Nutr 2009, 63:141-149.
21. Rehm J, Greenfield TK, Walsh G, Xie X, Robson L, Single E: Assessment
methods for alcohol consumption, prevalence of high risk drinking and
harm: a sensitivity analysis. Int J Epidemiol 1999, 28:219-224.
22. Tooze JA, Midthune D, Dodd KW, Freedman LS, Krebs-Smith SM, Subar AF,
Guenther PM, Carroll RJ, Kipnis V: A new statistical method for estimating Submit your next manuscript to BioMed Central
the usual intake of episodically consumed foods with application to and take full advantage of:
their distribution. J Am Diet Assoc 2006, 106:1575-1587.
23. WHO: Obesity: Preventing and Managing the Global Epidemic Geneva: WHO
• Convenient online submission
2000.
• Thorough peer review
doi:10.1186/1475-2891-9-36
• No space constraints or color figure charges
Cite this article as: Haftenberger et al.: Relative validation of a food
frequency questionnaire for national health and nutrition monitoring. • Immediate publication on acceptance
Nutrition Journal 2010 9:36.
• Inclusion in PubMed, CAS, Scopus and Google Scholar
• Research which is freely available for redistribution

Submit your manuscript at

www.biomedcentral.com/submit