Reliability is the consistency of a measure. If the same instrument, applied to the same object under the same conditions, yields the same result — whether repeated across time, across investigators, or across the items of a scale — the measure is reliable. Validity is a different and harder question: does the instrument capture the concept it is supposed to capture? A thermometer that reads three degrees high every time is perfectly reliable and quite invalid. Reliability concerns the stability of measurement; validity concerns its truthfulness.

The distinction matters because sociology measures things that do not sit still and cannot be seen. Alienation, empowerment, social capital, communal prejudice — none can be read off directly, so each must be approached through indicators, and every indicator can fail in either way. Reliability is a necessary but not sufficient condition for validity: an unreliable measure cannot be valid, but a reliable one may be measuring something else entirely.

Establishing reliability

Three procedures are conventional. Test-retest reliability administers the same instrument to the same respondents after an interval and correlates the results, with the obvious difficulty that genuine change is indistinguishable from instrument error. Internal consistency — usually reported as Cronbach's alpha — checks whether the several items composing a scale move together, on the assumption that they tap a single underlying dimension. Inter-observer reliability asks whether two trained coders, working independently on the same interview transcripts or the same village census, arrive at the same classification. In qualitative work reliability is often reframed as dependability: not identical results on repetition, but a research trail transparent enough that another scholar could follow the reasoning.

Types of validity

Face validity is the weakest claim — the measure looks, to an informed reader, as though it plausibly taps the concept. Content validity asks whether the indicators cover the full range of the concept's meaning rather than one convenient corner of it. Construct validity, formalised by Cronbach and Meehl, asks whether the measure behaves as theory predicts, correlating with what it should correlate with (convergent validity) and staying distinct from what it should not (discriminant validity). Criterion validity tests the measure against an external benchmark, either concurrent or predictive.

Campbell and Stanley added the pair that dominates discussion of research design. Internal validity concerns whether the causal inference within a study holds — whether the observed effect really follows from the supposed cause rather than from selection, maturation or history. External validity concerns generalisability: whether findings from this sample, this village, this year, travel beyond them.

The tension between them

The two standards pull against each other. Tightly standardised, closed-ended instruments maximise reliability by stripping away context, but they may sacrifice validity by forcing respondents into categories that do not fit their experience. Prolonged participant observation may capture meaning with great fidelity — high validity — while remaining unreplicable, since no second fieldworker will enter the same relationships. This is why quantitative traditions have historically prioritised reliability and interpretive traditions validity, and why triangulation, in Norman Denzin's formulation, is proposed as the remedy: combining methods, data sources, investigators and theories so that the weaknesses of one compensate for those of another. Convergent findings from a survey and an ethnography are stronger than either alone.

Problems in Indian survey research

Indian conditions sharpen every one of these difficulties. Translation is the first: a schedule drafted in English and administered in Hindi, Tamil or Bhojpuri may lose the conceptual edge of terms like autonomy or discrimination, and back-translation catches only the grosser errors. Respondent bias follows from steep social distance — a landless respondent facing an educated, city-dressed interviewer has strong reason to give the answer that seems safe, and courtesy bias inflates agreement with whatever the questioner appears to believe. The social identity of the investigator is not a nuisance variable but a condition of the data: as the essays in The Fieldworker and the Field record, the caste, gender and language of the researcher determine which households open their doors, who speaks in the room, and whether questions about untouchability or domestic violence receive any answer at all. Recall error in reporting income, the tendency of male household heads to answer for women, and the mismatch between census categories and lived jati identity all compound the problem.

For the UPSC answer

Define both terms in one line each and use the crooked-thermometer example — it fixes the asymmetry immediately, that reliability without validity is possible but not the reverse. List the types of validity by name, keeping internal and external validity distinct from construct and content validity, since questions often conflate them. Then argue the trade-off: standardisation buys reliability at the cost of meaning, ethnography the reverse, and triangulation is the methodological answer. Close with the Indian fieldwork problems — translation, courtesy bias, the caste and gender of the investigator — which show that these are not technical footnotes but sociological facts about the research relationship itself.

References & further reading

  1. Cronbach, L. J. & Meehl, P. E. (1955). Construct Validity in Psychological Tests. Psychological Bulletin, 52(4).
  2. Campbell, D. T. & Fiske, D. W. (1959). Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix. Psychological Bulletin, 56(2).
  3. Campbell, D. T. & Stanley, J. C. (1963). Experimental and Quasi-Experimental Designs for Research. Rand McNally.
  4. Denzin, N. K. (1970). The Research Act in Sociology. Butterworths.
  5. Srinivas, M. N., Shah, A. M. & Ramaswamy, E. A. (eds.) (1979). The Fieldworker and the Field. Oxford University Press.
  6. Bryman, A. (2012). Social Research Methods. Oxford University Press.