The Philadelphia Negro and the Invention of Empirical Sociology

In 1896 a twenty-eight-year-old scholar was hired by the University of Pennsylvania to study the Black population of Philadelphia's Seventh Ward. He was given a one-year appointment as "assistant instructor," a salary of $900, no office, no assistants, and no place on the faculty list.

He knocked on approximately five thousand doors.

The book that resulted, The Philadelphia Negro (1899), was the first large-scale empirical sociological study conducted in the United States — and it appeared before Chicago's sociology department had produced anything comparable, before the American Journal of Sociology was a decade old, and before most of the discipline's American founders had done any fieldwork at all.

Its author was W. E. B. Du Bois, and for most of the twentieth century, sociology did not count him as a sociologist.

The question he was hired to answer, and the question he asked instead.

Philadelphia in the 1890s had a "Negro problem." The city's reform-minded elite — the settlement-house movement, the Quaker philanthropists, the university trustees — were concerned about crime, disorder, poverty and disease in the Seventh Ward, the district where most of the city's Black population lived.

The commission was, in effect, an invitation to confirm what everyone knew. The reformers who arranged the funding, principally Susan Wharton and the Philadelphia Settlement, expected a study documenting the pathology of a population, which would then justify their programme of moral uplift. The University wanted the credibility of a scientific report and did not particularly want a Black scholar on its faculty, which is why the appointment carried no rank and no renewal.

Du Bois took the commission and inverted the question.

He did not ask "what is wrong with these people?" He asked: what are the actual conditions of life in this ward, what produced them, and what would the same conditions produce in anybody?

To answer, he spent fifteen months in the Seventh Ward. He and his wife Nina — married weeks before the work began — moved into the ward itself, above a cafeteria. He designed six separate schedules of questions: a family schedule, an individual schedule, a home schedule, a street schedule, an institution schedule, and one for the surrounding area. He administered them personally, house by house, to some 2,500 households and roughly 9,600 individuals — nearly the entire Black population of the ward.

He supplemented the survey with historical research going back to 1638, with institutional records, with interviews, and with what would now be called participant observation.

He was, by any reasonable definition, doing modern urban sociology, alone, in 1897 — before the methods had names.

The intellectual problem Du Bois faced was not primarily technical. It was that the entire scientific apparatus of his day was constructed to reach a conclusion he knew to be false.

The reigning explanation for Black poverty, crime and mortality in 1896 was biological. The dominant race science — Spencer's social evolutionism (3.3.2), the American School of ethnology, the whole apparatus of craniometry, the eugenics movement then gathering force (3.3.2) — held that the observed conditions reflected inherent racial capacity.

Du Bois's strategy was to accept the standard of evidence entirely and out-perform his opponents on it.

His stated view at the time: "the Negro problem was in my mind a matter of systematic investigation and intelligent understanding. The world was thinking wrong about race, because it did not know. The ultimate evil was stupidity. The cure for it was knowledge based on scientific investigation."

So he counted. He counted occupations, incomes, rents, rooms per family, ages at death, causes of death, literacy, church membership, club membership, marital status, ownership, criminal convictions and their circumstances. He mapped the ward block by block, house by house, colour-coding each dwelling by economic condition.

And then he asked what a fair comparison would show — comparing the Black population not with the white middle class but with white populations under comparable conditions, and comparing recent migrants with the long-settled.

The result is a demonstration that the conditions are explained by the conditions, and specifically by the interaction of migration, exclusion from employment, housing restriction, and the accumulated effects of slavery — not by anything inherent in the population.

What the book actually does

The Philadelphia Negro runs to 520 pages and contains, among other things:

A house-to-house survey of nearly 10,000 people, personally administered.

Detailed occupational analysis showing that Black Philadelphians were systematically excluded from skilled trades and industrial employment — not by aptitude but by union exclusion, employer refusal and customer objection — and were concentrated in domestic service and casual labour regardless of qualification.

A social class analysis of the Black population — Du Bois distinguishes four "grades," from a settled prosperous group through the respectable working poor to the very poor and the criminal. This was pointed: it demolished the assumption of a homogeneous "Negro problem" and showed the internal differentiation that any honest observer of any population finds.

Health and mortality analysis relating Black death rates to housing density, water supply, sanitation, occupation and access to medical care — with explicit comparison to white populations in comparable housing.

A study of crime that examines the composition of the arrested population, the effects of policing practice, and the relationship between recent migration and offending. He does not minimise crime and he refuses to detach it from its conditions.

Historical chapters tracing the Black population of Philadelphia from 1638, including the effects of the Fugitive Slave Act and the great migrations.

And the maps. Du Bois produced detailed social cartography of the ward, categorising every dwelling. This was pioneering — Charles Booth's poverty maps of London (3.2.2) were the only comparable precedent, and Du Bois's analysis of the relationship between physical space and social condition is arguably more sophisticated.

The central argument

Du Bois's causal claim, stated at the time when the opposite was scientific consensus:

The condition of Black Philadelphians is the product of their social situation, and specifically of three things:

One — the legacy of slavery, ended thirty-four years earlier, which had systematically destroyed property, education, family stability and accumulated advantage.

Two — present exclusion, which he documents in detail: from apprenticeships, from unions, from industrial employment, from most housing, from most public accommodation. He shows Black men with trades working as porters because no employer would hire them at their trade.

Three — migration and adjustment, the arrival of large numbers of rural Southern migrants into an urban environment with no institutions prepared for them — which produces the same disorganisation it produces in any migrant population, and which Du Bois compares explicitly to European immigrant experience.

And therefore: the problem is not in the population. It is in the situation. Change the situation and the outcomes change.

Du Bois's book is not a work of advocacy dressed as research, and it is important to be precise about this.

He was frequently critical of the population he studied. He wrote about crime without minimising it, about family instability, about what he considered the failures of the Black elite to take responsibility for the poor, and about behaviours he thought self-defeating. Some passages read harshly, and later scholars — including sympathetic ones — have noted the Victorian moralism and the class condescension of a Harvard-trained scholar towards the ward's poorest residents.

He also insisted, repeatedly and with evidence, that these were consequences and not causes.

The combination is what makes the book scientifically formidable. A study that found nothing uncomfortable would have been dismissed as special pleading. Du Bois documented everything, including what damaged his case, and then showed what produced it.

That is exactly the discipline Weber demanded of the teacher — confronting the audience with inconvenient facts, including facts inconvenient to one's own position (4.3.9) — being practised by someone with far more at stake than Weber ever had.

The methodological heart of the book, and the move that makes it modern.

The standard argument of 1896: Black death rates in Philadelphia are far above white death rates. Therefore Black people are constitutionally weaker.

Du Bois's response, in three steps.

First, disaggregate. He breaks down mortality by ward, by district within the ward, and by cause of death. The differences within the Black population are enormous — the prosperous streets and the alleys are not the same place.

Second, compare like with like. He compares the Black population to white populations living in comparable housing, at comparable density, with comparable water and sanitation and comparable occupations. The gap narrows dramatically.

Third, examine what remains. He attributes the residual to specific, identifiable factors: exclusion from the better housing stock regardless of ability to pay; concentration in occupations with high exposure; exclusion from hospitals and from medical training; and the effects of recent arrival.

Notice the structure of the argument. It is a controlled comparison, conducted without statistical software, in 1897, to test a causal hypothesis against a rival. This is what a modern epidemiological or sociological analysis does, and Du Bois is doing it before the discipline had agreed that it should be done.

And notice its rhetorical position. Du Bois cannot simply assert that the biological explanation is wrong; his audience will not accept his authority. He must defeat it on evidence, in public, using their own standards. The book's rigour is a function of his exclusion — he had no standing to be believed, so he built an argument that did not require him to be believed.

Go deeper

Du Bois's career, and why this lesson has to tell it

The biography is not decoration here. It is the mechanism by which the work was lost.

Born 1868 in Great Barrington, Massachusetts, three years after the Civil War. Fisk University, where he first encountered the Jim Crow South. Harvard, where he took a second bachelor's degree, then a master's, then in 1895 became the first African American to receive a PhD from Harvard. His dissertation, The Suppression of the African Slave-Trade to the United States of America, was published in 1896 as the first volume of the Harvard Historical Studies.

Two years at the University of Berlin (1892–94), where he studied with Gustav von Schmoller, Adolph Wagner and Heinrich von Treitschke, and was trained in the German historical school's methods of exhaustive empirical investigation. He was denied his Berlin doctorate on a residency technicality when his funding was withdrawn.

This matters for the intellectual history: Du Bois's method comes from the same German empirical tradition that produced Weber's early survey work on East Elbian agricultural labourers. He and Weber met in Berlin, corresponded afterwards, and Weber arranged for the German publication of one of Du Bois's essays in 1906 and called him the "most important sociological scholar anywhere in the Southern States."

Then: Wilberforce, the Pennsylvania appointment, and from 1897 to 1910, Atlanta University — where he directed the Atlanta Sociological Laboratory (4.5.5), producing sixteen annual empirical monographs on Black American life on a budget that never exceeded a few hundred dollars a year.

And then he could not stay. In 1910 he left the academy for the NAACP, which he had co-founded, and edited The Crisis for twenty-four years. He returned to Atlanta in 1934, was forced out in 1944, and spent his final decades increasingly isolated by the anti-communist politics of the period — his passport was revoked, he was indicted in 1951 as an unregistered foreign agent (and acquitted on a directed verdict), and in 1961 he joined the Communist Party and emigrated to Ghana at the invitation of Kwame Nkrumah.

He died in Accra on 27 August 1963 — the day before the March on Washington. Roy Wilkins announced his death to the crowd at the Lincoln Memorial.

The erasure, and how it happened

This is the part of the story that belongs in a lesson on method, because it is a finding about how disciplines work.

The facts are stark. The Philadelphia Negro preceded the Chicago School's urban studies by two decades and used comparable methods. The Atlanta Studies preceded and in some cases directly anticipated work later credited to others. Du Bois's The Souls of Black Folk (1903) contained concepts — the veil, double consciousness, the colour line — that the discipline eventually adopted (4.5.2, 4.5.3).

And for most of the twentieth century he appeared in sociology curricula, if at all, as a political figure rather than a founder. Standard theory textbooks through the 1980s treated Marx, Durkheim and Weber as the canonical trio and Du Bois as a topic in race relations, if that.

The mechanisms of the erasure were institutional, and Aldon Morris documents them in The Scholar Denied: W. E. B. Du Bois and the Birth of Modern Sociology (2015), which is the standard account.

Funding. The philanthropic foundations that financed American social science — principally the Rockefeller-funded bodies — directed money towards Robert Park at Chicago and towards Booker T. Washington's Tuskegee, both of whom offered accounts of race relations more acceptable to donors than Du Bois's. The Atlanta programme ran on almost nothing while Chicago was lavishly resourced.

Institutional position. Du Bois held appointments only at historically Black institutions with no graduate programmes of national reach. He trained no doctoral students who could carry his programme, which is the single most effective way a body of work is transmitted (compare Durkheim's Année Sociologique school, 4.2.8).

Citation. Park had encountered Du Bois's work — he had worked for Washington at Tuskegee — and the Chicago urban studies do not credit him. Whether this was deliberate or the ordinary invisibility of work by a Black scholar to a white profession, the effect was the same.

And the canon-formation mechanism identified in 3.3.3 for Harriet Martineau operates here identically: canons are assembled by people in institutions, with funding, students and journals, and exclusion from those does not merely reduce a scholar's influence — it removes them from the record of who did what first.

A caution on the revisionist account, because honest teaching requires it.

Morris's argument that Du Bois founded modern American empirical sociology has been widely accepted in its main lines and has been debated in its strong form. Some historians of the discipline argue that Morris understates the genuinely independent development of Chicago methods, and that the counterfactual — a Du Bois-centred discipline — is harder to establish than the fact of his exclusion.

What is not in dispute: that The Philadelphia Negro was a major empirical achievement that preceded comparable American work; that the Atlanta programme produced sixteen years of systematic research on almost no funding; that the discipline did not credit either for seventy years; and that the reasons were institutional and racial rather than scientific.

The lesson holds either way, and it is the one 3.3.3 taught: what a discipline knows about its own origins is a product of institutional history, not a record of it.

And a note on the data visualisation

An episode that has become widely known and belongs here.

For the 1900 Paris Exposition Universelle, Du Bois and his Atlanta students prepared an exhibit on Black American life: sixty hand-drawn charts, graphs and maps, produced with ink, gouache and pencil on card. They present data on population, literacy, occupation, property ownership, business, and the conditions of life in Georgia and across the United States.

They are, by common assent among designers and statisticians who have examined them, extraordinary — spiral bar charts, area charts, stacked comparisons and a striking use of colour, produced decades before most of these forms were conventional. They won a gold medal at the Exposition and then sat in the Library of Congress until they were digitised and republished a century later.

The point is not that they are beautiful, though they are. It is that they were made by a research programme with no money, to persuade an international audience of facts that audience did not want, using every available means of making evidence legible. They are what the argument of this lesson looks like when it is drawn rather than written.

"Du Bois was a civil rights leader who also wrote." He held a Harvard PhD, trained in the German historical school, produced the first major American empirical community study, and directed a sixteen-year research programme. The activism came after and partly because the scholarship was ignored.

"The Philadelphia Negro is a study of Black Philadelphia." It is also a study of white Philadelphia's employment, housing and policing practices — the exclusions are documented as carefully as their effects.

"Du Bois was uncritical of the community he studied." He was frequently and sometimes harshly critical, and the Victorian moralism has been noted by sympathetic scholars. He insisted the conditions he criticised were consequences.

"The Chicago School invented urban sociology." They systematised and institutionalised it. The Philadelphia Negro precedes their work by two decades, and Booth's London surveys precede both.

"Du Bois was excluded because his work was political." His early work was scrupulously empirical and deliberately so, precisely because he expected not to be believed. The exclusion was institutional: funding, appointments, students and citation.

"He has been restored to the canon now." He is far more taught than in 1980, and standard theory courses still frequently present a Marx–Durkheim–Weber core with Du Bois as an addition. The restoration is partial and recent.

Why it matters

It is the founding text of American empirical sociology, and knowing that changes what the discipline's history looks like. A field that thought it began in Chicago in the 1920s in fact had a comprehensive urban community study, with survey instruments, social mapping and controlled comparison, in Philadelphia in 1899.

It demonstrates the method under maximum pressure. Du Bois had no institutional authority, a hostile audience, and a scientific consensus against him. His response was to make the evidence so thorough that his own standing was irrelevant. That is a model of what empirical rigour is for.

It shows the sociological explanation defeating the biological one, in real time, on the evidence. The argument that conditions produce outcomes — and that a fair comparison must hold conditions constant — is the core move of the entire discipline, and here it is being made against the strongest version of its opposite.

And it is the discipline's clearest case of the canon problem. Lesson 3.3.3 showed a founder written out. This is the same finding, in a different country, a different generation, and with an author whose work is entirely available and was never in doubt. What a discipline remembers about itself is an institutional product, and this lesson is the proof.

W. E. B. Du Bois (1868–1963): first African American to receive a Harvard PhD (1895), trained at Berlin under Schmoller and Wagner in the German historical school's empirical methods — the same tradition that formed the early Weber, who later called him the most important sociological scholar in the American South.

The Philadelphia Negro (1899) is the first major American empirical community study: fifteen months in the Seventh Ward, six survey schedules, roughly 2,500 households and 9,600 individuals personally surveyed, plus historical research from 1638, institutional records, interviews, observation and detailed social mapping of every dwelling.

Its argument: the conditions of Black Philadelphians are produced by the legacy of slavery, present exclusion from trades, unions, employment and housing, and the disorganisation of recent rural-to-urban migration — not by anything inherent in the population. The method is controlled comparison: compare like with like, disaggregate, and account for what remains.

Du Bois documented uncomfortable findings fully — crime, family instability, the failures of the Black elite — and insisted they were consequences. The book's rigour is a function of his exclusion: having no standing to be believed, he built an argument that did not need him to be believed.

The Paris 1900 data visualisations — sixty hand-drawn charts produced by Du Bois and his Atlanta students — are the same argument, drawn.

The erasure, documented by Aldon Morris in The Scholar Denied (2015): philanthropic funding directed to Chicago and Tuskegee; appointments only at institutions without national graduate reach, so no doctoral students to carry the programme; and non-citation by the Chicago School. The revisionist account's strong form is debated; the fact of the exclusion and its institutional causes are not.

Same finding as 3.3.3: what a discipline knows about its own origins is a product of institutional history, not a record of it.

Community study — intensive multi-method empirical investigation of a bounded local population.

Social survey — systematic house-to-house data collection using standardised schedules.

Social cartography — the mapping of social conditions onto physical space; used by Booth in London and Du Bois in Philadelphia.

Controlled comparison — comparing populations matched on relevant conditions in order to isolate a causal factor.

The Atlanta Sociological Laboratory — Du Bois's research programme at Atlanta University, 1897–1910, producing sixteen annual monographs.

German historical school — the tradition of exhaustive empirical and historical economic investigation in which Du Bois trained at Berlin.

Canon formation — the institutional process by which a discipline decides whose work counts as foundational.

One — look at the Paris charts. Du Bois's 1900 data portraits are freely available from the Library of Congress. Spend ten minutes with them and note what each is arguing, not just how it looks.

Two — apply the fair-comparison test. Find a current statistic comparing two population groups on some outcome. Ask what conditions differ between them, and what the comparison would show if those conditions were held constant. This is the entire method of the book.

Three — check a syllabus. Find any introductory sociology theory syllabus online. See whether Du Bois appears, where, and in what company. The answer is itself data about canon formation.

Four — read the first chapter. The Philadelphia Negro is out of copyright and freely available. The opening chapter on the scope of the enquiry is a model of how to state what a study can and cannot establish.

Five — trace a transmission mechanism. Pick any scholar whose work you consider influential and ask: where did they teach, who funded them, whom did they train, and who cited them? Those four questions explain more about influence than the quality of the work does.