Mapping the conflicts between idiographic and nomothetic subcultures

The NS controversy is specifically about scale-freeness, a criterion Barabási and Albert use to characterize complex networks. In their model, as a network grows, its new nodes favor linking to the most connected nodes, a phenomenon known as preferential attachment. This phenomenon is similar to the Matthew effect (Merton, 1968), also known as “the rich get richer, and the poor get poorer.” The resulting network is called “scale-free” because it features the same properties across multiple scales. Specifically, the degree of the nodes, i.e., the number of neighbors, follows a power-law distribution (a statistical distribution itself scale-free). In practical terms, a few nodes get the most links (the “hubs”), while most nodes are poorly connected.

Pervasiveness is not a concept of NS, but a term I use to refer to one of its main rhetorical points: the empirical observation of the same phenomenon across many unrelated situations. Pervasiveness calls for an answer to an implicit question: Why do we observe the same thing in contexts that have a priori nothing in common? If your goal is to discover laws of nature, pervasiveness is a remarkable observation, because it suggests an underlying pattern.

Universality is a misleading but important concept, which one needs to understand at least superficially. The pervasiveness of the power law has been known in thermodynamics since the 1970s, under the pompous name of universality (Feigenbaum, 1976). Physicists explain this pervasiveness with the concept of self-organized criticality (Bak et al., 1987), which states that the power law naturally arises at the tipping point of a phase transition (called “criticality”) for mathematical reasons related to scale-freeness. Physicists such as Barabási translate this argument from thermodynamics to NS, based on three facts. (1) Phase transition and the Erdős–Renyi random graph model are instances of a process known as percolation (the emergence of a “strongly connected component” in a random graph is similar to the apparition of ice crystals in cooling water). (2) The power law and the complex network are pervasive. (3) The power law characterizes complex networks (via scale-freeness). From these premises, physicists deduce that the power law is the external sign of an underlying complex network. This idea took the name of universality because it extends Feigenbaum’s work, but it actually means something quite different. Feigenbaum’s universality is only an observation; NS’s universality states a law of nature.

I refer to this as the argument of universality, and the important point is to track how it changes. The argument appears under different forms in the publications of network scientists such as Barabási. In its strongest form, it affirms the existence of a universal law of complexity, explaining the pervasiveness of the power law and the complex network. The claim is supposed to be rooted in a mixture of graph theory and physics of complex systems, but experimental results gradually accumulated as evidence against it. The claim then degrades into a weaker form, stating the existence of unspecified laws, on the grounds of the pervasiveness of the properties of the complex network. Finally, in its weakest form, universality means only pervasiveness, which fits its colloquial meaning and Feigenbaum’s version. The strongest form claims the existence of a law; the weakest is only an empirical observation. The weak version requires only evidence, but the strong version also requires a theory. These different meanings must not be conflated because they have different validity conditions. Some researchers voice this criticism (Watts and Clauset as cited in Klarreich, 2018). As Barrat et al. (2008: 296) put it, “physicists know extremely well that the universality of some statistical laws is not to be confused with the ‘equivalence of systems’.” In the current state of the controversy, even the weakest form is contested.

Many researchers challenge the pervasiveness of the power law based on its resemblance to the log-normal distribution (a cousin of the famous normal distribution, aka the “bell curve”). Both are quite similar in practice, when it comes to matching empirical observations, but scale-freeness is related only to the power law, via the preferential attachment model. In this dispute, the empirical ground of scale-free pervasiveness is at stake. The heavy-tailed distribution is a notion introduced subsequently to subsume the log-normal and power-law distributions.

The controversy unfolds in two distinct moments, characterized by the accumulation of traces in the digital public space as network scientists debate on their blogs, Twitter, and arXiv, a platform hosting non-peer-reviewed pre-prints. The first dispute happens in 2005 and focuses on the resemblance between the log-normal and power-law distributions. The second happens in 2018 and challenges more directly the measure of scale-freeness.

My methodology is based on document analysis and draws on controversy mapping (Venturini, 2010). I “follow the actors” (Latour, 1987) to identify the points they see as controversial. I often refer to the researchers involved in the debates as actors or network scientists, for simplicity and because they all engage with NS, although they do not always qualify themselves as such.

The controversy has no clear boundaries. The NS concepts and methods are challenged and discussed over its two decades of existence, as for any scientific field. In this article, I specifically investigate the contentious points whose resolution is contested by actors: when they also disagree about their disagreement.

I coded a selection of 40 academic and non-academic publications related to the controversy. I explored the literature with snowballing sampling before reducing the corpus to a set of key documents. I selected explicit refutations, refuted articles, commentary, major publications citing the disputes (e.g., attempts at concluding them), and major references. See Table 1 for the list and Figure 3 for a contextualized visualization.

Before engaging with the matter of the two disputes, I summarize how the actors of the controversy frame it themselves. Their recurrent arguments are the following:

1. A disciplinary divide
2. A matter of how public relations are enacted by certain researchers
3. An issue of conceptual ambiguity

The dispute is about the claim that observed power laws are log-normal distributions. Some argue a flaw exists in the statistical procedure used to identify power laws, thus challenging their pervasiveness. The dispute unfolds as follows.

In May 2005, Barabási (2005) publishes an article on the presence of “heavy tails in human dynamics”.

In October, Stouffer et al. (2005: 1) publish on the online repository arXiv¹ (no peer review) a refutation of Barabási’s claim that “the dynamics of a number of human activities are scale-free.” They argue that “the reported power-law distributions are solely an artifact of the analysis of the empirical data.”

In the days following the release of Stouffer et al.’s pre-print, other researchers comment on the dispute on their respective blogs (Clauset, 2005a; Shalizi, 2005; Venkatasubramanian, 2005). Clauset (2005a) frames it as a matter of “good empirical research” and summarizes the main sticking point as such: “[Stouffer et al.] eliminate the power law as a model, and instead show that the distributions are better described by a log-normal distribution.”

Venkatasubramanian includes Mitzenmacher in the controversy. Venkatasubramanian interviews Mitzenmacher, defending the impossibility of contrasting the two distributions in practice, as he had established in a previous publication (Mitzenmacher, 2004).

In November, Barabási et al. (2005: 2) publish a response to Stouffer et al.’s rebuttal, on arXiv as well. First, they acknowledge that both distributions match observations, but add that Stouffer et al.’s claim stems from a misunderstanding of the original data set. Second, they argue that their work “fails to propose an alternative mechanism indicating that a lognormal distribution could also emerge,” and therefore, “is a mere exercise in statistics, one that has little hope to be conclusive” on a larger data set. From that point in time, Mitzenmacher’s argument on the relative irrelevance to contrast the log-normal and power-law distributions meets consensus in the community.

In a second blog post, Clauset (2005b) comments on the dispute, and like Mitzenmacher, reframes it to account for the role of Barabási’s rhetoric in the reception of his work:

Clauset criticizes the strong version of the argument of universality, and points at a specific problem with Barabási’s laws-seeking approach: It is not falsifiable. However, note that Clauset does not contest the method or its result, qualified as “suggestive” but empirically valid if “judged accordingly”.

After a second pre-print from Stouffer et al. (2006) refining their argument, the dispute moves to the classic academic space of peer-reviewed publications. At this point, most authors acknowledge that “whether or not a distribution is heavy-tailed is far more important than whether it can be fit using a power law” (Holme, 2019).

A series of publications contest specific claims on scale-freeness. Keller (2005, 2007) refines her epistemic critique of universality. Malmgren et al. (2008, 2009) show that heavy-tailed distributions in complex networks are not necessarily caused by preferential attachment and propose an alternative model. Muchnik et al. (2013) support the point. Alderson et al. (2019) keep refuting the scale-free model in the case of internet, a critique they formulate during every phase of the controversy, but without mentioning either dispute (Doyle et al., 2005; Willinger et al., 2004, 2009).

Clauset solidifies his position with the help of Shalizi and Newman (Clauset et al., 2009). They develop a rigorous framework for testing scale-freeness, confirming certain empirical observations of power laws and ruling out others.

Stumpf and Porter (2012) also publish an article presented in 2018 as the final point on the controversy.² “The most productive use of power laws in the real world will … come from recognizing their ubiquity … rather than from imbuing them with a vague and mistakenly mystical sense of universality” (666).

Despite critiques, Barabási keeps supporting the argument of universality, albeit under a weaker form (Barzel and Barabási, 2013). The situation leads Pachter (2014) to comment on his blog that “Barabási’s ‘work’ is a regular feature in the journals Nature and Science despite the fact that many eminent scientists keep demonstrating that the network emperor has no clothes,” echoing the recurrent claim that the pervasiveness of power laws is a “myth” (Lima-Mendez and Van Helden, 2009; Shalizi, 2010; Willinger et al., 2009).

Barabási publishes the book Network Science (2016) commenting on the first dispute that “as long as there is empirical data to be fitted, the debate surrounding the best fit will never die out” (151) and standing behind the pervasiveness of the power law despite a “decade-long crusade against network science” (16).

The second dispute makes visible why the disagreement persists. Before I get to this point, I must establish how the controversy differs from its depiction by actors, so that we can see beyond the hypothesis of a disciplinary divide.

I account for the position of key actors on scale-freeness across the two disputes by tracking the four following statements, systematically coded for the 40 publications of the corpus:

1. We CAN contrast the power-law and log-normal distribution in empirical situations.
2. We CANNOT contrast them.
3. Scale-freeness requires a better statistical characterization.
4. A basic test of scale-freeness is useful to science even if it is not perfectly rigorous.

I use these statements as an analytical grid of argumentative positions that key actors may or may not occupy during each dispute. I show this grid visually for ease of understanding. As only the first two statements are mutually exclusive, I juxtapose only them. According to the key actors’ narrative, “statisticians” defend the possibility to contrast distributions, and “physicists” argue that it does not matter in empirical situations. I position the statements accordingly, as illustrated in Figure 6.

Clauset actively contributes to reaching consensus until the second dispute. He refrains from framing the debate as a disciplinary matter and co-publishes with statisticians and physicists. His publications are respected and show an in-depth understanding of Barabási’s position. “Power-law Distributions in Empirical Data” (Clauset et al., 2009) is the second most-cited publication of the corpus, cited by Barabási (2016, 2018) twice. However, Clauset (2005b) also requires “falsifiable hypotheses”. It is this imperative of falsifiability, and not a disciplinary divide, that grounds his reopening of the controversy.

To understand how a criterion as consensual as falsifiability can become controversial, I need to introduce two approaches to knowledge that renders visible the epistemological commitment of network scientists, notably during the second dispute.

The type of knowledge Barabási produces determines his position in the controversy. His approach postulates the existence of universal structures: Phenomena obey laws, and the purpose of science is to find them. This position is what the philosopher Windelband introduced as the nomothetic approach to knowledge (Lindlof, 2008; see also Munk, 2019), from the Greek “proposition of the law”. From that perspective, regularities are how nature tells us its structure, and science tries to understand that language. Barabási is the textbook example of the nomothetic mind. See, for instance, how he presents the power law in his best-seller book Linked (2002: 77, emphasis added):

Windelband opposes the nomothetic to the idiographic approach to knowledge. This other perspective focuses on accounting for the specifics of phenomena. It is considered typical of the humanities, and the usual idiographic use of networks is to describe (see e.g. Grandjean, 2016; Venturini et al., 2018). Some researchers such as Clauset adopt this approach, as they try to stabilize metrics capable of characterizing scale-free phenomena. Their goal is to improve the descriptive process regardless of general implications. Their phenomena may well disobey laws—if that is what experiments find. This independence of generality is what makes their position idiographic.

Galison (1999) remarks that “[e]ach subculture has its own rhythms of change, each has its own standards of demonstration, and each is embedded differently in the wider culture of institutions, practices, inventions, and ideas” (143). He observes how “theorists trade experimental predictions for experimentalists’ results” (146). In the second dispute, I find similar interactions between nomothetic theorists, such as Barabási, and the idiographic practice of instrumentalists, such as Clauset.

The second dispute focuses directly on scale-freeness. It extends the first one, with similar arguments and involves some of the same actors, but unfolds differently.

In 1999, Barabási and Albert publish two famous articles. In the first one (Albert et al., 1999), they study the structure of the World Wide Web and measure that the probabilities of a page to cite or to be cited “follow a power-law over several orders of magnitude” (130). In the second (Barabási and Albert, 1999), they introduce the concept of preferential attachment and measure multiple data sets to conclude that “large networks self-organize into a scale-free state” (510). This article, considered a pillar of network science, states for the first time the disputed point: that scale-freeness is pervasive.

In January 2018, Broido and Clauset publish on arXiv the pre-print “Scale-free Networks Are Rare”. They retrace the origin and circulation of Barabási and Albert’s original statement:

To challenge the “scale-free hypothesis,” they “carry out a broad test of the universality of scale-free networks by applying state-of-the-art statistical methods to a large and diverse corpus of real-world networks” (2). They conclude that “genuinely scale-free networks are remarkably rare, and scale-free structure is not a universal” (7). The pre-print triggers an instant reaction on the web, notably on Twitter.³

In the following days, Holme (2018a) comments in a blog post that “if we could rewrite history and redefine power-laws as ‘something that follows a straight line in a log-log histogram if you squint from the side of a computer screen’, then they would, for sure, be abundant” (the reconciliation position, see Figure 8(b)).

In the following days, Barzel (2018) publishes a short piece online aligned with Holme’s position: “the meaningfulness of scale-free supersedes its detailed empirical accurateness.” He finds the discussion “roughly aligned along a disciplinary divide”.

One month after Broido and Clauset’s pre-print, Klarreich (2018) publishes a piece in Quanta Magazine, presenting and explaining the controversy at length and with great clarity. Despite the absence of peer review, Klarreich writes that “network scientists agree, by and large, that the article’s analysis is statistically sound.” Like Holme and Barzel, she sees two camps and the disciplinary influence of physics. On one side, “supporters of the scale-free viewpoint, many of whom came to network science by way of physics, argue that scale-freeness is intended as an idealized model.” On the other, “critics object that terms like ‘scale-free’ and ‘heavy-tailed’ are bandied about in the network science literature in such vague and inconsistent ways as to make the subject’s central claims unfalsifiable.”

Two months after the publication of the pre-print, Barabási (2018) issues a rebuttal on his website. This self-published piece is more didactic and more polemic than a typical academic publication. For him, Broido and Clauset fail to recognize the scale-free mechanism “[b]y insisting to fit a pure power law to every network, and ignoring what the theory predicts for any of them.” Barabási also challenges the relevance of their procedure: “And the real surprise? Even the exact model of scale-free networks, following a pure power law, fails their test. … The true failure is their methodology: It fails to detect that the gold standard is scale-free.” He concludes that “the study is oblivious to 18 years of knowledge accumulated in network science.” His response ignores the question of falsifiability, and challenges the relevance of the measurement procedure.

In November, Holme (2018b) exposes “the state of affairs” in another blog post: “Simply speaking, there are two camps: those seeing scale-freeness as an emergent property, and those seeing it as a statistical property.” On one side, “emergentists … view scale-free networks essentially as outlined in Barabási and Albert’s Emergence of scaling in random networks, … [and] didn’t lose faith in the buzzwords of the nineties’ complexity science: universality, fractals, self-similarity, criticality, emergence.” On the other, “statisticalists [argue] that scale-freeness is not scientifically important if it is not testable … [and] are on top of the latest data science trends.” He concludes that “[t]he disappointing realization is that whether scale-free networks are rare [or not] is really a choice that needs to be argued by words”—a nice example of Kuhn’s (1962) “paradigm incommensurability”.

On 4 March 2019, Nature Communication publishes two articles: Broido and Clauset’s (2019) revised article “Scale-free Networks Are Rare” and Holme’s (2019) complementary article, “Rare and Everywhere: Perspectives on Scale-free Networks.”

Broido and Clauset’s (2019) revised article is substantially the same, retaining the original data and analysis, making the argument clearer and more solid. They clarify that their definition of scale-freeness is not based on preferential attachment and add a “robustness analysis” section implicitly addressing Barabási’s technical and conceptual concerns.

Holme (2019) summarizes the controversy and reflects on it. For him, the “controversial topic” is to know whether “scale-free networks rare or universal” and “important or not”. He argues that “in the Platonic realm of simple mechanistic models, … the concepts of emergence, universality and scale-freeness are well-defined and clear. However, in the real world, … they become blurry. … Now we have one camp … thinking of scale-free networks as ideal objects …, and another seeing them as concrete objects belonging to the real world.” He suggests finding consensus by acknowledging the legitimacy of studying complexity-related notions, such as scale-freeness, and the need to build a better statistical framework. He remarks finally that “it often feels like the topic of scale-free networks transcends science.”

The argument of universality is divisive. My coding identified six publications stating it (four co-signed by Barabási), and nine publications criticizing it. Only one does both (Barrat et al., 2008), making a similar point as Holme: Universality is a defined concept “related to the identification of general classes of complex networks” (76), but “all knowledgeable physicists would agree” that “the quest for universal laws … cannot apply to network science” (296). The other publications mentioning universality pick a side.

Although a formal critique of universality for complex networks exists since at least 2005 (Keller), Barzel and Barabási (2013) disregard it. They acknowledge that “a mathematical framework that uncovers the universal properties of [complex networks] continues to elude us” (673) but do not question the idea itself. Network Science (Barabási, 2016) adopts the same position. This lack of dialogue suggests incompatible worldviews.

Keller develops the most precise argument against universality. She identifies a “clash of two cultures” with the “tradition … in which the search for universal laws has taken high priority,” i.e., the nomothetic approach (2007). For her, the argument of universality is invalid, and successful only for reasons external to the criteria for scientific truth. She explains the “faith in … ‘the unique and deep meaning of power laws’” by “the rapid growth of the sector of the publishing industry” and “the remarkably effective uses of language employed in presenting these ideas” (2005: 1067). However, as power laws “are not as ubiquitous as was thought,” and it “tells us nothing about the mechanisms that give rise to them,” the claim “that scale-free networks are a ‘universal architecture’ … are problematic” (2007).

The “clash” is not about the universality of scale-freeness; this is just a disagreement. The clash is about the scientific status of the disagreement. Some consider universality disproved; others consider it legitimate. For the first, “the network emperor has no clothes” (Pachter, 2014); the second disregard the critique. This controversy is a disagreement about a disagreement, and the parties lack common ground to settle their contention. Klarreich (2018) comments on Broido and Clauset’s (2018) pre-print, that it “seems to be functioning like a Rorschach test, in which both proponents and critics of the scale-free paradigm see what they already believed to be true.” Clauset and Barabási do not see scale-freeness from the same perspective.

Clauset and Watts focus on falsifiability, in the classic Popperian sense. It shows that they see universality as a hypothesis, an evaluable statement. Clauset remarks early that Barabási’s work does not “provide falsifiable hypotheses” and later, that the “scale-free hypothesis” (Broido and Clauset, 2018) “sounds like a nonfalsifiable hypothesis” (Klarreich, 2018). Watts criticizes that “the claim just sort of slowly morphs to conform to all the evidence, while still maintaining its brand label surprise factor” (Klarreich, 2018).

In contrast, Barzel and Barabási’s (2013) nomothetic approach proceeds by postulating universality, for instance, when they seek “a mathematical framework that uncovers the universal properties of [complex networks]”. Barabási’s position on modeling also shows this. I coded the argument that empirical measures only make sense when backed by a model: Only Barabási states it during each period (see Figure 5). He sees attempts to measure scale-freeness without a model as “a mere exercise in statistics” (Barabási et al., 2005), because for him, the meaning of the findings derives from the postulate of universality embodied in the model. The postulate is part of his way to know.

The different sides of the dispute implicitly disagree on the validity conditions applicable to universality. For Clauset and Watts, universality is a hypothesis that can be proven or disproven. For Barzel and Barabási, it is an epistemic device.

The nomothetic approach to knowledge postulates the existence of universal laws, but it does not state their empirical reality. Keller’s (2005) critique that it is a “faith” is a misinterpretation. The postulate of universality determines an experimental program: which experiments to conduct and how to interpret them. But they can fail. Universality may “elude us” (Barzel and Barabási, 2013). Even so, it drives the scientific process. Questioning the postulate of universality is questioning the entire nomothetic approach. As the latter has been undeniably successful in physics, it confers on universality a remarkably solid foundation. This may explain why Holme and Vespignani defend universality.

Conversely, Keller does not acknowledge universality as a constituent of the nomothetic epistemology. She presents the physicist’s “traditional holy grail of universal ‘laws’” (2007) as if it were a horizon, while it is, instead, part of their way. Asking Barabási to abandon his “faith in, as he says, ‘the unique and deep meaning of power laws’” (2005: 1066) can be only as successful as asking a physicist to reprove physics.

Keller’s position is idiographic, as characterized by Windelband. She opposes Barabási’s universalism with the importance of the specific. She defends the relevance of studying phenomena in their uniqueness and demands that we ponder “when it is useful to simplify, to generalize, to search for unifying principles, and when it is not” (2007).

Contrary to Keller, Clauset seems to fully understand the nomothetic approach, and to acknowledge it. As we have seen, he defends Barabási’s early “apparent arrogance” and legitimacy to publish a finding “that is merely suggestive so long as it is honestly made, diligently investigated and embodies a compelling and plausible story.” However, Clauset (2005b) also demands “falsifiable hypotheses by which [Barabási’s claims] could be invalidated.” His refutation in 2018 does not touch upon the postulate of universality, but its empirical validity conditions.

The nomothetic quest for universal laws requires by nature changing theory in the face of evidence: The better model replaces the worse. Evidence always has some degree of looseness in this context, as laws are only as good as the experiments. Barabási opposes this argument to Clauset, insisting that his “findings do not undermine the idea that scale-freeness underlies many or most complex networks” (Klarreich, 2018). But “Clauset doesn’t find this analogy convincing” and replies that “it is reasonable to believe a fundamental phenomenon would require less customized detective work” (Klarreich, 2018). Clauset et al. (2009) defend a decade-old agenda of assessing the experimental validity of scale-freeness.

Barabási’s experimental program is derived from theory (it is, nevertheless, empirical). His pioneering work on scale-freeness (Barabási and Albert, 1999) prompted multiple authors to seek it in various contexts. The subsequent wave of empirical findings reinforced his claim for the pervasiveness of complex networks (list in e.g. Lima-Mendez and Van Helden, 2009). As Galison (1999) observed in another context, theorists (Barabási) “trade experimental predictions” (pervasiveness) “for experimentalists’ results” (146).

The experimental program gradually affirmed by Clauset is that of an experimentalist. It leaves behind the model-based goals of theorists and focuses instead on experimental validity—Popperian falsifiability. By reclaiming the right to invalidate theory by experiment, Clauset challenged Barabási’s nomothetic program and set foot on idiographic ground.

Galison (1999: 146) makes relevant remarks on the situation:

The controversy reveals tensions between the agendas of different subcultures where distinct approaches to knowledge prevail. Galison suggests that such epistemic rifts are the norm rather than the exception. These epistemic tensions existed before the controversy, and I see no reason to doubt they can survive it.

Barzel (2018), Vespignani, Watts (Klarreich, 2018), and Holme (2019) acknowledge the importance of Broido and Clauset’s (2018, 2019) work. Of course, it promises better validity standards for the field. But more importantly, by declaring a new experimental program, their work shows the way out of the long-lasting controversy.

I suggest a plausible interpretation of the controversy. The accumulation of evidence against the universality of scale-freeness weakened Barabási’s empirical program. However, most actors still agree on the pervasiveness of complex networks—whatever that means. As Broido and Clauset’s program is resilient to the critique of universality, actors may adopt it to design their own experiments. I see theorists and experimentalists as the two legs of the field. When the theoretical leg weakened, the weight naturally shifted to the experimental leg. The controversy made visible an otherwise latent difference of perspective.