+ 
Case Selection
++ In order to be included, cases must fulfill three criteria. +
++ First, cases must involve allegations of foreign interference by primarily digital means. The Australian Government Department of Home Affairs defines foreign interference as activity by a foreign actor that is "coercive, corrupting, deceptive, or clandestine" in nature, distinguishing it from the more benign phenomenon of foreign influence. By focusing expressly on digital activity, this definition denotes a range of interference activities—including disinformation, media manipulation, and cyber intrusion—that are conducted by foreign actors to affect political outcomes. +
++ Second, cases must be novel. A novel case is one which involves a fresh foreign interference claim or which reveals new evidence to reinvigorate an old one. A novel case is also one in which significant newsworthiness is attached to the individual or organization making the claim. In general, a president or ex-president’s claim is novel regardless of the evidence presented. Meanwhile, an op-ed or report by a mid-level U.S. official is only novel if it contains previously undisclosed information. +
++ Third and finally, cases must be relevant to the 2020 U.S. election. This focuses case selection on alleged foreign interference that seems intended to influence voting behaviors, denigrate particular candidates, or engage in political or social issues of direct relevance to the election. It also bounds case selection to foreign interference claims that have occurred around or following the 2018 U.S. midterm elections. +
+Attribution Impact
++ Attribution Impact measures the spread of case-related articles and content over the seven days following a foreign interference allegation. It is a sum of the cumulative Facebook engagements, Twitter shares, and Reddit engagements of relevant articles, intended to capture the digital impact of a given case. It is best used as a comparative, rather than absolute, measure. +
++ This sum is computed by a standardized query process that utilizes SerpApi (which enables "Google dorks" using advanced Google search operators), BuzzSumo (which enables the query of a large database of social media content and collection of engagement data), and CrowdTangle (which enables the collection of engagement data for content not indexed by BuzzSumo). +
++ The following steps are followed in order to determine the Attribution Impact. This process is automated by Python and run by means of a Google Collab notebook: +
+-
+
- + A case's Link of Attribution is used by BuzzSumo to compute a list of backlinks, which includes social media engagement data. + +
- + To account for content not indexed by BuzzSumo, SerpAPI is used. The Google dork is standardized by way of several variables: Source, Disinformation, Disinformant Nation, and Date of Attribution. The standard Google dork is “intext:Source AND Disinformant AND Disinformant Nation.” (In some cases, alternate names are substituted. These substitutions are documented in the raw dataset.) The resulting Google dork is reviewed by coders and irrelevant results are deleted. + +
- + The edited Google dork is automatically reconciled with the list of BuzzSumo backlinks. + +
- + An additional BuzzSumo search is performed on remaining web links, importing social media engagement data where matches are found. + +
- + Where matches are not found, web links are visited manually and their social media engagement data is recorded via the CrowdTangle link checker extension for Google Chrome. + +
- + Social media engagement data is tallied from these three sources—BuzzSumo backlinks, the BuzzSumo search of web links, and manual CrowdTangle input—to arrive at the final Attribution Impact. + +
+ The fixed timeframe, standardized Google dork formula, and opacity of the tools themselves means that relevant social media engagement data may be excluded. Therefore, the Attribution Impact should be treated as a rough estimate, most useful for comparing between cases. +
++ It is the intention of the FIAT team to introduce additional dimensions of Attribution Impact as the tool evolves. +
+Attribution Score
++ The Attribution Score is a framework of eighteen binary statements (true or false) that assess foreign interference claims made by governments, technology companies, the media, and civil society organizations. The measure is intended to capture the reliability of the attribution as discernible through public sources rather than to serve as a fact-check of the attribution itself. If a statement is deemed applicable, a point is awarded. If a statement is deemed inapplicable or irrelevant, no point is awarded. Initial coding was reviewed by the FIAT research team and shaped by iterative discussion. +
++ This scoring system is based on the experience of DFRLab experts in assessing—and making—such allegations. It is also based on a review of work produced by the wider disinformation studies community, and particularly resources compiled by attribution.news. +
++ The Attribution Score is composed of four subsections: +
+Credibility
+-
+
- The source of the attribution does not have a direct financial interest in a certain attribution outcome. +
- The source of the attribution has a diversified and transparent funding stream. +
- The source of the attribution does not strongly endorse a specific political ideology. +
- The source of the attribution is in no way affiliated with a political campaign. +
- The source of the attribution has not previously promoted mis- or disinformation. +
Objectivity
+-
+
- The attribution avoids using biased wording. The attribution avoids high-inference or emotive language. +
- The headline accurately conveys the content of the attribution. +
- The attribution clearly distinguishes factual information from argumentative analysis. +
Evidence
+-
+
- The attribution provides a clear illustration of the methods, tactics, and platforms involved in the alleged information operation. +
- The attribution contextualizes the engagement with, and impact of, the alleged information operation. +
- The attribution identifies actors and states allegedly responsible. +
- The attribution clearly explains the strategic goal and rationale of the actors who conducted the alleged information operation. +
- The attribution relies on information which is unique to, or can only be procured by, the relevant actor. (e.g. classified information for U.S. federal agencies, back-end/developer information for technology companies) +
Transparency
+-
+
- The attribution provides open access to a dataset or archived links of alleged assets. +
- The attribution methodology is clearly explained. +
- The attribution is replicable through open-source evidence. +
- The attribution acknowledges relevant limitations or mitigating factors in its assessment. +
- The attribution has been corroborated by a third party or independent investigation. +
+ An attribution that scores fifteen points or higher is especially reliable. An attribution that scores six points or fewer should be carefully scrutinized. +
+
+
+