Q&A: Alexa Koenig on the potential and pitfalls of open source investigations

Sign up for The Media Today, CJR’s daily newsletter.

On October 17, the parking lot outside Al-Ahli Arab Hospital in Gaza, where hundreds of displaced Palestinians had amassed, was hit by an explosive of uncertain origin. The New York Times, among other outlets, quickly reported a claim by Hamas that an Israeli air strike was responsible. Israel denied this, claiming that a rocket launch by a Hamas-allied militant group inside Gaza had misfired. Soon, an Al Jazeera video emerged that seemed to show a launch from inside Gaza. Other outlets, including the Associated Press, began to cite the Al Jazeera video in their analysis. Then, more videos emerged. One, posted to X (formerly Twitter) by a data analyst, seemed to show that the rockets in the Al Jazeera video had nothing to do with the incident at all. The AP issued a retraction. The Times separately changed its initial headline and issued an apology for relying too heavily on Hamas’s claims.  

The media’s handling of the incident demonstrated both the promise and pitfalls of reporting based on video footage and other data sourced remotely from the internet, including satellite images, social media analytics, and publicly accessible datasets. The ability to turn to open source investigative techniques (often referred to as “OSINT”) has made it possible for reporters to challenge official accounts of the news and uncover stories that they could not have told otherwise. But reliance on this kind of analysis has also fueled misinformation by lending a false sense of certainty to claims based on impartial and, often, unverified material. 

When Alexa Koenig started the Investigations Lab at the University of California, Berkeley, Human Rights Center in 2016, she did so to teach future investigators not only how to leverage a growing arsenal of digital forensic tools, but how to do so responsibly. Students in the lab partner with organizations including Amnesty International and the United Nations to investigate human rights violations and potential war crimes. Their work often involves the arduous process of verifying footage, including cross-checking minute details against satellite data. As OSINT methods grew in popularity around the world, Koenig observed a need for a unified ethical framework and set of professional standards to guide practitioners and increase the effectiveness of the field. Koenig and her team convened experts across a range of disciplines for a series of workshops, producing a “Berkeley Protocol on Digital Open Source Investigations” that was published in 2020 in collaboration with the UN, and established best practices.

Koenig is co-faculty director of the Human Rights Center and director of its Investigations Lab, as well as an adjunct professor at the UC Berkeley School of Law. Recently, we spoke about the promises and challenges of OSINT for journalism, training students in the method, and why the field needs a code of ethics. Our conversation has been edited for length and clarity. 

YRG: What were you hoping to achieve when you founded the Investigations Lab in 2016?

AK: The idea was to bring together students from different disciplines to support our partner organizations conducting digital open source research. We’ve seen how new and emerging technologies are transforming fact-finding. We were part of an effort that began in the early 2000s to think through what these new forms of information meant for evidence collection, for identifying sources, getting background information on atrocities, and so on. Part of that came from our work looking at why a series of very high-profile cases in the International Criminal Court in 2011 were falling apart at very early stages of prosecution. We found that the judges were critiquing the prosecutors for relying on the stories of witnesses without bringing in the corroborating information needed to meet the court’s evidentiary standards. That kicked off thinking about how satellite imagery, drone footage, big-data analytics, and all this smartphone-generated content provided a new window into the world of people affected by human rights crises.

Journalism, social science, and human rights spaces really didn’t have personnel trained in how to mine these digital sources. We realized we could train students to understand the strengths and weaknesses of these new forms of data to be responsible researchers in getting information out to the world. We do projects that may last only a semester of two and projects that may last multiple years. We had a longstanding relationship with different fact-finders looking at the genocide in Myanmar. This semester we’re wrapping up a project that involves pulling together a guide for open source investigators on how these methodologies can be helpful in reporting on environmental crimes and climate change.

You mentioned the potential weaknesses of these methods. Can you give me some examples?

One thing that’s been lauded about open source investigations is that they can be democratizing. Most people, if they want to, can learn some of these skills fairly quickly. It takes a lot of practice to get really good at them, but they’re not out of reach for most people who have any proficiency with the internet. I think the challenge, however, is the community’s growth without the safeguards in place around professional ethics. Just because you can do something in digital spaces doesn’t mean you necessarily should. 

An example: a journalist came across a video of a woman being raped, and decided to write an article about it and post stills from the video, including some that included the woman’s naked body. [The journalist] was trying to draw attention to this really horrific crime and see if people could help identify the perpetrators depicted in the video. However, there was [pushback] over whenever it was appropriate to depict the victim, and the publication ultimately removed some of those photos. It’s [a question of]: what can I do versus what should I do? Of course you can post stills from a video that’s been publicly posted, but should you have the consent of the victim? Can you think about ways to protect that victim from further dignitary harms before you do so? 

Why did you feel the Berkeley Protocol needed to happen?

We began to think that there needed to be protocols or standards for doing this work, even just in the use of terminology. For example, a journalist would call information that they found online “open source evidence.” The lawyers who were working with the journalists would get upset because “evidence” is the information presented in a court of law to prove a fact in dispute. We were finding that people were talking across each other. I had also begun receiving calls from lawyers, investigators, and journalists who were looking at crises erupting in the Middle East and saying, I’ve been getting videos and photographs sent to me from people over WhatsApp, or, I found a bunch of videos that are relevant to my cases or story on YouTube. How do I download and preserve this content? How should I tag it and code it so other people can find it later?

Another concern was how misleading social media content can be. There can be misinformation or disinformation, fully fabricated data, stuff that’s been staged. We realized that everyone—from practitioners to potential judges who came into a courtroom to readers of a publication—needed to know how to assess the quality of these investigations and the underlying data. It’s very easy to show a satellite image, and draw a bunch of different colored boxes in circles around buildings and trees, and compare it to stills from a video to say where and when something happened. But a lot of people really don’t have the training to assess whether that’s correct.

What did the process of developing the protocol involve?

We pulled together a group including journalists, trial attorneys in the International Criminal Court, investigators from the ICC, and human rights practitioners to talk about international guidelines. We had three days of meetings. The first day I expected us to just get a few terms in place to make sure we were all talking about the same thing. Instead of taking the hour or two that I had allotted, it took eight hours. We realized we needed a tool to communicate across communities of practice. We interviewed about a hundred and fifty people across the globe about standard methodology and terminology. We also hosted a series of workshops about contested areas, like, when is it appropriate to use a false identity when doing online research? It turns out there are real tensions between the journalism community and the legal community about this. For law enforcement who also do digital open source investigations, you have a third set of considerations. 

One of the big workshops focused on ethics. It really came down to three things: What are the security considerations of everyone involved? That can be physical security, digital security, and psychological security. The second bucket is dignity. Are you violating the dignity of other people? And third, accuracy. Are you getting closer to the truth or are you actually just making it look like you are? One thing the Berkeley Protocol recommends is thinking through all these considerations in advance of starting an investigation and doing what’s called a “digital landscape analysis” of who is communicating in digital spaces—how that differs by gender, by geography, by age—so that you’re being inclusive. We also decided to make the Berkeley Protocol tool-agnostic. We didn’t name Facebook or Twitter or any tools for downloading and preserving digital footage because, if we did, the protocol would be obsolete by the time it was finished. The online environment shifts so quickly. It’s a foundational set of principles. Each investigative team really needs to build their standard operating procedures.

More and more newsrooms are adopting OSINT or establishing new teams to focus on the methods. How do you think this has been going? What do you think could be improved?

It’s been really exciting to watch the proliferation of news outlets doing this work. By telling stories in a visual format, they’re engaging to audiences in ways that print stories may not. I worry that there’s not much training around OSINT ethics yet. I think there’s a trap that people can fall into, of not thinking about the communities that have been affected by violence and how [exposure] can magnify that trauma for that individual or those communities. I would also love to see more training within journalism teams [on verifying digital material]. Journalists are working with such limited budgets and are working so quickly that they often don’t feel they have the luxury to do those additional modules beyond the techniques themselves. But ultimately, it’s what’s going to keep us all safe and make sure the reporting is high quality. 

One of the things that we recommend for verification in the Berkeley Protocol is that whenever you see a video or photograph that you found online, you go through a three-step, very formal verification process: you look at any technical data that is still attached, you do content analysis—look at the photo or video compared to satellite imagery and analyze if what you see is consistent with what you’ve been told about it—and then you do source analysis.

There is an infamous video called “Syrian Hero Boy” that claims to be of a young boy rescuing a young girl under fire in Syria. The media outlets that decided to run with that video, as a feelgood story from Syria, often hadn’t done that source analysis; they hadn’t really queried, who actually posted this and why would they be posting it? The ones who didn’t [run the story said], we don’t know enough about the source of that to really trust that this is what it claims to be. And they made the right call: this had actually been staged in Malta on the film set of Gladiator. When the director of this short film was asked why he had put this video up on YouTube anonymously, he said that he felt that the Western world wasn’t paying enough attention to the plight of Syrians. A lot of the media amplified his video as if it had been legitimate and ended up putting fake information into the world. You can see how easy it is when you don’t have the training to ultimately get it right.

If you just tack these methods on top of traditional journalism, you’re missing a lot of what has made the open source investigations community so strong, which is the spirit of innovation and collaboration across disciplines. I think we can learn so much from each other when we haven’t all gone through the same academic programs or the same ways of training. It’s going to be important for journalists and editors to work with people from other disciplines. To really stay ahead of the ways these methods can enrich storytelling—to reach new audiences or better engage audiences—will require being part of that community.

Other notable stories:

• Earlier this week, the father of Anas Al-Sharif, a reporter for Al Jazeera in Gaza, was killed in an Israeli air strike on his home in a refugee camp. Al-Sharif had previously reported receiving threats from the Israeli military; the Committee to Protect Journalists said that it was “deeply alarmed” by a “pattern of journalists in Gaza reporting receiving threats, and subsequently, their family members being killed.” Elsewhere, The Guardian’s Ruth Michaelson spoke with Plestia Alaqad, an Instagrammer in Gaza who became a citizen journalist during the war, and whose “personal touch found a niche that television news failed to capture.” And—following multiple reports confirming that Israeli tank fire killed a Reuters journalist and wounded others, including from Agence France-Presse, in southern Lebanon—the French government called for a probe.
• The Verge is out with a package of stories on “the year Twitter died”—blaming Elon Musk for killing it, and grappling with its legacy as a “news cycle accelerator, a tool of mass harassment, an idealistic money-losing workplace, and an infinite joke machine.” Nilay Patel looked back on how the platform shaped—and ultimately broke—the news industry. “I had countless conversations with reporters in the pre-Elon Musk Twitter era who could not fathom doing their jobs without the site, who insisted that Twitter was where sources and scoops and true insight lay,” Patel writes. “It was utterly wrong and also completely understandable in a way that made arguing about it futile: Twitter could feel like a direct connection to a constantly shifting force in the culture.”
• Recently, members of the News Leaders Association—which was formed out of a merger between the American Society of News Editors and the Associated Press Media Editors in 2019—voted to dissolve the organization by next summer, after devolving its functions and programs to other nonprofits. Yesterday, the NLA announced that the Joseph L. Brechner Freedom of Information Project at the University of Florida will now oversee the NLA’s Sunshine Week, an annual celebration of open government, while the Poynter Institute will inherit its awards program. The group has yet to announce new homes for its diversity survey, training programs, and historical records.
• Authorities in Russia have reportedly charged Alsu Kurmasheva, a Russian American journalist for the US state-backed broadcaster Radio Free Europe/Radio Liberty, with distributing “fake news” about the Russian military. Kurmasheva was already in detention in Russia after she was arrested earlier this year on charges of failing to register as a “foreign agent.” (The charges marked an escalation in Russia’s weaponization of that law, as we wrote at the time.) Over the weekend, protesters rallied in the Russian city of Kazan in solidarity with Kurmasheva, though law enforcement quickly broke up the rally.
• And Politico’s Jack Shafer offered some tips to Senator J.D. Vance, who beat Shafer to the “pinnacle of press criticism” last week when he urged the Justice Department to probe Robert Kagan over a Post op-ed in which Kagan discussed the prospect of resistance to a Trump dictatorship. “As much as I might want to sympathize with his takedown of a Washington Post contributor (we press critics must stick together!),” Shafer wrote, Kagan is not “an American Lenin out to smash the federal government.”

ICYMI: Azerbaijan cracks down on the free press. Again.

Has America ever needed a media defender more than now? Help us by joining CJR today.

Yona TR Golding was a CJR fellow.