Sign up for The Media Today, CJR’s daily newsletter.
On Friday, Ellen Weinstein, an award-winning illustrator based in New York, found herself in unusual company. She was in one of the reading rooms at the New York Public Library, working with the members of NYPL Labs, who are trying to “re-imagine the library for the digital age.”
Six other similarly incongruous pairs were at work throughout the city, from Dumbo to Morningside Heights. Like Weinstein and NYPL Labs, whose members include multimedia artists, designers, and developers, each group was tasked with crafting a “story” in the broadest sense of the term.
They were all part of “bit by bit,” a weeklong project in digital storytelling hosted by the Brown Institute for Media Innovation and the Columbia Journalism School. It connected storytellers, such as artists, writers, or photojournalists, with technologists, like data scientists, coders, and developers. The aim was to see how combining skillsets could lead to new ways to tell stories that stretched narrative beyond its usual prose or visual arts foundation. The project stems from the idea that algorithms and code are no less creative than words or art and can play an equally constructive role in storytelling.
Back at the library, Weinstein was at a table strewn with paper and paint, brainstorming with the lab’s crew. Their product or presentation was due Saturday afternoon. The teams had been introduced last week but they had begun work only on Thursday. Time, Weinstein said, was the only constraint.
“There are no limitations,” she said. “That’s what interested me because I am usually working within pretty strict parameters.”
But not this time. Her work with the NYPL Labs produced an interactive app for live chats, complete with animal-like figures whose shapes shifted based on the words, tone, grammar, and even timestamp of each chat. Given that the substance of emails, online chats, etc., is often pulled for targeted advertising or even alleged surveillance, Ben Vershbow, the founder of NYPL Labs, said this was an attempt to use the same digital content for something more creative and even whimsical.
The teams presented their work at Columbia University on Saturday before a large audience. First up were Nicholas Lemann, former dean of the J-school, and Hanna Wallach, a researcher at Microsoft and computer science professor at the University of Massachusetts. Wallach and Lemann developed an application that could help investigative journalists cull information from large volumes of data. They analyzed nearly 88,000 documents with a tool that grouped words, identified sets of related words, and represented the frequency of different words or sets over time. Journalists can then identify “spikes” in the frequency of certain word distributions and narrow in on the cause. This application was different from existing ones, Lemann said, because it was useful for analyzing data when “you have no idea what’s in there.”
There were two other journalists among the seven pairs: Asim Rafiqui and Nina Berman, both photojournalists.
Rafiqui had been paired with Kati London, a data researcher and coder. They developed an interactive, photo-based platform where the photo ceases to be a still image but is embedded with text, video, audio, more images, and even prompts that allow the viewer to not just interact but also edit certain components before sharing the final product. They said they hoped to make photos “touchable and playable,” an idea that attracted London, who said she has always been interested in how media interacts with people. When they were questioned about the risk of allowing viewers to “edit” content, which could lead to ethical problems involving authenticity and accuracy, Rafiqui said he saw such a product only as part of a reputed publication that could limit the extent to which users modified the content.
Berman and her teammate, Gilad Lotan, a data scientist, investigated what they suspected was a twitter bot–Wake Up Now, a venture promising quick money to investors. They followed the alleged bot’s social media and digital trail, traced it to a company’s website, and spoke to one of its CEOs and a few of its young investors. Berman said they still had plenty of questions, including why so many young people are drawn to this murky way of making quick money online. They hope to continue their investigation.
The rest of the “stories” included an application that matched sentences with similar underlying patterns by digital art practitioner Jer Thorp and Joel Glibb, a Canadian musician and songwriter; a surreal video by poet-cum-architect Vito Acconci and artist-cum- computer scientist Jonathan Harris; and an experimental installation on walking by landscape architect Diana Balmori and visual designer Phoenix Perry.
The “bit by bit” teams with journalists emerged with projects that had narrative potential. The ones with artists were more inventive and certainly less utilitarian. But all the teams seemed to have enjoyed the process–the collaboration, the chance to peek into a creative universe different from their own–which they spoke of with great enthusiasm. “It felt so far afield from anything I have ever done,” said Weinstein. That’s why she knew she had to do it. And she discovered that there was such a thing as “beautiful code.”
Has America ever needed a media defender more than now? Help us by joining CJR today.
