Combining Results of Multiple Twitter Searches into one File on the Cheap

Twitter is a great subject for social media research because 1) it is used by a lot of active and influential people and 2) its data is presumed public, obviating privacy concerns. Yet the sheer volume of Twitter data poses problems for researchers, especially those without thousands of extra dollars needed to harness insane amounts of computer power. Part of the solution for modest researchers at small institutions like myself is to study relatively small-scale subjects. Another part of the solution is to tie together multiple low-cost solutions and not look for one magic software package to address all needs.

I’m working on a project right now in which I’ve been following all tweets by and tweets mentioning members of the Maine State Legislature over time. I could write a program in PHP using the Twitter API to accomplish this… if I had a bit more time and know-how. I’ll try to get these later, but for now, I’m running multiple copies of the program Tweet Archivist Desktop, each of which captures and saves tweets by or regarding one Twitter account as they’re posted. Tweet Archivist Desktop costs just $9.99 for a perpetual license, which I consider well work the price.

Tweet Archivist Desktop creates a separate .csv dataset for each of the searches I’m saving. To gather them all together, I’m following advice shared helpfully by solveyourtech. On my Windows laptop, I’m entering the command prompt and combining all csv files in a folder into a single csv file with a variant of the “copy” command.

copy command in Windows command prompt combines multiple csv files into one

Remembering Pete Seeger 1-28-14: Collective Memory, Shared on Twitter

Activist folksinger Pete Seeger died at the age of 94 on January 27, 2014. As word of Seeger’s death spread on January 28, Twitter was flooded with tributes, including 28,226 posts made to the social media outlet’s #PeteSeeger hashtag channel by 9 PM. Of those posts, 21,617 (some 76.8%) were “re-tweets” of others’ posts. Pete Seeger wouldn’t have minded: he was a staunch believer in people forming publics to sing together, hearing a call and issuing a response, finding a tune and amplifying it not by microphones but in sheer numbers.

What did the world sing today about Pete Seeger? To answer that question, I tuned the Tweet Archivist Desktop (a handy $10 tool) to the #PeteSeeger hashtag, where it archived users’ public posts silently and efficiently in a background window on my computer. I used NodeXL (free and open-source) to find the most common word pairs in posts and to visualize them in the graphic you see below. When pairs are connected into chains and webs, the result is a semantic network that captures the spirit of the day.

Remembering Pete Seeger: a data visualization of a semantic network of the most common words and their connections in the 28,226 #PeteSeeger Twitter contributions from midnight to 9 PM on January 28 2014

In case you’re wondering, the word “communist” only appears 29 times in all those posts, far too rarely to reach the threshold required to appear in the image. “Thank” or “thanks” appears over 2,000 times.

A Hashtag Contested: Positive and Negative Social Media Reaction to the RSA-NSA Scandal

For some time now, public relations professionals have been worrying about “the bashtag problem.” Corporations may spend years cultivating positive conversations about their products over social media by developing and promoting a hashtag, only to see “their” hashtag fall into bashtag status when negative social media posts about that organization swamp the positive posts the organization seeks. Upset that public criticism may “ruin their brand,” some corporations have developed intimidation strategies to shut up and shut down isolated critics. But when large numbers of people join in the bashtagging, there’s no easy way to stop the dissent.

Through the fall of 2013, cybersecurity corporation RSA enjoyed positive references on its #RSAC hashtag on Twitter that it had developed to advertise its annual professional conference. In late December, however, it emerged that RSA’s data encryption products had a “back door” built into them that allowed the National Security Agency (NSA) to break users’ encryption and (possibly without a warrant) snoop on private communications. On December 23, RSA issued a “non-denial” that seemed to implicitly acknowledge the arrangement. On that day, the positive flavor of the #RSAC hashtag changed.

After collecting the Twitter posts (or “tweets”) of the #RSAC hashtag using the Tweet Archivist Desktop, I’ve looked at the content of each one, determining whether its attitude toward RSA or the RSA Conference (RSAC) was positive, negative or neutral. The following graph tracks the volume of positivity, negativity and neutrality in the #RSAC hashtag from December 21 2013 through January 14 2014 (today):

Volume of Tweets Positive, Negative and Neutral Toward RSA in the #RSAC hashtag, 12/21/2013 to 1/14/2014

After an initial burst in which some prominent conference speakers canceled their participation in protest, it appeared that negative tweets regarding the RSA Conference might abate over the end-of-year holidays, and RSA began to use the channel to promote its conference again. Then, on January 7, RSA let out a teaser of a Tweet about the identity of its keynote speaker:

RSA Tweets on January 7 2014: Click here to find out who has been announced as #RSAC closing keynote speaker for 2014

That speaker is Stephen Colbert. With a celebrity drawn into the story, public attention returned, generating a new peak of critical #RSAC tweets that seems to be continuing. Some of those tweets are original, but the bulk of them constitute just a few messages, tweeted and retweeted over and over again over the #RSAC hashtag channel. Anti-surveillance social movement organization Fight For the Future has deployed a special web page

Fight for the Future asks its followers to send out automated tweets to overwhelm the #RSAC hashtag

… on which it asks its followers to share this message on Twitter: "Surveillance is no joke! Tell @StephenAtHome to cancel his keynote at this NSA tainted conference. #RSAC"

15.4% of all Tweets on the #RSAC hashtag from December 21 2013 to January 14 2014 are this one Tweet, posted over and over. Another Fight for the Future mass tweet, "Does Stephen Colbert secretly love the NSA? There's only one way to find out: #RSAC," accounts for another 2.1% of #RSAC Tweets during the period.

Fight for the Future is part of a coalition of anti-surveillance groups who have announced a national day of protest on February 11. It’s called “The Day We Fight Back.” Where will the fight be? On the streets? Will there be a march? A picket? A rally in some square?

Apparently not. According to press materials, all activities will be taking place on the internet, where followers will be encouraged to share graphics on their blogs, to change their profile photos on Facebook, and to chant pre-written slogans over Twitter.

In American social movements, web banners are replacing cloth banners. Marches are giving way to orchestrated internet bashtagging. Yesterday’s gone, yesterday’s gone.