Social Media Data Mining with Raspberry Pi: 9 Videos for the Complete Beginner

Since the start of this year, I’ve been working on a project to take a $30 Raspberry Pi 2 computer turn it to create a social media data mining machine using the programming language Python. The words “programming language” may be off-putting, but my goal is to work through the process step-by-step so that even a complete beginner can follow along and accomplish the feat.

The inexpensive, adaptable $30 Raspberry Pi 2I’m motivated by two impulses. My first impulse to help people gain control over and ownership of the information regarding interaction that surrounds us. My second impulse is to demonstrate that mastery of social media information is not limited to the corporate, the government, or the otherwise well-funded sphere. This is not a video series for those who already are technologically wealthy and adept. It’s for anyone who has $30 to spare, a willingness to tinker, but the feeling that they’ve been left out of the social media data race. I hope to make the point that anyone can use social media data mining to find out who’s talking to whom. The powers that be are already watching down at us: my hope is that we little folks can start to watch up.

I’m starting the project by shooting videos. The video series has further potential, but has proceeded far enough along to represent a fairly good arc of skill development. Eventually I’d like to transcribe the videos and create a written and illustrated how-to pamphlet; these videos are just the start.

Throughout the videos, I’ve tried not to cover up the temporary mistakes, detours and puzzling bugs that are typical of programming. No one I know of hooks up the perfect computer system or writes a perfect program on the first try. Working through error messages and sleuthing through them is part of the process, and you’ll see that occasionally in these videos.

Please feel free to share the videos if you find them useful. I’d also appreciate any feedback you might have to offer.

Video 1: Hardware Setup for the Raspberry Pi

Video 2: Setting up the Raspberry Pi’s Raspbian Operating System

Video 3: Using the Raspberry Pi’s Text and Graphical Operating Systems

Video 4: Installing R

Video 5: Twitter, Tweepy and Python

Video 6: Debugging

Video 7: Saving Twitter Posts in a CSV File

Video 8: Extracting and Saving Data on Twitter URLs, Hashtags, and Mentions

Video 9: Custom Input

Feeling Like a Fraud in the First Year of College (and why that’s OK)

When I ask students in their first year of college how they’re doing, I often get a pained look in response. it’s common for students to feel a bit (or maybe a whole lot) like they don’t fit, even like they’re a fraud, not a “real student.” Do you ever feel that way? Well, here’s a confession: I felt that way myself when I got started as an undergraduate student, and it took me some time to shake that feeling. It turns out that this kind of feeling is absolutely normal and even a kind of standard part of getting used to a new role. As Erving Goffman explains in his dramaturgical theory of social interaction (Goffman 1959), we are all in a sense playing roles on a public stage, trying to pull off our scenes, trying to remember our lines. Difficulty in performing a role like that of student doesn’t mean you’re a bad human being — it just means you need to be patient with yourself and give yourself a bit of time to practice your new role before you can feel like you’ve truly nailed it. I share my thoughts on this subject in the video below:

William Shakespeare shared this sentiment in two of his plays:

“Life’s but a walking shadow, a poor player
That struts and frets his hour upon the stage
And then is heard no more: it is a tale
Told by an idiot, full of sound and fury,
Signifying nothing.”
– Macbeth

“All the world’s a stage,
and all the men and women merely players:
They have their exits and their entrances;
And one man in his time plays many parts.”
– As You Like It

Goffman would agree with the second quote, but disagree heartily with the second quote. For Goffman, the lines we deliver and the successful scenes we accomplish in interaction signify a great deal. Indeed, the whole point of our strutting and fretting our hour upon the stage is to learn how to signify to one another. An undergraduate’s role in the first year is hard, because a college or university education involves a whole new way of signifying what matters, how it matters, and how we know what matters.

The bottom line? Feeling like a fraud in the first year of the undergraduate experience is not just normal; it’s actually OK, a sign of growth, a sign that you’re extending yourself into a new role. As the popular slogan goes, you just have to “fake it till you make it,” to keep practicing the new role until you get the part down well. A die-hard dramaturgist might say that there’s really no difference between someone who perfectly impersonates a success and an actual success.

Just keep practicing, keep pretending until the pretense becomes real. You can do it.


Goffman, Erving. 1959. The Presentation of Self in Everyday Life. New York: Random House

“Hello, My Name Is…”: Gender in Sims 4 Choices

Gender socialization involves first the limits presented by the messages that we are sent, second our reaction to those limits, and third our efforts to reinforce or change those limits. Gender socialization often happens through media, and one of the newest and most popular forms of media is the computer game.  To examine gender socialization in computer games, I’ve recorded a video that walks through the various options presented for the development of “male” and “female” characters in a very popular game called Sims 4.

As you watch the brief (6 minute) video of the results below, consider: what messages do the options for the two available gendered categories of “male” and “female” identity send us? What choices does the player have?  What choices does the player not have?


Here’s what I noticed during my walkthrough:

  • Most crucially, only two named categories are presented.  They’re labeled “male” and “female,” although really what’s being performed here is not explicit biological sexuality but rather social indicators of masculinity and femininity.  This is indeed about socialized gender, not strictly sex.  If we’re dealing with gender, would you say there more than two gender categories put on display in the Sims 4 character design feature?
  • Walks: Both Male  and Female  Characters may have a “Perky” or “Snooty” Walk…
  • For both Males and Females, a particular walk is labeled “Feminine”
  • For males, available voices are “clear, “warm, ” and “brash.”
  • For females, available voices are “sweet, “melodic, ” and “lilted.”
  • The female character shows a “romantic” trait by drawing in, bending, and turning.
  • The male character shows a”romantic” trait by placing legs apart, stepping forward.
  • Both male and female characters display identical “Bro” behaviors.
  • Male stylized character packages display a wide stance, a direct forward gaze and thorough coverage…
  • …except for the “Emo” package, which displays feminized characteristics in dress and bodily arrangement.
  • Some female stylized character packages display a somewhat wide stance and a direct forward gaze…
  • …but some draw inward and back, have an indirect gaze. Most are less covered by clothing.
  • Available tops for male characters  cover the torso thoroughly.
  • Available tops for female characters  are more likely to expose the torso …and present more alternatives.
  • Female full-body clothing options for characters expose skin, as do bottom clothing choices.
  • Male full-body clothing options for characters are less likely to expose skin, and present fewer choices.
  • Male bottom clothing options for characters are also less likely to expose skin, and also present fewer choices.
  • 33 accessory clothing accessory choices are available for female characters.
  • 14 accessory clothing accessory choices  (less than half) are available for male characters.

That’s what I noticed.  What do you see?

New Lesson Plan: Frame Alignment Operations in Political Testimony

I’m really excited today to roll out my second lesson plan for the political transparency website Open Maine Politics. has been mashing together census, social media, newspaper and legislative data for some time now, and now it’s time for me to turn my attention to education. As a professor of social science, I have a vision of political data as a source for learning about issues of representation, gender, framing, and social network formation — but up to now it’s all been in my own head. Sharing lesson plans for undergraduate university students (and upper-level high school students) is a labor of love.

So, I’m glad to introduce Lesson Plan Two: Frame Alignment Operations in Political Testimony, complete with:

  • References to Erving Goffman’s and David Snow’s theoretical work on frames and frame alignment!
  • Examples drawing from obscenity laws and Lenny Bruce on trial!
  • Primary Source Documents for students to find frames: testimony on bills before the Maine State Legislature!
  • An Interactive Padlet where students can post their findings!

Oh, what fun. Give it a whirl, and if you like the gist of it, please feel free to use the lesson plan in your own work with students (a nice link for attribution is all I need.)

Stages of Teaching and Learning Social Media Analytics (Presentation Notes)

This afternoon, I’ll be making a short presentation of thoughts on teaching social media analytics at the 2015 conference of the International Communication Association as part of its BlueSky Workshop on Tools for Teaching and Learning of Social Media Analytics. While the workshop is focused on the experience of teaching using a series of particular tools, I am interested in rejecting the question, “Which tools are best for teaching?,” and supplanting it with the idea of building capability in students in a progressive strategy. At different stages in students’ development as social media researchers, different analytic platforms may be more or less appropriate as teaching tools.

Below is a copy of notes for my presentation; notes can also be downloaded as a PDF here.

Objective: To introduce unexperienced undergraduate students to the process of analyzing social media with sufficient breadth that they may continue to learn independently.

Teaching Challenges Provoking Implementation:

  • As the mandate for higher education continues to widen, undergraduate students tend more and more to be non-traditional, to lack preparation, to lack confidence, and to be fascinated by but intimidated by math, research and technology.
  • Social media platforms are in a state of constant change.
  • Social media analytics packages and methods are rapidly evolving now and are likely to experience significant change in the next decade.

Learning Outcomes: Students who complete a course in social media analytics will be able to:

  1. Find and navigate social media platforms
  2. Recognize the common elements of social media:
    1. Individuals
    2. Actions
    3. Memberships
    4. Relationships
  3. Extract observations of these elements into datasets:
    1. Individual-level
    2. 1-mode network
    3. 2-mode network
  4. To analyze data and report data visualizations, qualitative categorizations and quantitative statistics

Strategy: A gentle, stepwise series of stages taking students from where they are to where they need to be, introducing students to a variety of analytic platforms, and focusing on the social research skills that will remain constant despite changes in social media and social media analytic platforms.

Stages of learning social media analytics, from Consumer to Manager to Secondhand Gatherer to Primary Gatherer to Analyst

Teaching Challenges in Implementation:

  • Universal access for students who no longer share a common campus, common hardware and common software
  • Reasonable yet challenging entry for students who come to class with a variety of previous experience and capabilities
  • A variety of reasonable endpoints for students who vary in their level of progression and accomplishment

The University Without Walls: Splash Brings Teaching and Learning Together

The traditional model of university learning truncates students’ vision on both ends. High school students may be told that they should aspire to higher education, but unless family members are part of that tradition they may not know why.  Once admitted, undergraduates study academic subjects and are tested for signs of accomplishment, but have limited opportunities to take the next step of sharing their knowledge and skills with others.

MIT Splash 2014 scene at the registration desk in the Infinite Corridor, Memorial Lobby

In my role as a parent, I recently accompanied my ninth-grader to the Massachusetts Institute of Technology’s Splash program. Taking place every year on the weekend before Thanksgiving, Splash is a 2-day, 20-hour marathon of 1-hour classes taught by MIT undergraduates on subjects tending heavily toward the natural sciences, computer science and mathematics but also including the social sciences, humanities, arts and popular culture. Some classes are designed for attendees who are already advanced in mathematics, computer programming or other technical skills, but most classes require no prior knowledge at all. To get a full idea of the breadth of Splash, see 2014’s course catalog and its list of 618 unique classes. Any high school student may attend, and the cost of attendance is a relatively low $40 (with financial aid available).

MIT Splash Session on Systems of Voting and their functional outcomes -- November 2014

To keep Splash focused on high-schoolers and to let those high-schoolers spread their wings, parents are prohibited from attending sessions, but a separate session for parents led by Jordan Moldow ’14 was informative. Moldow’s “Behind the Scenes” presentation gave me a sense of the scale of this effort:

MIT Splash 2014 Statistics: 2500 students, 457 teachers, 618 unique classes, 30 student administrators

Put together 2500 students, 457 teachers, 30 administrators and many more volunteers and you’ve got a takeover of the MIT campus for a weekend. MIT donates space, which is helpful considering that this is a non-profit student effort. It’s also a smart move by MIT, considering that 2500 geek-minded young people every year have a chance to fall in love with the campus; you couldn’t dream of a better effort to recruit future applicants.

MIT’s Splash is not an overnight success; rather, it is the result of long, cumulative investment. MIT inaugurated its ESP (Educational Studies Program) for teens in 1957 with its High School Summer Project, and ESP launched its first Splash weekend in 1988. Splash is now in its 26th year, and has become such a phenomenon at MIT that according to Moldow, “a large portion of MIT students will at some point during their time here do something for Splash.”

The organizational effort to keep Splash is considerable. No Splash leaders or organizers are paid; all are volunteers. Chairs, treasurers, secretaries, administrative organizers, art directors, publicity directors, website administrators and directors of teacher development form a core group that meets twice a week during the school year, once to make group decisions and once more in a work session to carry out those decisions.

New Splash teachers are cultivated every year, months before the event itself. Veteran teachers act as directors of teacher development, Moldow explained to parents. Their role is to “communicate with teachers and critique their syllabi or class descriptions. We run 3-6 teacher trainings each year to talk about what are effective methods of presenting materials to a class, to make sure students absorb information, to make sure it is entertaining and to make sure that students are engaged.” A few members of the Cambridge, Massachusetts community volunteer to teach Splash classes, but most teachers are undergraduate MIT students. By learning to teach, MIT students improve their command of the subjects they study while practicing the important skill of communicating advanced knowledge in a concise and comprehensible way. “Just as we are trying to serve students by teaching them, so we are also trying to serve teachers by helping them to become better teachers,” Moldow said.

As Splash has become more and more popular at MIT, the organizers of Splash have sought to expand the program beyond that university’s walls to involve other campuses. In 2009, Splash alumni formed Learning Unlimited, a non-profit organization that organizes an annual “SplashCon” and supports over 20 universities that have started running their own local Splashes. If students at your university are interested in starting a Splash, Learning Unlimited will bring them into a nurturing network of advisors and supply them with the software they need to make a Splash run — at no charge. Get in touch with the leadership of Learning United here to spread this model of education that so spectacularly brings down the Ivory Tower’s walls.