For attendees and members of the general public who are interested in the research basis for the factual claims made by Assoc. Prof. Lorien Lake-Corral and myself in our presentation on discrimination and the KKK in Maine, please feel free to download the powerpoint file attached below, which contains not only our presentation slides but also an appendix on discrimination and a complete set of references.
Over the last five years, the social media platform Twitter has become a standard part of the communications package of U.S. senators. An analysis of Twitter activity by senators in the first four days of the 115th Congress (Tuesday January 3 to Thursday January 6) reveals a large amount of communication with a great deal of variety between members of the Senate. During this period, the 100 members of the Senate posted out 1,792 Twitter posts (“Tweets”). Many of these posts were accomplished impersonally (as with Senators’ speeches, statements and letters) through work delegated to hired communications staff.
The distribution of these Tweets is uneven. The office of New York Senator Charles Schumer posted the largest number of Tweets during the four days at 79, with Texas Senator John Cornyn not far behind at 69 Tweets. These two most voluminous Twitter users directed their posts in different ways: three out of five of Sen. Cornyn’s Tweets mentioned or replied to another Twitter user, while Sen. Schumer broadcasted his Tweets slightly more than half the time without any reference to any other Twitter user. While both senators had much to say, Sen. Schumer acted as more of a broadcaster and Sen. Cornyn acted as more of a communicator. The least communicative Senator on Twitter was Thad Cochran of Mississippi, who only posted one Tweet during the first four days of the 115th Congress:
For a member of the U.S. Senate, Sen. Cochran’s single Tweet went relatively unnoticed, with only 5 retweets, 24 likes, and 14 replies. Vice President-Elect Mike Pence did not respond to Sen. Cochran’s outreach.
Those senators who do not communicate tend not to be the recipient of communication. Sen. Cochran, for instance, was not mentioned by any other senator during the new Senate’s first week. Sens. Schumer and Cornyn, on the other hand, received multiple mentions from other senators during the period. The most mentioned senator during the first four days of the 115th Congress was Catherine Cortez Masto, the new Senator for Nevada. Most of these mentions by other senators were messages of welcome, although some noted her work, as in this retweeting message from Sen. Schumer regarding the new Senate’s plans to dismantle the existing health care structure:
Patterns of communication between members of the Senate via Twitter tended to be partisan, as the following social network graph of mentions and replies indicates. This network graph uses a “spring embedded” visualization technique so that ties (indicated via curved lines) draw connected nodes closer to one another:
Most Democratic senators’ accounts tend to cluster close to one another (although Senator Michael Bennet of Colorado is the network’s only “isolate,” not mentioning or referring to any other Twitter account during the period), and most Republican senators’ accounts also tend to cluster close to one another as well. Interestingly, the two Independents of the Senate, Senators Angus King of Maine and Bernie Sanders of Vermont, are clustered closely to Democrats’ accounts, Sen. Sanders most markedly so. Sen. King clusters with Democrats in this period because he mentions the same non-Senate Twitter account that they do (as indicated in gray).
There are exceptions to strict partisanship. Many members of the Senate refer to the same non-Senate accounts across party lines, as in the case of Senator Heidi Heitkamp of North Dakota, represented as the blue dot in the upper left of the network graph. While Sen. Heitkamp does not directly converse on Twitter with Republican senators, neither does she converse with her Democratic colleagues. Because Sen. Heitkamp mentions an account that is also mentioned by Senator Joni Ernst of Iowa, she is clustered with Republicans. Senator Tim Scott (represented as the red dot toward the bottom of the network graph) follows the same pattern, not directly mentioning Democratic senators’ accounts but mentioning a number of the same non-senate accounts that they do. Senator Joe Manchin takes this pattern of cross-partisanship through indirect contact to its fullest extent, referring to the Twitter accounts of Vice President Mike Pence as well as the news outlets Fox News and the news shows Fox & Friends and Morning Joe that are popular targets of communication for a number of Republican senators. In the network of Twitter communication, this places Sen. Manchin squarely in the midst of the Republican upper-half of the Senatorial network.
As the large number of gray-shaded accounts in the network indicate, members of the Senate spend considerable energy communicating to accounts outside the Senate. Some 471 accounts outside the Senate were targets of communication during the first 4 days of the 115th Congress. The most common target of communication during these days was President-Elect Donald Trump, who was mentioned in 17 senators’ Tweets. The next most common target was the account of Planned Parenthood, whose federal funding for poor women’s pap smears and contraception is under threat from Senate budget cutters. Rounding out the top ten most-referred to Twitter accounts by senators are four Democratic senators, the collective account of Senate Democrats, one Trump cabinet pick (Governor Rick Perry of Texas), and two national news outlets.
Patterns of reference to particular media outlets are highlighted in the network graph below, which is identical to the graph above but which features the accounts of national news outlets with graphic icons. The three most commonly referred-to media outlets during the period were MSNBC (10 references), Fox News (8 references), and C-SPAN (7 references). The location of some of these news outlets is unsurprising. Right-leaning Fox News, Politico, and The Hill are referred to most commonly by by Republicans, and left-leaning NPR is referred to exclusively by Democratic senators. However, the centrist CNN is surprisingly only referred to by Republican senators, and the right-leaning Washington Times is referred by by both Republicans and Democrats in the Senate.
Data collection and visualization for this post was carried out with NodeXL software.
The incredibly useful research program called R is available on many platforms — Linux, Windows and Apple computers — and can run the same scripts across all three of its different versions. That said, the experience of getting R to run those scripts is not quite the same on an Apple Mac. This seems to be some kind of unwritten rule for Macs — whatever your program, on a Mac the menus, procedures and names of commands will somehow end up being different.
So what? Well, if you’re just getting started with R, you’ll need to occasionally get some tips and tricks for making the program work. Most of the how-to blog posts and videos you can find out there use examples using a Linux or Windows system — and they just won’t work for a Mac. I found this out the hard way when teaching students to use the igraph package for R to perform social network analysis. A few of my students have Macs at home, and it didn’t take long for them to cry for help, because the R program they were dealing with looked very little like the R program I’d been showing them.
If you find yourself in the same boat, and are running into trouble using R and igraph, I hope the following video will be of some help. Using a screen capture of a Mac running OS X, I briefly demonstrate the experience of installing R and running a script with the igraph package on from an Apple vantage point. One difference is that there are a few menu options you’ll need to select when installing igraph to actually make it run. In another simple but crucial difference for Macs, you’ll need to select all the text in your script before running it. THEN, and only then, use the “Execute” command. That’s not necessary on a Windows computer, but it’s a make-or-break move on a Mac.
Why? Don’t ask me why. It’s the same old story that we’ve had for thirty years: it’s just different on a Mac.
The walkthrough video:
Please leave a comment if you have a question or need clarification, and I’d be glad to be of help if I can.
Since the start of this year, I’ve been working on a project to take a $30 Raspberry Pi 2 computer turn it to create a social media data mining machine using the programming language Python. The words “programming language” may be off-putting, but my goal is to work through the process step-by-step so that even a complete beginner can follow along and accomplish the feat.
I’m motivated by two impulses. My first impulse to help people gain control over and ownership of the information regarding interaction that surrounds us. My second impulse is to demonstrate that mastery of social media information is not limited to the corporate, the government, or the otherwise well-funded sphere. This is not a video series for those who already are technologically wealthy and adept. It’s for anyone who has $30 to spare, a willingness to tinker, but the feeling that they’ve been left out of the social media data race. I hope to make the point that anyone can use social media data mining to find out who’s talking to whom. The powers that be are already watching down at us: my hope is that we little folks can start to watch up.
I’m starting the project by shooting videos. The video series is not complete yet (I have yet to film videos on automation using cron jobs and input statements), but has proceeded far enough along to represent a fairly good arc of skill development. Eventually I’d like to transcribe the videos and create a written and illustrated how-to pamphlet; these videos are just the start.
Throughout the videos, I’ve tried not to cover up the temporary mistakes, detours and puzzling bugs that are typical of programming. No one I know of hooks up the perfect computer system or writes a perfect program on the first try. Working through error messages and sleuthing through them is part of the process, and you’ll see that occasionally in these videos.
Please feel free to share the videos if you find them useful. I’d also appreciate any feedback you might have to offer.
Video 1: Hardware Setup for the Raspberry Pi
Video 2: Setting up the Raspberry Pi’s Raspbian Operating System
Video 3: Using the Raspberry Pi’s Text and Graphical Operating Systems
Video 4: Installing R
Video 5: Twitter, Tweepy and Python
Video 6: Debugging
Video 7: Saving Twitter Posts in a CSV File
Video 8: Extracting and Saving Data on Twitter URLs, Hashtags, and Mentions
Every September, the U.S. Census Bureau releases data regarding the U.S. population from its annual American Community Survey. The American Factfinder website very handily archives this data and makes it available through guided or customized search.
I particularly encourage you to visit American Factfinder and search for a table titled “CHARACTERISTICS OF THE GROUP QUARTERS POPULATION BY GROUP QUARTERS TYPE.” That table sounds dry and uninteresting, but it contains a nugget of gold for any voter who wants to fact-check claims being made lately about immigrants. In press releases and in speeches this year, political officeholders and candidates have asserted that immigrants to the United States are dangerous and liable to commit crimes. Of course, it is possible to find tragic stories of crimes committed by immigrants to the United States, just as it is possible to find tragic stories of crimes committed by people born in the United States. But individual stories are not a good basis for policy. Claims about immigrants as a source of crime are strong in their accusation and as such need to be evaluated on the basis of systematic evidence.
To cut to the chase, data from this table reveal that immigrants make up a lower share of people held in adult corrections facilities in the United States than their share of the U.S. population. “Native born” Americans — those born in the United States — made up 86.5% of the U.S. population in the 2015, but made up 91.9% of those housed in adult correctional facilities in the United States in 2015. The “foreign born” immigrants to the United States made up 13.5% of the U.S. population in 2015, but made up only 8.1% of those housed in adult correctional facilities in the U.S. in 2015:
This data does not appear to be consistent with the claim that foreigners coming to the United States to live are a unique and concentrated source of crime. Trends for 2015 match findings for previous years compiled for the National Bureau of Economic Research. Those who wish to pursue policies against immigrants on the basis that doing so would cut crime rates in the United States need to explain how their assertions match these observations.
Deciding who to vote for in state legislative campaigns can sometimes be tricky because thorough coverage of local candidates can be hard to find. In the state of Maine, state legislators in Maine are known for their accessibility. This may be because Maine’s legislative districts tend to be small; it may also be due to the friendly nature of Maine folk in general. Whatever the reason, getting in touch with candidates for Maine political office is both important and possible.
In this day and age, the quickest way to learn about state legislative candidates and to find their contact information is through social media platforms like individual web pages, Facebook and Twitter. To help you in that process, the I’ve put together a spreadsheet with information about the social media presence of the 70 candidates for the Maine Senate in 2016, along with some additional contextual information. To download this information for personal use, click here for a Microsoft Excel file.
This sort of information changes all the time — if you have updated information about new accounts, please share a comment below to let me know, or write to firstname.lastname@example.org.
In tonight’s speech, Donald Trump will accept the presidential nomination of the Republican party. The text of Trump’s speech makes the following claims regarding crime in the United States:
“These are the facts:
“Decades of progress made in bringing down crime are now being reversed by this Administration’s rollback of criminal enforcement.
“Homicides last year increased by 17% in America’s fifty largest cities. That’s the largest increase in 25 years.”
Let’s look at these two claims and check the facts.
Fact Check of Claim 1: “Decades of progress made in bringing down crime are now being reversed by this Administration’s rollback of criminal enforcement.”
Response: The annual FBI report Crime in the United States provides the most recent data on crime, both in the United States overall and in particular communities. Annual reports are released every fall to describe crime in the year before, based on direct reports of police officers all over the country (the delay occurs because it takes time to gather all those reports and carefully tabulate them). The most recent report was released in 2015, describing crime in the year 2014. Anyone who tells you they know about U.S. national crime trends for any more recent year is fibbing — because 2016 isn’t over yet, and because final counts for 2015 are still being worked on.
The trends on violent and property crime victimization rates in the United States are shown below, from the very first page of the 2015 Crime in the United States report, released at the end of September 2015:
People can disagree about policy, but it is not possible for policy changes to have led to a reversal in progress in the crime rate in the United States, because there is no evidence that such a reversal exists.
Fact Check of Claim 2: “Homicides last year increased by 17% in America’s fifty largest cities. That’s the largest increase in 25 years.”
We don’t actually know whether this is the case, because final data for 2015 is not yet available. A preliminary count, that is not a final count, that is only for the first six months of 2015, and that is only for cities with over 100,000 in population — has been released. Here it is. Let’s realize, based on this data (look at Table 4), that:
First, we do not yet actually have a final count for 2015.
Second, on the basis that only the first six months of 2015 have been counted, it is not possible to make the conclusion that Donald Trump makes regarding the entire year.
Third, if we actually look at the fifty largest cities in the United States, and look at the preliminary count for the first six months of 2015 (not the entire year), we find that the homicides are up 8.4% in America’s fiftiest largest cities compared to 2014 — Donald Trump’s speech claims twice as much as this.
Fourth, it’s interesting that the speech only focuses on homicide, and not on violent crime in general. The increase in the violent crime rate from 2014 to 2015 is 3.1%.
Fifth, even these rises do not take into account the rise in population of America’s fifty largest cities, increasing the population, which will of course increase the number of murders.
Sixth, even this increase, in the context of the huge falls of the last twenty years, still marks a low crime rate in America’s fifty largest cities in recent history. The preliminary homicide rate in the fifty largest cities of the United States in the first six months of 2015 was 4.06 homicides per 100,000 people. In the first six months of 2015, the overall violent crime rate was 305.7 per 100,000 people. By comparison, in 2008, when Barack Obama was elected president and the decline in homicides was already well underway, the homicide rate was 12.1 per 100,000 people in America’s 50 largest cities, and the overall violent crime rate was per 852.9 per 100,000 people. In other words, since Barack Obama became president, if the 2015 preliminary data holds, the homicide rate is down 66.4% and the violent crime rate is down 64.2%.
It turns out that Donald Trump’s claim is based on a post made in very early estimate by a blogger using very early data in January 2016, less than a month after 2015 ended.
This second claim by Republican nominee Donald Trump, like the first, is not supported by the facts.
Another season, another new public plagiarism case.
It’s not a partisan thing. A different season, a different public plagiarism case:
David Greenberg: “Why Biden’s plagiarism shouldn’t be forgotten.”
Students, listen up: Of course there are moral implications when you steal others’ words and pass them off as your own. Of course the choice to plagiarize keeps you shallow because you haven’t bothered to do the work of thinking for yourself. Even if you don’t care about that, plagiarism is a horrible strategic choice. Your reputation will be destroyed. You’ll find yourself making the defense that you’re not malicious, “just” incompetent and sloppy. Any actual original work you do will be discounted. Your career will be stunted.
The best protection against plagiarism is to use your own brain to think up, then write, something truly original and your own. If you can’t manage that, then why are you writing or speaking in the first place? For written forms that require thorough research of others’ contributions — to which you then should add your own original thoughts — be polite: quote and cite. Although academic integrity policies can take many words to express the standard, avoiding plagiarism really is that simple.
The following are links to supporting materials for the presentation “Twitter Adoption in U.S. Legislatures: A Fifty-State Study” made to the 2016 International Conference on Social Media & Society on Wednesday, July 13 at Goldsmiths, University of London.
1. Free full-text access:
SMSociety ’16 Proceedings of the 7th 2016 International Conference on Social Media & Society, 2016
2. Download Powerpoint Presentation Slides from presentation
3. Abstract: This study draws theoretical inspiration from the literature on Twitter adoption and Twitter activity in United States legislatures, applying predictions from those limited studies to all 7,378 politicians serving across 50 American state legislatures in the fall of 2015. Tests of bivariate association carried out for individual states lead to widely varying results, indicating an underlying diversity of legislative environments. However, a pooled multivariate analysis for all 50 states indicates that the number of constituents per legislator, district youth, district level of educational attainment, legislative professionalism, being a woman, sitting in the upper chamber, holding a leadership position, and legislative inexperience are all significantly and positively associated with Twitter adoption and Twitter activity. Controlling for these factors, legislator party, majority status, partisan instability, district income, and the percent of households in a state with an Internet connection are not significantly related to either Twitter adoption or recent Twitter use. A significant share of variation in social media adoption by legislators remains unexplained, leaving considerable room for further theoretical development and the development of contingent historical accounts.
Please feel free to review these materials before or after my presentation. I look forward to your comments.
(cross-posted in revised form from an ongoing column for the Kennebec Journal)
Part I: No Limits?
As part of my work in social media research, I follow the Twitter accounts of Maine’s elected officials, studying patterns in their public communications.
I wasn’t surprised to see graduation messages from Rep. Justin Chenette, D-Saco, in my feed this month; ’tis the season, after all, and local officials are regularly invited to speak.
“The only limitations, are the ones we set for ourselves.”
“You can achieve anything you set your mind to.”
Ideas like these are commonly asserted: during the first half of the month of June 2016, the phrase “no limits” was posted to Twitter at a rate of more than 100 times per hour. That’s understandable. The idea of no limits in life, of a plain of possibilities open wide before us, is a hopeful idea. It’s an inspiring idea. It’s a popular idea.
Unfortunately, it’s also an incorrect idea.
Don’t believe me? Allow me to issue two challenges. If there really are no limits in life, try:
• Heading to Baxter State Park here Maine, then walking straight through the base of Mount Katahdin.
• Jumping so high into the sky that you launch yourself into orbit, without assistive technology.
No matter how much you wish, no matter how hard you try, you can’t accomplish these acts. We are all limited by laws of physics. Due to electromagnetic repulsion, you can’t walk through rock. Your legs just can’t generate enough kinetic energy to overcome the force of Earth’s gravity. Ignoring these limits can kill you.
Just as there are laws of physics, there are social laws that set limits in society. Consider the following image of a social network, in which circles represent people and lines represent relationships between them:
As you can see, there are two groups of people in this network: reds and blues. Sociologist Peter Blau considered the rate of “outgroup association” (the percent of all relationships by group members to people outside the group) and discovered a law: that smaller groups have a higher rate of outgroup association than larger groups.
The reds in the network have five relationships, two of which are to non-reds. 2/5 = 60 percent. The blues in the network have 10 relationships, two of which are to non-blues. 2/10 = 20 percent.
Go ahead, add and subtract as many ties from this network as you wish. You can’t make the pattern go away. It’s an inescapable law.
So what? Why does this social law matter? Stop thinking about reds and blues and start thinking about numerical minorities: women in male-dominated corporate boardrooms, immigrants in native-born communities, or black people in majority-white America. Due to math alone, no matter how much you wish it to be otherwise, such minorities will have contact with majority members more often than majorities have contact with minorities.
Because of this pattern, minority groups must devote more energy to knowing the habits of two cultures just to get by. The stakes for failure can be high. A woman who ignores male culture in the boardroom can lose a job. An immigrant who can’t speak the language loses business. When northerner Emmett Till whistled at a white woman in the South, he lost his life.
This is one social law, one inescapable limit among many in society. A sense of limitless potential may feel good to this year’s crop of graduates, but if that feeling is untempered by a reality check, the outcomes can be disastrous.
Part 2: Deadly Friendships
Sociology’s laws are subtler than the laws of physics, but they’re no less deadly.
For another example of sociological laws and their consequences, let’s consider Feld’s Law. Twenty-five years ago, sociologist Scott Feld demonstrated that on average, your friends have more friends than you do. This sounds impossible, but it’s true.
To show you what I mean, we’ll look at another social network. In this network, circles indicate people and lines indicate friendships between them:
Some people in this network, like Carol, have more friends than others, like Hal. Popularity and unpopularity is a normal part of human social life.
Also, some members of this network are friends with very friendly people; Don’s friends have 3.5 friends of their own on average (Carol with 4 friends, Ed with 3). On the other hand, the average friend of Carol has just 2.25 friends (Al, Betty and Don with 2 friends each, and Gina with 4). That sort of variation shouldn’t be too surprising either. But as Feld found, things get strange when we consider the overall trend.
Take a look at the table below, which shows the results for each person as well as the overall average for all people. The average person in our network has 2.5 friends. The average person’s average friend has 2.9 friends. That’s exactly the sort of result that Feld’s law predicts.
Go ahead, draw your own social networks and do the math for yourself. You’ll find that Feld’s law holds true for almost any network you can think of. That’s odd, but why should you care?
One reason to care is that Feld’s law explains that feeling many of us have that we’re less popular than our friends. Odds are, you’re right. Don’t take it personally; it’s just the way societies work.
Another reason to care is that Feld’s law is also true for any relationship in which people share something with one another. People share needles in the opiate epidemic facing not just the state of Maine where I work but significant swaths of the North American continent more broadly. People have shared sexual relations since the dawn of humanity. These kinds of sharing can also share deadly viruses like AIDS, hepatitis B and syphilis.
If you’re thinking of sharing a needle, Feld’s law tells us that on average, the people you share needles with share needles with more people than you do. Feld’s law tells us that on average, the people with which you have unprotected sex have unprotected sex with more people than you do. This means that even if you share a needle just once, even if you have unprotected sex just once, your chances of catching a killer virus are disproportionately high.
Understanding invisible social laws are crucial for getting by in life because they provide guides for safe and unsafe behavior, just as physical laws guide us while we climb a mountain or skirt a rooftop. When you support research to uncover sociological laws, the benefits aren’t abstract. Understanding how society works can save a life — and that life could someday be your own.