Detecting Spam in a Twitter Network- Bridget Gelms

Technology is changing the way people communicate. Whether it’s by instant messenger, blogging, video chatting, or e-mail, people need to adapt to the ever-progressing forms of communication that the digital age is providing us with. One such way is Twitter. Twitter is a website where people can send out of a version of a text message from virtually anyway. With many different devices applicable to Twitter, this site makes it simple for people to send out a message. Unfortunately, if it’s easy for legitimate users it’s also easy for illegitimate users like spammers.

            Spam and spammer are words that were born out of the creation of the internet. Spam, as defined by Google, is the “abuse of electronic messaging systems (including most broadcast media, digital delivery systems) to send unsolicited bulk message indiscriminately” while spammers is said to be “a person or organization that sends spam.” Twitter and its structure allow spam to evolve with the times meaning that those junk-messages that used to be filtered through our e-mail account can now reach us through different modes of communication. Spam and spammers are constantly coming up with new and innovative ways to trick spam detectors and appear like they are normal everyday internet users. This, in turn, creates a need for researchers to come up with new and innovative ways to detect spam messages. Previously, spam was detected based on the content of the actual message. Now, spammers are beginning to be identified by how their spam is sent, which is harder to detect but is an element to spam that spammers can’t hide. This piqued the interest of researchers Sarita Yardi, Daniel Romero, Grant Schoenebeck, and Danah Boyd who conducted a study in an attempt to learn about spam on Twitter. 

The structure and functions of Twitter have changed the spamming game and this article identifies three main reasons. First, Twitter allows you to “follow” someone, even though they aren’t “following” you. Therefore, spammers can be connected to copious amounts of people without having to go through the work of having others follow them. This is something that sets Twitter apart from other social networking sites like Facebook and Myspace where both parties consent to being “friends” is necessary for someone to see your activity. Secondly, the ability to “re-tweet” (or “tweet” something that someone else has “tweeted”) gives them the opportunity to alter links that were originally to legitimate sites so that they now direct someone to the spam. URL shorteners are used so that links don’t take up the majority of the tweet (Twitter only allows its users to use 140 characters per tweet). These shorteners obviously don’t reveal the full name of the website, so those who click on the link trust it’s legitimate. This gives spammers a new way to conceal their spam links. Third, the user-generated aspect of Twitter means that tweeting patters change in regularity, amount, and circulation depending on its users. This allows spammers to use what’s being said on Twitter to their advantage.

            Another feature of Twitter is “hashtags”. By placing a hash mark in front of a group of words that describe what your tweet is about, it creates a link to all those other tweets that are also marked with that same hashtag. For example, someone might tweet “What is the deal with that smoke monster?!” and tag it “#LOST” in order to participate in a discussion of the popular TV show Lost. Twitter tracks what the most popular hashtags are and posts the top 10 list on the main page, making it extremely easy for thousands of people to engage in a conversation about the same thing, even if they aren’t “following” each other.

            This article reports finding of various research conducted about what users use Twitter for. While reasons vary, it’s been found that most people use Twitter for its “common ground” aspect. Even though they might only be following a few people that they actually know they are able to connect to many people that they don’t based on some sort of common ground, made possible by the hashtags.

            Specifically, this article tracks the growth of a hashtag- one that is generated solely within Twitter meaning its birth, life and death occurs only with Twitter. They give examples of #michaeljackson and #iranianelection as not being applicable to this study because outside influences could affect the progression of the hashtag. They use #robotpickuplines, a hashtag that started with user grantimahara, a user who has many followers because they are a host of the popular television show Myth Busters (and also a robot builder, hence the joke about robot pick-up lines). They track #robotpickuplines from the very first one, to everyone who participated in the conversation, all the way to the extinction of #robotpickuplines.

            Within two hours of grantimahara tweeting #robotpickuplines, it had become a “trending topic” and appeared on Twitter’s homepage (and remained there for at least three more hours) which gave the hashtag more viewership which led to new tweets with the tag #robotpickuplines and even re-tweets of existing ones. The study found that the most commonly re-tweeted robot pick-up line was “Hi my name’s Vista. Can I crash at your place tonight?” This study reports that they tracked #robotpickuplines over its lifespan of four days and found 17,803 tweets involving this tag which were generated from 8,616 different users. They also found that “user participation followed a power law distribution where 6,021 users tweeted one time, 2,595 tweeted two or more times, and a dedicated 205 tweeted 10 or more times using the #robotpickuplines.”

So where does information about spammers fit into this process? As I’ve identified before, spammers use certain tactics in order to infiltrate legitimate tweets such as including links to a URL, shortening the URL’s so that the site name is not visible (this then puts the responsibility on host sites such as www.bitly.com to sift out the spam), using more than one hashtag to cover a larger base, and sometimes they use “suggestive keywords” that never fail to get some hits such as “naked”, “girls”, or “webcam”.  With these tactics in mind, the researches followed a simple set of rules to detect spammers within the #robotpickuplines trend. They manually went through a random sampling of 300 tweets with the hashtag #robotpickuplines and marked each one as spam or not spam. They then used this set to compare the rest of the tweets to. The study reports that “our algorithm matched 91 percent of the time, with 27 missed spam tweets and 12 false positives.”

Using this information, they were able to discover a correlation between the lifecycle of the #robotpickuplines and spam spread through utilizing the hashtag. The rise in the amount of spam messages containing the tracked hashtag ran in conjunction with legitimate tweets containing #robotpickuplines with a slight lag. Despite the spammers taking about five hours to being latching onto this hashtag after grantimahara’s first posting, 14% of all tweets containing the tag #robotpickuplines were generated by spammers. Here are two bar graphs from the study illustrating their findings; the first shows the first 24 hours of #robotpickuplines. The study reports that the hashtag “started at 11am CST and spiked around 3pm when it became a trending topic. It dropped around 4am and picked up again, although less heavily, the next day.” The next graph is the spam tweets using #robotpickuplines.

The study also addresses a series of questions that may or may not affect the results. For instance, they research whether the age of the Twitter account is different between spam accounts and actual accounts. They also determine whether or not spammers tweet more often than regular users of Twitter and if spammers have more friends than followers linked to their account. Finally, they also examine whether or not spammers are grouped together. While their research is thorough and their findings to these questions are interesting, I think that what we can relate to our work in our Digital Literacies class comes from the information already presented. This study is relevant to our work in English 213 for a number of reasons, but I want to focus primarily on themes found in Clay Shirky’s Here Come Everybody, specifically chapters five and seven because they relate to this study in regards to why events within the study progressed the way that they did and perhaps why Twitter is such an easy target for spammers.

            In chapter 5 of Here Comes Everybody, Shirky includes a figure (found on page 129) that illustrates the principles of Twitter very well by showing the relationship between an audience and the pattern of conversation. The larger the audience, the looser the conversation will be. We see this in the study of the hashtag #robotpickuplines on Twitter in that not everyone who participated in that conversation heard everything else that was said within that conversation. Had the researchers tracked the life of a similar hashtag within a medium that has a much smaller audience than Twitter, the conversation would have been much more contained and not been a target for spammers. 

            So how has Twitter changed the way spam works? For e-mail, spammers had to look for who to target and how to do so. In Twitter, they now look for what to target and when to do so. For example, the “Trending Topics” table on the main page of Twitter lets users (and spammers) know what are the most popular items being discussed. As they begin “trending”, they build momentum. This can be a definite indicator for spammers as this takes care of the “who” and “how” that they used to have to figure out with email.

            Chapter seven in Here Comes Everybody also addresses issues related to Twitter and this study. Shirky stresses the importance of how the speed of various processes affects our everyday lives and he begins his chapter titled “Faster and Faster” by saying that “collective action is different from individual action”. This is seen in the study on the evolution of #robotpickuplines on Twitter because it took a massive amount of people to link up to this hashtag enough that spammers took an active interest in utilizing it for their benefit. Shirky directly discussed Twitter in chapter seven and describes it as being “simple” yet “compelling”. Shirky explains that websites like Twitter are primarily used for the benefit of small groups like people who read their friends’ tweets. However, he also describes instances where Twitter can be far more important in its uses as he describes a democracy activist in Egypt who has recorded his arrest through his Twitter account. It’s through this “simple” process that human beings can create something “compelling”.

            Much of Shirky’s book hones in on the idea that people now possess the tools necessary to organize efficiently and create a revolution by using these tools to our advantage. Twitter can be one such tool. While it might not seem like tweets involving #robotpickuplines isn’t that important, this study does demonstrate the power that Twitter can hold. A nerdy joke about robots was sent out into the digital realm in less than 140 characters. This single action that probably only took its creator a few seconds to do sparked a form of “organization” that resulted in connecting thousands of people to the same joke. This group of people commanded enough attention that spammers took interest. It was because of one simple message that thousands of people, who were otherwise not connected to each other in any way, were able to take part in a conversation instantly. The researchers created a visual representation of that conversation:



With each pink dot representing a Twitter account, this picture makes a statement about our abilities to organize and adopt new behaviors as people who are living in a digital world. This picture astonishingly shows just how far something like #robotpickuplines can extend within the span of a few hours.

Works Cited

Boyd, Danah, Daniel Romero, Grant Schoenebeck, & Sarita Yardi. “Detecting spam in a Twitter network.” First Monday Volume 15 Number 1-4 (January 2010): web. http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2793/2431

Shirky, Clay. Here Comes Everybody. New York, New York: Penguin Books, 2008. Print.

 “define: spam”, “define: spammer”. Google (October 16, 2010): web. http://www.google.com

People and Social Networks

            In her graduate thesis paper, Jennifer Ryan discusses the far-reaching ramifications that social networks have on human communities. She discusses the history of social networks, the way people use them, the feelings they evoke, and the effects they have on people’s lives. Her research has lead her to the conclusion that social network participants:

Use social network sites to extend their offline communities into online practice in a manner more closely in line with the concept of "networked individualism," which suggests we are expanding our social networks (weak ties in particular) according to our cultural tastes and communities of membership. (189)

            Jennifer’s thesis is interesting in that she makes a bold argument over who is creating a social identity and the reasons for doing it. Unlike McLuhan’s prediction that society would move from an oral culture to a massive online collective consciousness, Jennifer claims that our online presence is used only to enhance our offline lives. Instead of creating whole new communities, we seek out our likes. In this way our offline lives are still more prevalent than online. We’re not using the Internet as this worldwide tool to improve mankind or connect as a whole species; instead we’re simply using it to connect and strengthen existing bonds.

            Jennifer looks at three social networks for her research: Facebook, Myspace, and Tribe. Her study shows that a certain demographic of people connect and use each network. Musicians and high school students have mostly claimed Myspace, Facebook a mixture of college, high school and young adult, and Tribe by the rest of the people leftover. Out of the three, Facebook and Myspace claim spots in the top five of most visited websites. All of the users on these sites manipulate technology to connect with people that exist within their real life communities. Instead of broadening out, social networks help us to focus in to the point where words like “Facebook stalking” are accepted norms. While Jennifer points out the positive ways that we connect and maintain these relationships, she also highlights the loss of privacy that pervades this new social media.

            People using these websites are mainly concerned with social and personal vices. The sharing of information is centralized around likes, friends, and lifestyle niches. If you look back at how Shirky talked about existing institutions losing control over the flow of data, you can see where social networks have helped corrode that control. With people joining sites and sharing information based on their likes and desires, web based technologies have been forced to create applications and feeds centered around their audiences. The information on the Internet is so pervasive that if a certain website doesn’t have the information or capabilities we desire we simply move on to the next available resource. An online social network’s capital gain depends on a large number of people constantly being on their site.

            The idea of spreadable media becoming the new norm for community communication has been a topic that we’ve covered extensively throughout the semester. Jennifer identifies social networks as the mainframe through which media is now being shared. According to Jennifer:

            The Internet provides a platform for the spread of information and ideas that can bring like-minded individuals into contact with one another regardless of temporal or spatial distance. Such perceived potential provides support for utopian ideologies, such as “neotribalism” and “technoshamanism,” that purport to promote the sanctity of humankind- the “sacred” campfire ritual and shamanistic practices described at the beginning of this thesis- utilizing modern technologies to tap into the “collective un/consciousness.” (138)

This idea that the Internet will provide a way for people across the globe to connect and share ideas that will ultimately improve the nature of humankind is an idea shared by Shirky. This idea of neotribalism simply means that the new tribe, the new communities we’re forming are through digital media.

            Jennifer’s thesis is centered on social networks and the way people use them to communicate and form identities. I find myself agreeing with her observation that currently we’re using social networks to help cultivate and maintain offline lives. However, I believe she overlooks the rapidly growing online identities that are being formed along side the offline ones. With every passing year I’ve found that more and more offline faucets of life are being invaded by technology. Academic life is now embedded with the need to have online identities and communities that are a separate entity than our offline ones. There are some people and things that are wholly dependent upon the Internet for existence. Professors can now establish and maintain relationships with others in their field without having to first connect offline. The ease with which the Internet allows people to form social bonds, not just strengthen existing ones, is a vital part of social networks.

            After reading our texts for the semester and Jennifer Ryan’s thesis I feel that the Internet is still largely underused as an educational tool. Most of the points that Selfe made seem to still be largely ignored or unanswered. We still are no closer to figuring out what technological literacy is, or how we should use it. People seem to be focused on personal benefits that technology provides more than humanistic benefits. I don’t think the question is if the Internet is making us stupid, I think the biggest question is what should we be expecting the Internet to do for us? Technology has evolved so rapidly and become so accessible that I don’t think we’ve had time to carefully examine how or what we really want to use it for. It can do anything. Jennifer writes “the Internet is a complex new medium that allows for the intimacy, interactivity, and casualness of speech as well as the permanency and permeability of writing.” (168) People are still too amazed with it that I don’t think anyone’s ready to stop and consider the points that Selfe or Shirky make. We don’t have rules or regulations for using it for business, education, or world markets. What we really need is a way to examine what we have, what technology and the Internet are capable of, and figure out the best way to manipulate it to become a useful tool and not just a social tool.

Ryan, Jenny. (2008). The Virtual Campfire: An Ethnography of Online Social Networking.

I could not figure out how to attach my report. So I am just copying and pasting it. Enjoy.

Does Education Care About Twitter?

            Education is currently battling with the issue of technology in the classroom. Recent studies have discussed how technology is used in the classroom, how prepared teachers are to use it, and how accessible it is in schools throughout the country. In Joanna Dunlap and Patrick Lowenthal’s article, Horton Hears A Tweet, several of these concerns are expounded upon along with how educational technology can make good use of Twitter in online and regular education courses.

            To begin the article, Dunlap and Lowenthal proclaim that, “social networking tools, have great potential for enhancing the social context in support of learning, especially in online education” (Dunlap and Lowenthal 1). It important to recognize that technology does bring about better ways for students to learn in the future; therefore, we need to make sure we are paying attention to these new technological items as we progress in the twenty first century. Dunlap and Lowenthal go on and argue that education can better itself through the use of social networking sites such as Twitter, even though there are drawbacks. Such potential problems with Twitter involve people becoming obsessed with it, it taking too much time, and the content, which is sometimes questionable (Dunlap and Lowenthal 1). However, they feel Twitter can be a great tool, which can “add value to online and face-to-face university courses (Dunlap and Lowenthal 1).

This notion of technology possibly harming us today has been brought up before. In his article Does the Internet Make You Smarter, Clay Shirky notes that new technologies have always caused people to question whether or not new ideas harm or help people. Shirky writes of the printing press coming about, which caused people to worry about the effects it would have on citizens. It did actually change how Europe ran because people could read their own literature and religious texts, which eventually did away with the reliance on Catholic Church because people could finally interpret their own texts. Essentially, Shirky says that new technologies such as the Internet actually restore reading and writing as central activities in our culture. He also feels that new media is not harmful because it brings about new institutions that allows our society to develop. This links to Dunlap and Lowenthal who see the concerns with Twitter and using it in education, but also sees the potential it brings to better our society as we move forward.

            Dunlap and Lowenthal also focus on how Twitter can help students to be engaged and aide in the educational process. For starters, Using Twitter or other online forms of communicating can “encourage cooperation among students, encourage effective learning, and enrich educational experiences (Dunlap and Lowenthal 1-2). Again, it is vital to see how these new technologies can help bring about new forms of learning and thinking. Dunlap and Lowenthal go on and discuss the importance of making sure that students feel the “social presence” when using these methods of teaching because it allows students to feel that their peers and teachers are real. Without the social presence, students could feel “isolated and disengaged” (Dunlap and Lowenthal 2). Basically, Dunlap and Lowenthal feel that we must make sure students do not just simply post questions and wait while others talk and debate issues. This would not really benefit the students because they are not activity engaged in the discussion. Before Dunlap and Lowenthal discuss Twitter’s characteristics, they argue that Twitter, “allows us to establish natural, free-flowing, just in time contact with students,” which helps students to bring about the social presence that is needed in classes.  

            These concerns again link to the concerns that Clay Shirky and Nicholas Carr made concerning new technologies possibly coming to distract us and possibly dumb society down. However, it is important to note Marshall McLuhan’s book, The Medium is the Massage. McLuhan wrote that we need to be flexible and adapt to the new environment or we will be pushed aside. He also feels that people need to accept the new ideas because they are the way of the future and we need to be prepared to use them.  In the end, we need to be flexible and attempt to use Twitter and other forms of technology in the classroom or we are limiting our ability to move forward and become educated.

            Finally, Dunlap and Lowenthal discuss why Twitter is so special. They assert that Twitter is perfect because it is, “part social networking and part microblogging” (Dunlap and Lowenthal 3). Another reason why Twitter is great for online education courses is that it allows students “to share ideas and resources, ask and answer questions, and collaborate on the problems of practice,” in a quick, efficient manner (Dunlap and Lowenthal 3). Twitter is also an interesting social site because professionals are on it. These professionals can bring about their own knowledge, which in turn enhances learning because these students get real life experiences and information.

However, Dunlap and Lowenthal note that certain guidelines must be put into place for students to understand and use Twitter effectively.  Students first need to be motivated to use it, and then need to be shown people that they can follow. This allows them to get a good start on other scholars who they can follow and learn more about Twitter. Then, students need to be shown “appropriate ways to use and engage in the Twitter community,” versus the basic way in which most users Tweet basic information about their lives (Dunlap and Lowenthal 5). Last, students need to be encouraged to post their own work, knowledge, questions, or concerns. This allows a community of learners to come together to help one another out and update ideas.

            Another interesting aspect of using Twitter for learning is that it promotes “a lifelong learning skill,” because these new sites will become the way of the future. (Dunlap and Lowenthal 6). The use of Twitter also brings about a sense of community where students can “share and get feedback on their ideas, work, and products” (Dunlap and Lowenthal 7). This relates to other authors who feel that technology can help us in the technological world that we live in today. Cynthia Selfe writes of the problems that are occurring today in schools when it comes to technology, such as inequities in schools concerning computers and the Internet, the use of technology in the classroom, and the lack of teacher training when it comes to using that technology. Overall, Selfe argues that we need to focus on new technology and new ways of learning because they will become the way of the future and we need to train our citizens to become technologically literate. Being able to use Twitter and new technology in schools benefits students because they are learning the ways in which new ideas and developments will shape the world.

            Dunlap and Lowenthal do not simply focus on using Twitter is distance courses, but in face-to-face classes as well. They note that Twitter can be used in the classroom because it is “an effective way to create a black-channel forum during a lecture or presentation,” where students can post comments and feedback without interrupting the speaker or presentation (Dunlap and Lowenthal 8). It is important to make sure that these black-channel comments are not harmful and negative, but positive and uplifting. Twitter can also “be used the way clickers are used for polling the classroom,” which can bring about student participation or engagement during a class because they can vote or send messages to the instructor (Dunlap and Lowenthal 9). This way, students can reflect their opinions and post answers to topics together in the class. These polling exercises also help the teacher because they can see what the students are thinking, what they might need help on, or what they would like to learn more about. 

            It is interesting to see how Twitter can be used in face-to-face classes and online courses. Overall, new ways of learning are coming about through the development of technology. This relates to Clay Spinuzzi when he writes about knowledge work and new forms of being educated. Usually, students learn vertically and specialize in one area. However, with new forms of technology, we need to be able to learn horizontally by crossing fields, trades, and disciplines so we can become well rounded as technology influences us daily. By using Twitter and other forms of technology, students can learn to embrace new ways of becoming educated and can also become technologically literate.

            To end the article, Dunlap and Lowenthal say that there must be more research performed by professionals concerning technology and education. They encourage researchers to examine the relationship between faculty and students and the learning strategies employed, rather than just the impact of the technology in isolation. In conclusion, they feel that Twitter, “allows us all to make noises in greater amounts and recognize that every voice counts,” but more inspection can help professionals to see what we need to do to make technology fit well into education so students can succeed in today’s world  (Dunlap and Lowenthal 11).

            Overall, Twitter and other social networking sites can enhance the learning experience for many. It allows people to come together and post questions, concerns, or problems. Once posted, fellow students, or professionals can come together and work to help one another learn new ideas. Finally, students could learn a vast amount of information from using these networking sites. With all of the new technologies that are out there, students would undoubtedly enjoy using these methods to work with their peers and discover new ideas and opinions. In conclusion, it is important to focus on new sites such as Twitter, as they can be a platform for new ways of learning and thinking about the issues of today.

Jordan