Emoji as Visual Literacy
In June of 2016, Karl Ahlrichs declared: “Communicating with millennials can be like learning a new language.” He continues, explaining that visuals are a critical portion of what fuels this communication, stating, “Visuals pack a greater punch than text and are more likely to draw attention.” Visual communication has become prevalent in our contemporary world, not just for millennials, but for any consumer of technology and the Internet. Visual communication has become so widespread in the Internet age that now we even supplement or replace written language through emoji, emoticons, stickers, or GIFs.
Not only used by early-adopters and millennials, a study by Emogi reports that 92% of the online population uses stickers or emoji in text messages, and though Facebook, Twitter, Instagram, GChat, and Snapchat, amongst other platforms. Between 30–35% of the population consider themselves to be “frequent users,” and this number is on an upward trajectory. Using emoji as an evolving form of communication is proving its impact and longevity, but what are the reasons for and consequences of their use?
In the PBS video “Can You Speak Emoji?”, host Mike Rugnetta calls emoji “a relatively restricted system… Emoji may include a small set of easily depicted emotions, but… ideas about the world and people in it are incommunicable with emoji, probably by design.” This statement creepily aligned emoji with Newspeak, the harrowing language concept invented by the author George Orwell in the novel 1984. In the novel, Newspeak was used by a dystopian society that limits freedom of thought and threatening ideas by limiting the number of words that could be used in the language. (One needs go no further than experiencing an awkward autocorrect to start thinking our computers are controlling our language!)
I began to worry: were my own communications devolving into a twisted form of Newspeak, governed by the companies that create these limited number of emoji?
There must be some reason for such a large portion of the population to use emoji instead of the nearly endless possibilities of written words. Adam Sternbergh in New York magazine writes that emoji “attempt to bridge the difficult gap between what we feel and what we intend and what we say and what we text.” Ryan Eyers agrees that emoji can add emotional sensitivity to conversation, writing for Babbel that “both emoji and emoticons are used to add emotional nuance and context to text-based communication.” Emoji (along with their precursor, the emoticon) contribute a unique and distinct emotional element to written or typed language in our contemporary culture.
While these may be advantages to communicating with emoji, can emoji attain the sensitivity and nuance that one can achieve through spoken or written language? What implications does the proliferation of emoji in contemporary conversation have to our ability to communicate with visuals? Is our ability to communicate and express our thoughts with one another devolving from using a Newspeak-esque emoji vocabulary, evolving into an innovative and neoteric form of communication, or, on the other hand, reconnecting us to our inherent abilities for visual communication? This paper will attempt to offer insights, if not answers, to these questions, addressing the history of emoji and communicating with visuals, emoji’s potential to become a language, and how their origins and usage have helped us re-access our abilities for visual communication and literacy.
Some of the earliest “emoji” I was exposed to were the drawing books I’d use when I was a kid, where you’d be given a page of eyes, a page of noses, and a page of mouths, and you could choose one of each from these plethora of choices, creating a myriad of unique and sometimes surprising emotions. Faces and recognition of emotions is an important element of our development and one of our earliest forms of communication. Some of our earliest drawings as children are of faces, emphasizing emotional indicators such as the eyes. We’re even hardwired to recognize faces and emotions in everyday objects through a phenomenon called pareidolia.
This illusion was certainly in play when Scott Fahlman, allegedly attributed to being the “father of the emoticon,” posted this character:
to the Carnegie Mellon online bulletin board in 1982. From this, further emoji and kaomoji, which are more complicated images “drawn” using multiple characters, began to be typed into the keyboards and showed up on the screens of people across the world.
It was not too long afterwards, on the other side of the globe, that Japanese programmer Shigetaka Kurita began creating an entire keyboard of what in Japanese were called “emoji,” pictorial characters that expressed certain emotions or feelings. These emoji were originally introduced by DoCoMo, a Japanese mobile communications company, but were later integrated into the mail and web services of all mobile carriers in Japan. Eventually, upon hearing of this secret keyboard of “picture language,” American programmers were able to release the keyboard to English language phones. Soon, Apple made emoji compatible with their products, and, in 2010, emoji were accepted into Unicode, which programmed the emoji keyboard into a standard keyboard. Voila! The worldwide emoji phenomenon had begun.
But, of course, this phenomenon had actually begun hundreds, if not thousands, of years before, with the beginning of written language itself. Ancient Sumerian cuneiform began as a system of pictographs, which became more abstract as the writing system developed. Although phonetic symbols eventually entered cuneiform, it still retained logograms, symbols that read as complete words or phrases, much like emoji read to us today. The ancient Egyptians, possibly adopting aspects of early cuneiform, also used logograms as part of their writing. The basis of today’s world languages rests in these early pictures. Even as these written forms turned into phonograms, some letter symbols, such as the letter O, retained a graphic depiction of the form our mouth makes while making the sound. By the time of the Middle Ages, rebuses, pictorial puzzles in which words represented homonyms, were in common use in coats of arms, stories, and later, during the 1800s, even greeting cards.
History shows us that subbing in a picture for a word has never been an unheard of idea. Pictographs are once again becoming part of our contemporary communication, but can they rise to the level of becoming a real language?
Journalists and bloggers throughout the world have no problem throwing around the word “language” when it comes to describing emoji. The Telegraph wrote that “Emojis are the fastest growing language in history” and Adam Sternbergh wrote that “Emoji are now available to you as an optional written language, just like any global language.” Dr. Tania Lombrozo, Professor of Psychology at Berkley, believes that “the recognition of a new language is a good reminder that language is a constantly evolving cultural creation. Like biological species, languages change over time and sometimes diverge, resulting in the formation of new dialects or even new languages.” Communicating with emoji could certainly be part of this linguistic evolution. Johanna Nichols, an emeritus professor of linguistics at UC Berkeley, offers a definition of language, stating, “The gold standard criterion [for language] is mutual intelligibility. If a speaker of only one of them hears the other, can they understand? (and vice versa). If so, they are the same language; if not, not.” While dialects and related languages can challenge this statement (for instance, speakers of Spanish and Italian can often understand one another), according to Nichols, if two people can understand each other, their communications could be a language. Rachel Peters addresses this in her blog entry for Launch Media, stating the fun element of emoji can be the “inside joke” amongst friends, who have assigned a particular meaning to ambiguous emoji. Emoji certainly have a range of possible meanings and could be interpreted many different ways depending on who the set of speakers are. This idea is evocative of the concept of “twin languages” used by siblings who are so close they learn to communicate on their own, regardless of an outside language influence. Peters elaborates on the idea of emoji containing arbitrary meaning by sharing an anecdote from the rapper Drake, who recently “tattooed the “person with folded hands” emoji onto his body and was confronted with its ambiguity when people asked why he got a tattoo of hands high-fiving. He responded via Instagram, “I pity the fool who high-fives in 2014.” Clearly, he sees the emoji as praying hands.”
I personally encountered this potential for emoji ambiguity in a conversation about this paper with my book club, when I got into an enthusiastic argument over the true meaning of the emoji with its tongue sticking out. I insisted it meant, “fed-up” or “annoyed,” but the rest of the group adamantly insisted it had a more humorous connotation, such as indicating a joking tone. Even different service providers interpret the same emoji in many different ways.
If emoji interpretations such as these are so different between two speakers, does this refute Nichols’ mutual intelligibility thesis?
An argument could be made that emoji are too ambiguous to be language, because they can be and so often are interpreted differently by a variety of people. Many words in the English language can also have different, or even contradictory meanings, depending on the context in which they are used, such as crane, date, or net. Word meanings can also evolve over time. For instance, the word “awful” used to mean “worthy of awe.” “Nice” historically meant silly, foolish, or simple. Ultimately, much like words, the meaning of an emoji depends on the context in which it is used. The ambiguity of emoji can also be privy to personal interpretation by the speaker, the same way a theater performer could change a line based on their inflection, making it sarcastic, sincere, exclamatory, or subtle. Cultural context is also significant, as many countries and cultures interpret emoji differently, such as the famed “poop” emoji, which is actually a sign of good luck in Japan. (Who knew?)
Anyone who knew Drake’s strong opinion excoriating the use of high fives would understand his emoji was, in fact, praying hands. Peters writes that, “some argue that the ambiguity of emojis is part of evolving language and can be powerful.” Jenna Wortham, as quoted by Peters’, agrees with her assessment of the “inside joke,” writing that emoji meanings “may change depending on who you’re talking to,” but this does not discredit their ability to communicate. In addition, as emoji themselves have evolved into more detailed GIFs and stickers, the nuances of the emotions being conveyed become more specific and less ambiguous. Just as two words can be similar but are subtley different, such as “fantastic” or “wonderful,” the sad fox sticker seems truly distraught while the sad Pusheen cat seems to be experiencing more of a… wistful ennui.
One argument against emoji functioning as a viable language is that they do not just have arbitrary meaning, but also have an arbitrary or absent grammatical structure. As written in TIME magazine, Katy Steinmetz contests this, explaining that emoji very clearly rely on the order of the symbols to communicate. Emoji tend to come at the end of sentences, respect linear time and action, and, in series of emoji, the “emotional stance” comes before any other action.
Even if emoji grammar can be arbitrary, this does not necessarily negate their potential to become a language. American Sign Language, or ASL, can be compared to emoji because it is a largely visual language that can contain context-specific meaning and has evolved a unique grammar. In ASL, often an SVO (subject-verb-object) order is normal (such as Boy — Love — Dog), although sometimes OSV (object-subject-verb) order can be used and is indicated by a forward head-tilt and a pause (Dog — Love — Boy). There can also be a SVOS (subject-verb-object-subject) structure, in which the subject is repeated twice (Boy — Love — Dog — Boy). Some hypothesize that ASL exhibits free word order, but, even so, there is not an argument against its validity as a language. Similarly, emoji can have a “normal” or usual order, but this order could vary depending on the context.
It is also possible that we are not attaining “mutual intelligibility” with emoji simply because we lack fluency in easily and accurately creating its symbols. Unlike emoji keyboards, our standard QWERTY keyboards are made up of a limited set of letters that can be combined into nearly limitless combinations. Taking a look at Chinese keyboards can help inform us that we could learn to efficiently type thousands, if not tens of thousands, of linguistic symbols such as emoji. Chinese consists of 56,000 independent characters, and it takes learning 3,000 to 4,000 characters to gain functional literacy. There are two types of Chinese keyboards: Pinyin and Wubi. In a Pinyin keyboard, the user types on a regular QWERTY keyboard, using an accepted Roman characters spelling of a Chinese word, which the computer then interprets into a Chinese character. Because Chinese is a tonal language, you can follow a spelling with a digit to indicate a different word. If all else fails, the computer will generate a pop-up window of numerous possibilities, invoking today’s texting technology, where if we begin typing in “H-A-P” into our phones, our autocorrect may suggest “happy,” “happen,” and, of course, :-) . The Wubi keyboard is thought to be a slight improvement on Pinyin typing, where each key indicates a type of pen stroke you would use if you were handwriting the letter. Each part of the keyboard is divided up into different types of pen strokes, including horizontal, vertical, hook, left-falling, and right-falling. By combining these strokes, one can form symbols on the computer just as you would write them on a page. The fastest Wubi typists can type up to 160 words per minute, showing that typing fluency of emojis is attainable.
A final argument against emoji as language is the limited number of available emoji. iOS currently includes 1,851 emoji in their keyboard, far less than the almost 200,000 unique words in the Oxford English Dictionary. If we had the same number of emoji as numbers of words in the English language, would we be able to more effectively communicate with them? Sranan Tongo is a language spoken in Suriname that contains less than 400 words, far fewer than the number of emoji at our fingertips, yet it is still capable of being used to communicate. How many “words” could a picture hold? (Previously verified scientific research says “a thousand.”) Ultimately, fewer words do not necessarily mean less communication.
In his PBS talk, Mike Rugella purports that emoji currently function as a paralanguage, or symbols used to annotate language, rather than replace it. While this may be the current state of emoji, their potential to develop into a language is evident. The 2015 study by Emogi shows that people intend to use emoji as a way to more effectively communicate than with language alone.
When asked why they used emoji, researchers found that, overall, the most popular response from the study participants was: “They help me more accurately express what I am thinking.” Regardless of their supposed ambiguity or lack of grammatical structure, emoji are effective at communicating a good deal of information and attaining mutual intelligibility. Rugella quotes Nick Stockton in his talk, stating, “But emoji aren’t a language. At least, not yet. They’re more like an embryonic language, a cluster of cells that might be a language some day.” Nevertheless, the seeds for language have been planted.
Are emoji language? Emoji don’t CARE if they’re language; they’re becoming as much of part of how we communicate as any language is. Some of our most important and relevant methods of communication exist outside of the realm of language. Emoji aren’t the only supposedly “new” language sweeping and reinterpreting our linguistic world, but the language of code is also, behind our backs, or really, behind our computer screens, entrenching itself into our lives like kudzu. The invasive species of these emerging languages are now becoming a part of language instead of an attack on it. Paul Ford writes in his essay in Bloomberg magazine: “In code as in life, ideas grow up inside of languages and spread with them;” and this definition of language is expanding. As Mike Rugella purports, we in a paralanguage society, in which forms of communication beyond language are gaining or regaining a level of significance and ubiquity in our discourse.
Emoji are a sign that the rise of Internet communication could be increasing our global society’s visual literacy and consciousness. Art and design are no longer reserved for the elite, as they have so often been for the past 600 years in many parts of the world. Instead, they proliferate in the lives of anyone with Internet access. Today, we no longer have to be scribes or artists or illustrators to effectively communicate with visual imagery. Visual illiteracy proliferates as well on the Internet (along with any illiteracy), but much more frequently, we are exposed to the Rembrandts of visual communication as well, as seen in the profusion of highly visually literate web and mobile user design. As more and more users experience the carefully curated (albeit not always perfect) design of the applications in which emoji are used, along with the images that are shared throughout these media, they are experiencing a training in design and art. However, we still need education to guide students in understanding the value and nuance of certain images and designs. In this Internet age, a newfound attention has been brought to the visual and communicative arts, previously abandoned by the STEM fields in college education for the past 50 years. Today, with the inclusion of art, the word is STEAM. Arts and visual communications programs, along with the institutional education of our contemporary lives, are teaching the world, instead of just academics and commercial artists, to READ in pictures. The proliferation and development of emoji communication could be part of this rebirth of visual communication. We’re coming out of the dark ages of visual literacy.
Our growing “visual vocabulary,” including emoji, makes it possible for us to communicate even abstract ideas in pictures. The proliferation of digital communication has included more than just text messages, but experimentation in creative and effective visual communication through games, websites, and apps. While the style of emoji tend to be playful and, as a family member of mine coined it, “emote down” and lack profundity and sincerity, their potential to visually communicate can be much more powerful. Wassily Kandinsky revealed to the Western world the parallels between music and art, that both can have timbre, rhythm, melody, and harmony, and the same parallels could be drawn to language.
Art can have a syntax, a grammar, a structure, and, certainly, an ability to communicate. If art has the ability to communicate abstract thought, emoji could as well. Using the metaphor of designer Bruno Munari, although we may currently use emoji more like a telegram, it could also be used to achieve the abstract qualities of poetry. Munari reflects that Paul Klee once wrote poems with colors before each line of text… an early and abstract paralinguistic emoji. Who’s to say that maybe “this”:
couldn’t gain the emotional and abstract weight of “this”:
Could “this”:
become “this”:
Mike Rugella says emoji could even be equated to the abstraction and ambiguity found in dance. In his book Understanding Comics, Scott McCloud heralds this abstraction found in both comics and emoji and calls it the “special power of comic imagery.”
He writes that the more ambiguous or abstract an image is, the more people it could describe. While greater abstraction may create greater ambiguity, as it does in all of art, this is part of the powerful mystique of visual communication.
We have agency in this ability to create our own visuals and the next generation of emoji. New words in the English language are developed daily, with many of them promptly being accepted by dictionaries, such as Merriam-Webster declaring “truthiness” to be its 2006 Word of the Year. In the tradition of Shakespeare and Lewis Carroll’s poetry, a popular collection of invented neologisms called The Dictionary of Obscure Sorrows gained popularity around 2015. Some of its words, including “vellichor: the strange wistfulness of used bookstores,” were mistakenly included in an Internet list of “the most beautiful words in the English language,” even though they were, of course, not actually part of the English language. With the quick communicative power of the Internet, new words are adopted daily, and emoji are part of that phenomenon. The joy of discovering a new emoji is akin to the delight of discovering a new word that, like vellichor, perfectly describes how you feel. Facebook now includes hundreds of emoji that they call “stickers,” and now there are programs to make your own. You can also integrate a cartoon version yourself into your emoji through Bitmoji. With GIFs now built into Facebook Messenger, it is possible to easily access a much larger visual vocabulary than the standard emoji keyboard. The Wubi method of typing reminded me of those early “emoji” drawing books I had, where I would choose a set of eyes, a mouth, and a nose from a series of choices. By combining portions of emoji visuals like a Wubi keyboard, we could even learn to build our own emoji characters, ala the art of Ed Emberly and as previously abounded through kaomoji.
The Apple watch and current iPhones allow you to “draw” a text to a friend. A chat conversation with a close friend revealed that we’re just a photograph away from creating our own emoji.
The creativity of kaomoji has not died with the introduction of emoji, but is growing. If we continue to have a restricted platform of emoji to use in our chat messages, as Mike Rugella worried, maybe we’re stuck in Newspeak, but emoji have opened the door for us to create our own visual communications. With new technology and the growing popularity of DIY, we’re creating the next “frabjous” or “vellichor” of the emoji world.
Emoji can help facilitate more creative engagement in communication as well. Linguist David Crystal uses the same “inside joke” language as Peters to describe rebuses, stating, “[They’re] an inside joke… [Rebuses] actually require a lot of cognitive effort, so they assume a level of playfulness on both the part of the writer and the reader.” This playfulness and creative engagement with visual language can help inspire even more creative ways to integrate different forms of communication.
The folk creativity that has emerged through this increasing visual literacy is obviously not just limited to emoji, as addressed by famed designer and branding artist Von Glitschka in his talk at the 2014 Creative South Conference. In his talk, Von Glitschka gave a number of examples of how not only artists, but every day people have embraced creativity and visual culture in their daily lives, reminding us that, “creativity isn’t limited to just creative people.” He recalls the story of Dale Price, a Gulf War vet and dad who wanted his kids to be more excited about getting on the bus in the morning. Dale dressed in different costumes every day, including a pirate, Batman, an Egyptian, and even Ariel.
Glitschka tells the story of Freya Jobbins, who, when injured and in physical rehabilitation from her career as a police officer, discovered a box of baby doll parts at a garage sale and began making hilarious… and a little bit creepy, faces out of them.
Glitschka also tells the story of Paul Howalt who began an Instagram feed called “Find a Face,” in which he searches for pareidolia, those hidden faces or emotions, in his every day environment.
Stephanie Posavec and Giorgia Lupi are two information designers who have reinvented the idea of being penpals. Instead of writing letters to one another, for one year, they sent infographics detailing their lives between their homes of London and New York. The themes of these infographics included “What time is it and why was I looking at it?”, “What kind of clocks do I see?”, “What I’ve purchased,” “What I’ve complained about,” “Types of doors I’ve seen,” “Indecisions,” and, most importantly, “When I’ve laughed and why.”
In our flattened global Internet world, these new possibilities for visual communication, literacy, and education are nearly endless.
Mimi Ito, a cultural anthropologist at UC Irvine, explains, “when people are given the capacity to communicate [with new technology], they’re picking them up and developing whole new forms of literacy.” Visual communication is changing because our changing culture and evolving technology is making visual communication and literacy more relevant than ever before. Our emoji paralanguage has many steps to go before achieving the moniker of “language,” but emoji are part of an evolving definition of language and communication. Soon more people will gain the fluency to get the “inside joke” that is emoji.
We are living in a contemporary age of visual culture and therefore using visual communication, including emoji, more than ever. Art does not just hang in a museum anymore, but it exists on our street corners, on our websites, and now, even in the communications on our phones. Visual communication and information is no longer a language of status and power, but a way to communicate amongst everyday people who are gleaning a visual education from the age of digital communication. Emoji have grown from and facilitate the development of THIS language as well, and are not the final step in its evolution.
Emoji Reading List
The 2015 Emogi Report
http://emogi.com/documents/Emoji_Report_2015.pdf
“Can You Speak Emoji?” by PBS
https://www.youtube.com/watch?v=vRNVf-1M7xQ
“Smile, You’re Speaking Emoji” by Adam Sternbergh in New York Magazine
http://nymag.com/daily/intelligencer/2014/11/emojis-rapid-evolution.html
“Do You Speak Emoji?” by Ryan Eyers in Babbel
https://www.babbel.com/en/magazine/do-you-speak-emoji
“Speaking Emoji” by Rachel Peters
http://www.slideshare.net/rachelhpeters/speaking-emoji-amux-presentation
“Why You Should Speak Emoji” by Rachel Peters in Launch Media
http://www.launchjourney.com/blog/why-you-should-speak-emoji
“Textless Texting: Why Emojis are the Future of Language” in Mission Media
http://www.missionmedia.com/blog-interior/textless-texting:-why-emojis-are-the-future-of-language
“Here Are Rules of Using Emoji You Didn’t Know You Were Following” by Katy Steinmetz in TIME
http://time.com/2993508/emoji-rules-tweets/
“Are Emojis a Language?” by John McWhorter and Gretchen McCulloch in Lexicon Valley/Slate
“A world-renowned Harvard linguist thinks emoji fill a gap in the English language” by Drake Baer in Business Insider
http://www.businessinsider.com/why-steven-pinker-loves-emojis-2015-8
“Emoji: Trendy Slang or a Whole New Language?” by Nick Stockton in WIRED
https://www.wired.com/2015/06/emojitrendy-slang-whole-new-language/
“Emoji Art & Design Show”
“Jon Hamm Accepted His Oh, Hello Understudy Role With a Text and Some Emojis, Because He’s a Very Professional Man Who Knows What He Wants” by Devon Ivie in Vulture
http://www.vulture.com/2016/10/jon-hamm-explains-his-oh-hello-understudy-role.html
