Ubiquity, Testing, Experience

Ubiquity: that is perhaps the one word contrast between the context of the previous generation of OWL testing and this new iteration. In my last post, I wrote about designing the students' experience of the class. This blog is perhaps clearest evidence of the transition from an OWL redesign project dedicated to Usability and one meant to focus on UxD. In 2006-2007, there were elementary blog hosting services, but now these template-driven hosting services (like this blogger site) are quite sophisticated. The design and function are "good enough" to support our class needs, as well as the demands of documenting and recording the project itself. I think about all the time saved both by students and me, and that time can be spent on reading more recent research as well in articulating this content. The blogging tools are ubiquitous. They're simply here: free, useful, and most importantly, usable.

Just as the resources available on the OWL and the different methods of accessing the information are changing, the methods we use for analysis need to change too. —Kaitlyn

I would be wasting my energy and students' time if we spent this class concentrating on usability to the exclusion of experience architecture. There are limits to the utility of revisiting usability precisely because usability has been so successful. In the middle 1990s, we were making arguments about including usability precisely because so many emergent designs were unusable, either pet projects not meant for widespread adoption or examples of what Norman refers to as "next bench" design, the design solutions engineers come up with that prove a concept at their workbenches but really aren't meant for general, mass use. Usability has broken that scheme and practice, and moved from emergent design (as Bjiker named it) and towards recognizable genres (which Spinuzzi has traced for some time in his work). Bob Johnson has recognized this Ubiquity Paradox and questions insistence on older usability, user-centered, and participatory lexical fixations. 

The OWL needs to make sense. Not just for me, a tech-savvy college student, but also my mom, a middle aged music teacher who does not work with a lot of technologies on a daily basis. It shouldn't make users wonder why they must take certain steps, it should be self-evident. —Jessica

Blogger is a tool we are employing to help create the communication environment of the OWL UxD project. Similarly, Purdue provides access to a sophisticated survey tool, Qualtrics. Other similar tools exist, some at low or no cost for educational use. I've been asked if I might be making students dependent on this tool: what happens when their first professional position requires data gathering and there is no access to Qualtrics? Putting aside whether this or another tool might be better to address the need, it seems silly to deny students use of a powerful tool for fear it might not be available at some unknown future point. Further, comparison with free tools allows students to decide: perhaps Survey Monkey has all the power and flexibility they need. Or perhaps they can then begin the challenging but necessary task of articulating need and gathering materials. 

What makes the Purdue OWL a great tool/resource is that it can be accessed by anyone with a computer and reliable Internet connection. Not only is it easily accessible, it's free. And while it contains the overarching theme of guidelines for writing, many different people access the site for different reasons. How the user experiences the site and his or her satisfaction with it depends heavily on whether or not the site helps in achieving there goals. Therefore, by redesigning the site, we are redesigning the user experience of the site so that it fulfills their needs and helps them achieve their goals. —Kristin

Besides blogging and surveying tools, students have numerous computing devices they carry with them to class, including a variety of smartphones, tablets, phablets, netbooks, laptops, and other writing tools. Most students log on to the classroom computers—which are last-generation iMacs with gigantic integrated screens. But many students rarely if ever log on to these computers, and by extension, also eschew resources like software and storage space available through the wired campus network. Students access materials through the less robust but more flexible wireless connectivity available just about everywhere on campus. WiFi is like oxygen on campus: everywhere and seemingly free, and only noticed in its absence. That is, ubiquitous.

There will be a wide range of things we have to consider while we are creating the user experience to accommodate all OWL users because they have so many different types of users with different access to certain technologies. —Kira

Behavior surrounding the campus network and computing resources has convinced me that one response to ubiquity needs to be a reconfiguration of campus computing facilities. I favor replacing the 20+ student machines with 5 or 6 higher end machines capable of and equipped for HD video editing, 3D sound editing and recording, and student- as well as teacher- controlled image projection. This is how we are reimagining our multimedia production space. And have proposed for our main technology rich classrooms. But I digress. 

The purpose of redesigning the Purdue OWL has shifted from usability to UXD design. I find it interesting ... that usability has come so far and that this is a major reason why we focus more on UXD design... I agree that some major changes could be made to the room. On our redesign of the room, we included more tables and more computers, which will definitely help shape a user's experience. —Andrew

Blogging and surveying tools are part of this new age of ubiquity. Similarly, students' own mobile technologies are part of a literate context that seems to define this new era. That these technologies can be seamlessly accessed and used interchangeably—that they are useful and usable—are part as well of the environmental dimension of technology. That WiFi is not only present but robust, but more importantly expected. In this context, usability testing is much less pressing than attention to the users' experience within this techno-cultural space and place.

We need to begin taking into account all of the different devices that were mentioned in this post. Looking at some of the network statistics from the OWL, very few hits are actually coming from the mobile devices that students carry with them everywhere. While this could be that the content may only be relevant when they are working on an actual computer, I think we need to begin considering how users would interact with a mobile app and what information they may need access to from a device that fits in their hand. Rather than just re-sizing the OWL to pocket form, we need to start thinking ahead to what the needs of the users are. —Tanner

Blogger and other Google docs; email and instant messaging; Qualtrics; the physical classroom itself with its installed computers and full suite of software; campus WiFi; mobile and laptop computers: this mix of devices, software, services, and places all contribute to the instructional technology—they are available to students but can become invisible, transparent—as technologies. These technologies are for the most part looked through to the work at hand precisely because usability as a movement has been so effective. These things just work. So what's at stake?

There are many systems that "just work" without a second thought for me. But as I read this post, I began to wonder whether technologies truly "just work" or if I, as a user, have adapted my interaction with a technology so that it can work. —Jessica

Well, the continued invention and design of new services, and of new categories of devices (I did mention phablets above, which are certainly evolutionary and not revolutionary, but still, a new micro-category). These are important. But I think the bigger rewards lie in taking successful processes and knowledge making from the software and web worlds and putting them to work in new contexts: from workplaces to architecture to manufacturing, there are innumerable contexts where knowledge and core competencies developed in the service of creating these now-ubiquitous technologies and literacy practices make training in technical communication very valuable. In an age of ubiquity, experience architecture is an important and valuable specialty in a wide variety of workplaces.

So, my question is: as different tools for increasing/measurability usability increase, does the assumption that these users immediately know how to use them increase as well? And how do we negotiate these kinds of expectations? How do we need to think more critically about accessibility and, indeed, willingness, to work with and learn from these types of technologies? — Mary

This classroom is the experience of UxD for these students. I have designed it in order to teach these students about usability, user-centered design theory, as well as involve them in ongoing research. I also ask them to think with me, and sometime struggle to make clear what we are doing ... precisely because  I do not yet know. But that's what I take to be my responsibility in this class: I share with students what I know (what I think I know) at this moment about UxD, offer access to ongoing discussions as well as a balanced mix of new, groundbreaking, and foundational research. I bring with me the expectation that their other teachers and the curriculum have prepared them to move from what they have been taught to  the challenges that lie ahead of them. As these quotes I've pulled from their responses show, they have prepared themselves not just to respond to the challenges of Purdue's OWL and its user experience needs, but to address a wider range of challenges posed by valuing user-involved research.

International Students and ESL Resources by Andrew Yim

I have been shadowing tutors in the Purdue Writing Lab for a class this semester.   Freshmen have come in frantically looking for help with documents like their English 106 papers.  Many freshmen feel overwhelmed since many have never written an essay for college before.  Freshman year, I struggled with English 106 papers because I did not develop a strong foundation for writing.   Staring at a blank page for hours, I struggled to come up with ideas, but thank heavens that I had an extraordinary English 106 professor.  His class improved my writing immensely by teaching principles that I could use for future papers.  After taking his class, I realized that speaking fluent English has always made me feel more confident about my writing. I have observed many international students come into the Writing Lab looking for help on their essays.
 
Observing these sessions were eye-opening as many international students lack confidence in their work; however, I have observed many students formulating great ideas for their papers.  However, they believe that their language skills are not up to par and this mentality has made it hard for them to communicate their ideas.

Figure 1

As a native English speaker, writing in English is second-nature to me.  I could only imagine how international students feel when they are trying to write an English paper.    As they begin to formulate ideas, they might struggle with communicating these ideas onto paper leaving  them feeling frustrated.  If I tried to write a paper in Chinese or Korean, I would definitely feel lost.  This previous summer, I served as a Boiler Gold Rush Team Leader and I had quite a few international students from other countries shown in the photo above. Also, I am currently a part of an international ministry and have become friends with many international students.  For many, this is their first time visiting the United States.  Many believed that they would struggle with their classes because they struggled to understand English.  Some asked me to slow down my talking or to help them with their English papers   Students who come to the United States  have to demonstrate foreign language proficiency, but students still struggle even after demonstrating minimal proficiency in understanding English.

Figure 2

A few months ago, I was not sure how I could help them; however, working on redesigning the OWL has provided me with a clearer answer.  As seen in Figure 2, the English as a Second Language pages provides more clarity to international students who want to write more efficient essays. These pages contain a wealth of information about ESL resources.  However, international students are not aware of the many resources out there for them.  These pages also have resources for tutors wanting to learn strategies to help international students learn English. Furthermore, they cover a variety of grammar topics like adjectives, adverbs, verbs, prepositions, pronouns, and other resources.    As our class are redesigning these pages, we need to target our audience and I believe that many of my friends will benefit from our redesigns.

Throughout my experiences in the Writing Lab, many international students have negative expectations of their own writing.  They are so focused on correcting their grammar that they doubt their own ideas.  As non-native English speakers, they will often feel frustrated and seek out resources that will help them improve their English.  Through my observations, I have seen many non-English students struggle with understanding certain phrases in English.  Many phrases that are common to native English speakers are completely foreign to non-native speakers.     This disconnect in language makes it hard for them to write essays in English.

With many of my international friends, I have practiced speaking in English with them.  I speak quite fast so I have learned to slow down my talking.  In return,  I see them improve their English through conversation.  Thus a great resource that international students can take advantage of is ESL conservation groups.   I believe that the OWL Staff needs to be aware that these conservation groups can help improve students' writing.  Posting the hours of these conversation groups will be immensely beneficial to students looking to improve their language skills.    I believe that helping international students learn the basics of the English language can lead to better developed essays. This will can improve their self-confidence and help them become better writers.

Thus the OWL staff should make non-native speakers more aware of these resources when international students first arrive at Purdue.  The OWL Staff needs to find ways to help direct users, especially non-native speakers of English,  to information that will be most helpful to them.  Students in 515 need to be aware of this audience as they are proposing suggestions to help redesign the website especially for the pages about ESL.   I believe that it is crucial to use certain words on these pages to help international students understand certain concepts.  If there is too much text, international students will feel overwhelmed reading these posts.  Thus students in 515 should suggest a way to redesign these pages through observations of international students.

I believe that an ethnography will be helpful for gathering data to help redesign the ESL pages.  An ethnography is the study of people and culture and studying international students will help the OWL staff and students in 515 understand how to tailor these pages for international students.   Some ways to study international students would be to set up a Qualtrics survey interviewing international students about how they view the English language and if they believe that the ESL pages help them learn more about the English language.  Also, doing in-depth interviews with international students can be beneficial.  All of this data can be used to help effectively redesign the ESL pages.    I will definitely be advertising the ESL Pages to all of my international friends and to students who I meet in the future.

Purdue OWL Usability – September/October Update

A lot of work has happened in recent weeks connected with the usability study. In early September Tammy, Michael, Cait, and I sat down for a meeting to discuss potential areas for the students in Michael’s Advanced Professional Writing course. There were a number of concerns that were brought to the table: concerns relating to navigation, the splash page, connecting users to resources, presenting resources to users, and the functionality of our contact forms. What was beneficial was having two members of the research team from the last round of usability studies present, as they could speak to what we’ve done before, what we chose to address at that time, and what might be worth reexamining. Having the technical coordinator present also opened up new areas for discussion, as she deals most often with the OWL’s innards—the code that keeps it soaring, ready to swoop in and assist users in need. One issue that was brought up was how users experience the OWL. Currently, we have what some have derided as a 1990’s-esque splash page, and certainly its actual usefulness is something that ought to be questioned. But, during the course of our discussions that day it was pointed out that perhaps the splash is even less effective than we had hoped. One of the meeting members pointed out that we need data on how people actually use the OWL, perhaps many people bypass the splash page all together. Here’s an anecdote for illustrative purposes: 

I use the OWL a lot for my own personal reference as I write for publication and the job market, activities not even remotely related to my position as the OWL coordinator. Now, when I’m looking for information on Academic Cover Letters—for those increasingly rare tenure-track jobs—I don’t go to owl.english.purdue.edu. Rather, I go to Google and I type in, “ ‘Purdue OWL’ + ‘Academic Cover Letters’”. The very first hit is the opening page for the Academic Cover Letters resource on the OWL. Then I start to click, click, click away. To be honest, there are some days that I completely forget that the splash page exists. The argument for it being there is that the OWL isn’t just the collection of writing resources; it’s not just the Orange OWL. It’s the Purdue OWL Family of Sites; it’s the Orange OWL, it’s the Purdue Writing Lab site (green pages), it’s the Purdue OWL News Service (purple pages). 

We also discussed the intersections of content development and usability. The Purdue OWL, over the years, has developed its own written identity. This is something that has been passed down orally from one content coordinator to another during training and on-boarding sessions. It is then something that is highly negotiated during between content coordinator and their developers during the content development process. The discussion with the students in Michael’s course became, what can we do to crystalize certain elements of our style into a Purdue OWL Style guide, a document that will help to streamline certain elements of the development process and that will also concretize institutional memory and best practices? At a meeting of members of the usability teams and Purdue OWL leadership, I was given a list of areas in which the student-team working on the Purdue OWL style guide would like to focus their work. I must admit that I’m both impressed by the thought that is going into this document and hopeful for the lasting contribution that it can make to the Purdue OWL—and perhaps to OWL designers and content developers working in other contexts. The student-team plans on not only solidifying the most salient points of OWL style, but also on redesigning the current OWL content developers manual to be more effective in transmitting information to the content development team members. During the course of the various meetings and conversations that have been had at the OWL about usability in the past two months, another key item has arisen. Designing and implementing usability studies can take time. We’re making plans that will carry use well into the 2015-2016 academic year. 

However, when you’re working with graduate staff this can present new challenges. Both myself and Cait are likely to complete our Ph.Ds. before the start of the next school year. This means that two individuals that have played some role in this round of testing will be exiting Purdue before it’s over. This means that we must consider how we will hand this project off to the incoming content and technical coordinators. This is a key consideration when working in any context that has this kind of turnover. Ensuring continuity becomes important in order to maintain quality and to provide the best service to our end-users. For us, we will rely heavily on the experiences of both Tammy and Michael—this is their second round of usability studies on the OWL, and we will also allot additional time to bring the new coordinators up to speed regarding the usability study, how it impacts their work at the OWL, and how they might contribute to this intensive practical and scholarly endeavor.. This will likely include a healthy stack of documents and intermediate reports, but it will help us to ensure that the new coordinators will be adequately prepared not only to join the OWL family, but also to continue to contribute to our usability studies and eventual revisions.

On-Campus Reconnaissance

I recently participated in user testing for one of the departments on campus revamping its heavily-used website.  I won’t identify the department to protect its study and work, but I will say that the site is an important resource and instructional tool on campus.  I’m a frequent user of the site, and as someone involved in the next usability study for the Purdue OWL, I wanted to provide what I hoped would be helpful feedback and learn about the usability work happening elsewhere on campus. 

I like participating as a user and giving feedback.  My first introduction to usability was as an early adopter and usability participant for the TOPIC project at Texas Tech University.  Since then, I’ve become an advocate, a learner who knows a few things though not an expert.  I’m generally the type who will respond to improvement and marketing surveys or provide user reviews, but usability is something else and something more.  When I have a chance to do more than respond to a site’s color scheme or some low-level feature, I will take the opportunity because it’s a learning experience for me. 

So I agreed to give an hour of my time to meet with the researcher and provide information about the department’s existing site and upcoming changes.  I was given a short demographic survey and then asked to perform a series of tasks on the site, mostly related to locating specific pieces of information or resources on the site.  I relied heavily on the search tool (which makes sense given this particular site), and I was introduced to features I didn’t know about.  The researcher made an audio recording of my responses and also recorded my movements on the screen.  The tasks were most interesting, and it became clear to me that the study was aimed at the site’s search functionality and menu structure. 

I tried to be as clear and detailed as possible in my responses because I know how difficult and yet valuable it is to know what users find helpful or not, what features they want and need, and what parts of the site have simply been overlooked.  It’s a vulnerable moment for site administrators, webmasters, designers and other staff when presented with users’ feedback.  Data can offer surprising results, including the fact that a feature lovingly created and nurtured by a designer is not helpful to—or worse, vehemently hated by—users.  Feedback can result in the need for significant revisions.  These are some reasons why usability research is given just a passing, suspicious glance by designers. 

Yet usability is extremely important, and because I valued the campus website I was asked to test, I wanted my feedback to lead to beneficial changes.  Nevertheless, I was very much aware that I had two purposes for participating in this study:  as a legitimate user and as someone doing reconnaissance for her own usability project. 

One of the most important things I learned from this experience is that test subjects can find usability studies as difficult as researchers do.  It’s not always possible to articulate a reason for a preference or a process, so researchers need to be clear and offer alternative explanations without leading the research subject toward a certain (preferred) response. Also, recording subjects’ responses can be helpful, since researchers cannot take detailed notes while paying close attention to what users are doing and saying, what their eyes focus on, or what the mouse clicks.  And tracking users’ clicks and mouse movements with with software can reveal information that may not be easily detectable by researchers.

And finally, usability is time-consuming work, even with technologies that can make tracking, recording, card sorting, and other activities easier.  The researcher spent nearly an hour with me just administering the tests.  He and his co-researchers will then have to compile data gathered from me and other participants.  They will have to wade through all that information to see what it yields about site users’ and their preferences and needs.  They will have to give that information to site designers and programmers.  Someone, somewhere will have to decide what revisions should be made to the site.  And then someone else has to make those changes. 

The amount of work, time, and resources involved is not surprising, especially after the previous OWL usability study.  Like the campus website I tested, the OWL is a heavily trafficked and complicated resource with many different stakeholders to consider.  I know from experience that we’re in for several semesters’ worth of work, and with that in mind, I eagerly but patiently await any changes to the site I tested, knowing that results can take time.  And I look forward to figuring out what this next OWL usability project will be like.

Preparing for the 2014-2015 Purdue OWL Usability Study

Usability and the Purdue OWL Content Coordinator We have been internally talking about usability studies for quite some time, since shortly after I came onboard as the OWL content coordinator in the summer of 2012. At the time, not discussion was had as I needed to acclimate myself to my new role. As the newest member of the Purdue OWL team, I had a lot to learn about my position. The content coordinator is responsible for overseeing content development for the Purdue OWL, as well as for interfacing with user's of the Purdue OWL. This means needing to identify and hire talent to develop our new content, revising existing content in response to user feedback, and sorting through thousands of link request from schools, universities, and government bodies from all over the global. It is, at time, a rather large job that is constrained to only ten hours per week. So, at the time very little was said about usability studies. However, it did remain at the fringes of our collective attention during the 2012-2013 and 2013-2014 academic years. 

 With the Advanced Professional Writing course that Dr. Salvo is teaching this year, the time is right for a new round of usability studies. Since starting in 2012, I have wondered what role I might be able to play in this project. While I am no tech novice, I'm far from the power user that I used to be in high school, when I was studying how to program routers and switches--how to make the internet work. And while usability and user experience design aren't often talked about in my field (applied linguistics), I'm hopeful that I can learn enough to be useful through following the 515 course readings and sitting in on the classes when I'm able. That being said, I do feel that my time both developing content for the Purdue OWL, and sitting in the OWL's nest helping to keep her soaring will be helpful in the completion of this project. I believe that I can help to offer additional perspective. It may also be that my previous OWL research, focusing on L2 writing teacher's use of the OWL and on content development best-practices may be helpful in the endeavor on which we are about to embark. 

Creating a Plan 

At the beginning of summer, Purdue OWL leadership and Dr. Salvo met to begin early discussions on how the next round of usability studies might unfold and on what our goals for this round might be. That was in early to mid-May. After that, Dr. Salvo and our associate director left the States for a summer study abroad and research trip to Scotland. Shortly after their return, we held our second strategizing meeting. It was a fruitful meeting, one of those meetings where you could see the progress bar fill a little more. The kind of meetings that make you feel good about meetings. During this meeting, Dr. Salvo and I discussed the direction his class will take in the fall, which included a list of readings and planning the sequencing for two of the course assignments. The reading list is unlike most reading lists that I see in English department classes, even disciplinarily focused ones. The readings for this course are predominantly on usability and design principles, useful readings for students at Purdue. I must admit that I’m a bit excited about these readings, too. Any time that we can create a space for students, especially graduate students, to engage with new readings--readings potentially outside their interest or their disciplines--we are providing students and young professionals with the opportunity to learn to read in new ways and to find possible interfaces between disparate bodies of knowledge. 

 We covered a lot of ground in this meeting. We began sketching out a multi-tiered usability test with both large- and small-scale tests and sub-tests. Currently, our ideal-targets include a large-scale demographics survey (10k target) and three smaller online usability tests (~100 ea.). this will be followed up by a number of in-lab usability tests (~50). There will be some similarity between select online and in-lab tests. For example, we plan on having both web-based and in-lab card sort activities. These activities will be similar to what the OWL and Dr. Salvo did a few years ago, allowing participants to reorganize components of the OWL, e.g., the landing page. It is possible that the web-based and in-lab card sorts will yield interesting results when compared to one another. Some of these tests will also include working with our new eye-tracker package. We’re fortunate to have a very energetic student working with us on this part of the testing. We’ll also be looking into shoestring testing--testing on budget--and distance testing. The findings here may be of particular use to scholars in the social sciences and the humanities. 

Professionalizing Graduate Students

Something that is of particular interest to me is the opportunity to foster professionalization of graduate students. During the meeting we discussed the possibility of having selected graduate students work as team leaders on the sub-test materials developments. They will sit in on the proposals presentations, and they will consult with the students and guide them through their materials creation. To me, this is a vital opportunity in a number of ways. First, it gives these graduate students an opportunity to gain leadership and project management skills. These are two skills that I think every academic needs to have. They’re also useful if one chooses to exit the academy for either industry or government. Also, this would represent the opportunity for the team leaders to test their leadership skills in an academic environment. Before entering the academy I had spent a few years in leadership roles in the retail and entertainment sectors. Taking a leadership role at OWL was a wake up call as to just how different leadership in the academy can be. It’ll be good for the team leaders to try on new hats. Second, if we are able to link the tests that they oversee to their personal research interests, it is likely they may be able to get some publications or presentations out of this that will increase their capital on the job market. We’re certainly moving forward, and linking this to a class is allowing the OWL to tap the energies and experiences of individuals we might not get to otherwise. It’s always exciting for these collaborations to happen. Over the coming months, it’ll be interesting to see how the project comes together and how all parties involved, especially the relative novices--like myself--develop over the course of the projects.