On The Nature of Feedback

Digital and internet based artists are always seeking feedback on their work, but how does this feedback affect them? Is there a secret sauce that makes your feedback better? Or more likely to be used? Well I attempted to find out, sort of.

Crowdsourcing feedback has emerged in the past decade as a popular way for designers and businesses alike to receive broad reactions on products and ideas. Online communities and message boards have been quick to add feedback tabs and sections to capitalize on the possible advantages of crowdsourcing. There has been a fair amount of research done in this time around collecting online feedback, but the research has mostly been reserved for graphic design. The clothing section of design has been largely left in the dark in terms of scholarly research. In my research I have found some factors that make for good feedback online by measuring what ideas are implemented and which ones are not, specifically in the field of clothing.

The art of streetwear has seen a resurgence over the last decade. A rise in popularity that coincides with the broader American Do-It-Yourself craft movement, it was merely a matter of time until the two fields collided. No community is a better picture of DIY streetwear than Streetwear Startups, a subcommunity on Reddit. For years now this community has served as a way for people to buy streetwear made by hand and as a way for designers to advertise their material. Most importantly for our research, it is used to get feedback on designs. The “design feedback” section of the community has existed in its current form since 2015 and serves as the database for my analysis. Artists can post their designs using the design feedback flair and then can receive feedback from other members of the community, some of which are designers. The use of a flair allows users to categorize the post underneath a topic within the community. themselves or streetwear enthusiasts. Each post consists of about 5–10 comments of feedback, not all of which is implemented into the designs. To identify which comments were more likely to be implemented I combed through 150 posts and pulled out every instance where the feedback was implemented. Sometimes the designer will update the designs or respond to comments; if they did not then I reached out to the designers myself to see what changes they made. The comments were then differentiated between positive and negative feedback. By studying these feedback posts we can determine what makes an idea more likely to be implemented into the designs.

Common ideology would posit that professionals are the go-to source for new ideas. They are after all, professionals. Nevertheless there has been an increase in scholarship dedicated to finding the validity of crowdsourcing new ideas. One study in particular from Marion K. Poetz and Martin Schreier of the Copenhagen School of Government sought to find the value in crowdsourcing marketing ideas. In this study, five marketing companies were asked to generate new ideas through professional consulting per usual, but to also open up an avenue for crowdsourcing ideas. All five companies set up websites to field the ideas and offered $500 cash prize incentives to those whose ideas were chosen. The ideas from the websites were analyzed by two executives at the company (the CEO and head of R&D). Ideas were first judged on two premises, whether it was a true idea and whether or not the idea could be evaluated properly. After this the quality was judged based on three criteria, the novelty of the idea, the value of the idea, and the feasibility of the idea. After compiling all of the research, the study found that there was far less novelty in the ideas that came from professionals, and that even though their ideas were more feasible, the ideas from the crowdsourcing scored higher on value (Poetz & Schreier). The findings suggest that while crowdsourcing ideas is not a one-to-one replacement for professionals, it can be an effective and worthwhile complement to them. Poetz and Schreier’s research has been cited over one thousand times and is imperative when discussing the value in crowdsourcing ideas.

In 2007, three researchers from Carnegie Mellon University sought to bring crowdsourcing scholarship into the internet era. Unlike previous scholarship, this study observed ideas from a database and was not an experiment. The idea database IdeaStorm was launched by Dell in 2007; it allowed users to upload their own ideas while other users could vote on the ideas afterwards. After evaluating the ideas, Dell marked them in three different ways: implemented, partially implemented, and not planned. The study included over 12,000 ideas in a three year span. The study found that as time passed from the launch of the service, the amount of ideas posted decreased but their implementation increased. Their results found that people tend to overestimate the feasibility of their own ideas at first, over time this subsides which leads to more implementation over time.

In recent research the internet has become the predominant method of gathering information on crowdsourcing. Most studies are done through the internet with online communities. “A Classroom Study of Using Crowd Feedback in the Iterative Design Process,” presented their findings at the ACM Conference on Computer Supported Cooperative Work & Social. This study in particular acknowledges that there is little research done on online feedback even though many feel it would be a useful resource. The study was conducted in an Intro to Design course with ten students who received feedback once and then had their designs examined at the end to see what changes were made. After examining the final projects, researchers found that students were willing to make deep changes, not just cosmetic ones to their designs and that free-form broad feedback actually led to more changes than specific feedback.

Much of the research done has obviated the question of what exactly makes specific instances of crowdsourced feedback more likely to be implemented. Another study presented at the ACM conference dealt more specifically with this question. Similar to my own research, this study employed an online community of like minded designers to field feedback. In this experiment, students were asked to make designs for a campaign poster which were shown on a forum for graphic design to both experts and amateurs. A follow up to the study found that ideas were more likely to be implemented when the feedback was more positive. Also found in the study was that experts provided longer and more often, clearer, advice that was easier to implement.

Scholars often link the use of crowdsourced feedback to an increase in design revisions. In “Almost an Expert: The Effects of Rubrics and Expertise on Perceived Value of Crowdsourced Design Critiques,” several scholars ran a study on how getting advice from online message boards leads to changes in a student’s design (Yuan, Luther, Krause, and Vennix 2016). Research on feedback however has not sufficiently explored what exactly makes crowdsourced feedback more useful in the design process. While there is a large literature on graphic design in this field, there is a lack of scholarship in the clothing start-up sector. Clothing start-ups and more specifically streetwear start-ups have become increasingly common in the last decade as they have become more accessible to the average designer. “Almost an Expert” focuses entirely on graphic design made by students. The study concludes that students are willing to make more changes to their designs when gathering feedback online. My research seeks to fill the gap here by examining what exactly goes into such feedback that makes it more likely to be acted upon.

A full understanding of the target of analysis is necessary to completely grasp my conclusions on this research. Reddit is a social website that offers thousands of smaller online communities, or subreddits, where users can post and browse content. These subcommunities are often denoted with an “r/” before the title in reference to the community’s link. The subcommunity in question for my research, r/streetwearstartup, has seen a surge in users over the last several years, likely a product of the popularity of DIY crafts in the United States. In 2016, the community had just under 5,000 subscribers, but today they boast over 130,000 members. Growth like the one seen on streetwear start-ups is no doubt an extension of the arts and crafts revival in America. The arts and crafts revival in America has been thoroughly documented by researchers like Mascia- Lees. In “American Beauty: The Middle Class Arts and Crafts Revival in the United States,” Mascia- Lees discusses how market capitalism has driven forward an urge to buy from smaller businesses and things made by hand. One sector that has seen an increase because of this movement has been the DIY clothing sector. Gildan brand blank T shirts and embroidered logos have become mainstays on r/streetwearstartup and more designers are posting their creations daily to seek feedback from fellow creators.

To begin my research I sorted through the subreddit by the design feedback flair and looked through all forms of streetwear. This included tops like t-shirts and hoodies as well bottoms and accessories like bags and hats. Including all these items helps to broaden the scope of my research. I then sorted through 150 posts that were serviceable for my research. Posts with no comments or no discernable feedback were not counted. For example, if a post included only reaction comments like “this is so fire!” or “love the embroidery” then it wasn’t counted. Also excluded from the set of 150 were posts that only included two or fewer comments of feedback. Once I had gathered what I felt like was an accurate sampling I then went through each post’s comments and categorized them as positive or negative. After this, I documented which comments of feedback were implemented into the designs and which ones were not. More often than not the designer would post their revised designs in the following days of the original post. If not I would follow up with the designers over direct messages to see what changes were made. After documenting all of the feedback I found a moderately strong correlation between feedback that was positive and that which was implemented. Feedback that was generally negative was only implemented 48 times out of 252 when negative feedback was given. On the other side, feedback that was more positive in nature was implemented 360 out of 420 times. I can conclude through my research that there is a correlation in the way feedback was delivered to designers and their decision to implement said changes on r/streetwear.

Through my own research on crowdsourced feedback, I have found a correlation in one case between positive feedback and implementation, but there are limits to my own research. The obvious error is that feedback is not always as simple as positive and negative- there are other factors and qualities of feedback that I did not measure. Professionalism and brevity are two factors that come to mind that I was forced to ignore. In future research, scholars should look to more fully examine all elements of feedback to gain a better understanding of it.

The last decade has seen an increase in research on crowdsourcing. Companies and artists alike have started to use crowdsourcing as a way to generate new ideas and to gather feedback on old ones. Less research however has been done on the nature of such feedback and what goes into a revision or idea that makes it more likely to be implemented. I sought to fill that gap and have found sufficient evidence that in my study, positive feedback was more likely to be implemented than if it was negative. Still, there is much work to be done in creating a full body of scholarship around this subject by incorporating larger studies and different fields.