If social, economic and environmental sustainability are linked, then support for “platform cooperativism”, or the increasing number of non-profit member-owned organizations, has never been more important. Together, these organizations: (1) tackle issues their members identify in the world of work, (2) provide network-driven collections of shared things (e.g., books, tools) and resources (e.g., woodworking spaces, fab labs) that benefit local communities; (3) change the use of resources at the community level and potentially the sosocio-economic structures on the ground. Yet, there is a scarce amount of design research aimed at the particular challenges of organizing in this way.

In contrast to the new “sharing economy” (e.g., Uber, Airbnb) and their well-served commercial needs, platform co-ops attempt to advocate ecological, economic, and social sustainability with the goal of promoting a fairer distribution of goods and labor, ultimately creating a stronger sense of community. While some Human-Computer Interaction (HCI) communities (e.g. Computer-Supported Collaborative Work) have started to leverage ethnographic research methods to explore this emergent phenomenon, researchers have called for more diverse HCI approaches to address the growing scope of challenges within platform co-ops, member-driven exchange systems, and cooperativism more broadly.

The first special interest group meeting on this theme at the HCI conference, CHI’19¹, brought together nearly a hundred researchers from different HCI communities to identify future research directions around cooperativism and platforms. We present the key takeaways and research questions that were raised from this group and open the door for further conversation.

Highlights and Takeaways

Trust

There are emergent challenges of interpersonal trust within co-ops as well as trust in the supporting systems. Traditionally, interpersonal trust is a crucial component in building a successful co-op. When it comes to trust in software systems, use of the platform or app does not necessarily engender trust of itself, conversely, it may actually hide the trustworthiness of people behind the systems. We can ask: once trust is established within one community/platform, how can it be transferred when moving from one community to another? Note that credible trustworthy relationships may change within the transition. Many platforms implement reputation review mechanisms and rating systems with a view toward improving trust. An interesting research avenue is to explore how those systems and mechanisms can be classified and categorized with respect to trust. Ultimately, there are institutional aspects to trust, that is, trust in structures and the organization of groups – trust in governance, as well as more interpersonal concerns. Where should co-op members turn to when something breaks? Who do they need to talk to? How might we design systems to facilitate the development institutional trust and to sustain it?

Sustainability

When it comes to studying co-ops’ sustainability, the issue can be viewed from the perspective of the process (Is the process sustainable?), effect (What are the outcomes? Are they sustainable?), and intervention (Is the intervention itself sustainable?). For example, looking at food co-ops, how can HCI facilitate food production and consumption practices to be more ecologically sustainable? In relation to that, social sustainability is of critical importance. There are significant challenges of access to co-ops for marginalized communities. How do we ensure that people aren’t being “left out”? HCI could look at the opportunity to connect people in the food context attending to food poverty issues and working to de-stigmatize the need for food for underprivileged groups. When it comes to economic sustainability, a key question is what are the business models for co-ops that would have a better chance of success. One central issue to consider is that cooperatives (including local volunteer-driven initiatives) cannot function without some economic/payment model – they must take a cut for operational costs. On the whole, it seems like more purpose-driven niche co-ops (e.g., tool sharing, nanny sharing) have been more successful than timebanking models. Prior work suggests that those seem to fail often because not all skills are valued in the same way. To further address this topic, HCI could look for opportunities to draw more from economics and political science, and Elinor Ostrom’s work specifically.

Participation and scale

When it comes to participation in co-ops, key challenges are related to member involvement, sustaining participation, and addressing economic needs. Furthermore, co-ops often struggle with doing things at scale. HCI may be able to help on this end. Specifically, as co-ops grow, it becomes harder to maintain a democratic member-involvement style. What is more, growth often brings hierarchy that may undermine the democratic principles of a co-op. Networks or federations of co-ops offer a way to work out some of those issues and, what is more, when it comes to the use of technology, collective decision-making tools like Loomio point to interesting opportunities to facilitate the timely involvement of broader groups of members. An interesting research opportunity, then, might be how to understand and support the configuring of tool ecologies for co-ops (especially for local low-tech organizations that are a part of the global movement). What tools could be shared across different (types of) co-ops?

Designing for global and local co-ops

While the co-op movement is global, a repeated point in our conversations was that often the most successful examples are local instances where a community has managed to secure the space, power, and money to engage in projects that they want to take on. However, “cooperatives federations” came up as an example of how small/local co-ops can join forces to rely on shared technology, identity, and branding, while maintaining relative autonomy in their activities and decisions. When it comes to designing platforms for co-ops, key questions include: What kind of flexibility do designers need to account for in their systems design in order to allow co-ops to shape a platform to fit their purposes? How do designers ensure that platforms have enough commonalities to be meaningful and sufficiently consistent for those using them? How should one account for the challenges of adapting platforms/systems to different countries, regulatory contexts, and markets?

Studying co-ops

When conducting research with co-ops, researchers need to be aware of the organizational intricacies involved. This may require a re-think of how partnering with an organization works: Even simple decisions in co-ops may take a lot of time. This can be difficult to align with the typically short research cycle in HCI. Thus, careful planning of the research intervention needs to account for the nature of organizing under study. Depending on the size of a co-op or and the level of researchers’ engagement in the co-op’s activities, researchers should make informed decisions regarding what approach (qualitative or quantitative) and, subsequently, what methodological toolkit to employ. Contextual, participatory design and co-design approaches seem like a good start to explore — and design for — co-ops’ needs and practices. We urge reflection on the following questions to better align researchers’ and co-ops’ agendas: How can researchers establish long-term partnerships with co-ops? What design methods and approaches can incorporate and facilitate sustainable knowledge and skills transfer within the community over time? How could research outputs look like to be genuinely useful for community members? Ultimately, do we need to redefine HCI when working with co-ops, in both a technical sense (who owns the infrastructure) and in terms of broader socio-political considerations?

Join the conversation

If you are a researcher who is interested in cooperativism and platforms or someone who is actively involved in running a co-op, we would be pleased to hear your thoughts! We encourage you to join the conversation on Twitter with the hashtag #HCIxCooperativism. We are hoping this can also serve as a step toward new collaborations, be that in the shape of further special interest group sessions, workshops, or other forms of activity.

This article was written by Anton Fedosov, Airi Lampinen, Tawanna Dillahunt, Ann Light, and Coye Cheshire. A more detailed report from the special interest group meeting can be found here: https://uc.inf.usi.ch/event/sig-chi-2019/

¹ The ACM CHI Conference on Human Factors in Computing Systems is the premier international conference of Human-Computer Interaction. CHI — pronounced ‘kai’ — is a place where researchers and practitioners gather from across the world to discuss the latest in interactive technology. We are a multicultural community from highly diverse backgrounds who together investigate new and creative ways for people to interact.

Talking Cooperativism and Human-Computer Interaction: A Review of the CHI 2019 SIG meeting. was originally published in ACM CHI on Medium, where people are continuing the conversation by highlighting and responding to this story.

The problem of inefficient or insufficient sleep has been acknowledged as an increasingly prevalent global health concern. Prior research has attributed much of this trend to technology use before sleep, prompting experts to advise the reduction of screen time before bed. Yet with the growing omnipresence of our electronic devices, this is becoming ever more difficult. In contrast, we argue that technology holds the potential to promote sleep by facilitating psychological states predictive of a good night’s sleep (positive pre sleep states). We explore this potential through the study of our system, “Inter-Dream”, a multi-sensory interactive artistic experience driven by neurofeedback.

INTER-DREAM AND THE STUDY OF PRE-SLEEP

Inter-Dream consists of several components, these being:

· An interactive bed that can provide vibratory stimulation and be positioned and at the discretion of the artist operating the installation.

· An EEG headset which collects electrophysiological data from the participants brain dissipated throughout the scalp, transformed into frequency bandwidths via Fast Fourier Transforms.

· Visuals projected into both a VR headset worn by the participant, and around the room. The visuals are procedurally generated in real time in response to the EEG data provided by the participant.

· A calming ambient score composed by the artists.

Inter-Dream was originally designed by its artists with the intention of producing an interactive public art instillation which explored the speculative-future concept of interpersonally sharing dreams. In appreciating the neurofeedback properties of the system, the researchers in turn sought to explore how this system may alternatively be applied in the promotion of positive pre-sleep, bringing us to the present study.

Participants individually rested within Inter-Dream, for a period of 10 minutes. To assess the systems interaction with pre-sleep, its factors (arousal; and mood) were measured psychometrically before and after the use of the system. EEG data was also collected as supplement to psychometric data. Qualitative interviews were also employed to help identify why the system promoted (or didn’t promote) positive pre-sleep states.

RESULTS

Regarding the pre-sleep factors arousal and mood, we found that there was a significant decrease in cognitive arousal, negative emotion, and negative affect. However, there were no significant differences in scores of somatic arousal, and positive emotion and affect. In terms of arousal, this demonstrated that while the system illustrated a propensity to “calm” participants mentally, it wasn’t entirely able to do the same physically. Similarly, in terms of mood, the system illustrated a propensity to dispel negative emotions and affect, but not bolster positive ones. Furthermore, when considering this together with the descriptive analysis of EEG activity across our participants, a high frequency in the delta bandwidth relative to other bandwidths during the experience reinforced the notion that these changes in arousal and mood were complicit in promoting positive pre-sleep.

Finally, thematic analysis of participant interviews revealed the following themes:

1. Passivity and Self-Exploration. Through the participants narrative retelling of their experiences with Inter-Dream, an alternating disposition of passivity and exploration was revealed. Most commonly, participants described passivity when discussing their initial interactions with the system, followed by a notable shift toward playful self-exploration as participants became habituated to the system.

2. Mindfulness. Another prevalent theme was the description of cognitive states consistent with those of mindfulness. This was often voiced as a redirection of thought away from life stressors and toward the present experience as a result of the systems neurofeedback reactivity.

3. Restorative Restfulness. A small number of participant responses indicated experiences of restorative restfulness, describing feelings of rejuvenation after the experience akin to those experienced after a short power-nap.

4. Neurocentric Agency. There was an overwhelming focus on describing the connection between the visual imagery, and their brains activity, and a contrasting disconnect with other components. In addition, responses also demonstrated the participants saw this connection as a form of artistic or creative expression.

BRINGING IT TOGETHER

Taken together, our work highlighted the potential of neurofeedback technologies to facilitate creative expression and playful exploration as a potential pathway for future research in supporting sleep. Taking from this notion, we additionally devised a series of design strategies to assist in the design of future neurofeedback driven systems.

1. Facilitate Exploration. Participant responses demonstrated a disposition of curiosity toward the depth of exploration the system allowed. With this considered, we propose it would be in the interest of designers to expand on the level of variability and uniqueness that can be achieved with subsequent or prolonged use, to reward that exploration.

2. Promote Neurocentric Agency. Participants were more inclined to engage with and appreciate stimuli or components which they have agency over, voicing feelings of disconnect or disparity between the components of the system not responsive to their thought. We propose that the future design of multisensory neurofeedback driven systems should consider avoiding the inclusion of non-reactive elements as core components of the experience.

3. Facilitate Self-expression. Furthermore, this appreciation of agency was often paired with appraisals of artistic creativity toward the system. As such, we recommend the exploration of means in which users can interpersonally express and share their creativity generated by electrophysiological output. This could be further fostered by, for example, designing toward the integration of multiple users in a neurofeedback driven system, thereby providing a means for sharing and mutually appreciating the individuality of mind.

For more information, please considering reading our full paper.

Full Citation: Semertzidis, N. A., Sargeant, B., Dwyer, J., Mueller, F. F., & Zambetta, F. (2019, April). Towards Understanding the Design of Positive Pre-sleep Through a Neurofeedback Artistic Experience. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (p. 574). ACM.

Towards Understanding the Design of Positive Pre-Sleep Through a Neurofeedback Artistic Experience was originally published in ACM CHI on Medium, where people are continuing the conversation by highlighting and responding to this story.

This article summarizes a paper that will be presented at CHI 2019 on Tuesday 7th May 2019 at 11:00 in the session Kids And Health.

Machine learning techniques for automation are plentiful and powerful — but it is important to assess whether and how it is possible to use them well. Here we explore our assessments for automation-readiness in a baby milestone tracking app currently relying on human experts, with lessons for any projects considering integrating automation with high requirements for classification accuracy.

Background: Why Child Development Tracking Matters and the babyTRACKS Solution

One in six children has a developmental delay that impairs attainment of critical life skills in motor, language, cognitive, and/or social-emotional abilities. Early childhood developmental screening is critical for timely detection and intervention, which leads to better outcomes for kids and their families. The problem is especially acute in socioeconomically disadvantaged communities where regular access to healthcare is limited, but even in countries with adequate medical resources, 70% of children with a developmental delay are diagnosed late.

To empower parents to better partner with healthcare professionals to monitor their children’s development, we created babyTRACKS, a free, live, interactive developmental tracking app (available at the Apple App Store, and Google Play) with over 3,000 children since 2015. Parents write or select short milestone texts, like “began taking first steps,” to record their babies’ developmental achievements, and receive crowd-based percentiles to evaluate development and catch potential delays.

Scaling Curated Crowd Intelligence (CCI)

Behind the scenes in babyTRACKS, an expert-based Curated Crowd Intelligence (CCI) process manually groups incoming novel parent-authored milestone texts according to their similarity to existing milestones in the database (for example, “starting to walk”) or determining that the milestone represents a new developmental concept not seen before in another child’s diary.

To help CCI scale, we want to use machine learning to automate part or all of the current manual process. We stepped back to assess our automation readiness through three studies investigating:

(1) the scalability limitations of our CCI process, by analyzing the human cost of CCI, how the work is currently broken down, and which areas are (and are not) ripe for automation.

(2) the consistency of our dataset by testing the inter-rater reliability of curators and hence the validity of our milestone data for algorithmic training and evaluation purposes; and

(3) the value of the dataset, by appraising the “real world” clinical value of milestones when assessing child development.

We conclude that automation can indeed be appropriate and helpful for a large percentage, though not all, of CCI work. We further establish realistic upper bounds for algorithm performance; confirm that the babyTRACKS milestones dataset is valid for training and testing purposes; and verify that it represents clinically meaningful developmental information.

Pre-Automation Lessons Learned

Our work illustrates several important benchmarks to check prior to plunging in to developing a machine learning algorithm to automate an existing manual process (adapted of course to the technical specifics of the task):

• Assess if / how much automation can actually help
• Establish target for algorithm performance, based on best human agreement, to know what to aim for (may be less than 100%).
• Ensure your training/evaluation gold-standard dataset is indeed valid and meaningful.
• Remember that automation need not be all or nothing: algorithms can aid humans (for example, through narrowing down options), substantially speeding up manual tasks even if not able to fully replace people

Learn More

• Check out our full research paper to be presented at the ACM CHI Conference on Human Factors in Computing Systems 2019.
• Read our previous work on the babyTRACKS system (formerly called Baby CROINC).
• Try out the app on the Apple App Store or Google Play.
• Contact us at info@babytracks.org with any questions or collaboration ideas.

Citation:

Ayelet Ben-Sasson, Eli Ben-Sasson, Kayla Jacobs, Elisheva Rotman Argaman, Eden Saig. 2019. Evaluating Expert Curation in a Baby Milestone Tracking App. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ‘19). ACM, New York, NY, USA, Paper 553, 12 pages. DOI: 10.1145/3290605.3300783

Preparing for Automation and Evaluating Expert Curation in a Baby Milestone Tracking App was originally published in ACM CHI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Takeaway: We explored how a swarm of small wheeled robots could provide different touch patterns to users. We then ran two studies: one to capture how people perceive them and another study to explore how users would generate different touch patterns to express various types of social touch.

Touch Interaction with Robots

Touch is an integral part of our daily lives. It allows us to not only discriminate shapes and textures but also is embedded in our social interactions (e.g. handshake and huddle) and affective communications (e.g. hug and holding hands).

On the other hand, more and more robots of various forms and sizes are appearing in our daily lives. While they are getting better and better at object manipulation and sensing, little work has been done on enabling robots to use touch for interaction with people. This is due to various factors such as user’s safety and a lack of understanding on both how to design these touch (haptic) interactions and how people would perceive these touches.

Design of SwarmHaptics

To investigate this, we began with one of the simplest types of robot: a wheeled robot without any face or limbs. Leveraging its motion, we explored what types of touch it can provide as shown below. We then expanded to what a swarm of robots could do by leveraging the temporal, spatial, and force coordination among them as shown below.

Study 1: Perception Study

To first understand how people would react to these touches, we ran a perception study where we had the robots provide different touches to the users and recorded how users felt as shown below. For more study details, refer to our paper.

Study 2: Social Touch Elicitation Study

To better capture the expressivity with the robots, we employed a participatory design where users generated the appropriate interaction, or touch patterns with the robot, to convey different types of social touch. As shown below, participants controlled up to four robots simultaneously through a multi-touch screen to convey prompted messages such as “happy” or “move over”. This study allowed us to learn both how people convey specific social touches and a larger picture of how people use different features of the robots to convey information such as contexts and emotion.

Actionable Conclusions

1. Based on our study, there is a trade-off when increasing the number of robots for touch interactions. While more robots increase the perceived arousal and urgency, it comes at the cost of perceived likeability and safety. Thus, depending on the application, you may only want to have a small number of robots provide the haptic feedback even if you have access to a larger number of robots.

2. Visual motion of the robots can be used to provide context especially for abstract social touches. For instance, participants used how the robots move away/toward to convey abstract emotions like surprised and afraid. Thus, it is important to take into account how the robots move even when designing haptic (touch) interactions with people.

Full Citation:

Lawrence H. Kim and Sean Follmer. 2019. SwarmHaptics: Haptic Display with Swarm Robots. In CHI Conference on Human Factors in Computing Systems Proceedings (CHI 2019), May 4–9, 2019, Glasgow, Scotland Uk. ACM, New York, NY, USA, 13 pages. https://doi.org/ 10.1145/3290605.3300918

SwarmHaptics: Haptic Display with Swarm Robots was originally published in ACM CHI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Takeaway

VizNet is a large-scale corpus of over 31 million datasets compiled from the web, open data repositories, and online visualization platforms. Researchers can use VizNet to conduct experiments with real-world data, assess the ecological validity of synthetic data, and compare design techniques against a common baseline.

The Need for Visualization Repositories

Large-scale databases such as WordNet [1] and ImageNet [2] provide the data needed to train and test machine learning models, as well as a common baseline for evaluation, experimentation, and benchmarking. They have proven instrumental in pushing the state-of-the-art forward in language modeling and computer vision.

Research on graphical perception, however, often relies on ad hoc or synthetically generated datasets that do not display the same characteristics as data found in the wild. To date, insufficient attention has been paid to design and engineer a centralized and large-scale repository for evaluating the effectiveness of visual designs. This heightens the need for building a large scale corpus to learn, evaluate, and benchmark various measures of perceptual effectiveness.

Characterizing Real-World Data

We introduce VizNet, a large-scale corpus of over 31 million datasets compiled from the web, open data repositories, and online visualization platforms.

We find that real-world datasets typically consist of 17 rows and 3 columns. 51% of the columns in the corpus are categorical data, 44% quantitative, and only 5% temporal. About half of the columns are best described by a normal, lognormal, or power law distribution. Summary statistics and distributions (bottom) are shown below.

Utility of VizNet as a resource for data scientists and visualization researchers

We demonstrate VizNet’s viability as a platform for conducting online crowdsourced experiments at scale by replicating the Kim and Heer (2018) study assessing the effect of task and data distribution on the effectiveness of visual encodings [3], and extend it with an additional task: outlier detection.

While largely in line with the original findings, our results do exhibit several statistically significant differences as a result of our more diverse backing datasets. These differences inform our discussion on how crowdsourced graphical perception studies must adapt to and account for the variation found in organic datasets.

As the VizNet corpus grows, assessing the effectiveness of these (data, visualization, task) triplets, even using crowdsourcing, will quickly become time- and cost-prohibitive. To contend with this scale, we conclude by formulating effectiveness prediction as a machine learning task over these triplets. Our results suggest that machine learning offers a promising method for efficiently annotating VizNet content.

Conclusions

1. VizNet provides the common baseline for comparing visualization design techniques and developing benchmark models and algorithms for studying graphical perception at scale.
2. We demonstrate how machine learning models can offer a promising method for efficiently annotating (data, visualization, task) triplets at scale.
3. VizNet research provides an important direction to understand the opportunities and challenges faced in replicating prior work in human-computer interaction and visualization research.

Acknowledgments

We thank Alex Johnson for providing access to the Plotly API, Robert Kosara for providing the Many Eyes data, and the authors of [4] for scraping and providing access to open data repositories.

References

[1] George A Miller. 1995. WordNet: a lexical database for English. Commun. ACM 38, 11 (1995), 39–41.

[2] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In CVPR.

[3] Younghoon Kim and Jeffrey Heer. 2018. Assessing Effects of Task and Data Distribution on the Effectiveness of Visual Encodings. Computer Graphics Forum (Proc. EuroVis) (2018).

[4] Sebastian Neumaier, Jürgen Umbrich, and Axel Polleres. 2016. Automated Quality Assessment of Metadata across Open Data Portals. Journal of Data and Information Quality (2016).

VizNet: Towards a Large-Scale Visualization Learning and Benchmarking Repository was originally published in ACM CHI on Medium, where people are continuing the conversation by highlighting and responding to this story.

At the heart of the computing revolution lies a printed circuit board (PCB). PCBs are the final stage of most circuit designs thanks to compactness, robustness, and ease of mass production. These strengths come at a cost, however — PCBs are invariably harder to debug than more malleable form factors, such as breadboarded circuits. While our HCI community has focused recent attention to the prototyping stages of circuit design, great challenges remain in our abilities to effectively debug the PCBs in proliferation around us.

We classify these observed challenges into three primary areas:

First we highlight issues of access. The compactness of PCBs creates difficulties in simply accessing signals of interest. To debug effectively, designers need to operate as closely as possible to their hardware, readily observing behaviors while perturbing the circuit in measured ways. Instead, designers often find themselves manually probing the pins of small integrated circuits, hunting faults that lie hidden beneath components, or manually adding in test pads to afford themselves greater access.

We also observe challenges of isolation. Connections on a PCB are fixed with solder and therefore are not readily disconnected. In debugging, however, many of the more powerful tools at our disposal require testing a component or subcircuit in isolation. Many of the greatest strides in PCB debugging, such as the widely adopted JTAG standard, improve on our ability to perform “in-circuit” testing (on a board in its fully connected state).

Finally, PCBs suffer from difficulties in iteration. The fixed nature of traces on a board raises barriers to modification. Often, if a designer modifies their design they must refabricate the entire board. This limited alterability limits debugging as well — there is often no better way to explore a potential solution than to simply produce a new PCB.

Our Vision

In consideration of these challenges, we produced a vision for our ideal debugging session:

After encountering an issue, a designer sets their newly fabricated PCB on an augmented test platform. The system immediately scans the board for short circuits, loose connections, and other deviations from the intended design.

For bugs escaping automatic detection, the test platforms allows the designer to inspect any signal in their circuit with a click. With similar ease, they can inject test signals at any point to observe responses throughout the circuit, or load a library of tests to verify properties of each component.

Without modifying their hardware, the platform can temporarily manipulate connectivity within the circuit, allowing the designer to isolate regions to test a component or subcircuit outside of the larger circuit context. And with similar ease, the designer can interactively replace parts of the circuit, allowing them to explore modifications without fabricating a new board.

In these interactions, the test platform is both automated tester and debugging aid, offering the designer control over and visibility into otherwise immutable and impenetrable hardware. This vision guided our design of Pinpoint.

Pinpoint

Pinpoint is an end-to-end pipeline enabling in-circuit PCB debugging techniques like those described above at low-cost and with minimal overhead. We designed Pinpoint to integrate into a designer’s existing workflow: After designing a PCB, Pinpoint’s instrumentation stage automatically adds test pads to their board. When fabricating their PCB, the user fabricates also fabricates a “jig board” — a bespoke hardware interface that connects their PCB to test software. Then, debugging proceeds as in the vision proposed above: the designer secures their board to the jig, connects the jig to Pinpoint’s Control Board, and uses the test software to explore their circuit.

Pinpoint’s software interface enables debugging interactions in alignment with our vision. Users can probe or inject signals at any instrumented point in their circuit, and they can record and replay these signals to study reproducible test examples. Rather than locating connections of interest on potentially complex board layout, users can instead indicate points on the schematic diagram. They can author functional tests in a custom scripting language to build up a suite of unit tests that quickly detect deviations from intended behavior. Pinpoint can also generate tests automatically, such as pairwise tests to detect unintended shorts in the circuit.

Pinpoint’s most powerful features utilize built-in solid state relays that control the connectivity of instrumented points in the circuit. Users can independently toggle any of these switches, effectively isolating parts of their circuit on demand. This enables tests which normally cannot easily run in-circuit (e.g. measuring current flow) and facilitates structured approaches to debugging (e.g. integration testing). Pinpoint also breaks out these instrumented locations for easy access by external connections. As a result, users can engage in what we call “splicing” — introducing external circuit elements to explore modifications to the circuit. Instead of desoldering a problematic component, the user can simply disconnect it from the rest of the circuit and replace it with an external component via jumper cables. These capabilities introduce sorely needed control over an otherwise static PCB.

Our vision calls for access, isolation, and iteration — for increased control over and a tighter coupling with hardware for more effective debugging. Pinpoint is a step towards that vision, providing in-circuit testing capabilities to all via automated instrumentation, programmatic probing/injection, high-level test authoring, subcircuit isolation, and more. For more details about its design, implementation, and evaluation, see our paper at CHI 2019 or see our project page!

Full citation: Evan Strasnick, Sean Follmer, Maneesh Agrawala. 2019. Pinpoint: A PCB Debugging Pipeline Using Interruptible Routing and Instrumentation. In CHI Conference on Human Factors in Computing Systems Proceedings (CHI 2019), May 4–9, 2019, Glasgow, Scotland UK. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/3290605.3300278

Pinpoint: In-circuit PCB debugging for all was originally published in ACM CHI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Summary: We designed and evaluated three methods of watching a sign language interpreted natural history documentary. One was a traditional in-vision method with signed programme content on TV. The other two were AR-enabled methods in which an AR sign language interpreter (a ‘half-body’ version and a ‘full-body’ version) is projected just outside the frame of the TV presenting the documentary. We invited participants with hearing impairments to our labs, in the UK and Germany, to tell us which method of delivery they preferred. In the UK, participants were split 3-ways in their preferences while in Germany, half the participants preferred the in-vision method followed closely by the ‘half-body’ version. Full details of the design of the AR interpreters and our explorative study have been published at the conferences ACM TVX 2018 and ACM CHI 2019 respectively.

Context: In collaboration with industry partners, BBC R&D and IRT have been exploring how to customise and personalise the experience of viewing programme content on connected TVs by, in tandem, delivering additional (companion) content, to personal devices via IP.

We have been working with the industry to put in place the technical standards that will be needed to get devices to ‘talk’ to each other and synchronise media presentation at DVB and HbbTV. HbbTV technologies are a part of TV platforms such as Freeview Play and will enable audiences to benefit from these technologies. Our various demonstrators include synchronising content on a connected TV with content on a tablet or with a toy. We have also conducted user experience studies and collaborative projects, such as 2-IMMERSE, help us understand how we might create connected personalised experiences that might appeal to our audience.

So far, our investigations have focused on devices like a mobile phone or tablet. However, with the rise in popularity of optical head mounted displays, AR applications enable a way to make experiences that might spill out beyond the frame of a TV thereby extending the real estate of the TV screen and enabling personalisation of the media experience in an intriguing way. As part of a closed laboratory study, we wanted to investigate how AR can be used for the delivery of sign language interpretations to audiences with hearing impairments while they watched a TV programme. Traditional access services undoubtedly enhance our TV programme offerings for many users but they are an intrusive option which can’t be personalised or controlled beyond turning them on/off. How might we allow users watching TV in a group, personalise their viewing experience without imposing their preferences on the whole group? There is merit in being able to present personalised augments, like a synchronised AR sign language interpreter, to enhance the TV experience without using up the screen real estate of the primary viewing device.

Design Problem: How do we go about designing an AR sign language interpreter?

The design of traditional in-vision signage provided us with guidance on the creation, capture and presentation of our AR sign language interpreters. Several transferrable pieces of knowledge were identified from the UK-based industry regulatory authority Ofcom’s ‘Code on Television Access Services’, and prior work in Germany and the UK, in organisations such as the BSL Broadcasting Trust and the Deaf Studies Trust.

We settled on two designs — the ‘half-body’ and the ‘full-body’ AR interpreters. Full details of the design process and key principles we used to create the final designs are available in our ACM TVX 2018 work-in-progress paper. A summary of the guidelines are:

1. The heads of AR interpreters have to be aligned to the top of the TV,
2. The bottom edge of ‘half-body’ AR interpreter have to be aligned with the bottom of the TV; almost sitting on the TV stand,
3. The feet of the ‘full-body’ AR interpreter grounded in the physical room,
4. The direction of gaze of the interpreter has to be horizontally facing towards TV while in resting phase and has to be directly facing towards the viewer during the interpreting phase, and
5. The interpreter needs to be positioned slightly overlapping the TV.

Method: We invited 23 participants (11 in the UK, 12 in Germany) to watch natural history documentaries in our user labs and evaluate three methods of delivering synchronised sign language interpretations — one traditional (picture-in-picture) in-vision method and two AR-enabled methods with the ‘half-body’ and ‘full-body’ AR sign language interpreters, presented through a HoloLens.

Findings: Participants were asked to fill questionnaires and we conducted a semi-structured interview to elicit their thoughts on the three methods we evaluated. In the UK, participants were split 3-ways in their preferences while in Germany, half the participants preferred the traditional method followed closely by the ‘half-body’ version.

Unsurprisingly, participants found wearing an optical head mounted display, while watching TV, uncomfortable. They also thought the field of view afford by the device to be too narrow to cover the TV and the AR sign language interpreter just outside to the right of the TV frame. Although, not all discomfort could be attributed to technical or physical limitations.

“Having glasses on your head for two hours? No way! And I can imagine, you put them on and everyone will look at you and they will recognise you are deaf. I don’t like showing the outside world that I am deaf. I feel too vulnerable”

— P10 (UK)

Despite the limitations, participants liked the idea of using novel technologies to solve a real world problem in TV watching and were open to the idea of using AR technologies to receive synchronous sign language interpretations in future iteration of these devices. Our qualitative data doesn’t reveal outright preferences between the three methods of sign language delivery presented but participants always had a clear favourite with clear reasoning behind their choice.

“She was sharper and she felt closer to you. It felt like I had my own personal interpreter in the room with me. I wanted to offer her a cup of tea.”

— P7 (UK)

We got a deeper insight into our participants’ perceptions and implications for improving the potential design of AR interpreters. Full details of our findings are in our paper. In summary, the importance of clarity of interpreter, maintaining the connection between interpreter and content, placement of interpreter relative to the content, and maintaining the balance between the interpreter overlapping over the content without obscuring the content was stressed. The underlying advantage in using AR technologies — the ability to tailor interpreters to the viewers needs — was seen as very exciting.

Although, we did not see a consensus on the preferred interpreter presentation, our results show that the demands made by deaf viewers on the sign-language service are very individual. Participants in our study highlighted the importance of control when it comes to personalising the experience to their needs.

Personalisation is key!

Paper citations

Vinoba Vinayagamoorthy, Maxine Glancy, Christoph Ziegler, and Richard Schäffer. 2019. Personalising the TV Experience using Augmented Reality: An Exploratory Study on Delivering Synchronised Sign Language Interpretation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ‘19). ACM, New York, NY, USA, Paper 532, 12 pages. DOI: https://doi.org/10.1145/3290605.3300762

Vinoba Vinayagamoorthy, Maxine Glancy, Paul Debenham, Alastair Bruce, Christoph Ziegler, and Richard Schäffer. 2018. Personalising the TV Experience with Augmented Reality Technology: Synchronised Sign Language Interpretation. In Proceedings of the 2018 ACM International Conference on Interactive Experiences for TV and Online Video (TVX ‘18). ACM, New York, NY, USA, 179–184. DOI: https://doi.org/10.1145/3210825.3213562

Personalising the TV Experience using Augmented Reality was originally published in ACM CHI on Medium, where people are continuing the conversation by highlighting and responding to this story.

What do children mean when they say technology is creepy?

This article summarizes a paper authored by Jason C. Yip, Kiley Sobel, Xin Gao, Allison Marie Hishikawa, Alexis Lim, Laura Meng, Romaine Flor Ofiana, Justin Park, and Alexis Hiniker. This paper will be presented at CHI 2019, a conference of Human-Computer Interaction, on Wednesday 8th May 2019 at 9:00 in the session Privacy and Young People.

What makes children think certain technologies are creepy but view others as benign?

To investigate this question, we conducted four participatory design sessions with 11 children (ages 7 -11) to design and evaluate creepy technologies, followed by interviews with the same children.

What Do Children Fear?

Children’s fears about the risks technology fell into two different categories:

Physical Harm. Children in our study often used morbid words such as “kill,” “murder,” and “death” to convey their fears about creepy technologies. They constantly referred to physical harm when describing creepy or unknown technologies. Children described, for example, technologies that try to punish their users and technologies to stalk others and cause harm.

Loss of Attachment. Children’s second recurring fear was that creepy technologies would take them away from their parents or otherwise intrude on relationships with people they love. A child in our interview stated that creepy technologies take you away from mom and dad. The children in our study had this fear of technology “taking over your life” so that they would be unable to be with their real parents.

What Makes Children Think a Technology is Creepy?

Kids told us that certain signals make technology seem creepy and arouse fears of physical harm or separation from their parents. The six signals that came up repeatedly were:

Deception. The children in our study frequently expressed fears about technology intentionally deceiving them. For instance, one child noted: “Like I’ll say -call Jan Smith [mom, pseudonym] and it [digital voice assistant] will call that person. Okay, it will call them. Then when I ask -will you kill me in my sleep? It says -I can’t answer that.” Here, the child wanted a direct NO in the answer to his question on killing him in his sleep, rather than an “I don’t know.”

Mimicry. Children expressed concerns about technology mimicking them or other people, potentially giving it the power to subsume their identity. Children in our study worried about too much information taken from technology so that it could, “steal your identity,” replace you, or take you away from your family.

Control. Children expressed concerns about the ability to control the flow of information, the actions of technology, and its output. For instance, one child expressed that if they could not control Amazon Alexa, the technology would seem creepy: “Yeah, so, it’s like Alexa is in this room and she starts interrupting this conversation.”

Unpredictability. Children explained that systems whose behavior they could not predict — such as a digital assistant that no longer responds to its wake word — led them to worry that something that seems harmless might become sinister.

Ominous Physical Appearance. The superficial look, sound, and feel of a technology is key to how children assess if the technology is creepy. In some instances, children were willing to look past other creepy signals if a technology had a charming appearance.

Mediating between signals and fear

Children referred to their parents as the most important factor in determining whether technologies were creepy or not.

One child noted that smartphones, laptops, and other consumer electronics were not creepy because their parents frequently used them without anxiety.

In contrast, another child expressed that consumer electronics had the potential to be creepy because his parents put a paper cover over their laptop camera to prevent intruders.

What does this all mean for parents/guardians and designers?

As a starting point for designing technologies children trust and enabling children to decide what to trust, we came up with a set of questions that both parents/guardians and designers can think about and discuss together with kids, such as:

● What information is okay for people to know about children? Do we think technology could deceive people? Why and how?

● What do we want a technology to look like? Do technologies that look and feel nice always act nice?

● Could technology ever replace people?

● What do you think about parent/guardian usage of technology? How does technology usage in parents affect children’s views of that relationship?

For a complete list, see our full paper.

Children today have core fears that are overlooked in the design process and
have the potential to be invoked by technology.

We believe that it is essential to understand more deeply children’s fears of technology and how they make sense of the digital world around them.

Full citation:

Yip, J., Sobel, K., Gao, X., Hishikawa, A.M., Lim, A., Meng, L., Ofiana, R.F., Park, J., & Hiniker, A. (2019). Laughing is scary, but farting is cute: A conceptual model of children’s perspectives of creepy technologies. In Proceedings of ACM SIGCHI Human Factors in Computing Systems (CHI 2019).

What makes children think certain technologies are creepy but view others as benign? was originally published in ACM CHI on Medium, where people are continuing the conversation by highlighting and responding to this story.

This article summarizes a paper authored by Shunichi Kasahara, Jun Nishida, and Pedro Lopes. This paper will be presented at CHI 2019, a conference of Human-Computer Interaction, on Monday 6th May 2019 at 14:00 in the session Direct Bodily Interaction.

More than years have passed since researchers started using electrical muscle stimulation (EMS) in interactive systems. Kruijff et al. (VRST'06) explored how these tiny medical devices could change gaming on a desktop by allowing the user's muscles to contract in response to game events. This fueled researchers such as Emi Tamaki & Jun Rekimoto to open the doors of EMS to the CHI community (CHI'11).

Ten years after… I'm astonished at all the creative uses the HCI community found for EMS: Max Pfeiffer & Michael Rohs explored how to steer participants, Jun Nishida & Kenji Suzuki enabled communicating gestures from person to person, and myself together with Patrick Baudisch explored how to turn the user's body into input and output devices (just to cite a few, for a more detailed timeline of the many contributing faces of EMS in HCI, see here).

But, ten years later, it's about time we talk about agency. EMS systems offer a compact and wearable form factor (when compared to their mechanical haptic counterparts, such as exoskeletons) but being moved by an external force: feels weird. If you were ever moved by some haptic actuated device (be it EMS, exoskeleton or a robotic arm) you probably felt how strange it is to see and feel your body being moved by an external cause.

So for this year's CHI, my group at the University of Chicago (with the help of our collaborators) decided to take some first steps in understanding what is going on with the loss of agency in haptic actuation. We asked ourselves the question: how do we understand what is going on in the user's brain when one is moved by an external force?

To answer this we turned to two core questions: (1) can haptic systems actuate us to provide a significant faster reaction time without always entirely compromising my agency? (2) How does our brain integrate haptic feedback when moved by an external force such as EMS?

In our first #CHI2019 paper, together with Shunichi Kasahara (Sony CSL) and Jun Nishida (University of Chicago), we explore how delaying the onset of the haptic actuation dramatically improves the sense of agency! Despite being a first step to understand the relationship between agency and preemptive, we think these results are really exiting; they allow us to build a model to choose how much to sacrifice agency to gain reaction time speed ups, and vice versa!

The next steps will involve understanding how this might scale to more complex tasks, as insofar, we only explored simple scenarios such as high-speed-photography or hitting an object in motion! If you want to try this, consider coming to our demonstration at SIGGRAPH'19.

Now, going deeper, one might ask: but how does our brain integrate and process these haptic signals? This is a question we are constantly debating and digging deeper. One possible angle to explore this came from our second #chi2019 paper, which was a collaboration with Klaus Gramann's group at TU Berlin and C.T Lin's group at UTS. We uncovered another small piece of the haptic agency puzzle while trying to understand how to detect mismatches in virtual reality (VR) without having to ask user's about their subjective experience; i.e., can we evaluate the coherence of a visuo-haptic VR experience without having to show you a presence questionnaire? Here, instead of asking the user, we measured their brain's responses, using EEG, as their interacted with VR objects that exhibited also haptic feedback (in fact, vibration and/or EMS). We found out that when there is a mismatch between visuals and haptics (e.g., out of sync) our brain's event related potential (ERP) looks very different from when things feel right. In fact, there is a pronounced negative valley in the user's ERP when things are off from our expectations.

It is precisely this mismatch in expectations, which we observed even when user's did not consciously realize this were off, that might help us understand the puzzle of agency and EMS. These unrealistic situations might be very similar to when we are moved by an inexplicable external force, such as EMS.

Going even deeper into the neural processes, with the assistance of functional magnetic resonnance imaging (fMRI) we can examine our brain at work when we are moved by external forces like EMS. This is precisely what I did with my collaborators Jakub Limanowski (main author at UCL), Janis Keck (FU Berlin), Patrick Baudisch (HPI), Karl Friston, and Felix Blankenburg (FU Berlin). In our upcoming Cerebral Cortex paper, we used fMRI to examine` how agency (i.e., whether you moved yourself consciously or EMS moves you) impacts how our brain interpret and integrates sensory information such as touch sensations!

These three projects just scratched the surface of all the questions around agency and haptics. We hope to foster more discussion on agency next week at #CHI2019. Come talk to us! Lastly, I am very grateful to work with all these brilliant minds, with a special and humble thanks to Shun, Jun, Sezen, Lukas, Jas and Klaus!

Electrical Muscle Stimulation in HCI: 10 years later… what about the question of agency? was originally published in ACM CHI on Medium, where people are continuing the conversation by highlighting and responding to this story.

This article summarizes a late-breaking work presented at CHI 2019, a conference on Human-Computer Interaction, on Tuesday 7th May 2019.

The current food system feeds 7.3 billion people and very few of them are involved in food production, while more than half live in cities. By 2030 the world population living in cities will rise to 5 billion. This creates a significant challenge in providing global urban populations with sufficient food in a sustainable way making necessary for people to start growing their own food at home. The aim of this project is to create a technological kit that will help citizens in the process and understanding of plants’ growth.

Field research in Community Gardens

To start our research we decided to focus on already successful urban farming initiatives, as community gardens. Community gardens are areas in cities reserved for non-commercial horticulture where people meet each week to work together in all the processes involved in growing food, and then share the produce among members. They have been shown to be successful in motivating people to be more involved in food production and sustainable ways of growing food. Our aim was to take inspiration from these communities to see if we could replicate some elements in a distributed manner, helped by technology.

We scheduled multiple visits to ‘Anna’s Tuin en Ruigte’ at Amsterdam Science Park within the city of Amsterdam where we observed the organization, participated in maintaining the garden, talked to members of the community, and conducted 1-to-1 interviews. The outcomes were used to draw an Empathy Map, the first step to define a brief for our prototype.

Growkit prototype

During the process of gathering information from urban gardening communities, several hardware explorations were conducted. Input from the community was used in each hardware iteration.

The final Growkit prototype design is made of the following components:

The sensors stick

The sensor stick (Catnip Electronics) measures the moisture of the soil, the amount of sunlight the plant receives and the room temperature. The stick sends the measured data to the central unit, where the received data from the sensors are compared to the optimal conditions for that specific species of plant and feedback is given to the users through the central unit.

The central unit

The central unit displays the health of the plants to which each stick is connected to the user through lights. The central unit has a circular shape with three circles. The different circles represent the three different measurements of the plant (i.e. water, light and temperature) through an integrated LED strip. The LEDs light up depending on which measured aspect of the plant needs attention. For the conditions to be optimal, the circles need to be fully lightened.

The app

The app receives all the data from the central unit and gives detailed information to the user about the plant that is being monitored. Each user has a personal profile showing the plants they are growing and the milestones they want to reach. The community of users can interact and encourage each other by celebrating successes and giving advice.

Next steps

The main focus of future work will be on building up the virtual community aspects further. One idea is to integrate data obtained from home growers with data from vertical farming and urban gardens to obtain richer data sets regarding specific plants’ growth conditions. We continue to iterate on the current Growkit hardware as we continue to take inspiration from urban farming communities to build towards more sustainable urban food production.

The Digital Society School is a growing community of learners, creators and designers who create meaningful impact on society and its global digital transformation. Check us out at digitalsocietyschool.org.

Growkit: Using Technology to Support People Growing Food at Home was originally published in ACM CHI on Medium, where people are continuing the conversation by highlighting and responding to this story.