More and more, individuals make decisions based on information they find online. Increasing numbers of technology-based HIV interventions have provided evidence that suggest online behavioral interventions are acceptable and efficacious in promoting behavior change (Hightow-Weidman et al., 2015; Muessig et al., 2015; Conserve et al., 2017; Garett et al., 2016). However, compared to in-person and supervised computer-based interventions, researchers have less control over participants’ engagement with online interventions. Researchers can look to “auxiliary data that capture details about the process of the interaction with the online intervention,” also known as paradata (Couper et al., 2010) to understand how and when participants are using an intervention (Sowan & Jenkins, 2010). Paradata can be used for myriad purposes, but is most often used to measure the number of participants who accessed the intervention, their frequency of access, time spent interacting with the intervention, and the specific activities they engaged in (Donkin et al., 2013; Kelders et al., 2013).
Paradata can shed light on the specific components of an intervention that were used by participants and can help determine if components should be kept, removed, or redesigned between versions or prior to scale-up and dissemination (Brouwer et al., 2011; Morrison & Doherty, 2014). Ultimately, paradata is a tool that can facilitate data-driven decision making about the design and functionality of an online intervention. Researchers can use paradata to make informed decisions in an effort to maximize participant engagement, satisfaction, and – ideally – to increase intervention efficacy.
With most randomized control trials of online interventions, it would be difficult or impossible to claim that all participants, even those within the same treatment arm, were exposed to the exact same intervention if they were able to choose when, how, and for how long they engaged with the intervention. The possibility exists that one’s time with or type of engagement with an intervention is associated with certain behavioral effects. Without paradata, researchers can’t account for differences in engagement (i.e. time spent on intervention, activities completed in intervention, etc.) that may affect the outcomes of their intervention. Simply comparing outcomes between study arms in a randomized control trial of an online intervention does not adequately account for participants’ “dose” of the intervention and any “dose-response” relationships that may be affecting the findings.
The Tech Core supports researchers in the iTech network in systematically collecting paradata that can enrich traditional outcome analyses and illuminate any dose-response relationships that may exist. Additionally, paradata analyses may help researchers identify if certain intervention components were tied to behavior change, enable data-driven iterations of the online intervention, and allow for efficacy comparisons of intervention components across studies.