HOME > Detail View

Detail View

PRESTO : experience sampling with on-line reconfiguration of experimental parameters for comparative evaluation of mobile applications

PRESTO : experience sampling with on-line reconfiguration of experimental parameters for comparative evaluation of mobile applications

Material type
학위논문
Personal Author
이명희 李明熙
Title Statement
PRESTO : experience sampling with on-line reconfiguration of experimental parameters for comparative evaluation of mobile applications / Myunghee Lee
Publication, Distribution, etc
Seoul :   Graduate School, Korea University,   2017  
Physical Medium
xii, 86장 : 삽화, 도표 ; 26 cm
기타형태 저록
PRESTO: Experience Sampling with On-line Reconfiguration of Experimental Parameters for Comparative Evaluation of Mobile Applications   (DCOLL211009)000000077116  
학위논문주기
학위논문(박사)-- 고려대학교 대학원: 컴퓨터·전파통신공학과, 2017. 8
학과코드
0510   6YD36   337  
General Note
지도교수: 金政鉉  
Bibliography, Etc. Note
참고문헌: 장 81-86
이용가능한 다른형태자료
PDF 파일로도 이용가능;   Requires PDF file reader(application/pdf)  
비통제주제어
ESM , A/B Testing , Comparative Usability Evaluation , Usability Test , Usability Evaluation System,,
000 00000nam c2200205 c 4500
001 000045915421
005 20171012173101
007 ta
008 170705s2017 ulkad bmAC 000c eng
040 ▼a 211009 ▼c 211009 ▼d 211009
085 0 ▼a 0510 ▼2 KDCP
090 ▼a 0510 ▼b 6YD36 ▼c 337
100 1 ▼a 이명희 ▼g 李明熙
245 1 0 ▼a PRESTO : experience sampling with on-line reconfiguration of experimental parameters for comparative evaluation of mobile applications / ▼d Myunghee Lee
260 ▼a Seoul : ▼b Graduate School, Korea University, ▼c 2017
300 ▼a xii, 86장 : ▼b 삽화, 도표 ; ▼c 26 cm
500 ▼a 지도교수: 金政鉉
502 1 ▼a 학위논문(박사)-- ▼b 고려대학교 대학원: ▼c 컴퓨터·전파통신공학과, ▼d 2017. 8
504 ▼a 참고문헌: 장 81-86
530 ▼a PDF 파일로도 이용가능; ▼c Requires PDF file reader(application/pdf)
653 ▼a ESM ▼a A/B Testing ▼a Comparative Usability Evaluation ▼a Usability Test ▼a Usability Evaluation System
776 0 ▼t PRESTO: Experience Sampling with On-line Reconfiguration of Experimental Parameters for Comparative Evaluation of Mobile Applications ▼w (DCOLL211009)000000077116
900 1 0 ▼a Lee, Myung-hee, ▼e
900 1 0 ▼a 김정현 ▼g 金政鉉, ▼e 지도교수
900 1 0 ▼a Kim, Grerard J., ▼e 지도교수
945 ▼a KLPA

Electronic Information

No. Title Service
1
PRESTO : experience sampling with on-line reconfiguration of experimental parameters for comparative evaluation of mobile applications (18회 열람)
View PDF Abstract Table of Contents

Holdings Information

No. Location Call Number Accession No. Availability Due Date Make a Reservation Service
No. 1 Location Science & Engineering Library/Stacks(Thesis)/ Call Number 0510 6YD36 337 Accession No. 123056949 Availability Available Due Date Make a Reservation Service B M

Contents information

Abstract

With the advent of mobile computing and communication devices, the Experience Sampling Method (ESM) has been increasingly used as a convenient and effective approach to evaluate mobile and context-dependent applications and capture their users’ ecological behaviors. 
Similar to any field-based in situ testing methods, the ESM is prone to biases from unreliable and unbalanced data, also A lot of economical cost is required for operating experiments and data analysis, especially for A/B testing and context-specific situations. In fact, ESM has rarely been applied to A/B testing, and literatures revealing its operational details and potential difficulties are very rare. 
In this thesis, as a step toward establishing concrete guidelines, we describe a case study of applying ESM to evaluating two competing interfaces for a mobile application. Based on the gathered data and direct interviews with the participants, we highlight the difficulties experienced and lessons learned. 
And then, we present an enhanced ESM (and an associated tool) called Parameter REconfiguring experience Sampling and analysis TOol (PRESTO), which has the capability to dynamically reconfigure evaluation parameters to help manage an experiment to be more reliable, economical, and convenient. 
With PRESTO, the experimenter can continuously analyze the incoming data for their sufficiency, balance, power (of the test) and reliability, and thereby, e.g. shorten or pay more attention on certain segments of the experiment. To validate its effectiveness, a case study of applying PRESTO to the A/B testing and evaluation of two competing interfaces for a mobile application, as compared to using a conventional ESM, was carried out. Based on the gathered data and direct interviews with the participants, we demonstrate how it could help manage ESM-based A/B testing more efficiently and reliably, and even derive different outcomes.

Table of Contents

Abstract………………………………………………………………………………...iAcknowledgementsiiiCONTENTSvList of FiguresviiiList of TablesxiiChapter 1.Introduction11.1 Motivation11.2 Experience Sampling Method/Tools and Its Evolution61.3 Potential Problems with Experience Sampling Method9Chapter 2.On Applying Experience Sampling Method to A/B Testing of Mobile Applications: A Case Study142.1 Test Application and Evaluated Interfaces142.2 ESM based A/B Testing Process and the Support Tool152.3 Detailed Experiment Procedure172.3.1 Participants and Experiment Duration182.3.2 Data Collection Schedule and Alarms182.3.3 Survey, Contextual, and Gold-Standard Questions192.3.4 User Behavior212.3.5 Compensation212.3.6 Questionnaires about the ESM procedure itself.212.3.7 Data collected from participant232.4 Data Sufficiency and Balance242.5 Participant Reliability262.6 Experiment Extension and the Utility of ESM272.7 Participant Responses about the ESM Process Itself292.8 Identified issues31Chapter 3.PRESTO: Experience Sampling with On-line Reconfiguration of Experimental Parameters for Evaluation of Mobile Applications: An Overview333.1 Intermediate analysis and evolved ESM process333.2 PRESTO Architecture37Chapter 4.An Overview Comparison Study: Conventional ESM vs. PRESTO for A/B Testing424.1 Type of A/B Testing424.2 Test Application and Evaluated Interfaces424.3 Detailed Experiment Procedure434.3.1 Participants and Experiment Duration464.3.2 Data Collection Schedule and Alarms484.3.3 Survey, Contextual, and Gold Standard Questions514.3.4 User Behavior524.4 Intermediate Analysis and Parameter Reconfiguration (PRESTO Only)554.4.1 Data Sufficiency554.4.2 Data/Subject Filtering564.4.3 Analysis Method584.4.4 Analysis Frequency and Rescheduling594.5 Results and Observations614.5.1 Data Sufficiency and Balance – Non-PRESTO614.5.2 Data Sufficiency and Balance – PRESTO664.5.3 Participant Reliability704.5.4 Economy and Utility of PRESTO734.5.5 Actual Experimental Results74Chapter 5.DISCUSSION78Chapter 6.CONCLUSION AND FUTURE WORK80References………………………………………………………………………………81