1887
Volume 24, Issue 1
  • ISSN 1572-0373
  • E-ISSN: 1572-0381
USD
Buy:$35.00 + Taxes

Abstract

Abstract

Phone calls are an essential communication channel in today’s contact centers, but they are more difficult to analyze than written or form-based interactions. To that end, companies have traditionally used surveys to gather feedback and gauge customer satisfaction. In this work, we study the relationship between self-reported customer satisfaction (CSAT) and automatic utterance-level indicators of emotion produced by affect recognition models, using a real dataset of contact center calls. We find (1) that positive valence is associated with higher CSAT scores, while the presence of anger is associated with lower CSAT scores; (2) that automatically detected affective events and CSAT response rate are linked, with calls containing anger/positive valence exhibiting respectively a lower/higher response rate; (3) that the dynamics of detected emotions are linked with both CSAT scores and response rate, and that emotions detected at the end of the call have a greater weight in the relationship. These findings highlight a selection bias in self-reported CSAT leading respectively to an over/under-representation of positive/negative affect.

Loading

Article metrics loading...

/content/journals/10.1075/is.22038.bol
2023-08-28
2025-02-09
Loading full text...

Full text loading...

References

  1. Ando, A., Masumura, R., Kamiyama, H., Kobashikawa, S., and Aono, Y.
    (2017) Hierarchical LSTMs with Joint Learning for Estimating Customer Satisfaction from Contact Center Calls. InProc. Interspeech 2017, pages1716–1720. 10.21437/Interspeech.2017‑725
    https://doi.org/10.21437/Interspeech.2017-725 [Google Scholar]
  2. Ando, A., Masumura, R., Kamiyama, H., Kobashikawa, S., Aono, Y., and Toda, T.
    (2020) Customer satisfaction estimation in contact center calls based on a hierarchical multi-task model. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 281:715–728. 10.1109/TASLP.2020.2966857
    https://doi.org/10.1109/TASLP.2020.2966857 [Google Scholar]
  3. Auguste, J., Charlet, D., Damnati, G., Bechet, F., and Favre, B.
    (2019) Can we predict self-reported customer satisfaction from interactions?InICASSP 2019 – 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages7385–7389. 10.1109/ICASSP.2019.8683896
    https://doi.org/10.1109/ICASSP.2019.8683896 [Google Scholar]
  4. Bockhorst, J., Yu, S., Polania, L., and Fung, G.
    (2017) Predicting self-reported customer satisfaction of interactions with a corporate call center. InAltun, Y., Das, K., Mielikäinen, T., Malerba, D., Stefanowski, J., Read, J., Zitnik, M., Ceci, M., and Dzeroski, S., editors, Machine Learning and Knowledge Discovery in Databases, pages179–190, Cham. Springer International Publishing. ISBN978-3-319-71273-4. 10.1007/978‑3‑319‑71273‑4_15
    https://doi.org/10.1007/978-3-319-71273-4_15 [Google Scholar]
  5. Chowdhury, S. A., Stepanov, E. A., and Riccardi, G.
    (2016) Predicting user satisfaction from turn-taking in spoken conversations. InINTERSPEECH. 10.21437/Interspeech.2016‑859
    https://doi.org/10.21437/Interspeech.2016-859 [Google Scholar]
  6. Deschamps-Berger, T., Lamel, L., and Devillers, L.
    (Sept. 2021) End-to-End Speech Emotion Recognition: Challenges of Real-Life Emergency Call Centers Data Recordings. In2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII), Nara, Japan. 10.1109/ACII52823.2021.9597419
    https://doi.org/10.1109/ACII52823.2021.9597419 [Google Scholar]
  7. Erden, M. and Arslan, L. M.
    (2011) Automatic detection of anger in human-human call center dialogs. InProc. Interspeech 2011, pages81–84. 10.21437/Interspeech.2011‑21
    https://doi.org/10.21437/Interspeech.2011-21 [Google Scholar]
  8. Eyben, F., Weninger, F., Gross, F., and Schuller, B.
    (2013) Recent developments in opensmile, the munich open-source multimedia feature extractor. InProceedings of the 21st ACM International Conference on Multimedia, MM ’131, page835–838, New York, NY, USA. Association for Computing Machinery. ISBN9781450324045. 10.1145/2502081.2502224
    https://doi.org/10.1145/2502081.2502224 [Google Scholar]
  9. Galanis, D., Karabetsos, S., Koutsombogera, M., Papageorgiou, H., Esposito, A., and Riviello, M.-T.
    (2013) Classification of emotional speech units in call centre interactions. In2013 IEEE 4th International Conference on Cognitive Infocommunications (CogInfoCom), pages403–406. 10.1109/CogInfoCom.2013.6719279
    https://doi.org/10.1109/CogInfoCom.2013.6719279 [Google Scholar]
  10. Graves, A., Fernandez, S., Gomez, F., and Schmidhuber, J.
    (2006) Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. InProceedings of the 23rd International Conference on Machine Learning, ICML’061, page369–376, New York, NY, USA. Association for Computing Machinery. ISBN1595933832. 10.1145/1143844.1143891
    https://doi.org/10.1145/1143844.1143891 [Google Scholar]
  11. Kim, Y., Levy, J., and Liu, Y.
    (2020) Speech sentiment and customer satisfaction estimation in socialbot conversations. InINTERSPEECH. 10.21437/Interspeech.2020‑2890
    https://doi.org/10.21437/Interspeech.2020-2890 [Google Scholar]
  12. Luque, J., Segura, C., Sánchez, A., Umbert, M., and Galindo, L. A.
    (2017) The Role of Linguistic and Prosodic Cues on the Prediction of Self-Reported Satisfaction in Contact Centre Phone Calls. InProc. Interspeech 2017, pages2346–2350. 10.21437/Interspeech.2017‑424
    https://doi.org/10.21437/Interspeech.2017-424 [Google Scholar]
  13. Miao, Y., Gowayyed, M., and Metze, F.
    (2015) Eesen: End-to-end speech recognition using deep rnn models and wfst-based decoding. In2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), pages167–174. 10.1109/ASRU.2015.7404790
    https://doi.org/10.1109/ASRU.2015.7404790 [Google Scholar]
  14. Morrison, D., Wang, R., and De Silva, L. C.
    (2007) Ensemble methods for spoken emotion recognition in call-centres. Speech Communication, 49(2):98–112. ISSN0167-6393. 10.1016/j.specom.2006.11.004
    https://doi.org/10.1016/j.specom.2006.11.004 [Google Scholar]
  15. Mower, E., Matarić, M. J., and Narayanan, S.
    (2011) A framework for automatic human emotion classification using emotion profiles. IEEE Transactions on Audio, Speech, and Language Processing, 19(5):1057–1070. 10.1109/TASL.2010.2076804
    https://doi.org/10.1109/TASL.2010.2076804 [Google Scholar]
  16. Petrushin, V. A.
    (1999) Emotion in speech: Recognition and application to call centers.
    [Google Scholar]
  17. Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., Hannemann, M., Motlicek, P., Qian, Y., Schwarz, P., Silovsky, J., Stemmer, G., and Vesely, K.
    The kaldi speech recognition toolkit. InIEEE 2011 Workshop on Automatic Speech Recognition and Understanding. IEEE Signal Processing Society, Dec. 2011. IEEE Catalog No.: CFP11SRW-USB.
    [Google Scholar]
  18. Russell, J. A.
    (1980) A circumplex model of affect. Journal of personality and social psychology, 39(6):1161. 10.1037/h0077714
    https://doi.org/10.1037/h0077714 [Google Scholar]
  19. Schuller, B., Steidl, S., Batliner, A., Burkhardt, F., Devillers, L., Müller, C. A., and Narayanan, S. S.
    (2010a) The interspeech 2010 paralinguistic challenge. InINTERSPEECH. 10.21437/Interspeech.2010‑739
    https://doi.org/10.21437/Interspeech.2010-739 [Google Scholar]
  20. Schuller, B., Weninger, F., Zhang, Y., Ringeval, F., Batliner, A., Steidl, S., Eyben, F., Marchi, E., Vinciarelli, A., Scherer, K., Chetouani, M., and Mortillaro, M.
    (2019) Affective and behavioural computing: Lessons learnt from the first computational paralinguistics challenge. Computer Speech & Language, 531:156–180. ISSN0885-2308. 10.1016/j.csl.2018.02.004
    https://doi.org/10.1016/j.csl.2018.02.004 [Google Scholar]
  21. Schuller, B. W., Steidl, S., Batliner, A., Burkhardt, F., Devillers, L., Müller, C. A., and Narayanan, S. S.
    (2010b) The INTERSPEECH 2010 paralinguistic challenge. InINTERSPEECH 2010, 11th Annual Conference of the International Speech Communication Association, Makuhari, Chiba, Japan, September 26–30, 2010, pages2794–2797. 10.21437/Interspeech.2010‑739
    https://doi.org/10.21437/Interspeech.2010-739 [Google Scholar]
  22. Schuller, B. W., Steidl, S., Batliner, A., Vinciarelli, A., Scherer, K. R., Ringeval, F., Chetouani, M., Weninger, F., Eyben, F., Marchi, E., Mortillaro, M., Salamin, H., Polychroniou, A., Valente, F., and Kim, S.
    (2013) The INTERSPEECH 2013 computational paralinguistics challenge: social signals, conflict, emotion, autism. InINTERSPEECH 2013, 14th Annual Conference of the International Speech Communication Association, Lyon, France, August 25–29, 2013, pages148–152. 10.21437/Interspeech.2013‑56
    https://doi.org/10.21437/Interspeech.2013-56 [Google Scholar]
  23. Segura, C., Balcells, D., Umbert, M., Arias, J., and Luque, J.
    (2016) Automatic speech feature learning for continuous prediction of customer satisfaction in contact center phone calls. InAbad, A., Ortega, A., Teixeira, A., Mateo, C. Garcia, Hinarejos, C. D. Martínez, Perdigão, F., Batista, F., and Mamede, N., editors, Advances in Speech and Language Technologies for Iberian Languages, pages255–265, Cham. Springer International Publishing. ISBN978-3-319-49169-1. 10.1007/978‑3‑319‑49169‑1_25
    https://doi.org/10.1007/978-3-319-49169-1_25 [Google Scholar]
  24. Vaudable, C. and Devillers, L.
    (2012) Negative emotions detection as an indicator of dialogs quality in call centers. In2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages5109–5112. 10.1109/ICASSP.2012.6289070
    https://doi.org/10.1109/ICASSP.2012.6289070 [Google Scholar]
  25. Viikki, O. and Laurila, K.
    (1998) Cepstral domain segmental feature vector normalization for noise robust speech recognition. Speech Communication, 25(1):133–147. ISSN0167-6393. 10.1016/S0167‑6393(98)00033‑8
    https://doi.org/10.1016/S0167-6393(98)00033-8 [Google Scholar]
  26. Zweig, G., Siohan, O., Saon, G., Ramabhadran, B., Povey, D., Mangu, L., and Kingsbury, B.
    (2006) Automated quality monitoring for call centers using speech and nlp technologies. InProceedings of the 2006 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: Companion Volume: Demonstrations, NAACL-Demonstrations’061, page292–295, USA. Association for Computational Linguistics. 10.3115/1225785.1225796
    https://doi.org/10.3115/1225785.1225796 [Google Scholar]
/content/journals/10.1075/is.22038.bol
Loading
/content/journals/10.1075/is.22038.bol
Loading

Data & Media loading...

  • Article Type: Research Article
Keyword(s): affective computing; customer satisfaction; emotions; real-world applications
This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error