COTA: Improving the Speed and Accuracy of Customer Support through Ranking and Deep Networks


    For a company looking to provide delightful user experiences, it is of paramount importance to take care of any customer issues. This paper proposes COTA, a system to improve speed and reliability of customer support for end users through automated ticket classification and answers selection for support representatives. Two machine learning and natural language processing techniques are demonstrated: one relying on feature engineering (COTA v1) and the other exploiting raw signals through deep learning architectures (COTA v2). COTA v1 employs a new approach that converts the multi-classification task into a ranking problem, demonstrating significantly better performance in the case of thousands of classes. For COTA v2, we propose an Encoder-Combiner-Decoder, a novel deep learning architecture that allows for heterogeneous input and output feature types and injection of prior knowledge through network architecture choices. This paper compares these models and their variants on the task of ticket classification and answer selection, showing model COTA v2 outperforms COTA v1, and analyzes their inner workings and shortcomings. Finally, an A/B test is conducted in a production setting validating the real-world impact of COTA in reducing issue resolution time by 10 percent without reducing customer satisfaction.


    Piero Molino, Huaixiu Zheng, Yi-Chia Wang


    KDD 2018

    Full Paper

    ‘COTA: Improving the Speed and Accuracy of Customer Support through Ranking and Deep Networks’ (PDF)

    Uber AI

    Previous articleVariational Bayesian dropout: pitfalls and fixes
    Next articleMultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving
    Piero Molino
    Piero Molino is a Senior Research Scientist at Uber AI. He works on natural language understanding and conversational AI. He is a co-founder of Uber AI.
    Huaixiu Zheng
    Huaixiu Zheng is a senior data scientist at Uber, working on projects in the domains of deep learning, reinforcement learning, natural language processing and conversational AI systems.
    Yi-Chia Wang
    Yi-Chia Wang is a Research Scientist at Uber AI, focusing on the conversational AI. She received her Ph.D. from the Language Technologies Institute in School of Computer Science at Carnegie Mellon University. Her research interests and skills are to combine language processing technologies, machine learning methodologies, and social science theories to statistically analyze large-scale data and model human-human / human-bot behaviors. She has published more than 20 peer-reviewed papers in top-tier conferences/journals and received awards, including the CHI Honorable Mention Paper Award, the CSCW Best Paper Award, and the AIED Best Student Paper Nomination.