Chatbot Solutions

The increasing availability of data, advances made in Artificial Intelligence, and in particular, natural language processing techniques, together with the widespread use of mobile apps and web applications, create new opportunities to deliver better and more efficient digital public services. At the same time, societies face important global challenges related to sustainability and climate change and governments face pressure from citizens to provide innovative and effective solutions that can help alleviate these problems.

Conversational agents are software applications that communicate with a human using either spoken or written natural language [1]. Chatbots are a type of conversational agents used in a wide range of domains [2]. Due to the demand for better customer service, the increased use of messaging applications, advances in natural language processing techniques, and the availability of platforms to support their implementation, chatbots have become popular in the public service domain [3] [4]. In this context, chatbots can reduce the administrative burden and allow citizens to communicate with the government in their everyday natural language [5], saving manpower expenses by delivering service 24/7 [3]. However, in this domain, there is a significant focus on information provision towards citizens, rather than facilitating chatbot-transaction-making [6], which, while providing valuable information to citizens, does not simplify traditional service procedures. This paper presents our experience in running a research project supporting the development of suitable chatbot technologies for sustainable public service use-cases. The “Emerging Technologies for Greener Communities” or EMERGREEN project is an interregional project funded under the Northern Periphery and Arctic Programme, bringing together public service partners from Ireland, Sweden, Finland, United Kingdom and Faroe Islands to address the common challenge of delivering quality sustainable public services aimed at generating greener communities in remote areas. EMERGREEN is focused on the exploration of open, user-friendly channels of communication that would be readily adopted by end-users, adaptable and sustainable. Specifically, two of the services developed apply chatbot technologies to addressing Waste Management and Green Energy Advisory Services. Interestingly, these two chatbot use cases meet two separate and opposing goals for the public service partners involved – one to reduce the need for human interaction and the other to encourage it. The RIA (Recycling Information Assistant) chatbot was created with Derry City and Strabane County Council in Northern Ireland in order to answer citizens’ recycling and other waste-related questions in real-time, with the goal of improving connectivity with users, providing access to information outside of office hours and reducing calls to the council on these topics. The SolcEllen chatbot was created with the Regional Council of Västernorrland in Sweden to provide consistent information about solar energy and solar panels to the inhabitants of this sparsely populated region. The chatbot not only answers questions, based on updated and reliable data but also encourages users to contact local energy advisors with more specific enquiries.

The development approach adopted in the project reflects the dynamics of developing emerging technologies for public services. Considering the strong emphasis on the community-driven initiative in this project, we followed user-centred design techniques to ensure that any technology developed in the project had citizens’ needs set as the top priority. That design approach was supported by the agile development process, yielding 10 iterations of technology releases in close collaboration with our stakeholders. Every cycle was initiated with requirements analysis and finalised with relevant evaluation by our users in the form of dedicated focus groups. The feedback loops were supported on the technical side by leveraging an established GitLab solution for managing versions and tracking issues. For our non-functional requirements analysis, we adopted the framework by Mallios et al. [1] who list 14 features to evaluate different dialogue systems initiatives methodologies. We extended the framework with four extra features: 1) Integration, 2) Open-source-based, 3) Modifiability and 4) Behavioural Change Promotion.

Figure 1: Chatbots Agile Development Pipeline

We conducted a state-of-the-art evaluation of the available Open-Source frameworks for chatbot development and selected Rasa1 as the main conversational AI framework. To initiate the data collection efforts, a series of chatbot demonstration sessions were performed, aimed at showcasing the potential of the technology for the EMERGREEN partners, emphasizing not only the chatbots’ capacity to provide information but also their ability to perform transactions. After the presentations and discussions around the benefits of this technology and its application in different domains, the team initiated the functional requirements elicitation process in parallel with data collection efforts. Although available AI-powered chatbot frameworks provide the means to exploit pre-trained language models for text classification and named-entity recognition (NER), data collection is still required to finetune these models for specific domain usage. Two different instruments were developed: 1) “intents_template”, this was used for AI-training and implemented as a CSV file and consists of a tabular document in which the public service partners list the users’ goals or users’ intents the chatbot solution should address and text examples that will trigger them. 2) “chatbot_behaviour_template” a document aimed at exploring the chatbot functional requirements in-depth, while at the same time getting an overview of the organisation’s maturity in terms of software infrastructure and API services availability. Following the data gathered with “chatbot_behaviour_template” instrument, conversation stories between the chatbot and the users were written and used as training data. Data augmentation techniques were utilised to artificially augment training datasets using programming languages and tools such as Chatito. This methodology allowed for creating AI training samples by including predefined structures and sets of entity examples and significantly increased the capacity of the chatbot to generalise well when presented with previously “unseen” queries. As the functional requirements indicated a need for identifying street names, addresses, postcodes, recycling items, bin colours and synonyms, relevant lists of entities were gathered from different sources, such as OpenStreetMaps, CSV files, databases, or created from scratch using human annotators and regular expressions.

RASA Open Source Framework

Each iteration in the development process followed the pipeline described in Figure 1.
After the chatbot model training phase, end-to-end testing was performed with different user groups, who interacted with the chatbots through a dedicated user interface, testing not only its overall friendliness or ease of use, but also providing suggestions in terms of the information provided, design, usability, and suggested areas for improvement. All the conversations were stored for further analysis, and users referred to their conversation IDs when providing feedback. This conversation monitoring provided a key instrument to analyse technical details, point out chatbot weaknesses, and to explore and inspire new user stories previously not considered. As the iterative process evolved, the need for a dynamic knowledge base system that would allow a quick and easy methodology to update chatbots’ answers in different scenarios became imperative. This need motivated the creation of a REST API that would provide specific data upon request, and that would allow for information modification without impacting on the chatbots’ deployment.

Figure 2: Technical Architecture Overview – It is not our intention to provide detail of the technical architecture in this paper, however, we would like to present some major components of our solution. Our chatbot technology has been stratified to reflect the three-layered, micro-service oriented architectural pattern. Therefore, our highly modular solution can be deployed on any major website or web portal independently on the specific web technology or framework applied. Each of the components responsible for the logic of the solution and the dynamic knowledge base layer can be deployed separately.