Commit 7703b71c authored by  Joel  Oksanen's avatar Joel Oksanen
Browse files

Finished background draft

parent cbca8c9d
\chapter{Background}
We begin this chapter by detailing the methodology of the ADA proposed by Cocarascu et al. \cite{RefWorks:doc:5e08939de4b0912a82c3d46c}. We then evaluate the limitations of ADA in relation to Amazon reviews, and suggest extensions to address them. We then move onto considering current research in the fields of aspect-level sentiment analysis and conversational systems in order to establish a basis for our enhancements to the agent.
We begin this chapter by detailing the methodology of the ADA proposed by Cocarascu et al.\ \cite{RefWorks:doc:5e08939de4b0912a82c3d46c}. We then evaluate the limitations of ADA in relation to Amazon reviews, and suggest extensions to address them. We then move onto considering current research in the fields of feature-level sentiment analysis and conversational systems in order to establish a basis for our enhancements to the agent.
\section{Argumentative Dialogical Agent}
......@@ -9,7 +9,7 @@ ADA is designed around a \textit{feature}-based conceptualisation of products. R
\begin{figure}[b]
\centering
\includegraphics[width=14cm]{images/ADA}
\caption{ADA as proposed by Cocarascu et al. for the Rotten Tomatoes movie review domain}
\caption{ADA as proposed by Cocarascu et al.\ for the Rotten Tomatoes movie review domain}
\label{fig:ADApipeline}
\end{figure}
......@@ -293,6 +293,7 @@ ADA can generate dialogical explanations for arguments based on the extracted QB
\mathcal{X}(\alpha) &= \{[\textrm{phrase from $u \in \mathcal{U}$ constituting $\mathcal{V}(u,a)=-$}]\}.
\end{flalign*}
\end{itemize}
\label{def:argdialogue}
\end{definition}
When a user requests an explanation for the sentiment towards an argument $\alpha$, ADA responds by providing another argument $\beta$ that supports $\alpha$, and possibly also contrasts it with a third argument $\gamma$ that attacks $\alpha$. A user can also request a direct quote from one of the reviewers on an argument. For example, a conversation between a user and ADA on the \textit{Canon IXUS 185} digital camera\footnote{https://www.amazon.co.uk/Canon-IXUS-185-Digital-Camera/dp/B01N6JP07Y} could take the following form:
......@@ -375,6 +376,11 @@ Based on these two differences, we propose two extensions to ADA in order to acc
\section{Feature-level sentiment analysis}
\begin{center}
\fbox{\parbox{13cm}{\textbf{Note:} This section will be extended with details about the particular NLP methods I will use once I've went through neural networks in my machine learning course.}}
\end{center}
\noindent
In this section, we will examine state of the art research in \textit{aspect-level sentiment analysis} \cite{RefWorks:doc:5e2b0d8de4b0711bafe4fba8}, which attempts to determine people's opinions on \textit{entities} and their \textit{aspects}. We have already been introduced to aspect-level sentiment analysis in ADA's review aggregation, where the entities are \textit{products} and the aspects are their \textit{features}. For consistency, we will refer to entities as products and aspects as features. Further research into this area will guide our implementation of the extensions proposed in section 2.1.
Consider the following review for a particular model of the \textit{Philips Sonicare} electric toothbrush range:
......@@ -391,7 +397,7 @@ Since ADA would associate \textit{Sonicare toothbrush} with \textit{terrible}, i
\item will calculate only a single sentiment for the entire review, while it actually represents a negative sentiment towards other Sonicare toothbrushes and a positive sentiment towards this particular toothbrush.
\end{enumerate}
The first error has to do with \textit{feature extraction}, which accounts for the first part of feature-level sentiment analysis where the agent extracts features from the text. The second error has to do with \textit{feature sentiment analysis}, where the extracted features are assigned with sentiment polarities. The following sections evaluate research in these two areas in hopes of finding a way to resolve these errors.
The first error has to do with \textit{feature extraction}, which accounts for the first part of feature-level sentiment analysis where the agent extracts features from the text. The second error has to do with \textit{feature-dependent sentiment analysis}, where the extracted features are assigned with sentiment polarities. The following sections evaluate research in these two areas in hopes of finding a way to resolve these errors.
\subsection{Feature extraction}
......@@ -423,13 +429,13 @@ Semantic information can be obtained from \textit{ConceptNet} \cite{RefWorks:doc
%Mined (deep learning) \newline
%Representation tiers
\subsection{Feature sentiment analysis}
\subsection{Feature-dependent sentiment analysis}
After we have extracted opinion targets (arguments) from a review, we wish to discern whether the opinions towards the arguments are positive or negative through sentiment analysis. Perhaps the main difficulty in feature-dependent sentiment analysis is to distinguish which opinions are acting on which arguments.
ADA attempts to tackle this issue by diving the review into phrases at specific keywords, such as the word \textit{but} in \textit{I liked the acting, but the cinematography was dreadful}, after which it assumes each phrase contains at most one sentiment. However, there are many cases where such a simple method will not work, like the example at the start of this section. This is particularly true for Amazon reviews where the text tends to be less formal compared to Rotten Tomatoes reviews.
More advanced methods using deep learning have been proposed in literature, although the task is deemed difficult and there is currently no dominating technique for this purpose \cite{RefWorks:doc:5e2b0d8de4b0711bafe4fba8}. Dong et al. \cite{RefWorks:doc:5e2e107ce4b0bc4691206e2e} proposed an \textit{adaptive recursive neural network} (AdaRNN) for target-dependent Twitter sentiment classification, which propagates the sentiments of words to the target by exploiting the context and the syntactic relationships between them. The result were promising, and the domain of Twitter is similar to Amazon reviews in terms of formality. The results were compared with a re-implementation of \textit{SVM-dep} proposed by Jiang et al. \cite{RefWorks:doc:5e2e1e23e4b0e67b35d1c360}, which uses target-dependent syntactic features in a SVM classifier instead of a neural network. As SVM-dep performed nearly as well as AdaRNN, either could be used to improve ADAs sentiment analysis accuracy.
More advanced methods using deep learning have been proposed in literature, although the task is deemed difficult and there is currently no dominating technique for this purpose \cite{RefWorks:doc:5e2b0d8de4b0711bafe4fba8}. Dong et al.\ \cite{RefWorks:doc:5e2e107ce4b0bc4691206e2e} proposed an \textit{adaptive recursive neural network} (AdaRNN) for target-dependent Twitter sentiment classification, which propagates the sentiments of words to the target by exploiting the context and the syntactic relationships between them. The result were promising, and the domain of Twitter is similar to Amazon reviews in terms of formality. The results were compared with a re-implementation of \textit{SVM-dep} proposed by Jiang et al.\ \cite{RefWorks:doc:5e2e1e23e4b0e67b35d1c360}, which uses target-dependent syntactic features in a SVM classifier instead of a neural network. Since SVM-dep performed nearly as well as AdaRNN, either could be used to improve ADAs sentiment analysis accuracy.
%Sentiment analysis vs. argument modelling \newline
......@@ -439,7 +445,46 @@ More advanced methods using deep learning have been proposed in literature, alth
\section{Conversational systems}
Speech to query \newline
QBAF to spoken explanations \newline
Extending conversational system proposed in \cite{RefWorks:doc:5e0de20ee4b055d63d355913} with RRIMS properties \cite{RefWorks:doc:5e31af55e4b017f1b5fb8684} \newline
QBAF may need to be modified
\ No newline at end of file
The main advantage of ADA is its ability to provide dialogical explanations for review aggregations, which help the user understand why a product is liked or disliked. This is a beneficial feature for example in the recommender system domain, where it has been shown to improve the general acceptance, perceived quality, and effectiveness of the system \cite{RefWorks:doc:5e2f3970e4b0241a7d69e2a4}.
Although an argumentation dialogue has been defined for ADA, a conversational user interface through which a user can participate in this dialogue remains to be implemented. In this section, we will investigate two novel approaches to designing such an interface: \textit{Botplications} proposed by Klopfenstein et al.\ \cite{RefWorks:doc:5e395ff6e4b02ac586d9a2c8} for textual interfaces and \textit{Quantised Dialog} proposed by Gunasekara et al.\ \cite{RefWorks:doc:5e39660de4b07263888fb37d} for speech interfaces. We will conclude by evaluating the two methods using the RRIMS properties proposed by Radlinski et al.\ \cite{RefWorks:doc:5e31af55e4b017f1b5fb8684} for conversational search systems.
\subsection{Botplications}
Klopfenstein et al.\ define a \textit{Botplication} as 'an agent that is endowed with a conversational interface accessible through a messaging platform, which provides access to data, services, or enables the user to perform a specific task'. To extend ADA with such functionality, we would implement a messaging interface through which the user can request review explanations in line with the argumentation dialogue in Definition \ref{def:argdialogue}. Instead of demanding the user to type out a request, the interface might implement structured message forms, such as preset replies or an interactive list of available commands. The advantage of structured messages is that they constrain the conversation into a limited number of expected outcomes and assist the user in using the interface.
The alternative to structured messages would be to use NLP to extract commands and intent from the user’s messages. However, Klopfenstein et al.\ argue that for a single bot, natural language should be avoided where possible, as 'going after AI is mostly excessive and counterproductive, when the same results can be obtained with simple text commands using a limited structured language'.
% + memory
\subsection{Quantised Dialog}
In a speech interface, structured messages are no longer possible. Although advances in deep learning have recently enabled speech recognition to reach human parity\footnote{https://www.microsoft.com/en-us/research/blog/microsoft-researchers-achieve-new-conversational-speech-recognition-milestone/}, understanding the intent behind the extracted natural language continues to be a challenging task and an active area of research. Many approaches to this problem in conversational systems are based on heuristics and handcrafted rules, which may not adapt well to unexpected input.
Gunasekara et al.\ recently proposed the more general method of \textit{Quantised Dialog} as a way to tackle this issue. It relies on the semantic quantisation and clustering of the user's requests in order to reduce the otherwise limitless interaction space of natural language dialog. It is then possible to teach a neural network in this simplified dialog space to predict the appropriate response to a query.
Although implementing Quantised Dialog for the relatively simple domain of ADA would be excessive, we could combine some of its features with our existing semantic analysis methods developed for ADA's review aggregations. The user will ask semantically loaded questions about the product and its features, which should be covered by the same feature extraction and feature-dependent sentiment analysis methods as the review texts. If we can extract the semantics behind the queries and group them with one of the explanation requests of Definition \ref{def:argdialogue}, we can answer them with the predefined responses. However, this assumes that the user has sufficient information about the kind of responses the agent can produce.
\subsection{Evaluation using the RRIMS properties}
Radlinski et al.\ propose that a conversational search system should satisfy the following five properties, termed the \textit{RRIMS properties}:
\begin{itemize}
\item \textbf{User Revealment} The system helps the user express, and possibly discover, their true information need.
\item \textbf{System Revealment} The system reveals to the user its capabilities and commands, building the user’s expectations of what it can and cannot do.
\item \textbf{Mixed Initiative} Both the system and the user can take initiative as appropriate.
\item \textbf{Memory} The user can reference past statements, which implicitly also remain true unless contradicted.
\item \textbf{Set Retrieval} The system can reason about the utility of sets of complementary items.
\end{itemize}
%In this project we will not concern ourselves with the \textit{memory} property, as the task of ADA is relatively simple and does not require extensive conversations in which the ability to reference past statements would be beneficial.
ADA's argumentation dialogue fairs well in terms of \textit{user revealment}, as it allows the user to request additional information about the product and its features. ADA also provides additional information to contrast with the requested information via its secondary \textit{although\dots} phrases, allowing the user to discover more about the product. For the user interface methods, Botplications perform better than Quantised Dialog in terms of the \textit{system revealment} and \textit{memory} properties. In terms of \textit{system revealment}, the structured messages of a Botplication provide more information about the system's capabilities. In terms of \textit{memory}, the chronological log of past interactions kept by Botplications' messaging interface again seems more useful. We will not concern ourselves with the \textit{mixed initiative} and \textit{set retrieval} properties, as we will not be implementing ADA as a full-fledged conversational search system in this project.
As a Botplication evaluates better Quantised Dialog in terms of fulfilling the RRIMS properties, the only reason we should implement the latter is if we specifically want a speech interface. Such an interface could be useful in for example a smart speaker, which could then provide the user with explainable product recommendations in situations where they are unable to interact with a physical device, such as while cooking.
%Speech to query \newline
%QBAF to spoken explanations \newline
%Extending conversational system proposed in \cite{RefWorks:doc:5e0de20ee4b055d63d355913} with RRIMS properties \cite{RefWorks:doc:5e31af55e4b017f1b5fb8684} \newline
%QBAF may need to be modified
\ No newline at end of file
......@@ -56,6 +56,27 @@
url={https://doi.org/10.1145/2507157.2507163},
doi={10.1145/2507157.2507163}
}
@article{RefWorks:doc:5e39660de4b07263888fb37d,
author={R. Chulaka Gunasekara and David Nahamoo and Lazaros C. Polymenakos and Ciaurri, David Echeverr ' ia and Jatin Ganhotra and Kshitij P. Fadnis},
year={2019},
title={Quantized Dialog - A general approach for conversational systems},
journal={Computer Speech {\&} Language},
volume={54},
pages={17-30},
url={https://doi.org/10.1016/j.csl.2018.06.003},
doi={10.1016/j.csl.2018.06.003}
}
@inproceedings{RefWorks:doc:5e395ff6e4b02ac586d9a2c8,
author={Lorenz Cuno Klopfenstein and Saverio Delpriori and Silvia Malatini and Alessandro Bogliolo},
year={2017},
title={The Rise of Bots: A Survey of Conversational Interfaces, Patterns, and Paradigms},
booktitle={Proceedings of the 2017 Conference on Designing Interactive Systems, {DIS} '17, Edinburgh, United Kingdom, June 10-14, 2017},
pages={555-565},
note={DBLP:conf/ACMdis/2017},
url={https://doi.org/10.1145/3064663.3064672},
doi={10.1145/3064663.3064672}
}
@article{RefWorks:doc:5e2f3970e4b0241a7d69e2a4,
author={Fatih Gedikli and Dietmar Jannach and Mouzhi Ge},
year={2014},
......
......@@ -9,7 +9,7 @@ Clear explanations of review aggregations are also central to the success of e-c
\section{Objectives}
There have already been some attempts to improve explanations for review aggregations, some of which are discussed in Chapter 2. One such attempt is what is called an Argumentative Dialogical Agent (ADA), proposed by Cocarascu et al. \cite{RefWorks:doc:5e08939de4b0912a82c3d46c} and implemented for the Rotten Tomatoes and Trip Advisor\footnote{https://www.tripadvisor.com/} platforms \cite{RefWorks:doc:5e0de20ee4b055d63d355913}. The goal of this project is to extend upon the work of Cocarascu et al. in order to design and implement a more generalised ADA to provide explanations for Amazon product reviews. The main objectives for the extended agent are threefold:
There have already been some attempts to improve explanations for review aggregations, some of which are discussed in Chapter 2. One such attempt is what is called an Argumentative Dialogical Agent (ADA), proposed by Cocarascu et al.\ \cite{RefWorks:doc:5e08939de4b0912a82c3d46c} and implemented for the Rotten Tomatoes and Trip Advisor\footnote{https://www.tripadvisor.com/} platforms \cite{RefWorks:doc:5e0de20ee4b055d63d355913}. The goal of this project is to extend upon the work of Cocarascu et al. in order to design and implement a more generalised ADA to provide explanations for Amazon product reviews. The main objectives for the extended agent are threefold:
\begin{itemize}
\item \textbf{Generalise} the agent to work with a larger variety of different products. Currently ADA has only been implemented for movie and hotel reviews, two highly homogeneous domains in which there is not much variance in key features and review language from one product to another. Implementing ADA for Amazon reviews will require more general NLP methods in extracting review aggregations.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment