Skip to content

GitLab

Commit cc03c3a2 authored by Joel Oksanen's avatar Joel Oksanen
Browse files

Added eval trees

parent ca6bd0ee
Hide whitespace changes
Inline Side-by-side
Showing with 304 additions and 9 deletions
individual_project_report/appendix/appendix.tex
View file @ cc03c3a2
\appendix
\chapter{Extracted ontologies}
\label{sec:ontology_appendix}
\ No newline at end of file
\label{sec:ontology_appendix}
\begin{sidewaysfigure}[h]
\centering
\begin{subfigure}{\textheight}
\centering
\footnotesize
\begin{forest}
for tree = {draw, l=2.5cm}
[{\textbf{watch}, product, item, piece, model}
[band
[links]
[leather]
[clasp]
]
[{\textbf{face}, dial}
[size]
[color]
[numbers]
[hands]
]
[price]
[{\textbf{quality}, value}]
[{\textbf{design}, style}]
[battery]
[look]
[box]
[weight]
[seller]
[brand]
[display
[case]
[bezel]
]
[value]
[crown
[date]
]
[movement
[second hand]
]
]
\end{forest}
\caption{Extracted ontology}
\label{fig:full_watch_ontology}
\end{subfigure}
\vspace{1cm}
\begin{subfigure}{\textheight}
\centering
\begin{forest}
for tree = {draw, l=1.5cm}
[{\textbf{watch}, ticker, analog watch, digital watch, wristwatch, wrist watch, \dots}
[{\textbf{crystal}, watch crystal, watch glass}]
[face]
[movement]
[watch case]
[{\textbf{digital display}, alphanumeric display}
[{\textbf{light-emitting diode}, led}]
]
]
\end{forest}
\caption{WordNet ontology}
\label{fig:wordnet_watch_ontology}
\end{subfigure}
\caption{Full ontologies for \textit{watch}}
\end{sidewaysfigure}
\begin{sidewaysfigure}[h]
\centering
\begin{subfigure}{\textheight}
\centering
\begin{forest}
for tree = {draw, l=2.5cm}
[{\textbf{television}, tv, unit, product, model}
[{\textbf{picture}, picture quality, image}
[settings]
]
[price]
[screen
[size]
]
[remote
[volume]
[channels]
]
[box]
[speakers
[{\textbf{sound}, audio}]
]
[stand]
[service]
[apps]
[warranty]
[brand]
[inputs
[hdmi]
]
[display
[quality]
]
[lcd
[{\textbf{color}, colors}]
]
]
\end{forest}
\caption{Extracted ontology}
\label{fig:full_tv_ontology}
\end{subfigure}
\vspace{1cm}
\begin{subfigure}{\textheight}
\centering
\begin{forest}
for tree = {draw, l=1.5cm}
[{\textbf{television}, television system, television receiver, tv, tv set, \dots}
[audio]
[{\textbf{cathode-ray tube}, crt}
[electron gun]
[{\textbf{screen}, CRT screen}]
]
[monitor]
[{\textbf{television equipment}, video equipment}]
[television transmitter]
]
\end{forest}
\caption{WordNet ontology}
\label{fig:wordnet_tv_ontology}
\end{subfigure}
\caption{Full ontologies for \textit{television}}
\end{sidewaysfigure}
\begin{sidewaysfigure}[h]
\centering
\begin{subfigure}{\textheight}
\centering
\begin{forest}
for tree = {draw, l=2.5cm}
[{\textbf{necklace}, product, locket, cross, charm, piece, item, pendant, chain}
[{\textbf{size}, length}]
[quality
[price]
]
[clasp]
[{\textbf{design}, style}]
[weight]
[heart]
[{\textbf{stone}, stones}
[{\textbf{color}, colors}]
]
[packaging
[box]
]
[metal
[silver]
[gold]
[detail]
]
[set
[look]
[{\textbf{seller}, company}]
[shipping]
[pearl]
]
]
\end{forest}
\caption{Extracted ontology}
\label{fig:full_necklace_ontology}
\end{subfigure}
\vspace{1cm}
\begin{subfigure}{\textheight}
\centering
\begin{forest}
for tree = {draw, l=1.5cm}
[{\textbf{necklace}, chain, string, strand, choker, collar, dog collar, neckband},
[{\textbf{pendant}, pendent, lavaliere, lavalier, lavalliere}]
]
\end{forest}
\caption{WordNet ontology}
\label{fig:wordnet_necklace_ontology}
\end{subfigure}
\caption{Full ontologies for \textit{necklace}}
\end{sidewaysfigure}
\begin{sidewaysfigure}[h]
\centering
\begin{subfigure}{\textheight}
\centering
\begin{forest}
for tree = {draw, l=2cm}
[{\textbf{stand mixer}, mixer, machine, product, item, unit, model, blender}
[{\textbf{bowl}, bowls}
[{\textbf{beaters}, beater}]
[sides]
[size]
]
[price]
[color]
[motor
[speed]
[power]
]
[quality]
[box]
[design]
[base
[plastic]
[head]
]
[attachment
[dough hook]
]
]
\end{forest}
\caption{Extracted ontology}
\label{fig:full_mixer_ontology}
\end{subfigure}
\vspace{1cm}
\begin{subfigure}{\textheight}
\centering
\begin{forest}
for tree = {draw, l=1.5cm}
[{\textbf{mixer}, blender, liquidiser, eggbeater, eggwhisk, electric mixer, whisk}
[electric motor
[armature]
[brush]
[field magnet]
[{\textbf{rotor}, rotor coil}]
[{\textbf{stator}, stator coil}]
]
]
\end{forest}
\caption{WordNet ontology*}
\label{fig:wordnet_mixer_ontology}
\end{subfigure}
\caption{Full ontologies for \textit{stand mixer} (*the term \textit{stand mixer} returned no results in WordNet, so \textit{mixer} was used instead)}
\end{sidewaysfigure}
\begin{sidewaysfigure}[h]
\centering
\begin{subfigure}{\textheight}
\centering
\begin{forest}
for tree = {draw, l=2cm}
[{\textbf{video game}, game, product, title}
[graphics]
[gameplay]
[controls]
[price]
[storyline]
[sound
[music]
]
]
\end{forest}
\caption{Extracted ontology}
\label{fig:full_game_ontology}
\end{subfigure}
\vspace{1cm}
\begin{subfigure}{\textheight}
\centering
\begin{forest}
for tree = {draw, l=2cm}
[{\textbf{video game}, computer game, virtual reality}]
\end{forest}
\caption{WordNet ontology}
\label{fig:wordnet_game_ontology}
\end{subfigure}
\caption{Full ontologies for \textit{video game}}
\end{sidewaysfigure}
\ No newline at end of file
individual_project_report/background/background.tex
View file @ cc03c3a2
......@@ -407,7 +407,7 @@ Some cross-domain feature extraction methods have been proposed \cite{RefWorks:
\subsubsection{ConceptNet and WordNet}
Semantic information can be obtained from \textit{ConceptNet} \cite{RefWorks:doc:5e382bf3e4b0034ec2324aed}, which is a graph connecting words and phrases with labelled and weighted edges expressing semantic relations between the words. As our goal is to obtain features of a product, the relations we are most interested in are:
Semantic information can be obtained from \textit{ConceptNet} \cite{RefWorks:doc:5e382bf3e4b0034ec2324aed}, which is a common-sense knowledge graph connecting words and phrases with labelled and weighted edges expressing semantic relations between the words. As our goal is to obtain features of a product, the relations we are most interested in are:
\begin{itemize}
\item \textit{CapableOf} for capabilities of products;
\item \textit{HasA} for parts of products;
......
individual_project_report/feature_extraction/feature_extraction.tex
View file @ cc03c3a2
......@@ -279,13 +279,13 @@ def get_tree(R, synsets):
\section{Evaluation}
We evaluate our ontology extraction method using human annotators both independently and against ontologies extracted using ConceptNet and WordNet. We evaluate the ontologies extracted for a variety of five randomly selected products which were not included in the training data for the classifier: \textit{watches}, \textit{televisions}, . The full ontologies extracted for these products are included in Appendix \ref{sec:ontology_appendix}.
In this section, we evaluate our ontology extraction method using human annotators both independently and against ontologies extracted using ConceptNet and WordNet. Furthermore, we independently evaluate the generalisation of the masked BERT method by experimenting with the number of the product categories used for its training.
\subsection{Ontology evaluation}
Furthermore, we independently evaluate the generalisation of the masked BERT method by experimenting with the number of the product categories used for its training.
We evaluate five ontologies extracted for a variety of randomly selected products which were not included in the training data for the classifier: \textit{watches}, \textit{televisions}, \textit{necklaces}, \textit{stand mixers}, and \textit{video games}. For each product, we use 100,000 review texts as input to the ontology extractor, except for \textit{stand mixer}, for which we could only obtain 28,768 review texts. The full ontologies extracted for each of the products are included in Appendix \ref{sec:ontology_appendix}.
\subsection{Ontology evaluation}
We also extract ontologies for the five products from ConceptNet and WordNet for comparison. For ConceptNet, we observe
\subsection{Generalisation evaluation}
......
individual_project_report/introduction/introduction.tex
View file @ cc03c3a2
......@@ -61,8 +61,7 @@ For both 1.\ and 2.\ we use BERT, a language model proposed by Devlin et al.\ \c
]
[price]
[quality]
[size]
[look]
[design]
[battery]
[\dots]
]
......
individual_project_report/main.tex
View file @ cc03c3a2
......@@ -13,13 +13,17 @@
\usepackage{graphicx}
\usepackage[colorinlistoftodos]{todonotes}
\usepackage[colorlinks=true, allcolors=blue]{hyperref}
\usepackage[edges]{forest}
\usepackage{forest}
\usepackage{subcaption}
\usepackage{multirow}
\usepackage{listings}
\lstset{basicstyle=\ttfamily\footnotesize,breaklines=true}
\renewcommand{\figurename}{Listing}
\usepackage{float}
\usepackage{rotating}
\usepackage{amsthm}
%% \DeclareMathSymbol{\Alpha}{\mathalpha}{operators}{"41}
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment