oli-dissertation.tex

% \documentclass{report}
\documentclass[tocprelim,
    % 11pt,
    % normalsizeheadings,
    % cornellheadings
    ]{cornellmodified}
% \input{custom-cornell-preamble.tex}


\input{dissertation.preamble}
\input{pdg-preamble-v4}
\input{labelmatrix}

\setlength{\skip\footins}{4ex}

% problem with tocloft: makes list of figures & table of contents too big
% \usepackage{tocloft}
% \renewcommand\cftpartpresnum{Part~}

%%%%%%%%%%%%%%%% FLAGS %%%%%%%%%%%%%%%%%
\newif\iffoundations\foundationsfalse
\newif\ifqqtheme\qqthemefalse
\newif\ifjoewording\joewordingtrue

\newif\ifprecompiledfigs
	\precompiledfigsfalse
	% \precompiledfigstrue

\newif\ifexternalizefigures\externalizefiguresfalse

\ifexternalizefigures
	\usetikzlibrary{external}
	\tikzexternalize[prefix=tikz/]  %
\fi

%%%%%%%% DEFAULT PACKAGES FROM CORNELL THESIS TEMPLATE
% \usepackage{graphicx,pstricks}
% \usepackage{graphics}
% \usepackage{moreverb}
% \usepackage{subfigure}
% \usepackage{hangcaption}
%
% \usepackage{txfonts}
\usepackage{palatino}
% \usepackage{minitoc}

%if you're having problems with overfull boxes, you may need to increase
%the tolerance to 9999
\tolerance=9999



\newsavebox\dpineqbox

\renewcommand{\caption}[1]{\singlespacing\hangcaption{#1}\normalspacing}
\renewcommand{\topfraction}{0.85}
\renewcommand{\textfraction}{0.1}
\renewcommand{\floatpagefraction}{0.75}

\title {%
    % Probabilistic Dependency Theory
    % A Theory of Inconsistency
    % Probabilistic and Structural Inconsistency
    % A Unified Perspective on Probabilistic Consistency
    % Relative Entropy Soup
    % A Unified Theory of Probabilistic Modeling 
    % Learning, Inference, and the Pursuit of Consistency: A Unified Theory of Probabilistic Modeling
    % A Unified Theory of Probabilistic Modeling and Self-Consistency
    % A Unified Theory of (Possibly Inconsistent) Probabilistic Modeling
    % A Unified Theory of Probabilistic Modeling and Internal Conflict
    % A Unified Theory of Probabilistic Modeling and Epistemic Conflict
    % A Graphical Model of Belief, Dependence, and Inconsistency
    A Unified Theory of Probabilistic Modeling, Dependence, and Inconsistency
    %  I also want to point out that while I don't use the bigram "epistemic conflict" very often, I do use both the terms "epistemic" and "conflict / conflicted" a great deal (and often in close proximity) so it is not so far from the terminology used in the thesis. 
}
\author {Oliver E. Richardson}
\conferraldate {August}{2024}
\copyrightyear{2024}

\begin{document}
\maketitle
\makecopyright

\begin{abstract}
% Your abstract goes here. Make sure it sits inside the brackets. If not,
% your biosketch page may not be roman numeral iii, as required by the
% graduate school.
%
% TODO
% (was not trying to self-aggrandize above; the ``Ph.D.'' is part of the template!)

% It is often the case that 
% It is common to have to deal with conflicting information
What should you do with conflicting information?
To be \emph{rational}, you must immediately resolve the inconsistency,
    so as to maintain a consistent (probabilistic) picture of the world.
But how?
% And what if there is no time to process?
% And is it really critical always have fully consistent beliefs?
And is it really critical to do so immediately?
% But is that always necessary?
% Epistemic conflict is an important aspect of human thought.
% ---and, as we shall see, it is already bein
% Yet we build artificial agents using probabilistic models that, by construction, cannot represent conflicted beliefs.
Inconsistency is clearly undesirable, but we stand to gain a lot by representing it.
%
% You must find a way to consult 
% But is that how you aspire to think? 
% But what makes information so much more dangerous when it's outside of you than inside?
% What is so special about the borders of an agent? 
% What about conglomerates made up of smaller actors? Why are 

This thesis develops a broad theory of how to approach probabilistic modeling with possibly-inconsistent information, unifying and reframing much of the literature in the process.  
The key ingredient is a novel kind of graphical model, called a \emph{Probabilistic Dependency Graph (PDG)},
which allows for arbitrary (even conflicting) pieces of probabilistic information. 
In \cref{part:univ-model}, we establish PDGs as a generalization of other models of mental state, including traditional graphical models such as Bayesian Networks and Factor Graphs, as well as causal models, and even generalizations of probability distributions, such as Dempster-Shafer Belief functions. 
% We also investigate a general analogue of a BN's conditional independencies for arbitrary directed hypergraphs---i.e., the underlying structure of a PDG. 
In \cref{part:univ-objective}, we show that PDGs also capture modern neural representations.
Surprisingly, standard loss functions can be viewed as the inconsistency of a PDG that models the situation appropriately. 
Furthermore, many important algorithms in AI are instances of a simple approach to resolving inconsistencies. 
In \cref{part:algo-logic-complexity}, we provide algorithms for PDG inference, and uncover a deep algorithmic equivalence between the problems of inference and calculating a PDG's numerical degree of inconsistency. 
We also develop powerful yet inutuitive principles for reasoning with (and about) PDGs. 
\end{abstract}

\begin{biosketch}
% Your biosketch goes here. Make sure it sits inside
% the brackets.
Oliver Richardson is a computer scientist 
    with broad interests and expertise,
    who loves to make beautiful things out of math and code.
He was born in Buffalo, NY and grew up in Salt Lake City, UT. 
% Oliver Richardson is a PhD candidate in Computer Science (CS) at Cornell University, advised by Joe Halpern.  
% He works on mathematical foundations of agents, sewing together probabilistic graphical models, information theory, programming languages, logic, and machine learning.  
Oliver earned bachelor's degrees in Computer Science, Mathematics, and Biology 
    (and minors in Cognitive Science, Physics, and Chemistry)
    from the University of Utah in 2018;
    there, he worked in applied machine learning 
        (structured prediction for natural language processing, with Vivek Srikumar),
    and pure math 
        (tropical geometry, with Aaron Bertram). 
He also holds an M.Phil. in Advanced Computer Science from the University of Cambridge, where he worked on diagrammatic reasoning with Mateja Jamnik. 
%
After recieving his Ph.D. in Computer Science under the supervision of Joseph Halpern, he will join the Montr\'{e}al Institute for Learning Algorithms (MILA) as post-doctoral fellow under Yoshua Bengio.
\end{biosketch}

\begin{dedication}
% This document is dedicated to all Cornell graduate students.
% To my friends and family, who 
To my mother, 
    who taught me to probe everything with questions;
to my father,
    % who became an expert in education to give
    % me the best possible learning environment.
    who learned as much as possible about learning in order to facilitate mine.
\end{dedication}

\begin{acknowledgements}
% Your acknowledgements go here. Make sure it sits inside the brackets.

\input{fragments/ack}

\end{acknowledgements}

% \dominitoc% do I want mini toc?
% {%
% \renewcommand{\thepart}{\Roman{part} :}
% \renewcommand{\thepart}{Part~\Roman{part} :}
% \renewcommand\cftpartpresnum{Part~}
\contentspage
% }%
% \tablelistpage
\figurelistpage

\normalspacing \setcounter{page}{1} \pagenumbering{arabic}
\pagestyle{cornell} \addtolength{\parskip}{0.5\baselineskip}


\chapter{Introduction}
    % \cite{KF09}
    % \cite{kingma2013autoencoding}
    \input{chapters/intro}
\chapter{Background and Mathematical Preliminaries}
    \input{chapters/prelim}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% \part{A Universal Modeling Language} \label{part:univ-model}
\part{A Natural and General Modeling Language}
    \label{part:univ-model}
% \part{Structured Modeling} \label{part:univ-model}

\chapter{Probabilistic Dependency Graphs (PDGs)} 
    \input{chapters/pdg-repr}
    
% \chapter{Capturing Other Knowledge Representations}
% \chapter{Everything is a PDG}
\chapter{Representing Things with PDGs}
    \input{chapters/kr}
     
\chapter{Qualitative Mecahnism Independence}
    \input{chapters/qim}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% \part{A Universal Objective} \label{part:univ-objective}
\part{A Universal Objective} \label{part:univ-objective}

% \chapter*{Introduction to Part II: }
\chapter*{The Meaning of Universality}
    \addcontentsline{toc}{chapter}{The Meaning of Universality}
    \input{fragments/part2intro}

\chapter{Loss as the Inconsistency of a PDG: 
    Choose your Model, not your Loss}
    \input{chapters/loss}
\chapter{ The Local Inconsistency Resolution (LIR) Algorithm }
    % \label{chap:lir}
    \input{chapters/lir}
    
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% \part{Algorithms and Complexity}
\part{Algorithms, Logic, and Complexity%
    % : Reasoning about (and with) PDGs
    } \label{part:algo-logic-complexity}

\chapter{Inference for PDGs, via Exponential Conic Programming}
    \input{chapters/infer}

\chapter[Lower Bounds, and the Deep Connection\texorpdfstring{\\}{}between Inconsistency and Inference]%
    {Lower Bounds, and The Deep Connection between Inconsistency and Inference}
    \input{chapters/infer-inconsist-connect}

\chapter{Reasoning with PDGs}
    % \label{sec:monotone}\label{chap:reason}
    \input{chapters/rwpdg}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\iffoundations
\part{Foundations} \label{part:foundations}

\chapter{Learner's Confidence}
    % \label{chap:conf}
    \input{chapters/conf}

\chapter{Relative Entropy Soup}
    % \label{chap:relent-soup}
    \input{chapters/relent-soup}

\chapter{The Category Theory of PDGs}
    % \label{chap:PDG-cat}
    \input{chapters/cat}
\fi

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\part{Wrapping Up}  \label{part:conclusions}
\chapter{Conclusions, Discussion, and Future Work}
    \label{chap:end}
    \input{chapters/end}

% \part{Appendices}

% \bibliography{pdg-aaai,z,qdg-refs,joe}
% \bibliography{pdg-aaai-refs,qdg-refs,loss-refs,joe,z}
\bibliography{merged}

\end{document}