denote-explore.info

This is denote-explore.info, produced by makeinfo version 7.1.1 from
denote-explore.texi.

INFO-DIR-SECTION Emacs misc features
START-INFO-DIR-ENTRY
* Denote Explorer: (denote-explore). Explore, visualise and analyse Denote files.
END-INFO-DIR-ENTRY


File: denote-explore.info,  Node: Top,  Next: Descriptive Statistics,  Up: (dir)

Denote Explorer: Explore your Denote digital garden
***************************************************

The Denote package by Protesilaos (Prot) Stavrou provides extensive
functionality for creating, retrieving, managing, and linking files in
plain text, Markdown, and Org Mode.  The most redeeming qualities of
this package are its filename convention and modular simplicity.  Due to
its reliance on file names, you can also use the package to access other
file types, such as PDFs or multimedia files (which we call Denote
attachments).  In this way, Denote becomes a fully-featured personal
knowledge management system.

   The Denote-Explorer package came into existence as my collection of
Denote files grew.  I created some auxiliary functions to manage and
explore my burgeoning Denote files.  This package provides four types of
commands:

  1. Descriptive statistics: Count notes, attachments and keywords.
  2. Random walks: Generate new ideas using serendipity.
  3. Janitor: Maintain your denote collection.
  4. Knowledge Graphs: Visualise and analyse your Denote files as a
     network.

* Menu:

* Descriptive Statistics::
* Random Walks::
* Janitor::
* Knowledge Graphs::
* Installation and Package Configuration::
* Acknowledgements::
* License::

-- The Detailed Node Listing --

Janitor

* Duplicate notes::
* Missing Links::
* Managing Keywords::
* Synchronising Meta Data::

Knowledge Graphs

* Community of Notes::
* Note Neighbourhood::
* Sequences Network::
* Keyword Network::
* Network Layout and Presentation::
* D3.js: D3js.
* GraphViz::
* Graph Exchange XML Format::
* Analysing the Denote Network::



File: denote-explore.info,  Node: Descriptive Statistics,  Next: Random Walks,  Prev: Top,  Up: Top

1 Descriptive Statistics
************************

The Denote-Explorer package distinguishes between Denote files (notes)
and attachments.  Denote files are either Org Mode, Markdown, or Plain
Text.  All other files, such as photographs, PDFs, media files, LaTeX,
and HTML, are attachments.

   After a day of working hard on your digital knowledge garden, you can
count the notes and attachments in your collection.  Two functions
provide some basic statistics of your Denote files:

  1. ‘denote-explore-count-notes’: Count the number of notes and
     attachments.
  2. ‘denote-explore-count-keywords’: Count the number of distinct
     Denote keywords.

   These functions are informative, but a graph says more than a
thousand numbers.  The built-in ‘chart.el’ package by Eric M.  Ludlam is
a quaint tool for creating bar charts in a plain text buffer.  Two
commands are available in Denote-Explorer to visualise basic statistics:

  1. ‘denote-explore-barchart-filetypes’: Visualise used file
     extensions.  With universal argument ‘C-u’ visualises only
     attachments.
  2. ‘denote-explore-barchart-keywords’: Visualise the top-n keywords
  3. ‘denote-explore-barchart-timeline’: Visualise notes created by year

   The ‘denote-excluded-files-regexp’ variable can contain a regular
expression of files that are excluded from these statistical functions.


File: denote-explore.info,  Node: Random Walks,  Next: Janitor,  Prev: Descriptive Statistics,  Up: Top

2 Random Walks
**************

Creativity springs from a medley of experiences, emotions, subconscious
musings, and connecting random ideas.  Introducing random elements into
the creative process generates avenues of thought you might not have
travelled otherwise.  Random walks through your notes can be beneficial
when you're stuck in a rut or just like to walk through your files
randomly.

   A random walk is an arbitrary sequence without a defined relationship
between the steps.  You take a random walk by jumping to a random note,
connected or unconnected to the current buffer.

   The Denote-Explorer package provides three commands to inject some
randomness into your explorations:

  1. ‘denote-explore-random-note’: Jump to a random Denote file.
  2. ‘denote-explore-random-regex’: Jump to a random Denote file that
     matches a regular expression.
  3. ‘denote-explore-random-link’: Jump to a random linked note (either
     forward or backward) or attachments (forward only).
  4. ‘denote-explore-random-keyword’: Jump to a random Denote file with
     the same selected keyword(s).

   The default state is that these functions jump to any Denote text
file (plain text, Markdown or Org-mode).  Prefixing the universal
argument (‘C-u’) includes attachments in the sample for a random jump,
otherwise the walk remains within the collection of notes.  The
‘denote-explore-random-regex-ignore’ variable can contain a regular
expression of files excluded from the sample of files to jump to.

   Jumping to a note that matches a regular expression lets you find
random notes matching a search string.  For example, to find a note you
wrote in May 2022, use ‘^202205’ and using ‘202305.*_journal’ jumps to a
random journal entry in May 2023.

   Jumping to a randomly linked file naturally only works when the
current buffer is a Denote file.  A warning appears when the current
buffer is an isolated note (no links or backlinks available).

   When jumping to a random file with one or more matching keywords, you
can choose one or more keywords from the current buffer, or override the
completion options with free text.  The asterisk symbol ‘*’ selects all
keywords in the completion list.  This section process is skipped when
the current buffer only has one keyword.  When the current buffer is not
a Denote file, you can choose any available keyword(s) in your Denote
collection.

   Jumping to a random note and matching multiple keywords only works
when ‘denote-sort-keywords’ is enabled, or when the selected keywords
are in the same order as in the target file.  You can alphabetise
keywords in your Denote files with ‘denote-explore-sort-keywords’,
explained in the next section.


File: denote-explore.info,  Node: Janitor,  Next: Knowledge Graphs,  Prev: Random Walks,  Up: Top

3 Janitor
*********

After hoarding Denote files for a while, you may need a janitor to
organise your collection.  A janitor ensures cleanliness, orderliness,
and sanitation in a building, so this role is also perfect for managing
to your Denote files.  The Denote-Explorer package provides a series of
commands to assist with cleaning, organising, and sanitising your files.

* Menu:

* Duplicate notes::
* Missing Links::
* Managing Keywords::
* Synchronising Meta Data::


File: denote-explore.info,  Node: Duplicate notes,  Next: Missing Links,  Up: Janitor

3.1 Duplicate notes
===================

The Denote package prevents duplicate identifiers when creating a new
note, but when assigning filenames manually, or when exporting org
files, duplicates might occur.

   The Denote identifier is a unique string constructed of the note's
creation date and time in ISO 8601 format (e.g., ‘2024035T203312’).
Denote either uses the current date and time when generating a new note
or the date and time the file was created on the file system.

   The file's creation date and time are not always relevant for
attachments.  For example, when adding scanned historical records, the
identifier might be centuries ago, so it must be added manually.

   The ‘denote-explore-identify-duplicate-notes’ command lists all
duplicate Denote files in a popup org buffer, which includes links to
the suspected duplicate notes and attachments.

   Additionally, the ‘denote-explore-identify-duplicate-notes-dired’
command displays files with duplicate identifiers in a Dired buffer.
You can directly change filenames in the Dired buffer with
‘dired-toggle-read-only’ (‘C-x C-q’) or remove duplicates with ‘D’
(‘dired-do-delete’).  Note that this function shows files in the denote
directory and not its subdirectories or symbolic links.

   With the universal argument (‘C-u’), these commands ignore any
duplicated identifiers created when exporting Denote Org mode files.

   The ‘denote-excluded-files-regexp’ variable can contain a regular
expression of files that are excluded from duplicate detection.  When
using the archive functionality for Denote files you might set this
variable to ‘"^.+\.org_archive"’ so they are not marked as duplicates.

   Be careful when changing the identifier of a Denote file, as it can
destroy the integrity of your links.  Please ensure that the file you
rename does not have any links pointing to it.  You can use the
‘denote-find-link’ and ‘denote-find-backlink’ commands to check a file
for links, or use the Denote Explorer link checker.


File: denote-explore.info,  Node: Missing Links,  Next: Managing Keywords,  Prev: Duplicate notes,  Up: Janitor

3.2 Missing Links
=================

The file identifier in Denote is the bit of information that keeps links
active even when you change their name.  But missing or dead links might
still appear in your network of notes when you delete redundant
information or you manually rename a file and change its identifier.

   Using ‘denote-explore-dead-links’ lists all links to non-existing
notes or attachments in your Denote directory.  This function creates a
read-only Org mode file with a table of source documents and the missing
linked document.  You can click on the links to jump to the source file
at the missing link location and either remove or edit it.  Links will
appear in their literal form, i.e.  ‘[[<link>][<description>]]’.

   The link contains an Elisp function and Emacs will ask for
confirmation every time you click a link.  You can disable these
warnings by temporary setting ‘org-link-elisp-confirm-function’ to
‘nil’.

   When no link is found a message pops up in the echo area.

   The ‘denote-excluded-files-regexp’ variable can contain a regular
expression of files that are excluded from the search for missing links.


File: denote-explore.info,  Node: Managing Keywords,  Next: Synchronising Meta Data,  Prev: Missing Links,  Up: Janitor

3.3 Managing Keywords
=====================

Denote keywords connect notes with similar content.  Keywords should not
exist in solitude because a category with only one member is not
informative.  Single keywords can arise because topics need to be fully
developed or due to a typo.

   The ‘denote-explore-single-keywords’ command provides a list of file
tags that are only used once.  The list of single keywords is presented
in the minibuffer, from where you can open the relevant note or
attachment.

   You can also find notes or attachments without keywords with the
‘denote-explore-zero-keywords’ command.  This command lists all notes
and attachments without keywords in the minibuffer, so you can open them
and consider adding keywords or leaving them as is.

   You can rename or remove keywords with
‘denote-explore-rename-keyword’.  Select one or more existing keywords
from the completion list and enter the new keyword.  This function
renames all chosen keywords or removes the original keyword from all
existing notes when you enter an empty string as new keyword.  This
function cycles through all notes and attachments containing one or more
selected keywords and asks for confirmation before making any changes.
The new keyword list is stored alphabetically.  This function uses the
front matter as the source of truth for notes and the file name for
attachments.

   The ‘denote-excluded-files-regexp’ variable can contain a regular
expression of files that are excluded from the purview of these
functions.


File: denote-explore.info,  Node: Synchronising Meta Data,  Prev: Managing Keywords,  Up: Janitor

3.4 Synchronising Meta Data
===========================

Denote stores the metadata for each note in the filename using its
ingenious format.  Some of this metadata is copied to the front matter
of a note, which can lead to discrepancies between the two metadata
sources.

   The ‘denote-explore-sync-metadata’ function checks all notes and asks
the user to rename any file where the front matter data differs from the
file name.  The front matter data is the source of truth for the title
and keywords.  This function also enforces the alphabetisation of
keywords, which assists with finding notes.

   The ‘denote-excluded-files-regexp’ variable can contain a regular
expression of files that are excluded from this synchronisation.


File: denote-explore.info,  Node: Knowledge Graphs,  Next: Installation and Package Configuration,  Prev: Janitor,  Up: Top

4 Knowledge Graphs
******************

Emacs is a text processor with limited graphical capabilities.  However,
committing your ideas to text requires a linear way of thinking since
you can only process one word at a time.  Visual thinking through tools
such as mind maps or network diagrams is another way to approach your
ideas.  One of the most common methods to visualise interlinked
documents is in a network or a personal knowledge graph, or in more
general terms, a network diagram.

   Denote implements a linking mechanism that connects notes (either
Org, Markdown, or plain text files) to other notes or attachments.  This
mechanism allows the user to visualise all notes as a network diagram.

   Network visualisation in Denote is not just a feature but a powerful
tool that visualises how notes are linked, helping you discover
previously unseen connections between your thoughts and enhancing your
creative process.

   It's important to note that Denote-Explorer does not offer live
previews of your note collection.  This deliberate choice prevents the
'dopamine traps' of seeing your thoughts develop in real-time.  Instead,
Denote-Explorer provides a focused tool for the surgical dissection of
your second brain, while the main user interface remains text-based.

   A network diagram has nodes (vertices) and edges.  Each node
represents a note or an attachment, and each edge represents a link
between them.  A link between file is directed and the arrow indicates
the source and target of the link.  The diagram below shows the basic
principle of a knowledge graph.  In the actual output, nodes are
circles.

     ┌──────────────┐        ┌──────────────┐
     │     node     │  edge  │     node     │
     │    (note)    ├───────►│    (note)    │
     │ (attachment) │ (link) │ (attachment) │
     └──────────────┘        └──────────────┘

   Denote-Explorer provides three types of network diagrams to explore
the relationships between your thoughts:

  1. Community: Notes matching a regular expression
  2. Neighbourhood: Search n-deep in a selected note
  3. Sequence: Visualise a hierarchical sequence
  4. Keywords: Relationships between keywords

   The package exports and displays these in one of three formats, with
JSON displayed in HTML / D3.js files as the default.  Other options are
GraphViz and GEXF.

   You create a network with the ‘denote-explore-network’ command.  This
command will ask the user to select the type of network to create.  Each
network type requires additional inputs to analyse to a defined part of
your Denote files.

   The ‘denote-explore-network-regenerate’ command recreates the
previous graph with the same parameters, which is useful when changing
the structure of your notes and you like to see the result visualised
without having to re-enter the parameters.

   Using the universal argument ‘C-u’ before issuing these two command
(re)generates a network excluding attachments.  The
‘denote-excluded-files-regexp’ variable can contain a regular expression
of files that are excluded from visualisation.

* Menu:

* Community of Notes::
* Note Neighbourhood::
* Sequences Network::
* Keyword Network::
* Network Layout and Presentation::
* D3.js: D3js.
* GraphViz::
* Graph Exchange XML Format::
* Analysing the Denote Network::


File: denote-explore.info,  Node: Community of Notes,  Next: Note Neighbourhood,  Up: Knowledge Graphs

4.1 Community of Notes
======================

A community graph displays all notes matching a regular expression and
their connections.  The example below indicates the community that
contains the ‘_emacs’ regular expression, within the dashed line.  The
algorithm prunes any links to non-matching notes, which in the example
below is the note with the ‘_vim’ keyword.

     ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─┐
        _emacs community
     │ ┌──────┐  ┌──────┐ │  ┌────┐
       │_emacs│  │_emacs│───►│_vim│
     │ └──┬───┘  └──────┘ │  └────┘
          │
     │    ▼               │
       ┌──────┐
     │ │_emacs│           │
       └──────┘
     └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─┘

   To generate a community graph, use ‘denote-explore-network’, choose
'Community' and enter a regular expression.  When no matching files are
found or there are only solitary nodes, then the network is not
generated and you will see this warning: ‘No Denote files or (back)links
found for regex’.

   The ‘denote-explore-network-regex-ignore’ variable defines a regular
expression to exclude certain notes from community networks.  For
example, if you create meta notes with long lists of dynamic links and
they have the ‘_meta’ keyword, then you could exclude these nodes by
customising this variable to the relevant regular expression.


File: denote-explore.info,  Node: Note Neighbourhood,  Next: Sequences Network,  Prev: Community of Notes,  Up: Knowledge Graphs

4.2 Note Neighbourhood
======================

The neighbourhood of a note consists of all files linked to it at one or
more steps deep.  The algorithm selects members of the graph from linked
and back-linked notes.  This visualisation effectively creates the
possible paths you can follow with the ‘denote-explore-random-link’
function discussed in the Random Walks section above.

   The illustration below shows the principle of the linking depth.
Notes B and C are at linking depth 1 from A and notes D and E are at
depth 2 from A.

     Depth 1      2
          ┌─┐    ┌─┐
       ┌─►│B│◄───┤D│
       │  └─┘    └─┘
      ┌┴┐
      │A│
      └─┘
       ▲  ┌─┐    ┌─┐
       └──┤C├───►│E│
          └─┘    └─┘

   To generate a neighbourhood graph from the current Denote note
buffer, use ‘denote-explore-network’ and enter the graph's depth.  The
user enters the required depth, and the software searches all notes
linked to the current buffer at that depth.  When building this graph
from a buffer that is not a Denote file, the system also asks to select
a source file (A in the diagram).  The system issues a warning when you
select a note without links or backlinks.  You can identify Denote files
without any links with the ‘denote-explore-isolated-notes’ function
describe above.

   The complete set of your Denote files is most likely a disconnected
Graph, meaning that there is no one path that connects all nodes.
Firstly, there will be isolated notes.  There will also exist isolated
neighbourhoods of notes that connect to each other but not to other
files.

   A depth of more than three links is usually not informative because
the network can become to large to read, or you hit the edges of your
island of connected notes.

   The ‘denote-explore-network-regex-ignore’ variable lets you define a
regular expression of notes exclude from neighbourhood networks.


File: denote-explore.info,  Node: Sequences Network,  Next: Keyword Network,  Prev: Note Neighbourhood,  Up: Knowledge Graphs

4.3 Sequences Network
=====================

Denote signatures can define a hierarchical sequence of notes, for
example a note with signature ‘1=1’ is the child of a note with
signature ‘1’ and a note with signature ‘1=2’ is its sibling.  The note
with signature ‘1=1=a’ is the child of ‘1=1’ and the grandchild of ‘1’,
and so forth.  In a sequence network, links exist independent of any
Denote links inside a note, the relationship is only based on the
hierarchy of the signatures.

     ┌─────┐    ┌─────┐    ┌─────┐
     |  1  ├───►│ 1=1 ├───►│1=1=a│
     └──┬──┘    └─────┘    └─────┘
        │       ┌─────┐
        └──────►│ 1=2 │
                └─────┘

   The content of the signatures can be either numbers or letters as the
order of children is not taken into consideration.  These sequences can
go on to many generations, building a family tree of your notes.  These
sequences are the basic building block of the popular Zettelkasten
methodology.

   To generate a sequence graph, use ‘denote-explore-network’ and select
the signature of the root node (note ‘1’ in the diagram).  When not
selecting any signature, all Denote files with a signature are included
in the visualisation.

   The ‘denote-explore-network-regex-ignore’ variable lets you define a
regular expression of notes exclude from neighbourhood networks.


File: denote-explore.info,  Node: Keyword Network,  Next: Network Layout and Presentation,  Prev: Sequences Network,  Up: Knowledge Graphs

4.4 Keyword Network
===================

The last available method to visualise your Denote collection is to
develop a network of keywords.  Two keywords are connected when used in
the same note.

   All keywords in a note form a complete network.  The union of all
complete networks from all files in your Denote collection defines the
keywords network.  The relationship between two keywords can exist in
multiple notes, so the links between keywords are weighted.  The line
thickness between two keywords indicates the frequency (weight) of their
relationship.

   While the first two graph types are directed (arrows indicate the
direction of links), the keyword network is undirected.  These links are
bidirectional associations between keywords.  The diagram below shows
three notes, two with two keywords and one with three keywords.  Each
notes forms a small complete network that links all keywords.

     ┌─────┐ ┌─────┐   ┌─────┐ ┌─────┐   ┌─────┐ ┌─────┐
     │_kwd1├─┤_kwd2│   │_kwd1├─┤_kwd2│   │_kwd3├─┤_kwd4│
     └─────┘ └─────┘   └─┬───┘ └───┬─┘   └─────┘ └─────┘
                         │ ┌─────┐ │
                         └─┤_kwd3├─┘
                           └─────┘

   The union of these three networks forms the keyword network for this
collection of notes.  The example generates the following keyword
network.

     ┌─────┐ ┌─────┐
     │_kwd1├─┤_kwd2│
     └─┬───┘ └───┬─┘
       │         │
       │ ┌─────┐ │  ┌─────┐
       └─┤_kwd3├─┴──┤_kwd4│
         └─────┘    └─────┘

   When generating this graph type, you will need to enter a minimum
edge weight (n).  The graph then will only show those keywords that are
at least n times associated with each other.  The default is one.

   Some keywords might have to be excluded from this graph because they
skew the results.  For example, when using the Citar-Denote package, you
might like to exclude the ‘bib’ keyword from the diagram because it is
only used to minimise the search space for bibliographic notes and has
no further semantic value.  The ‘denote-explore-network-keywords-ignore’
variable lists keywords ignored in this visualisation.


File: denote-explore.info,  Node: Network Layout and Presentation,  Next: D3js,  Prev: Keyword Network,  Up: Knowledge Graphs

4.5 Network Layout and Presentation
===================================

Emacs cannot independently generate graphics and thus relies on external
software.  This package can use three external mechanisms to create
graphs (configurable with ‘denote-explore-network-format’), set to D3.js
/ JSON output by default.  Other available formats are GraphViz SVG and
GEXF, discussed in detail below.

   The Denote-Explorer network algorithm consists of four steps:

  1. The ‘denote-explore-network’ function determines the relevant
     functions based on user input.
  2. The code generates a nested association list for the selected
     graph:
        • Metadata e.g.: ‘(meta (directed . t) (type . "Neighbourhood")
          (parameters "20210104T194405" 2))’
        • Association list of nodes, e.g., ‘(((id . "20210104T194405")
          (name . "Platonic Solids") (keywords "geometry" "esotericism")
          (type . "org") (degree . 4)) ...)’.  In the context of Denote,
          the degree of a network node is the unweighted sum of links
          and backlinks in a note.
        • Association list of edges and their weights: ‘(((source .
          "20220529T190246") (target . "20201229T143000") (weight . 1))
          ...)’.  The weight of an edge indicates the number of time the
          two files are linked, or the number of times two keywords
          appear in the same note in case of a keyword graph.
  3. The package encodes the association list to a either a JSON,
     GraphViz DOT, or GEXF file.  The location and name of this file is
     configurable with the ‘denote-explore-network-directory’ and
     ‘denote-explore-network-filename’ variables.
  4. Relevant external software displays the result (in most cases a web
     browser).


File: denote-explore.info,  Node: D3js,  Next: GraphViz,  Prev: Network Layout and Presentation,  Up: Knowledge Graphs

4.6 D3.js
=========

D3.js (https://d3js.org/) is a JavaScript library for visualising data.
This method provides an aesthetically pleasing and interactive view of
the structure of your notes.  Denote-Explorer stores the desired network
as a JSON file.  This JSON file is merged with a HTML / JavaScript
template to visualise the network.  Emacs invokes your default internet
browser to view the network.

   Hover over any node to reveal its name and relevant metadata.  For
neighbourhood and community graphs, when the note is an image or PDF
file, a preview appears in the tooltip.  Clicking on a node opens the
relevant file in the browser, or whatever application the browser
associates with the relevant file type.

   For community and neighbourhood graphs, the diameter of nodes is
proportional to their degree.  Thus, the most linked note in your query
will be the most visible.  The colours indicate the file type of each
node.  The size of nodes in a network graph is the same for all.

   For nodes with a degree greater than two, the name is displayed
outside the node.

   In keyword graphs, the thickness of the edges indicates the number of
times two keywords are associated with each other.

   The info button shows the type of network and provides some basic
statistics, such as the number of nodes (files) and edges (links) and
the network density.  The density of a network is the ratio between the
number of edges and the potential number of edges.  A density of zero,
as such means that no nodes are connected.  In a network with a density
of one all nodes are connected to each other.

   For community graphs the panel also provides the option to show or
hide isolated nodes to increase clarity.  Neighbourhood and keyword
graphs by their definition do not have isolated nodes.

   For community and neighbourhood graphs, the info panel also shows the
distribution of keywords for the visualised network.

   You can customise the output of the network files by modifying the
template.  The ‘denote-explore-network-d3-template’ variable contains
the location of the HTML/JavaScript template file so you can craft your
own versions.  This file contains several shortcodes:

   • ‘{{graph-type}}’: Type of graph, community, neighbourhood or
     network
   • ‘{{d3-js}}’: Content of the ‘denote-explore-network-d3-js’
     variable, which contains the URL of the D3 source code, which has
     to be version 7 or above.  The default template fetches the
     JavaScript code from the ‘d3js.org’ website.
   • ‘{{json-content}}’: The generated JSON file with the network
     definition
   • ‘{{d3-colourscheme}}’: Content of
     ‘denote-explore-network-d3-colours’.  this variable assigns a
     colour palette for the node file types.  You can choose between any
     of the available categorical colour schemes in the D3 package.
     Colours are assigned in the graph in order of appearance in the
     JSON file, so file types can have different colours in different
     graphs.


File: denote-explore.info,  Node: GraphViz,  Next: Graph Exchange XML Format,  Prev: D3js,  Up: Knowledge Graphs

4.7 GraphViz
============

GraphViz (https://graphviz.org/) is an open-source graph visualisation
software toolkit, ideal for this task.  The Denote-Explorer software
saves the graph in the DOT language as a ‘.gv’ file.  The GraphViz
software converts the DOT code to an ‘SVG’ file.

   You will need to install GraphViz to enable this functionality.
Denote-Explorer will raise an error when trying to create a GraphViz
graph without the required external software available.

   Hover over any node to reveal its name and relevant metadata.
Clicking on any node in a community or neighbourhood graph opens the
relevant file in the browser, or whatever application the browser
associates with the relevant file type.

   For community and neighbourhood graphs, the diameter of nodes is
proportional to their degree.  Thus, the most linked note in your query
will be the most visible.  When generating a neighbourhood, the source
node is marked in a contrasting colour.

   For nodes with a degree greater than two, the name is displayed
outside the node.  In keyword graphs, the thickness of the edges
indicates the number of times two keywords are associated with each
other.

   The diameter of nodes are sized relative to their degree.  Thus, the
most referenced note in your system will be the most visible.  For nodes
with a degree greater than two, the name is displayed outside the node
(top left).

   The configurable ‘denote-explore-network-graphviz-header’ variable
defines the basic settings for GraphViz graphs, such as the layout
method and default node and edge settings.

   The ‘denote-explore-network-graphviz-filetype’ variable defines the
GraphViz output format.  SVG (the default) or PDF provide the best
results.


File: denote-explore.info,  Node: Graph Exchange XML Format,  Next: Analysing the Denote Network,  Prev: GraphViz,  Up: Knowledge Graphs

4.8 Graph Exchange XML Format
=============================

The first two formats an insight into parts of your knowledge network,
but there is a lot more you can do with this type of information.  While
GraphViz and D3 are suitable for analysing sections of your network,
this third option is ideal for storing the complete Denote network for
further analysis.

   Graph Exchange XML Format (‘GEXF’) is a language for describing
complex network structures.  This option saves the network as a ‘GEXF’
file without opening it in external software.

   To save the whole network, use the Community option and enter an
empty search string to include all files.

   You can analyse the exported file with Gephi Lite
(https://gephi.org/gephi-lite/), a free online network analysis tool.
The ‘GEXF’ file only contains the IDs, names and degree of the nodes;
and the edges and their weights.


File: denote-explore.info,  Node: Analysing the Denote Network,  Prev: Graph Exchange XML Format,  Up: Knowledge Graphs

4.9 Analysing the Denote Network
================================

A well-trodden trope in network analysis is that all people are linked
within six degrees of separation.  This may also be the case for your
notes, but digging more than three layers deep is not very informative
as the network can become large and difficult to review.

   It might seem that adding more connections between your notes
improves them, but this is not necessarily the case.  The extreme case
is a complete network where every file links to every other file.  This
situation lacks any interesting structure and wouldn't be informative.
So, be mindful of your approach to linking notes and attachments.

   Your Denote network is unlikely to be a fully connected graph.  In a
connected graph, there is a path from any point to any other point.
Within the context of Denote, this means that all files have at least
one link or backlink.  Your network will most likely have isolated nodes
(files without any (back)links) and islands of connected notes.

   The previously discussed ‘denote-explore-isolated-files’ command
lists all files without any links and backlinks to and from the note in
the minibuffer.  You can select any note and add links when required.
Calling this function with the universal argument ‘C-u’ includes
attachments in the list of lonely files.

   The number of links and backlinks in a file (in mathematical terms,
edges connected to a node) is the total degree of a node.  The degree
distribution of a network is the probability distribution of these
degrees over the whole network.  The ‘denote-explore-barchart-degree’
function uses the built-in chart package to display a simple bar chart
of the frequency of the total degree.

   This function might take a moment to run, depending on the number of
notes in your system.  Evaluating this function with the universal
argument ‘C-u’ excludes attachments from the analysis.

   The importance of a note is directly related to the number of
backlinks.  The ‘denote-explore-barchart-backlinks’ function visualises
the number of backlinks in the top-n notes in a horizontal bar chart,
ordered by the number of backlinks.  This function asks for the number
of nodes to visualise and then analyses the complete network of Denote
notes (attachments are excluded because they don't have links from
them), which can take a brief moment.


File: denote-explore.info,  Node: Installation and Package Configuration,  Next: Acknowledgements,  Prev: Knowledge Graphs,  Up: Top

5 Installation and Package Configuration
****************************************

This package is available through MELPA.

   The configuration below customises all available variables and binds
all available commands to the ‘C-c e’ prefix.  To get started you don't
need to configure anything.  You should modify this configuration to
suit your needs, as one person's sensible defaults are another person's
nightmare.

     (use-package denote-explore
       :custom
       ;; Where to store network data and in which format
       (denote-explore-network-directory "<folder>")
       (denote-explore-network-filename "denote-network")
       (denote-explore-network-keywords-ignore "<keywords list>")
       (denote-explore-network-regex-ignore "<regex>")
       (denote-explore-network-format 'd3.js)
       (denote-explore-network-d3-colours 'SchemeObservable10)
       (denote-explore-network-d3-js "https://d3js.org/d3.v7.min.js")
       (denote-explore-network-d3-template "<file path>")
       (denote-explore-network-graphviz-header "<header strings>")
       (denote-explore-network-graphviz-filetype 'svg)
       :bind
       (;; Statistics
        ("C-c e s n" . denote-explore-count-notes)
        ("C-c e s k" . denote-explore-count-keywords)
        ("C-c e s e" . denote-explore-barchart-filetypes)
        ("C-c e s w" . denote-explore-barchart-keywords)
        ("C-c e s t" . denote-explore-barchart-timeline)
        ;; Random walks
        ("C-c e w n" . denote-explore-random-note)
        ("C-c e w r" . denote-explore-random-regex)
        ("C-c e w l" . denote-explore-random-link)
        ("C-c e w k" . denote-explore-random-keyword)
        ;; Denote Janitor
        ("C-c e j d" . denote-explore-duplicate-notes)
        ("C-c e j D" . denote-explore-duplicate-notes-dired)
        ("C-c e j l" . denote-explore-dead-links)
        ("C-c e j z" . denote-explore-zero-keywords)
        ("C-c e j s" . denote-explore-single-keywords)
        ("C-c e j r" . denote-explore-rename-keywords)
        ("C-c e j y" . denote-explore-sync-metadata)
        ("C-c e j i" . denote-explore-isolated-files)
        ;; Visualise denote
        ("C-c e n" . denote-explore-network)
        ("C-c e r" . denote-explore-network-regenerate)
        ("C-c e d" . denote-explore-barchart-degree)
        ("C-c e b" . denote-explore-barchart-backlinks)))

   You can use the most recent development version directly from GitHub
(Emacs 29.1 or higher):

     (unless (package-installed-p 'denote-explore)
       (package-vc-install
        '(denote-explore
          :url "https://github.com/pprevos/denote-explore/")))


File: denote-explore.info,  Node: Acknowledgements,  Next: License,  Prev: Installation and Package Configuration,  Up: Top

6 Acknowledgements
******************

This code would only have existed with the help of Protesilaos Stavrou,
developer of Denote.

   In addition, Jakub Szczerbowski, Samuel W.  Flint, Ad (skissue),
Vedang Manerikar, Jousimies, Alexis Praga, and Dav1d23 made significant
contributions and suggestions.

   Noor Us Sabah on Fiverr wrote the first version of the D3.JS template
file.  All enhancements were generated with the assistance of ChatGPT.

   Feel free to raise an issue here on GitHub if you have any questions
or find bugs or suggestions for enhanced functionality.


File: denote-explore.info,  Node: License,  Prev: Acknowledgements,  Up: Top

7 License
*********

This program is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
Free Software Foundation, either version 3 of the License or (at your
option) any later version.

   This program is distributed in the hope that it will be useful but
WITHOUT ANY WARRANTY, INCLUDING THE IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
General Public License for more details.

   For a full copy of the GNU General Public License, see
<https://www.gnu.org/licenses/>.



Tag Table:
Node: Top251
Node: Descriptive Statistics1968
Node: Random Walks3470
Node: Janitor6325
Node: Duplicate notes6901
Node: Missing Links9045
Node: Managing Keywords10327
Node: Synchronising Meta Data11994
Node: Knowledge Graphs12839
Node: Community of Notes16447
Node: Note Neighbourhood18090
Node: Sequences Network20232
Node: Keyword Network21892
Node: Network Layout and Presentation24564
Node: D3js26493
Node: GraphViz29664
Node: Graph Exchange XML Format31532
Node: Analysing the Denote Network32571
Node: Installation and Package Configuration35100
Node: Acknowledgements37870
Node: License38576

End Tag Table


Local Variables:
coding: utf-8
End: