Repositories list

• awesome-web-archiving

  Public
  An Awesome List for getting started with web archiving

  awesomeawesome-listwebarchiving

  Creative Commons Zero v1.0 Universal

  •158•2.1k•3•1•Updated Jan 4, 2025Jan 4, 2025

• warcaroo

  Public
  Java

  •

  Apache License 2.0

  •0•9•5•0•Updated Dec 20, 2024Dec 20, 2024

• webarchive-commons

  Public
  Common web archive utility code.

  Java

  •

  Apache License 2.0

  •72•52•15•4•Updated Dec 20, 2024Dec 20, 2024

• jwarc

  Public
  Java library for reading and writing WARC files with a typed API

  Java

  •

  Apache License 2.0

  •10•49•14•0•Updated Dec 19, 2024Dec 19, 2024

• warc-specifications

  Public
  Centralised repository for WARC usage specifications.

  HTML

  •31•104•41•1•Updated Nov 21, 2024Nov 21, 2024

• javaswf

  Public
  Fork of JavaSWF2 for building Heritrix

  Java

  •

  Other

  •0•0•0•0•Updated Oct 17, 2024Oct 17, 2024

• warc2html

  Public
  Converts WARC files to static HTML

  Java

  •

  Apache License 2.0

  •4•42•5•0•Updated Jun 27, 2024Jun 27, 2024

• openwayback

  Public
  The OpenWayback Development

  Java

  •

  Apache License 2.0

  •276•490•100•5•Updated Jan 3, 2024Jan 3, 2024

• openwayback-access-control

  Public
  web access control (exclusion oracle) tools for optional use with wayback machine

  JavaScript

  •

  Apache License 2.0

  •5•6•0•7•Updated Jan 2, 2023Jan 2, 2023

• logtrix

  Public
  Java library/tool for parsing and summarising Heritrix crawl logs

  Java

  •

  Apache License 2.0

  •1•3•3•3•Updated Nov 16, 2022Nov 16, 2022

• urlcanon

  Public
  url canonicalization library for python and java

  Java

  •8•33•3•0•Updated May 22, 2022May 22, 2022

• heritrix-dependencies

  Public
  Dependencies needed to build Heritrix that aren't in Maven Central

  0•0•0•0•Updated Sep 1, 2021Sep 1, 2021

• robustlinks

  Public
  Links on the web break all the time, robustify them!

  linkswebarchivesjavascript+ 2htmlrobust-links

  JavaScript

  •6•52•2•0•Updated Jan 4, 2021Jan 4, 2021

• training

  Public
  Inventory of Web Archiving Training Resources

  0•4•0•0•Updated Oct 24, 2019Oct 24, 2019

• archive-crawler-yg-archive

  Public
  An 'archive' of the Yahoo-hosted archive-crawler group

  1•3•0•0•Updated Oct 17, 2019Oct 17, 2019

• qa2019

  Public
  Resources for the 2019 IIPC QA hackathon

  HTML

  •2•3•14•0•Updated May 3, 2019May 3, 2019

• crawl-recipes

  Public
  A place to share practical bits of crawling experiences

  Apache License 2.0

  •0•0•0•0•Updated Dec 12, 2018Dec 12, 2018

• iipc.github.io

  Public
  IIPC Open Development

  Apache License 2.0

  •4•7•0•0•Updated Jun 16, 2017Jun 16, 2017

• travis

  Public
  Shared config for Travis CI for IIPC.

  Shell

  •

  Apache License 2.0

  •3•1•0•0•Updated May 3, 2017May 3, 2017

• heritrix3

  Public
  Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.

  Java

  •761•11•0•1•Updated Mar 9, 2017Mar 9, 2017

• cdx-cli

  Public
  Command line utility for working with CDX files

  Java

  •

  Apache License 2.0

  •4•1•0•0•Updated Sep 29, 2016Sep 29, 2016

• iipc-parent

  Public
  IIPC Parent POM

  Apache License 2.0

  •2•0•0•0•Updated May 24, 2016May 24, 2016

• twittervane

  Public archive
  Using social media to steer web archiving and curation.

  JavaScript

  •5•15•1•0•Updated Nov 20, 2015Nov 20, 2015

• openwayback-sample-overlay

  Public
  Sample Wayback Config using OpenWayback

  7•3•0•0•Updated Feb 7, 2014Feb 7, 2014