Reddit History Preserved: New Tool Archives 2.38B Posts Offline

Reddit's ecosystem has undergone a seismic shift in recent years. With the effective death of the public API and the disappearance of third-party applications, access to the platform's vast repository of discussions has become increasingly restricted. The Pushshift dataset, a critical resource for researchers and archivists, has faced repeated threats of being cut off, leaving the future of Reddit's collective knowledge in jeopardy.

Now, a new open-source project offers a definitive solution. A developer has built a tool capable of transforming the entire 3.28TB torrent of Reddit history into a fully functional, offline-accessible archive. This innovation ensures that once the data is downloaded, it belongs to the user forever—immune to corporate decisions, API keys, or internet connectivity.

The core function of the tool is deceptively simple yet powerful. It ingests compressed data dumps from Reddit (in .zst format), as well as archives from Voat and Ruqqus, and generates static HTML files. This approach eliminates the need for complex server infrastructure or constant internet access. Users simply open the generated index.html file in any browser to navigate through posts and comments.

For those requiring advanced functionality, an optional Docker stack with PostgreSQL can be deployed. This remains entirely on the user's machine, providing full-text search capabilities without external requests. The system is designed for maximum flexibility and privacy:

• No JavaScript or external tracking
• Works on air-gapped machines
• Serves content over a local LAN (e.g., Raspberry Pi)
• Can be distributed via USB drive

The primary value proposition is data sovereignty. Once the Pushshift torrent is downloaded and processed, the user owns the data. There are no API keys to manage, no rate limits to navigate, and no Terms of Service changes that can revoke access. This is a critical development for anyone relying on Reddit data for long-term projects or research.

Once you have the data, you own it. No API keys, no rate limits, no ToS changes can take it away.

The tool scales efficiently. The PostgreSQL backend ensures that memory usage remains constant regardless of dataset size. While a single instance can handle tens of millions of posts, the full 2.38 billion post dataset can be managed by running multiple instances segmented by topic. This architecture makes preserving the entirety of Reddit's history a feasible task for individuals and small organizations.

Beyond simple browsing, the archive is built for integration and automation. It ships with a full REST API featuring over 30 endpoints. Users can query posts, comments, users, subreddits, and perform aggregations directly against their local database.

Perhaps most notably, the project includes a Model Context Protocol (MCP) server with 29 tools. This allows AI applications to query the local Reddit archive directly, opening up new possibilities for AI-driven analysis and data mining without relying on cloud services. The developer built the tool using Python, PostgreSQL, Jinja2 templates, and Docker, utilizing Claude Code in an experiment of AI-assisted development.

The tool is designed to be accessible to users with varying levels of technical expertise. It supports a wide range of hosting scenarios, from the simplest to the most secure. The available self-hosting options include:

• USB Drive / Local Folder: The most basic setup; just open the HTML files.
• Home Server (LAN): Serve the archive to devices on a Raspberry Pi or similar hardware.
• Tor Hidden Service: Two commands enable access via Tor without port forwarding.
• VPS with HTTPS: Standard web hosting for public or private access.
• GitHub Pages: Suitable for hosting smaller archives.

A live demo of the archiver is available online, showcasing the static browsing experience. The project code is released into the Public Domain via GitHub, encouraging widespread adoption and contribution.

The release of this archiver tool represents a significant step in the preservation of digital culture. As platforms evolve and restrict access, the ability for individuals to maintain their own archives becomes increasingly valuable. This project provides a robust, scalable, and private method for ensuring that the 2.38 billion posts that constitute Reddit's history remain accessible for future generations.

By democratizing access to massive datasets, the tool empowers researchers, developers, and enthusiasts to continue their work without fear of platform instability. It stands as a testament to the open-source community's ability to respond to centralized control with decentralized solutions.