Reddit-Extractor

An effective Reddit post fetcher, bypassing JSON API restrictions with proxies.

Overview

Reddit's JSON API is publicly available, no authentication is needed.

However, Rate Limits obviously apply to this endpoint, so my wrapper aims to work around this issue with the fully automated use of randomized cookies per request, and proxies if a rate limit is applied

Installation

npm i reddit-extractor

Usage

import { Scraper, Post } from 'reddit-extractor';

// Proxies are not required, but recommended for large applications (Only http(s) proxies are supported)
// Reddit's JSON API rate limits if you make ~100 requests within quick succession
const proxyConfig = {
	protocol: 'http',
	host: '',
	port: 12321,
	auth: {
		username: '',
		password: '',
	},
};

const RedditExtractor = new Scraper('./tempFiles', proxyConfig);

Get a single Post

const postUrl = 'https://www.reddit.com/r/mac/comments/1fcx4p2/macos_sequoia_will_be_released_on_september_16th/';
const postData = await Scraper.fetchPost(postUrl);

if ('error' in post) {
	return console.error('Error in post', post.error);
}

console.log(postData);

Get recent posts from a subreddit

// Will return the 5 most recent posts from r/memes
const subreddit = 'memes';
const latestFivePosts = await redditScraper.fetchPosts(subreddit, 5);
if (!latestFivePosts.length) return console.error('No posts found');

const mostRecentPostData = latestFivePosts[0];

if ('error' in mostRecentPostData) {
	return console.error('Error in most recent post', mostRecentPostData.error);
}

console.log(mostRecentPostData);