Nes
/
neo-scraper
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Scrape a booru-like post object from the DOM.
64 Commits
1 Branch
2 Tags
693 KiB
TypeScript 99.6%
JavaScript 0.4%
 
 
Go to file
GlobalGit3 8d215729f3 Map and Arrays for prefixes 
Categorize tags and remove prefixes using Maps and Arrays.
A quick write-up, but works for the time being to get an idea for what is wanted.

Due to the data-tag-category's 'origin' being used for more than artist's names, prefixes are used to categorize the artist tag. There are also other prefixes such as 'editor' that are the same as 'artist'. This should be no problem as all artists' tags start with the 'artist' or 'editor' prefix.

Suffixes only applies when removing a prefix, this will probably be changed later to apply without removing the prefix.
It was coded to give an idea of wanting to change tagNames from "oc:originalcharacter" to "originalcharacter_(oc)".
 		2023-07-02 01:49:40 -05:00
docs Update Pixiv engine and tests 2022-08-07 15:24:04 +02:00
scripts Fix ES module crap 2023-01-15 16:42:30 +01:00
src Map and Arrays for prefixes 2023-07-02 01:49:40 -05:00
.eslintrc.cjs Add feature info to engines + project cleanup 2022-08-07 20:48:28 +02:00
.gitignore Update tests 2021-11-20 15:19:19 +01:00
.prettierrc Fix gelbooru + grab original source when available 2021-03-11 00:09:03 +01:00
LICENSE Initial commit 2021-03-11 00:08:54 +01:00
README.md Update README.md 2023-07-01 08:32:35 -05:00
jest-puppeteer.config.cjs Add feature info to engines + project cleanup 2022-08-07 20:48:28 +02:00
jest.preset.cjs Add feature info to engines + project cleanup 2022-08-07 20:48:28 +02:00
package.json Bump version to 0.6.0 2023-06-19 21:31:19 +02:00
tsconfig.json tsconfig.json change which shouldn't break anything 2023-06-19 21:30:02 +02:00
tsconfig.test.json Upgrade dependencies and maybe fix tests 2023-02-11 00:27:39 +01:00
yarn.lock Add experimental e-hentai engine 2023-06-19 21:05:24 +02:00

README.md

neo-scraper

Scrape a booru-like post object from the DOM.

Supported sites

Engine Sites Features Notes
animepictures anime-pictures.net content, resolution, tags, tag_category Rating is assumed to be safe.
danbooru danbooru.donmai.us, safebooru.donmai.us content, rating, resolution, tags, tag_category, source, notes
e621 e621.net, e926.net content, rating, resolution, tags, tag_category, source
furaffinity www.furaffinity.net content, rating, resolution, tags, tag_category
gelbooru safebooru.org, gelbooru.com, rule34.xxx, tbib.org, xbooru.com content, rating, resolution, tags, tag_category, source, notes Supported features might vary between hosts.
inkbunny inkbunny.net content, rating, resolution, tags
moebooru yande.re, konachan.com content, rating, resolution, tags, tag_category, source, notes
philomena derpibooru.org, trixiebooru.org, ponybooru.org, furbooru.org, ponerpics.org, manebooru.art, twibooru.org content, rating, tags, source
pixiv pixiv.net, www.pixiv.net content
reddit reddit.com, new.reddit.com, old.reddit.com content
rule34us rule34.us content, resolution, tags, tag_category Rating is assumed to be unsafe.
sankakucomplex chan.sankakucomplex.com content, rating, tags, tag_category, notes Hotlink protection currently prevents szurubooru from importing from this site.
shimmie2 rule34.paheal.net, rule34hentai.net content, tags, source Rating is assumed to be unsafe.
shuushuu e-shuushuu.net content, rating, resolution, tags, tag_category Rating is assumed to be safe.
twitter twitter.com, mobile.twitter.com content
zerochan www.zerochan.net content, resolution, tags, tag_category Rating is assumed to be safe.
fallback content Tries to find the largest image or video on the current page.