Nes
/
neo-scraper
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Updated Features & Added resolution. 

Resolution now supported.
Feature list was updated to reflect both resolution and tag category being used now.
pull/15/head
GlobalGit3 2023-07-03 16:43:39 -05:00
parent 8d215729f3
commit e208d69695
2 changed files with 27 additions and 20 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

38
README.md
View File

@ -4,22 +4,22 @@ Scrape a booru-like post object from the DOM.
## Supported sites ## Supported sites
| Engine | Sites | Features | Notes | | Engine | Sites | Features | Notes |
| -------------- | -------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | ------------------------------------------------------------------------------- | | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | ------------------------------------------------------------- |
| animepictures | anime-pictures.net | content, resolution, tags, tag_category | Rating is assumed to be safe. | | animepictures | anime-pictures.net | content, resolution, tags, tag_category | Rating is assumed to be safe. |
| danbooru | danbooru.donmai.us, safebooru.donmai.us | content, rating, resolution, tags, tag_category, source, notes | | | danbooru | danbooru.donmai.us, safebooru.donmai.us | content, rating, resolution, tags, tag_category, source, notes | |
| e621 | e621.net, e926.net | content, rating, resolution, tags, tag_category, source | | | e621 | e621.net, e926.net | content, rating, resolution, tags, tag_category, source | |
| furaffinity | www.furaffinity.net | content, rating, resolution, tags, tag_category | | | furaffinity | www.furaffinity.net | content, rating, resolution, tags, tag_category | |
| gelbooru | safebooru.org, gelbooru.com, rule34.xxx, tbib.org, xbooru.com | content, rating, resolution, tags, tag_category, source, notes | Supported features might vary between hosts. | | gelbooru | safebooru.org, gelbooru.com, rule34.xxx, tbib.org, xbooru.com | content, rating, resolution, tags, tag_category, source, notes | Supported features might vary between hosts. |
| inkbunny | inkbunny.net | content, rating, resolution, tags | | | inkbunny | inkbunny.net | content, rating, resolution, tags | |
| moebooru | yande.re, konachan.com | content, rating, resolution, tags, tag_category, source, notes | | | moebooru | yande.re, konachan.com | content, rating, resolution, tags, tag_category, source, notes | |
| philomena | derpibooru.org, trixiebooru.org, ponybooru.org, furbooru.org, ponerpics.org, manebooru.art, twibooru.org | content, rating, tags, source | | | philomena | derpibooru.org, trixiebooru.org, ponybooru.org, furbooru.org, ponerpics.org, manebooru.art, twibooru.org | content, rating, resolution, tags, tag_category, source | |
| pixiv | pixiv.net, www.pixiv.net | content | | | pixiv | pixiv.net, www.pixiv.net | content | |
| reddit | reddit.com, new.reddit.com, old.reddit.com | content | | | reddit | reddit.com, new.reddit.com, old.reddit.com | content | |
| rule34us | rule34.us | content, resolution, tags, tag_category | Rating is assumed to be unsafe. | | rule34us | rule34.us | content, resolution, tags, tag_category | Rating is assumed to be unsafe. |
| sankakucomplex | chan.sankakucomplex.com | content, rating, tags, tag_category, notes | Hotlink protection currently prevents szurubooru from importing from this site. | | sankakucomplex | chan.sankakucomplex.com | content, rating, tags, tag_category, notes | |
| shimmie2 | rule34.paheal.net, rule34hentai.net | content, tags, source | Rating is assumed to be unsafe. | | shimmie2 | rule34.paheal.net, rule34hentai.net | content, tags, source | Rating is assumed to be unsafe. |
| shuushuu | e-shuushuu.net | content, rating, resolution, tags, tag_category | Rating is assumed to be safe. | | shuushuu | e-shuushuu.net | content, rating, resolution, tags, tag_category | Rating is assumed to be safe. |
| twitter | twitter.com, mobile.twitter.com | content | | | twitter | twitter.com, mobile.twitter.com | content | |
| zerochan | www.zerochan.net | content, resolution, tags, tag_category | Rating is assumed to be safe. | | zerochan | www.zerochan.net | content, resolution, tags, tag_category | Rating is assumed to be safe. |
| fallback | | content | Tries to find the largest image or video on the current page. | | fallback | | content | Tries to find the largest image or video on the current page. |

9
src/engines/Philomena.ts
View File

@ -3,7 +3,7 @@ import { SafetyRating, TagCategory } from "../BooruTypes.js";
export default class Philomena extends ScrapeEngineBase { export default class Philomena extends ScrapeEngineBase {
name = "Philomena"; name = "Philomena";
features: ScrapeEngineFeature[] = ["content", "rating", "tags", "source"]; features: ScrapeEngineFeature[] = ["content", "rating", "resolution", "tags", "tag_category", "source"];
notes = []; notes = [];
supportedHosts = [ supportedHosts = [
"derpibooru.org", "derpibooru.org",
@ -47,6 +47,13 @@ export default class Philomena extends ScrapeEngineBase {
// Set default rating (Will be determined later) // Set default rating (Will be determined later)
post.rating = "unsafe"; post.rating = "unsafe";
// Set resolution
const resolutionEl = ((<HTMLElement>document.querySelector("span[class='image-size']"))?.innerText).split(" ")[0];
const [width, height] = (resolutionEl?.replace("&nbsp;", "").split("x") || []).map(Number);
if (!isNaN(width) && !isNaN(height)) {
post.resolution = [width, height];
}
// Set tags // Set tags
const tagEls = Array.from(document.querySelectorAll(".tag.dropdown")).map((x) => x as HTMLSpanElement); const tagEls = Array.from(document.querySelectorAll(".tag.dropdown")).map((x) => x as HTMLSpanElement);