Seed URLs List
Domain Processor Control
Running
Background Processor:
Active and monitoring for new domains
Check Interval:
30
seconds
Domain Processor: This service automatically
monitors the TempSeedUrl table for new domains. When a new domain
is detected, it validates the website, checks if it's active, uses
the region from TempSeedUrl, analyzes JavaScript usage, and
inserts valid domains into the GenericWebsite table.
Pending domains: 3277
Auto-restart enabled - processor will automatically start when new domains are added.
Pending domains: 3277
Auto-restart enabled - processor will automatically start when new domains are added.
Total Websites
22999
Crawled
563
JS Heavy
12253
Total PDFs
24175048
Generic Website Data
Status:
Region:
Date Range:
| Domain URL | Status | Region | Spider Name | Crawled | Last Crawl Ended | Total PDFs Found | JS Heavy | Actions |
|---|---|---|---|---|---|---|---|---|
|
https://media.digikey.com
|
DONE | Global | generic_crawler_one_domain | Yes |
2025-12-19
18:46:20
|
0 | No | |
|
https://sentura.de
|
NEW | DE | Not Set | No | Never | 0 | Yes | |
|
https://vivareal.com.br
Query: "Sicherheitsdatenblatt" "08/01/25"
|
NEW | BR | Not Set | No | Never | 0 | Yes | |
|
https://diligent.community
|
NEW | Global | Not Set | No | Never | 0 | Yes | |
|
http://zakulisi.cz
Query: "Sikkerhetsdatablad" "05/09/2024"
|
NEW | CZ | Not Set | No | Never | 0 | Yes | |
|
https://exchangerates.org.uk
Query: "Sikkerhetsdatablad" "27/02/2024"
|
NEW | UK | Not Set | No | Never | 0 | No | |
|
https://intacct.com
Query: "Sikkerhetsdatablad" "20/03/24"
|
NEW | US | Not Set | No | Never | 0 | No | |
|
https://www.kbctools.ca
|
DONE | Global | generic_crawler_one_domain | Yes |
2025-12-20
06:41:35
|
105132
|
No | |
|
https://hygiene-medizinprodukte.de
|
NEW | DE | Not Set | No | Never | 0 | No | |
|
http://media.hiscoinc.com
|
DONE | Global | generic_crawler_one_domain | Yes |
2025-12-20
06:41:43
|
0 | No | |
|
https://neff-gewindetriebe.de
Query: Sicherheitsdatenblatt 10/25/25 filetype:pdf
|
NEW | DE | Not Set | No | Never | 0 | Yes | |
|
https://optima-ph.no
|
DONE | Global | generic_crawler_one_domain | Yes |
2025-12-20
06:56:17
|
638
|
No | |
|
https://qmi.no
|
DONE | Global | generic_crawler_one_domain | Yes |
2025-12-20
08:23:41
|
253
|
No | |
|
https://dealer.bestoilgroup.com
|
DONE | Global | generic_crawler_one_domain | Yes |
2025-12-20
08:42:03
|
2032
|
No | |
|
https://ddb.gov.ph
|
NEW | PH | Not Set | No | Never | 0 | No | |
|
https://cbssports.com
Query: "Sikkerhetsdatablad" "01/05/24"
|
NEW | US | Not Set | No | Never | 0 | Yes | |
|
https://www.permatex.com
|
DONE | Global | generic_crawler_one_domain | Yes |
2025-12-22
05:01:47
|
139244
|
No | |
|
www.tapaniplumbing.com
|
DONE | Global | generic_crawler_one_domain | Yes |
2025-12-22
05:40:10
|
14
|
No | |
|
https://jianshu.com
Query: "Sicherheitsdatenblatt" "12/06/2024"
|
NEW | US | Not Set | No | Never | 0 | Yes | |
|
www.ourprosolutions.com
|
DONE | Global | generic_crawler_one_domain | Yes |
2025-12-22
06:27:37
|
0 | No |
Showing 22941
to 22960
of 22999 websites
Show: