Seed URLs List
Domain Processor Control
Running
Background Processor:
Active and monitoring for new domains
Check Interval:
30
seconds
Domain Processor: This service automatically
monitors the TempSeedUrl table for new domains. When a new domain
is detected, it validates the website, checks if it's active, uses
the region from TempSeedUrl, analyzes JavaScript usage, and
inserts valid domains into the GenericWebsite table.
Pending domains: 3277
Auto-restart enabled - processor will automatically start when new domains are added.
Pending domains: 3277
Auto-restart enabled - processor will automatically start when new domains are added.
Total Websites
22999
Crawled
563
JS Heavy
12253
Total PDFs
24175048
Generic Website Data
Status:
Region:
Date Range:
| Domain URL | Status | Region | Spider Name | Crawled | Last Crawl Ended | Total PDFs Found | JS Heavy | Actions |
|---|---|---|---|---|---|---|---|---|
|
https://skive.dk
|
NEW | DK | Not Set | No | Never | 0 | No | |
|
https://ieee.org
|
NEW | US | Not Set | No | Never | 0 | No | |
|
www.gemini-coatings.com
|
DONE | Global | generic_crawler_one_domain | Yes |
2025-12-23
04:09:35
|
19753
|
No | |
|
eazygleam.com.au
|
DONE | Global | generic_crawler_one_domain | Yes |
2025-12-23
05:35:33
|
2591
|
No | |
|
shopsense-online.co.uk
|
DONE | Global | generic_crawler_one_domain | Yes |
2025-12-23
05:44:12
|
0 | No | |
|
https://ichgcp.net
Query: "Sikkerhetsdatablad" "09/05/2025"
|
NEW | US | Not Set | No | Never | 0 | No | |
|
www.midwestbusparts.com
|
DONE | Global | threaded_playwright_spider | Yes |
2025-12-23
09:01:57
|
0 | No | |
|
biosoilsolutions.com.au
|
DONE | Global | threaded_playwright_spider | Yes |
2025-12-23
09:02:17
|
0 | No | |
|
cloroxautodish.com
|
DONE | Global | threaded_playwright_spider | Yes |
2025-12-23
09:02:32
|
0 | No | |
|
interscience.cn
|
DONE | Global | threaded_playwright_spider | Yes |
2025-12-23
09:02:52
|
0 | No | |
|
mk0baladhesives6cm53.kinstacdn.com
|
DONE | Global | threaded_playwright_spider | Yes |
2025-12-23
09:03:07
|
0 | No | |
|
shop.chequerscontracts.co.uk
|
DONE | Global | threaded_playwright_spider | Yes |
2025-12-23
11:24:46
|
0 | No | |
|
www.biobasic.com
|
DONE | Global | generic_crawler_one_domain | Yes |
2025-12-23
15:22:20
|
188284
|
No | |
|
https://cma-cgm.com
Query: safety data sheet 30/04/25 filetype:pdf
|
NEW | FR | Not Set | No | Never | 0 | No | |
|
www.crystal-clean.com
|
DONE | Global | generic_crawler_one_domain | Yes |
2025-12-23
16:32:42
|
4782
|
No | |
|
https://hm.com
Query: "Sicherheitsdatenblatt" "13/08/24"
|
NEW | SE | Not Set | No | Never | 0 | Yes | |
|
mutualscrew.com
|
DONE | Global | generic_crawler_one_domain | Yes |
2025-12-23
18:42:46
|
103670
|
No | |
|
https://usdoj.gov
Query: "safety data sheet" "07/20/24"
|
NEW | Global | Not Set | No | Never | 0 | No | |
|
www.mgnewell.com
|
DONE | Global | generic_crawler_one_domain | Yes |
2025-12-23
20:55:52
|
1857
|
No | |
|
www.pecosales.com
|
PROCESSING | Global | Not Set | No | Never | 0 | No |
Showing 22961
to 22980
of 22999 websites
Show: