Detect Google AdSense with Python: From Fast HTTP Scraping to Playwright Stealth

Detecting whether a website is running Google AdSense is a common task for digital marketers, SEO researchers, and competitive intelligence analysts. AdSense works by injecting a specific JavaScript library into the page, accompanied by a unique Publisher ID (formatted as pub-xxxxxxxxxxxxxxxx) and ad container tags (<ins class="adsbygoogle">).

Depending on whether the target website uses static HTML, lazy loading, or Web Application Firewalls (Cloudflare/Akamai), you need different strategies ranging from lightweight HTTP requests to full browser automation with Playwright.

---

Core AdSense Markers and Fingerprints

There are four primary fingerprints an AdSense-enabled site leaves behind:

1. Script Signatures: Script tags loading adsbygoogle.js or pagead2.googlesyndication.com.
2. DOM Containers: The presence of <ins class="adsbygoogle"> ad unit containers.
3. Publisher ID: Regex patterns matching pub-\d{16} in the page source.
4. The ads.txt File: A public file at https://domain.com/ads.txt declaring authorized Google digital sellers (google.com, pub-XXXXXXXXXXXXXXXX, DIRECT, f08c47fec0942fa0).

---

Method 1: Fast Static HTML and ads.txt Verification (requests + BeautifulSoup)

For websites without aggressive bot protection or client-side JavaScript rendering, a lightweight scraper using requests and BeautifulSoup offers the highest throughput.

Requirements

pip install requests beautifulsoup4

Static Detection Script

import requests
from bs4 import BeautifulSoup
import re
from urllib.parse import urlparse

def check_adsense_static(url: str) -> dict:
    if not url.startswith(('http://', 'https://')):
        url = 'https://' + url

    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36'
    }

    results = {
        'url': url,
        'has_script': False,
        'publisher_id': None,
        'has_ads_txt': False,
        'verdict': False
    }

    try:
        # 1. Fetch Homepage HTML
        response = requests.get(url, headers=headers, timeout=10)
        html_content = response.text.lower()

        # Check script signatures
        signatures = ['googlesyndication.com', 'adsbygoogle.js', 'pagead2']
        results['has_script'] = any(sig in html_content for sig in signatures)

        # 2. Extract Publisher ID
        pub_id_match = re.search(r'pub-\d{16}', html_content)
        if pub_id_match:
            results['publisher_id'] = pub_id_match.group(0)

        # 3. Check ads.txt
        domain = urlparse(url).netloc
        ads_txt_url = f"https://{domain}/ads.txt"
        try:
            ads_txt_resp = requests.get(ads_txt_url, headers=headers, timeout=5)
            if ads_txt_resp.status_code == 200 and 'google.com, pub-' in ads_txt_resp.text.lower():
                results['has_ads_txt'] = True
        except requests.RequestException:
            pass

        results['verdict'] = results['has_script'] or results['has_ads_txt'] or bool(results['publisher_id'])

    except requests.RequestException as e:
        print(f"Error fetching {url}: {e}")

    return results

if __name__ == "__main__":
    res = check_adsense_static("https://example.com")
    print(res)

---

Method 2: Dynamic Browser Automation for Protected and Lazy-Loaded Sites (Playwright)

When standard requests scripts fail with 403 Forbidden, Cloudflare challenges, or when ads are loaded via client-side lazy loading (injected only after scrolling), browser automation is required.

Playwright launches a real browser instance, executes JavaScript, simulates user interactions (like scrolling), and inspects the live DOM.

Requirements

pip install playwright playwright-stealth
playwright install chromium

Playwright Detection Script

import asyncio
import re
from playwright.async_api import async_playwright
from playwright_stealth import stealth_async

async def detect_adsense_advanced(url: str):
    async with async_playwright() as p:
        browser = await p.chromium.launch(headless=True)
        context = await browser.new_context(
            user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
            viewport={"width": 1920, "height": 1080}
        )

        page = await context.new_page()
        await stealth_async(page)
        print(f"Navigating to {url}...")

        try:
            # Navigate and wait for network to settle
            await page.goto(url, wait_until="networkidle", timeout=30000)

            # Scroll down to trigger lazy-loaded ad tags
            await page.mouse.wheel(0, 2000)
            await page.wait_for_timeout(2000)

            # 1. Check live DOM for AdSense scripts
            adsense_scripts = await page.locator('script[src*="adsbygoogle"]').count()

            # 2. Check for AdSense container elements
            adsense_ins_tags = await page.locator('ins.adsbygoogle').count()

            # 3. Extract Publisher ID from rendered content
            content = await page.content()
            pub_id_match = re.search(r'pub-\d{16}', content)
            pub_id = pub_id_match.group(0) if pub_id_match else "Not Found"

            print("\nPlaywright Scan Results:")
            print(f"AdSense Scripts Loaded: {adsense_scripts}")
            print(f"Ad Containers (<ins>): {adsense_ins_tags}")
            print(f"Publisher ID: {pub_id}")

            detected = adsense_scripts > 0 or adsense_ins_tags > 0 or pub_id != "Not Found"
            print(f"Verdict: {'AdSense DETECTED' if detected else 'No AdSense found'}")

        except Exception as e:
            print(f"Error during scan: {e}")
        finally:
            await browser.close()

if __name__ == "__main__":
    asyncio.run(detect_adsense_advanced("https://example.com"))

---

Comparison: Static Scraping vs. Playwright

Feature	Static (requests + BS4)	Headless (Playwright)
Execution Speed	Extremely fast (under 500ms)	Slower (2-5s per page)
JavaScript Rendering	No (Static HTML only)	Yes (Full V8 engine)
Lazy-Loaded Ads	Fails to detect	Detects on scroll/wait
Bot Detection / WAF	Vulnerable to 403 blocks	High bypass rate with stealth
Resource Usage	Minimal CPU/RAM	Higher (Browser instance)

---

Detection Accuracy Comparison

Detection Marker	Reliability	Mechanism
ads.txt Verification	Highest	IAB standard file declaring official advertising seller IDs.
Live DOM <ins> & Script	High	Confirms active ad placement tags loaded by browser.
Regex Publisher ID	Medium-High	Matches pub- identifiers within scripts and configuration blocks.
Static HTML Scan	Medium	Quick initial check, susceptible to SPA/lazy-load omissions.

---

Sources & Technical References

• [1.1] Google AdSense Help: How to find your Publisher ID - Official Publisher ID format documentation.
• [2.1] IAB Tech Lab: ads.txt Specification - Authoritative technical standard for ads.txt files.
• [3.1] BeautifulSoup Documentation: Searching the tree - Parsing and locating HTML elements.
• [4.1] Playwright Python Docs: Locators and Selectors - DOM element selection and waiting strategies.
• [5.1] GitHub: playwright-stealth - Evasion techniques for browser automation fingerprints.
• [6.1] Google AdSense Docs: Ad Units Code - Details on <ins class="adsbygoogle"> tags.