Node JS Puppeteer throws error at url with proxy

Solution for Node JS Puppeteer throws error at url with proxy
is Given Below:

I’m making a automation script for filling a form online with puppeteer, and to not blacklist ip’s I decided to use proxies for each request, this is the part which gives me error

            console.log(`profile ${ii} started`)
            let proxy = await proxy_rotation(proxy_data, ii);
            console.log("using proxy: ", proxy);
            
            let exec_path = path.resolve(path.dirname(process.execPath) + "/node_modules/puppeteer/.local-chromium/win64-869685/chrome-win/chrome.exe")
            const browser = await puppeteer.launch({ 
                executablePath: exec_path,
                args: ['--disable-web-security']
            });
            const page = await browser.newPage();
            console.log("1");
            await page.setRequestInterception(true);
            await useProxy(page, proxy);
            console.log("2");
            await page.goto(data[ii][0]); //this is where the error gets thrown
        

this part below doesn’t get to run when using a proxy, without it, runs smotthly

            console.log("3");
            await page.type("#name", data[ii][1]);
            await page.type("#yourEmail", data[ii][2]);
            await page.type("#phone", data[ii][3]);
            await page.type("#street", data[ii][4]);
            await page.type("#city", data[ii][5]);
            await page.type("#psc", data[ii][6]);
            await page.select('select#state', data[ii][7]);
            await page.select('select#prefered_size_sel', data[ii][8]);
            await page.$eval('input[name="agreed_personal_info_tiny_contact_form"]', check => check.checked = true);
            await page.evaluate(() => {
                document.querySelector('input[name="agreed_personal_info_tiny_contact_form"]').click();
            });

I just console logged a few numbers, to debug where the script is getting stuck. I also tested the proxy and website I’m trying to access both with a proxy tester and manually, and had no problem accessing it
proxy tester

but when I run my script I get this script error
I understand it pretty much says it cannot access the url, but there should be no reason for that. Do I need to change the way I’m acessing the url when using a proxy? Or add some extra args to the browser? Can I get a more specific error message somehow? Thanks for any suggestions

Also this is the puppeteer function that throws the error

async function navigate(client, url, referrer, frameId) {
            try {
                const response = await client.send('Page.navigate', {
                    url,
                    referrer,
                    frameId,
                });
                ensureNewDocumentNavigation = !!response.loaderId;
                return response.errorText
                    ? new Error(`${response.errorText} at ${url}`)
                    : null;
            }
            catch (error) {
                return error;
            }
        }

That error indicates that something is off how you are using your proxy. Is useProxy your own function or the one from puppeteer-page-proxy? You mention setting the proxy per-request but seems you are setting it for the whole page, is that intentional? The way your proxy is formatted also seems off- check how I do it below.

You can try launching the browser with your proxy server and using page.authenticate() to handle auth. Like this:

let proxy = await proxy_rotation(proxy_data, ii);
const [host, port, username, password] = proxy.split(':');

const parsedProxy = new URL(`http://${username}:${password}@${host}:${port}`);
const browser = await puppeteer.launch({
    executablePath: exec_path,
    args: ['--disable-web-security', '--ignore-certificate-errors', `--proxy-server=${parsedProxy.host}`]
});
const page = await browser.newPage();
await page.authenticate({
    username: parsedProxy.username,
    password: parsedProxy.password,
});

Before doing that I would change what you pass to useProxy such that it looks like http://username:[email protected]:port (Lines 2-3).