Our training courses

Book any course as classroom, onsite or online training, or as one-to-one consultancy
Excel Training Basic Excel Advanced Excel VBA Macros Office Scripts Excel Introduction Excel Intermediate Excel Advanced Excel Business Modelling Excel VBA macros Advanced VBA Fast track Excel VBA Basic Office Scripts Advanced Office Scripts Power Platform Training Pure Power BI DAX and fast-track Power Automate Power Apps Introduction to Power BI Adv. Power BI Reports Adv. Power BI Data DAX for Power BI Fast track Power BI Fast track Power BI/DAX Power Automate Desktop Basic Power Automate Advanced Power Automate Fast track Power Automate Power Apps Programming and AI SQL training Visual C# training Python training AI training Introduction to SQL Advanced SQL Fast track SQL Introduction to MySQL Introduction to Visual C# Intermediate C# Fast track C# Introduction to Python Advanced Python Fast track Python Using AI tools Using the OpenAI API SQL Server training Reporting Services Report Builder Integration Services Azure / SSAS Reporting Services Advanced SSRS Fast track SSRS Report Builder Introduction to SSIS Advanced SSIS Fast track SSIS SQL in Azure Data Studio SSAS - Tabular Model
Other training resources

Free resources

Read our blogs, tips and tutorials

Try our exercises or test your skills

Watch our tutorial videos or shorts

Take a self-paced course

Read our recent newsletters

Paid services

License our courseware

Book expert consultancy

Buy our publications

Getting help

Get help in using our site
Our training venues

Where we are

London

Manchester

Where you are

Across the UK

At your site

Online

At your computer
Why we are different

Transparent reviews

551 attributed reviews in the last 3 years

Delivery of courses

Refreshingly small course sizes

Outstandingly good courseware

Whizzy online classrooms

Wise Owl trainers only (no freelancers)

Nicer to work with

Almost no cancellations

We have genuine integrity

We invoice after training

Reliable and established

Review 30+ years of Wise Owl

View our top 100 clients

Search our website

GO
+44 (161) 883 3606 sales@wiseowl.co.uk Web enquiry form
We also send out useful tips in a monthly email newsletter ...

Sign up to newsletter

Our training courses

AI - Articial Intelligence

Power BI and DAX

Other training resources

Tips and tricks

Guides and tutorials

YouTube tutorials

Test your skills

Self-paced courses

Our training venues

Why we are different

Hundreds of published reviews

Small class sizes

Outstanding courseware

Online classroom

(Almost) no cancellations

Wise Owl trainers only

Genuine integrity

Invoices after training

30+ years of Wise Owl

Our top 100 clients

This page has 4 threads | Add post

Video tutorials

Video tutorials

Finding our videos useful? Please consider helping us to share more!

VBA - scraping websites videos | Excel VBA Part 47.5 - Basic HTTP GET and POST Requests

Posted by Andrew Gould on 05 June 2019

Excel VBA Part 47.5 - Basic HTTP GET and POST Requests

This video explains the basic differences between the GET and POST methods and how you can use each method to send data along with an HTTP request. You'll learn about adding a query string to a URL for GET requests and how to send values in the body of a POST request. You'll also see how to ensure the values you pass to a request are properly encoded using the Excel EncodeURL function. Throughout the video you'll also learn the basics of using the Chrome Developer Tools to work out how requests are sent by the Chrome browser which helps when translating this to VBA code.

See our full range of VBA training resources, or test your knowledge of VBA with one of our VBA skills assessment tests.

This video has the following accompanying files:

File name	Type	Description
Basic HTTP GET and POST requests COMPLETED.xlsm	Excel workbook with macros
Basic HTTP GET and POST requests.xlsm	Excel workbook with macros

Click to download a zipped copy of the above files.

There are no exercises for this video.

Making a video bigger

You can increase the size of your video to make it fill the screen like this:

Full screen view

Play your video (the icons shown won't appear until you do), then click on the full screen icon which appears as shown above.

When you've finished viewing a video in full screen mode, just press the Esc key to return to normal view.

Improving the quality of a video

To improve the quality of a video, first click on the Settings icon:

Settings icon

Make sure yoiu're playing your video so that the icons shown above appear, then click on this gear icon.

Choose to change the video quality:

Changing video quality

Click as above to change your video quality.

The higher the number you choose, the better will be your video quality (but the slower the connection speed):

Different speeds

Don't choose the HD option shown unless your connection speed is fast enough to support it!

Is your Wise Owl speaking too slowly (or too quickly)? You can also use the Settings menu above to change your playback speed.

This page has 4 threads

From:	a_sh
When:	29 Mar 22 at 19:41

Hello again and let me ask another question.

I wrote some function to search in header text but I have trouble to find destination url when I use MSXML and search an item in a website search form and that search guides me to a new url.

Is there any way to obtain the destination address? I really need that.

Another side issue is that using html with MSXML doesn't allow us to use two or more getelementby... in one line of the code. for example when I use IE I can write following code:

Set HTMLDiv = HTMLDoc1.getElementsByClassName("Class1")(0).getElementsByClassName(" Class2")(0)

but when using MSXML I have to break it into the following lines:

HTMLDoc2.body.innerHTML = HTMLDoc1.getElementsByClassName("Class1")(0).innerHTML

Set HTMLDiv = HTMLDoc2.getElementsByClassName(" Class2")(0)

Is there a simpler way to write it?

From:	Andrew G
When:	30 Mar 22 at 07:42

Hi!

It sounds as though you're attempting to make use of features of the web page as provided by a web browser - unfortunately an XMLHTTP request doesn't provide that. If you need to interact with search forms and other parts of the web page I'd recommend automating a browser, either IE or Chrome using Selenium (we have several videos on using Selenium to automate Chrome).

I hope it helps!

From:	a_sh
When:	30 Mar 22 at 09:49

Ok thank you. I couldn't believe that there is no way to obtain destination url while using XMLHTTP.

Reply

Reply

Reply

From:	a_sh
When:	29 Mar 22 at 10:25

Hello, thank you for your lessons that I can say are the best among all of the relative tutorials I have seen.

I have a problem with parsing data from the "HEAD" part of html.

by using "HTMLDoc.body.innerHTML = XMLRequest.responseText" we lose this part of html that I exactly need to scrape data from.

I searched the whole internet but I couldn't find the solution.

I need to extract a lot of data from this part so I can't use the InStr function and ...

I would be very grateful if you could help me with this.

From:	Andrew G
When:	30 Mar 22 at 08:24

Hi! Have you tried the getResponseHeader and getAllResponseHeaders methods of the XMLHTTP object?

From:	a_sh
When:	30 Mar 22 at 09:46

I tried but didn't work.

Reply

Reply

Reply

From:	cfunk
When:	07 Oct 21 at 22:14

I'm trying to use this code to login to a website.

VBA Code:

Sub GetTable2() Dim req As New MSXML2.XMLHTTP60 Dim url As String url = "https://1305081.app.netsuite.com/app/common/search/searchresults.nl?searchid=1704&whence=" req.Open "Post", url, False req.setRequestHeader "Content-Type", "application/x-www-form-urlencoded" req.send "email=(myemail)&password=(mypassword)" 'check if page loaded If req.Status <> 200 Then MsgBox req.Status & " - " & req.statusText Exit Sub End If saveHTMFile req.responseText End Sub

When I get to the point of 'check if page loaded, the message box says "Internal Server Error." Any idea what could be wrong?

From:	Andrew G
When:	08 Oct 21 at 07:16

I'm sorry, I don't know.

Reply

Reply

From:	jpduffy3
When:	09 Jul 20 at 01:50

Thank you for your very helpful tutorial entitled "Excel VBA Introduction Part 47.5 - Basic HTTP GET and POST Requests." I have run into a problem that stumps me. Following the tutorial, I get the following result:

***************************
<!doctype html>
<html lang="en">
<head>
<base href="/">
<meta charset="UTF-8">

<meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1,user-scalable=no">
<meta name="mobile-web-app-capable" content="yes">
<title>Profits Run</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:400,500,600,700,800">

<link rel="apple-touch-icon" sizes="60x60" href="/assets/images/favicons/apple-touch-icon.png">
<link rel="icon" type="image/png" sizes="32x32" href="/assets/images/favicons/favicon-32x32.png">
<link rel="icon" type="image/png" sizes="16x16" href="/assets/images/favicons/favicon-16x16.png">
<link rel="manifest" href="/assets/images/favicons/site.webmanifest">
<link rel="mask-icon" href="/assets/images/favicons/safari-pinned-tab.svg" color="#5bbad5">
<meta name="msapplication-TileColor" content="#da532c">
<meta name="theme-color" content="#ffffff">

<link href="app.12d5769f220fe4fa467b.css" rel="stylesheet"></head>
<body>
<div id="root"></div>
<script type="text/javascript" src="vendor-d7a24874.js"></script><script type="text/javascript" src="app-4c8acbc2.js"></script></body>
</html>
***************************
However, when I inspect the elements of the website, I get the information I am looking for. I believe the problem is that the java script is not running, and I have been unable thus far to find out how to make the java script run so that I can scrape the portions of it that interest me.

If you are able to give me a suggestion or a work around, I would be very grateful.

Thank you

From:	Andy B
When:	09 Jul 20 at 09:14

Hi, the easiest way to get the javascript to run is to use VBA to open the page in a web browser. We have a few videos on how to do this using Internet Explorer (which you may have already seen). If you’re interested in using Chrome you might like to investigate the Selenium library. We don’t have any resources on Selenium but you can find a basic tutorial here. I hope that helps!

Reply

Reply

Head office

Kingsmoor House

Railway Street

GLOSSOP

SK13 2AA

London

Landmark Offices

99 Bishopsgate

LONDON

EC2M 3XD

Manchester

Holiday Inn

25 Aytoun Street

MANCHESTER

M1 3AE

Visual C# training

Excel training courses

Power Automate training

Power BI / DAX training

Python training courses

SQL training courses

VBA (macros) training

© Wise Owl Business Solutions Ltd 2024. All Rights Reserved.

Terms & Conditions

End of small page here

Please be aware that our website uses cookies!

I'm OK with this Tell me more ...