Myujth

I am trying to scrape the contents of a webpage so I can check whether a
Stata dataset exists.

I have put together a few lines of code but they don't work:

tempfile page
copy "https://www.stata-press.com/data/r15/u.html" "`page'"
tempname fh
file open `fh' using "`page'", read
file read `fh' line
while r(eof)==0 
if "`line'"=="regsmpl.dta" dis "Dataset exists"
else dis "Dataset doesn't exit"
file read `fh' line

file close `fh'

Any ideas will be highly appreciated.

You could first feed the entire page into a scalar variable using the fileread() function:

local dataset regsmpl
scalar page = fileread("https://www.stata-press.com/data/r15/u.html")

After the scalar is successfully created, you can go about this using two approaches.

Solution 1: Check if the dataset is mentioned in the page

if strmatch(page, "*`dataset'.dta*") display "Page mentions dataset"
else display "No trace of dataset in page"

Solution 2: Check if there is an actual link pointing to the dataset

local link = ustrregexm(page, `"<a [^>]*bhrefs*=s*"([^"]*`dataset'.dta[^"]*)"')
local url = trim(ustrregexs(1))

if "`url'" != "" display "The link is: `url'"
else display "There is no such link"

Your approach can also work using both strmatch() and a regular expression:

tempname fh
file open `fh' using "https://www.stata-press.com/data/r15/u.html", read
file read `fh' line

local tag = 0
while r(eof) == 0 
 if strmatch(`"`line'"', "*regsmpl.dta*") local tag = 1
 file read `fh' line


if `tag' == 1 display "Dataset exists"
else display "Dataset doesn't exit" 

tempname fh
file open `fh' using "https://www.stata-press.com/data/r15/u.html", read
file read `fh' line

local tag = 0
while r(eof) == 0 
 local link = ustrregexm(`"`line'"', `"<a [^>]*bhrefs*=s*"([^"]*`dataset'.dta[^"]*)"')
 if `link' == 1 
 local url = trim(ustrregexs(1))
 local tag = 1
 
 file read `fh' line


if `tag' == 1 display "The link is: `url'"
else display "There is no such link"