# Julia-Gumbo-webscraping
**URL:** https://discourse.julialang.org/t/julia-gumbo-webscraping/30203
**Category:** Data
**Tags:** question
**Created:** [October 23, 2019, 5:03am UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203 "2019-10-23T05:03:26Z")
**Posts on this page:** 17
**Page:** 1
### Author:  [@Anil\_Mathews](https://discourse.julialang.org/u/Anil_Mathews)
#### Post date: [October 23, 2019, 5:03am UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/1 "2019-10-23T05:03:26Z")
Is it possible to convert structure type to julia string type.
i have scraped some data from website with Gumbo.jl and i want to push this data to a Array but when i do that i get this error:Cannot `convert` an object of type Array{Any,1} to an object of type HTMLNode
---
### Author:  [@onetonfoot](https://discourse.julialang.org/u/onetonfoot)
#### Post date: [October 23, 2019, 6:19am UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/2 "2019-10-23T06:19:36Z")
Should be possible, can you post a snippet of code? It’ll be easier to help you
---
### Author:  [@Anil\_Mathews](https://discourse.julialang.org/u/Anil_Mathews)
#### Post date: [October 23, 2019, 7:59am UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/3 "2019-10-23T07:59:13Z")
I has been resolved.
But could you please tell me what is HTMLNode in Gumbo.jl HTML Type.
---
### Author:  [@onetonfoot](https://discourse.julialang.org/u/onetonfoot)
#### Post date: [October 23, 2019, 8:11am UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/4 "2019-10-23T08:11:16Z")
It’s an [abstract type](https://docs.julialang.org/en/v1/manual/types/#Abstract-Types-1), you can see it’s subtypes using:
```julia
subtypes(HTMLNode)
3-element Array{Any,1}:
HTMLElement
HTMLText
NullNode
```
You might also find this [link](https://en.wikibooks.org/wiki/Introducing_Julia/Types) on types useful
---
### Author:  [@Anil\_Mathews](https://discourse.julialang.org/u/Anil_Mathews)
#### Post date: [October 23, 2019, 8:20am UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/5 "2019-10-23T08:20:05Z")
Sir, i have scraped some data with gumbo.jl and i want to convert data to string. I am not able to do that.How can i convert?
```julia
using HTTP
using Gumbo
using Cascadia
using DataFrames
Base_Url="https://www.moneycontrol.com/india/stockpricequote/A"
Active_Url=Base_Url
println(Active_Url)
h=HTTP.request("GET",Active_Url)
#println(h)
html=parsehtml(String(h))
From_Website=eachmatch(sel".bl_12",html.root)
#println(From_Website)
Name=From_Website[5][1]
println(Name)
a=String(Name)
```
---
### Author:  [@onetonfoot](https://discourse.julialang.org/u/onetonfoot)
#### Post date: [October 23, 2019, 9:28am UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/6 "2019-10-23T09:28:42Z")
```julia
using HTTP, Gumbo, Cascadia
url = "https://www.moneycontrol.com/india/stockpricequote/A"
response = HTTP.get(url)
html = response.body |> String |> parsehtml
elements = eachmatch(sel".bl_12", html.root)
elements[5][1].text
```
---
### Author:  [@Anil\_Mathews](https://discourse.julialang.org/u/Anil_Mathews)
#### Post date: [October 23, 2019, 9:33am UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/7 "2019-10-23T09:33:09Z")
Could you please explain this
html = response.body |\> String |\> parsehtml
---
### Author:  [@kevbonham](https://discourse.julialang.org/u/kevbonham)
#### Post date: [October 23, 2019, 10:41am UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/8 "2019-10-23T10:41:12Z")
Hi @Anil_Mathews, please take a look at [this post](https://discourse.julialang.org/t/psa-make-it-easier-to-help-you/14757), which in particular provides information on how to get the code in your parts to be formatted (makes it easier to read). There’s some other useful information about how to write questions to make it easier to help you.
> [@Anil\_Mathews](#):
>
> Could you please explain this
> `html = response.body |> String |> parsehtml`
This is pipe syntax, and is equivalent to `html = parsehtml(String(response.body))`. [Here’s the relevant](https://docs.julialang.org/en/v1/manual/functions/index.html#Function-composition-and-piping-1) section of the manual.
---
### Author:  [@Anil\_Mathews](https://discourse.julialang.org/u/Anil_Mathews)
#### Post date: [October 23, 2019, 10:59am UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/9 "2019-10-23T10:59:52Z")
Sir, I want scrape all company names from a site. When i the second for loop is not working and .text is no showing no object text
using HTTP
using Gumbo
using Cascadia
using DataFrames
using StatsBase
letters=[‘A’, ‘B’, ‘C’, ‘D’, ‘E’, ‘F’, ‘G’, ‘H’, ‘I’, ‘J’, ‘K’, ‘L’, ‘M’, ‘N’, ‘O’, ‘P’, ‘Q’, ‘R’, ‘S’, ‘T’, ‘U’, ‘V’, ‘W’, ‘X’, ‘Y’, ‘Z’]
#print(letters)
list=
Base\_Url=“[Stock Quotes|Company Stock Price quotes|NSE/ BSE Listed Company Stocks|Indian Stock Market](https://www.moneycontrol.com/india/stockpricequote/)”
lst=Array{String,1}()
for letter in letters
url=Base\_Url\*letter
response = HTTP.get(url)
html = response.body |\> String |\> parsehtml
elements= eachmatch(sel".bl\_12", html.root)
```
for ele in [3:length(elements)]
print(ele)
a=elements[ele][1].text
push!(lst,a)
end
```
end
---
### Author:  [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [October 23, 2019, 2:49pm UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/10 "2019-10-23T14:49:11Z")
Please quote your code:
> [@Please read: make it easier to help you](https://discourse.julialang.org/t/psa-make-it-easier-to-help-you/14757):
>
> Welcome to the Julia Discourse! We are enthusiastic about helping Julia programmers, both beginner and experienced. This public service announcement (PSA) outlines best practices when asking for help. Following these points makes it easier for us to help you and more likely you’ll get a prompt, useful answer. Keywords are highlighted to make it easier to refer to specific points. Choose a descriptive title that captures the key part of your question, eg “plots with multiple axes” instead of …
Normally this is a matter of etiquette and getting help more effectively, but in this case because you link to an external site, it’s getting you flagged as spam.
---
### Author:  [@Anil\_Mathews](https://discourse.julialang.org/u/Anil_Mathews)
#### Post date: [October 23, 2019, 3:40pm UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/11 "2019-10-23T15:40:31Z")
I want to create Structure to store my details
````nohighlight
name:: String
age ::int
Father_Name:: String
end```
Then, I want to store these data in Array type anil
```data= Array{Anil,1}("anil",22,"Mathew)```
Unfortunately, it's wrong, how to code
2)how to find a particular type from an array?
From my data Array, i want to find String only, how to do that?
````
---
### Author:  [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [October 23, 2019, 4:02pm UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/12 "2019-10-23T16:02:12Z")
Again, please quote your code.
---
### Author:  [@Anil\_Mathews](https://discourse.julialang.org/u/Anil_Mathews)
#### Post date: [October 23, 2019, 4:30pm UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/13 "2019-10-23T16:30:03Z")
I was scraping company names from
````julia
using HTTP, Gumbo, Cascadia```
```url = "https://www.moneycontrol.com/india/stockpricequote/A" ```
#getting the data
```response = HTTP.get(url)```
#parsing the data
```html = response.body |> String |> parsehtml```
#the page contain 747 elements
```elements = eachmatch(sel".bl_12", html.root)```
#iterating each to get company name
```for ele in Array{Int,1}(range(3,length(elements)))
try
println(ele,"========", elements[ele][1].text)
finally
break
end
end```
#But the problem is Array element is 747,However, after 745 element left two are 0 element. So, the loop breaks. I want to manage that if the 746 element is None break and iterate another page. Please help me.
````
---
### Author:  [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [October 23, 2019, 4:50pm UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/14 "2019-10-23T16:50:24Z")
That’s still not properly quoted I’m afraid and the system is telling us (mods) that you’re still spamming… Try backticks: it’s the character in the upper left corner of most keyboards.
---
### Author:  [@BLI](https://discourse.julialang.org/u/BLI)
#### Post date: [October 23, 2019, 7:50pm UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/15 "2019-10-23T19:50:21Z")
“Quote” the code doesn’t mean to use the quotation symbol ". If you insert code _in-line_, “quote the code” means to insert the code between _a pair of back-ticks_ , e.g., leading to `using Gumbo;` .
If you have longer code that you want to _display_, “quote the code” means to insert the code between a _pair of lines_ with triple back-ticks, e.g.:
```julia
using Gumbo;
x = range(0,2pi,length=100)
```
I don’t know whether it is mandatory, but you can specify the programming language at the end of the first line with triple back-tics, e.g., you add `julia` just after the triple back-ticks of the first line.
Such “quotation of the code” with back-ticks makes your questions much more readable, and increases the chance of someone deciphering your code.
---
### Author:  [@BLI](https://discourse.julialang.org/u/BLI)
#### Post date: [October 23, 2019, 7:56pm UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/16 "2019-10-23T19:56:44Z")
… to be more precise, here is the back-ticking… `using Gumbo;`… is set up by:
while the displayed code,
```julia
using Gumbo;
x = range(0,2pi,length=100
```
is set up by:
---
### Author:  [@kevbonham](https://discourse.julialang.org/u/kevbonham)
#### Post date: [October 24, 2019, 1:54pm UTC](https://discourse.julialang.org/t/julia-gumbo-webscraping/30203/17 "2019-10-24T13:54:16Z")
> [@BLI](#):
>
> … to be more precise, here is the back-ticking… `using Gumbo;` … is set up by:
> 
Meta help with code formatting - in discourse (though not all markdown-based things), you can actually use 4 backticks to get formatting of triple backticks:
`````julia
````
```
using Gumbo;
x = range(0,2pi,length=100
```
````
`````
(note: in the source, the above is surrounded by 5 backticks… one can do this _ad infinitum_)
For in-line formatting, you can surround single-back ticks in double backticks. “This is a ``test``” is achieved with `"This is a `` `test` ``"`
EDIT: I didn’t realize, but Tamas made a detailed post about this, [linked in the PSA above](https://discourse.julialang.org/t/psa-how-to-quote-code-with-backticks/7530)
> [@Anil\_Mathews](#):
>
> Sir, I want scrape all company names from a site. When i the second for loop is not working and .text is no showing no object text
Please do read the post that Stefan and I linked you to - I know when you’re trying to solve a particular problem, it can be frustrating to be sent down seemingly unrelated paths, but taking a little time now to learn how to ask questions in a way that will make it easier for people on this forum to help you will pay off in the long run, I promise.
A couple of other points of etiquette in addition to quoting code:
1. it’s also good practice to ask separate questions in separate threads. If your original question was answered, you should ask follow-ups in separate threads.
2. Use a title that reflects the question you’re asking (you can actually edit the title after the fact if you wish). Your first question is not really about web scraping, but rather about type conversion
3. Only one post can be marked as the solution. Since my post was the answer to a follow up, you really shouldn’t mark it as the answer. One of @onetonfoot’s answers would be more appropriate.