How can I convert this type of dictionary into a dataframe?

General Usage
1

Hi

I have used a Python SDK to extract data from Splunk.
In the image you can see an example of the output.

image

I want to send that information to a DataFrame.

Any clues will be helpful.

Thank yoi

2

Assuming that you have those in a vector Dv,

vcat(DataFrame.(Dv)...)

would work. You could also convert them to NamedTuples instead of Dicts and then construct the DataFrame.

3

I am iterating. This is the original code:

image
image885Ă—486 117 KB

4

That doesn’t change anything. You can either collect the Dicts into a vector during your loop and use my suggestion afterwards, or you can create one-row DataFrames while you iterate (using df = DataFrame(item)) and collect them into a single one afterwards (again using vcat).

5

This is the second option you can create one-row DataFrames while you iterate:

image

It seems to be working.

But how can I see the dataframe?

6

That is not going to be doing what you want, you aren’t actually accumulating anything. Try this

Dv = Vector{Dict}()
for item in reader
    push!(Dv,item)
end
df=vcat(DataFrame.(Dv)...)

And, just taking a guess here,

df=vcat(DataFrame.(reader)...)

would probably work (since I see you are just printing with the loop anyway).

7

This is what I get

image

8

You need to using DataFrames

9

I think is the same problem I have with Python:
image

JULIA
image

10

Okey. It has worked in Python.
Now I will check Julia.

image
image1375Ă—606 172 KB

11

image

The iteration never accumulates anything.

12

Can you give a MWE with code written in backticks, as follows, rather than screenshots?

```
Your code here
```

It seems like reader might be empty. Having a MWE will help determine that that’s the problem.

2 Likes
13

This is the code. It is now working!!!
I used “collect”.

results=pyimport("splunklib.results")

kwargs_oneshot = (earliest_time= "2019-09-07T12:00:00.000-07:00",
              latest_time= "2019-09-09T12:00:00.000-07:00",
              count=0)

searchquery_oneshot = "search index=iis | lookup geo_BST_ONT longitude as sLongitude, latitude as sLatitude | stats count by featureId | geom geo_BST_ONT allFeatures=True | head 2" 

oneshotsearch_results = service.jobs.oneshot(searchquery_oneshot; kwargs_oneshot...)

# Get the results
reader = results.ResultsReader(oneshotsearch_results)

# collect them into an array
Dv = collect(reader)

using DataFrames
df=vcat(DataFrame.(Dv)...)