How to globally change a data frame inside for loop?

Specific Domains Data
, , ,
1

Hi there!

How to apply changes in for loops to be kept outside of for loop too?
I have a dataframe that is read inside a for loop via a function and and after bunch of computation, I apply a filter command over it and hope to have the changed dataframe for the next iterations of for loop. But the dataframe df does not change. How can I ask julia to keep changes and read the chnaged df at each iteration?

for  i in I
      # read df
      # do some computation...
     elements= find_removeable_indices(x) # this function returns some indices that we want to remove from df later 
    # do some other computation...
    filter!(row -> (row.Index ∉ elements) && row.Price >= current_Price + shift , df)  # df is the original dataframe with 1000 rows and 50 columns. One of the columns is `index`, the other is `Price` and shift is a constant
end
2

Maybe a clean way to do this would be to put the for loop in a function that gives the data frame as return value. But it would help if you could give a complete example: in your code example it’s not clear where the df variable is defined and where it’s used.

3

@sijo The df is defined outside of this for loop and it is read from a csv file.
My understanding of ! is that it should change anything regardless of its local domain. Is that right?
And filter should return those rows from df that met the condition. In this case return those rows in df where their index columns are not in elements and also their price are greater than the current shifted price. Is this a correct interpretation?

Is there any alternative to filter to allow for removing some rows from a dataframe?

4

Indeed an example would be good, in the REPL:

julia> df = DataFrame(x = rand(1_000));

julia> for i ∈ 1:10
           println(nrow(df))
           filter!(:x => >(i/10), df)
           println(nrow(df))
       end
1000
899
899
798
798
695
695
600
600
505
505
393
393
296
296
188
188
92
92
0
1 Like
5

One probable problem derive from reading df internally the for loop, as seams you have done.