# LightQuery 0.7.0
**URL:** https://discourse.julialang.org/t/lightquery-0-7-0/45183
**Category:** Package Announcements
**Tags:** data
**Created:** [August 18, 2020, 11:29pm UTC](https://discourse.julialang.org/t/lightquery-0-7-0/45183 "2020-08-18T23:29:07Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@bramtayl](https://discourse.julialang.org/u/bramtayl)
#### Post date: [August 18, 2020, 11:29pm UTC](https://discourse.julialang.org/t/lightquery-0-7-0/45183/1 "2020-08-18T23:29:07Z")
LighQuery 0.7 is out. It uses Base named tuples and as such is much much easier to use. I’m hoping that to have reached some level of interface stability, and that future changes will focus on performance. You should already find it extremely performant for tables with a small number of columns and no missing data.
See the docs here, which I’ve tried to make a little bit nicer. [Usage and performance notes · LightQuery.jl](https://bramtayl.github.io/LightQuery.jl/dev/)
---
### Author:  [@derekmahar](https://discourse.julialang.org/u/derekmahar)
#### Post date: [August 19, 2020, 1:25pm UTC](https://discourse.julialang.org/t/lightquery-0-7-0/45183/2 "2020-08-19T13:25:14Z")
Thank you for creating this package. Do you plan to implement non-equi-joins?
---
### Author:  [@bramtayl](https://discourse.julialang.org/u/bramtayl)
#### Post date: [August 19, 2020, 1:28pm UTC](https://discourse.julialang.org/t/lightquery-0-7-0/45183/3 "2020-08-19T13:28:57Z")
I’ve thought about it. You can already sort of do it by combining `Iterators.product` and `Iterators.filter`. Is there a way to do a non-equi-join without visiting all possible pairs?
---
### Author:  [@derekmahar](https://discourse.julialang.org/u/derekmahar)
#### Post date: [August 19, 2020, 1:52pm UTC](https://discourse.julialang.org/t/lightquery-0-7-0/45183/4 "2020-08-19T13:52:16Z")
According to [_Join (SQL) - Inner join_](https://en.m.wikipedia.org/wiki/Join_(SQL)#Inner_join), SQL implementations use a [hash join](https://en.m.wikipedia.org/wiki/Hash_join) or [sort-merge](https://en.m.wikipedia.org/wiki/Sort-merge_join) algorithm in order to avoid computing the full Cartesian product of both input tables.
---
### Author:  [@bramtayl](https://discourse.julialang.org/u/bramtayl)
#### Post date: [August 19, 2020, 2:01pm UTC](https://discourse.julialang.org/t/lightquery-0-7-0/45183/5 "2020-08-19T14:01:45Z")
Hmm, well, LightQuery essentially uses a sort-merge, but I’m not sure how to implement it for non-equi-joins (or even if it’s possible).
Consider finding all pairs in a list that sum to 10. 1 and 9 and 9 and 1 would both work, but they are on opposite sides of the list.
---
### Author:  [@derekmahar](https://discourse.julialang.org/u/derekmahar)
#### Post date: [August 19, 2020, 2:23pm UTC](https://discourse.julialang.org/t/lightquery-0-7-0/45183/6 "2020-08-19T14:23:47Z")
How about creating an index for the join-predicate (sum) that points to the corresponding rows in each table that make the predicate true? Create, insert, and delete functions would have to be aware of the join-predicate and maintain the index.
---
### Author:  [@bramtayl](https://discourse.julialang.org/u/bramtayl)
#### Post date: [August 19, 2020, 5:54pm UTC](https://discourse.julialang.org/t/lightquery-0-7-0/45183/7 "2020-08-19T17:54:47Z")
Hmm, still a bit confused but Id be very happy to take a PR
---
### Author:  [@derekmahar](https://discourse.julialang.org/u/derekmahar)
#### Post date: [August 19, 2020, 8:52pm UTC](https://discourse.julialang.org/t/lightquery-0-7-0/45183/8 "2020-08-19T20:52:13Z")
I think I misunderstood your example.
Did you mean, given a sorted list of integers from 1 to 10, find all pairs that sum to 10? In that case, for each item `n` in the list, you could perform a binary search on the same list for item `10 - n` which would require a total of `n log n` visits, or complexity `O(n log n)`.
I agree that deriving a single general algorithm for all cases would be difficult if not impossible. You’d have to use a different algorithm for different scenarios.
---
### Author:  [@bramtayl](https://discourse.julialang.org/u/bramtayl)
#### Post date: [August 19, 2020, 9:04pm UTC](https://discourse.julialang.org/t/lightquery-0-7-0/45183/9 "2020-08-19T21:04:45Z")
I meant roughly
```julia
collect(Iterators.filter(((x, y),) -> x + y == 10, Iterators.product(1:10, 1:10)))
```
---
### Author:  [@derekmahar](https://discourse.julialang.org/u/derekmahar)
#### Post date: [August 19, 2020, 9:58pm UTC](https://discourse.julialang.org/t/lightquery-0-7-0/45183/10 "2020-08-19T21:58:44Z")
Yes, I think that’s what I described. You’re joining together the same list of consecutive integers from 1 to 10 and selecting those pairs whose sum is 10. While you can’t avoid visiting all of the list items at least once, for each of those items, if the list is sorted, you need revisit only `log n` items for a total of `n log n` iterations which is better than the `n^2` iterations that your expression performs. If the list isn’t sorted, the join couldn’t benefit from a binary search and would require `n^2` iterations. If your program had to perform this join multiple times, the join algorithm would benefit from pre-sorting the list and using the sorted list instead.
So, performance of equi-joins and non-equi-joins can both benefit from sorted data. Granted, this is easier to imagine than implement.
---
### Author:  [@bramtayl](https://discourse.julialang.org/u/bramtayl)
#### Post date: [August 20, 2020, 12:55pm UTC](https://discourse.julialang.org/t/lightquery-0-7-0/45183/11 "2020-08-20T12:55:53Z")
It seems to me that the optimizations you could make would depend heavily on the join function being used?
---
### Author:  [@derekmahar](https://discourse.julialang.org/u/derekmahar)
#### Post date: [August 20, 2020, 2:31pm UTC](https://discourse.julialang.org/t/lightquery-0-7-0/45183/12 "2020-08-20T14:31:16Z")
Yes, almost certainly, so it would likely require multiple algorithms to optimise such joins. Clearly, the highest cost-benefit would come from optimising equi-joins since these are most common, but the performance of many non-equi-join conditions might also benefit from equi-join optimisations.
In [DataFrames.jl](https://github.com/JuliaData/DataFrames.jl) issue [#2340](https://github.com/JuliaData/DataFrames.jl/issues/2340), @bkamins and @andyferris are exploring ways using [SplitApplyCombine.jl](https://github.com/JuliaData/SplitApplyCombine.jl) to improve performance of joins between DataFrames. Though it doesn’t address non-equi-joins in particular, @bkamins [commented](https://github.com/JuliaData/DataFrames.jl/issues/2340#issuecomment-667676531) that he does intend to address them in the future.