# Write to the file the structure of the form (key =\> value) followed by reading using Mmap.mmap

**URL:** <https://discourse.julialang.org/t/write-to-the-file-the-structure-of-the-form-key-value-followed-by-reading-using-mmap-mmap/18680>\
**Category:** General Usage\
**Tags:** mmap\
**Created:** [December 15, 2018, 6:44am UTC](https://discourse.julialang.org/t/write-to-the-file-the-structure-of-the-form-key-value-followed-by-reading-using-mmap-mmap/18680 "2018-12-15T06:44:10Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![1112](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/1112/32/9325_2.png) [@1112](https://discourse.julialang.org/u/1112)\
**Post date:** [December 15, 2018, 6:44am UTC](https://discourse.julialang.org/t/write-to-the-file-the-structure-of-the-form-key-value-followed-by-reading-using-mmap-mmap/18680/1 "2018-12-15T06:44:10Z")

</div>

There are two numbers B and A. You must save their file as [B =\> A]. Then, using Mmap.mmap, read this file and search B, if B is found to get a value A.

Please tell me how to do it. I can not understand how nested arrays in Julia. And as I understand it Mmap.mmap works only with an array.?!

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [December 15, 2018, 7:44am UTC](https://discourse.julialang.org/t/write-to-the-file-the-structure-of-the-form-key-value-followed-by-reading-using-mmap-mmap/18680/2 "2018-12-15T07:44:42Z")

</div>

Is this a homework problem?

---

<div class="post-metadata">

**Author:** ![1112](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/1112/32/9325_2.png) [@1112](https://discourse.julialang.org/u/1112)\
**Post date:** [December 15, 2018, 8:01am UTC](https://discourse.julialang.org/t/write-to-the-file-the-structure-of-the-form-key-value-followed-by-reading-using-mmap-mmap/18680/3 "2018-12-15T08:01:45Z")

</div>

What do you mean by “homework”?  
This task is my personal …

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [December 15, 2018, 8:04am UTC](https://discourse.julialang.org/t/write-to-the-file-the-structure-of-the-form-key-value-followed-by-reading-using-mmap-mmap/18680/4 "2018-12-15T08:04:11Z")

</div>

The wording sounds like a homework problem (“you must”).

Can you post the solution you have tried?

---

<div class="post-metadata">

**Author:** ![1112](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/1112/32/9325_2.png) [@1112](https://discourse.julialang.org/u/1112)\
**Post date:** [December 15, 2018, 8:12am UTC](https://discourse.julialang.org/t/write-to-the-file-the-structure-of-the-form-key-value-followed-by-reading-using-mmap-mmap/18680/5 "2018-12-15T08:12:45Z")

</div>

Understood, this is a translator’s cost, since I speak English poorly …  
Especially I have not tried. this is not what i need

```julia
	setFile = function()
		a = 5
		b = 6
		a1 =7
		b2 =9
		A = Array{Int64,2}
		A = [a b;a1 b1]
		s = open("mmap1.bin", "w+")
		write(s, size(A,1))
		write(s, size(A,2))

		write(s, A)
		close(s)
		
		
	end
	readFile = function()
	# Test by reading it back in
		s = open("mmap1.bin") # default is read-only
		m = read(s, Int)
		n = read(s, Int)
		A2 = Mmap.mmap(s, Matrix{Int}, (m,n))
		println(A2)
		finalize(A2)
		
	end
	setFile()
	readFile()

```

---

<div class="post-metadata">

**Author:** ![1112](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/1112/32/9325_2.png) [@1112](https://discourse.julialang.org/u/1112)\
**Post date:** [December 15, 2018, 8:24am UTC](https://discourse.julialang.org/t/write-to-the-file-the-structure-of-the-form-key-value-followed-by-reading-using-mmap-mmap/18680/6 "2018-12-15T08:24:23Z")

</div>

Need something like this

```julia
	ArBP = Dict();
		B = 8
		A = 12
		ArBA = Dict();
		push!(ArBA,(B=>A))
		s = open("mmap1.bin", "w+")
			write(s, ArBA)
		close(s)
		
		s = open("mmap1.bin"
		A2 = Mmap.mmap(s, Dict(),...)
		if(A2[8])
			return A2[8]
		end	

```

But, mmap does not work with dict.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [December 15, 2018, 8:27am UTC](https://discourse.julialang.org/t/write-to-the-file-the-structure-of-the-form-key-value-followed-by-reading-using-mmap-mmap/18680/7 "2018-12-15T08:27:53Z")

</div>

Use `collect` and `Dict` to convert between `Dict` and a `Vector` of `Pair`s.

```julia
julia> d = Dict(1 => 3, 2 => 4)
Dict{Int64,Int64} with 2 entries:
  2 => 4
  1 => 3

julia> collect(d)
2-element Array{Pair{Int64,Int64},1}:
 2 => 4
 1 => 3

julia> Dict(collect(d))
Dict{Int64,Int64} with 2 entries:
  2 => 4
  1 => 3

```

---

<div class="post-metadata">

**Author:** ![1112](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/1112/32/9325_2.png) [@1112](https://discourse.julialang.org/u/1112)\
**Post date:** [December 15, 2018, 1:09pm UTC](https://discourse.julialang.org/t/write-to-the-file-the-structure-of-the-form-key-value-followed-by-reading-using-mmap-mmap/18680/8 "2018-12-15T13:09:42Z")

</div>

Thank you!  
It really works!

```julia
	setFile = function()
		d = Dict(1324 => 3234, 2879 => 4564)
		A = collect(d)
		s = open("mmap4.bin", "w+")
		write(s, size(A,1))
		write(s, size(A,2))
		write(s, A)
		close(s)
		
	end
	readFile = function()
		s = open("mmap4.bin")
		m = read(s, Int)
		A2 = Mmap.mmap(s, Array{Pair{Int64,Int64},1}, (m,))
		arBp = Dict(collect(A2))
		println(arBp[1324])
		finalize(A2)	
	end
	setFile()
	readFile()

```

Also tell me, with a very large file, about a billion records. Will I have performance problems?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [December 15, 2018, 1:19pm UTC](https://discourse.julialang.org/t/write-to-the-file-the-structure-of-the-form-key-value-followed-by-reading-using-mmap-mmap/18680/9 "2018-12-15T13:19:30Z")

</div>

> [@1112](#):
>
> about a billion records. Will I have performance problems?

It may not be fast, but if you don’t have enough memory then I guess this is the best solution.

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [December 15, 2018, 2:32pm UTC](https://discourse.julialang.org/t/write-to-the-file-the-structure-of-the-form-key-value-followed-by-reading-using-mmap-mmap/18680/10 "2018-12-15T14:32:54Z")

</div>

Wait, no, this is not how memory mapping works. By reading it into a dictionary that way you’ll defeat the whole purpose of memory mapping, and no it won’t work with billions of objects (unless you have an absurd amount of RAM). You probably want to just work with arrays (not dictionaries), and loop over the entire memory mapped array until you find the element you’re looking for. There is a description of memory mapping and some code examples which are very similar to what you want to do [available here](https://docs.julialang.org/en/latest/stdlib/Mmap/index.html).

Note of course that if you’ll be doing this operation more than once, you’d probably want to index or sort the list in some way to avoid the linear time lookup.

---

<div class="post-metadata">

**Author:** ![1112](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/1112/32/9325_2.png) [@1112](https://discourse.julialang.org/u/1112)\
**Post date:** [December 15, 2018, 5:04pm UTC](https://discourse.julialang.org/t/write-to-the-file-the-structure-of-the-form-key-value-followed-by-reading-using-mmap-mmap/18680/11 "2018-12-15T17:04:38Z")

</div>

Indeed, I need to work with arrays, but as I wrote in the first post, I do not understand how to convert the structure I need into an array in Julia. For example, in php I would do this:

```julia
$ar[0][b][0][a]

```

And would have sorted by b

If I understand you correctly, this code will try to load all the data into RAM?

```julia
Dict(collect(A2))

```

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [December 15, 2018, 5:37pm UTC](https://discourse.julialang.org/t/write-to-the-file-the-structure-of-the-form-key-value-followed-by-reading-using-mmap-mmap/18680/12 "2018-12-15T17:37:56Z")

</div>

Yes, it would.

In my response, I was assuming you serialize for storage only.

If you want data larger than memory, and also fast lookup, you need to do something more complicated. Eg you could sort on the key column, then use the `searchsorted*` functions.

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [December 16, 2018, 1:35am UTC](https://discourse.julialang.org/t/write-to-the-file-the-structure-of-the-form-key-value-followed-by-reading-using-mmap-mmap/18680/13 "2018-12-16T01:35:02Z")

</div>

> [@1112](#):
>
> I do not understand how to convert the structure I need into an array in Julia

`A2` in your example is an array already no? So if you need to search for a value, you could loop over the array, row by row, until you find the value you’re looking for.

But what is your use case exactly? For a billion records, this will be a very slow operation. If you’ll be doing repeated reads, you probably want to preprocess your data (e.g. sort the numbers), or perhaps use an existing database.