[In Development] Bitcoin.jl - A Julia bitcoin library

I have spent the last days testing Channel, Task, TCPSocket, error handling and despite a dirty implementation I have now a much more responsive code to handle network communication.
I have also added a getheaders function which works… The network stack will definitely need to be enhanced before being extended to other functionalities.

Bottom line: I did learn a lot about Julia these last days, and the more it goes, the more I like this language :slight_smile:

7 Likes

v0.1.9 bas been released and now allows for MerkleBlock validation.

I’m now working on bloom filters, but first need to update Murmur3 package to make it compatible with julia 1.1

3 Likes

I did realize another package handling MurmurHash3 was available but did not succeed to make it work, see related issue
On the other hand, I did manage to update Murmur3 package for 32 bits hashes (pull request) which allowed me to implement bloom filters for Bitcoin.jl :slight_smile:

I have also updated network functions which now make use of Channel type and which unfortunately broke node tests while it is all running fine in REPL.

Would anyone have some example using testset macro with Channel? I suspect a variable scope issue somewhere with Channel but can’t figure that out at the moment…

1 Like

With version 0.1.10 just released, Bitcoin.jl adds support for SegWit and brings all function seen in Programming Bitcoin book by Jimmy Song to Julia.

Subsequent version will focus on improving code structure to ease maintenance and improvements. JuliaDB support will then be added to allow for blockchain analysis in pure Julia.

2 Likes

How do you plan to analyze a blockchain with JuliaDB? Just treat it as one very long table? And what sort of analyses did you have in mind?

Maybe a chain of blocks? :chains::troll:

This is a fun project to watch :+1::eyes::blush:

JuliaDB, or any other database, would allow lookups of UTXOs and past transactions by address, calculation of bitcoin days destroyed, transaction value, volume distribution and many other metrics that helps at determining network “health” and potential impact on valuation.

On top of the above use case, blockchain are by design meant to provide security, history security which requires anyone to keep a copy of all blocks. On the other hand all these blocks and transactions may not be relevant if you remove the security aspect, and which allows you to store only relevant information.
Not to mention performance, clearly at the advantage of “classic” databases

I was thinking creating several tables such as blocks, transactions, addresses, UTXOs… I have to think this idea a little further in anycase, but that’s the plan, in big lines

1 Like

If you have the block chain stored locally, that is exactly why I created
https://github.com/gdkrmr/BTCParser.jl

If you indexed the Blockchain, there is also
https://github.com/gdkrmr/LevelDB.jl
I also have a BTCParser.jl integration for this, I am not sure if the code is already public.

1 Like

It is obvious that both Bitcoin.jl and BTCParser.jl do overlap.
Considering my code needs to be restructured, I’d be glad to contribute to and use your parser instead of my implementation.
In any case I would always perform “in house” validation of all blocks and transaction before storing any of them rather than relying on third party software, which is the whole point of bitcoin: don’t trust, verify!
It would also be nice to allow your parser to be used with P2P network data directly and maybe split the parser itself from the file operation functions ?
I’ll have a deeper look and contact you via GitHub accordingly :wink:

With regards to LevelDB, I just did not know about that one but was initially planning on a pure Julia solution. Maybe we could think of common performance tests and compare which of LevelDB or JuliaDB performs better? There’s a lot of work to be done on my side before we can actually compare though :smiley:

Never used JuliaDB, but I think JuliaDB is more like python pandas or R data.frame, LevelDB is a key-value storage — like a dictionary.

v0.1.12 is released bringing along get_headers and get_blockhashbyheight functions.
It also makes use of BitConverter and Secp256k1 packages instead of ECC, which should make it easier to maintain.

Next version should see Bitcoin primitive types such as block and transaction moved out to a stand alone package - so that anyone could use it as a basis for any Bitcoin application.

2 Likes