State of DSP?


#1

looks like it might be in need of some help

  • the examples could be improved.

  • it’s not currently running any tests for 0.5

i’m volunteering :slight_smile:


#2

I think it does run the tests for 0.5 and even 0.6, just that the badges are not displayed on Github:
http://pkg.julialang.org/detail/DSP.html
You could make a PR for that, and also add examples.


#3

Thanks, I’ll get started !

I do a fair amount of DSP work and I am looking forward to contributing.


#4

Is Image Processing included in Signal Processing or are they separated (As I don’t see a sub forum dedicated to Image Processing)?


#5

Images.jl
Pkg.add("Images")
Not complete and with performance sometimes worse than Matlab (bwlabel regionprops)
But a good basis for developing other Image Processing operations.

So it is a good place to contribute in mosaicing, optical flow, registrations, object detection, bilateral filtering etc
+
There are several deep learning toolboxes , which is the main focus of the image processing world lately.
+
There is levenberg-marquet somewhere in the Optimization packages

Good luck , and it will be interesting to hear about your experience with it with regards to algorithmic RnD


#6

Open an issue for those?


#7

How come there is no dedicated Sub Forum for Image Processing?


#8

@RoyiAvital, see the discussion here: Creation of Images domain.

I do think that an image processing domain would be worthwhile, but there hasn’t been a large volume of discussion in all of the JuliaLang Discourse regarding image processing or computer vision. It could be a chicken and egg thing–there’s not much discussion because there’s no forum, and there’s no forum because there hasn’t been much discussion. But it could also be that not many people are using Julia for image processing yet (despite the, IMHO, very well designed Images.jl and related packages).

Cheers,
Kevin


#9

There are a number of packages related to Images.jl that have been getting a bit of work recently, especially from Summer of Code participants. Check out the packages under https://github.com/JuliaImages. In particular, ImageFeatures.jl and ImageSegmentation.jl have been getting a lot of new algorithms.

(There is still a lot missing compared with, e.g., OpenCV, so contributions are still needed/welcome.)

Cheers,
Kevin


#10

@kevin.squire, Well I think that since there is a Signal Processing forum with low traffic, to the least Image Processing is entitled to have Low Traffic (Sub) Forum :-).

I’d be happy to add filters and functions.
Yet I would write it with Single / Float array in mind which I think won’t be good enough.

Thank You.


#11

The best thing would be if we could have a “Julia code exchange” a place to share codes and algorithms even if they are partial and do not cover the whole range of possibilities.

If you wrote something for array of Float32 which is useful for me but I need it to support other types , it’s fairly easy.
Or someone might get inspired to take an algo and cater for all the corner cases and publish it as part of Package.

We can benefit greatly if we share code and algorithms that do not yet warrant a new package or are in state clean enough to publish as a package.

In works great in Matlab file exchange.

Preferably each shared code will be in file or will have a single file to include, and some documentation on what and how.

By the way , algorithms written to support only single float are way easier to understand and modify than algorithms that have extra spaghetti of overloads to cater for all cases.
Actually algorithms are better shared that way …


#12

Github exists. It is free, it’s setup with version control, lets you share associated data and figures, and discuss the code in detail. It also pushes you to license your code. You also get continuous integration testing so that way you know the code continues to work, and Julia has functions so setting this up takes about 2 minutes.

If you don’t want to use Github, then you can share code via Gists and get many of the benefits as well.

Lots of codes on the MATLAB file exchange are not appropriately licensed, and that’s not easily recognized before downloading and opening up files. Thus the MATLAB file exchange is a great way to have people accidentally break laws… there’s no way to track bugs, no continuous integration testing, etc. This is objectively a huge step backwards in just about every way from what we currently have.

Nothing about this requires a new service. Github/Gists does all of this fine.

The reason why more people don’t share the simple code is because, as you noted, it’s really easy to support more types: you just do things like change z = 0.0 to z = zero(typeof(x)). Usually there’s no extra types or lines of code even needed to do this (except for supporting units… and StaticArrays…). But this doesn’t stop you or others from sharing codes which only work on Float64.


#13

@ChrisRackauckas I appreciate your good intentions , but I find it annoying that you try to educate me on GitHub or Gist , without giving me the credit that I might just know about those and still aiming at something different.


#14

@TsurHerman, I totally agree with you.
It would be great to have a place to share code which isn’t validated to to the level of Package.

It comes from my preference of having things operate on simple arrays (Float / Double).
I’m not a programmer, I don’t like using all those “Elegant” abstractions.
For me, the best code is the one easiest to first day programmer to understand.

This is why I never liked Python.
What most heavy users of the language consider “Elegant” I find “Being over smart”.
It is completely personal thing, but I like the simplicity of MATLAB approach over Python “Flexibility”.

@ChrisRackauckas, It seems you missed the whole point.
The idea is to have centralized place to share code simple as that (Not to manage code, just to let other, Hi I have a specific function doing this and that, you may use it…).
The need is for a centralized platform which Julia users can go and find and not only store in public (as GitHub).
Another thing is that you always suggest people something instead of their ideas, why is that?
Give people credit they know exactly what they want.


#15

This is a description of Github. Sure, Julia puts packages there. But most Git repos are just code placed on a centralized public platform for others to search for.

By pointing out that every feature you described is part Github and its normal uses, it’s clear that if you did not mean to describe Github then the description is not complete. It sounds like you want something different, but I have no idea how what you’re thinking of is supposed to be different. I am curious to hear what a service would be, which is why I made the comparison for clarification. I was hoping that such a comparison to the current solution would bring out the missing details for how it’s supposed to be different.

BTW, the thread should probably be split at State of DSP? post 11 (title Julia Code Exchange) since this is all off-topic from the original discussion.


#16

Just my 2 cents on this. I think we can all agree that there is a lot of code out there that is using existing packages for certain purposes but is not necessarily big or mature enough to turn into a fancy package. I suppose the impression people have (me included) of Github is that it has this high standard of code quality and accountability, with a whole system for version control, forking, issues and pull requests to facilitate group and open source package development. While this may sound like heaven to a developer or someone who enjoys impersonating one every now and then (like me), it may be just a tad bit more than what other people are comfortable dealing with, at least for their day-to-day potentially one-time-use pieces of code.


#17

Since I follow a lot of this stuff very closely, I do add a bit of interpretation. Take the example above about “There is levenberg-marquet somewhere in the Optimization packages” where I suggested LsqFit.jl. The Levenberg-Marquet solver was in Optim.jl with the optimization packages, but was recently moved to LsqFit.jl, and Optim.jl + LsqFit.jl were moved out of JuliaOpt and to JuliaNLSolvers even more recently. So while at face value I may have responded to a question looking for LM in an optimization package with a least-squares fitting library in an org that is named after nonlinear solvers, that really is the answer.

Other times I suggest something based on the description which is given, but the description is incomplete which then spawns a productive conversation about what the current approaches are missing. This is a nice way to really pinpoint what the community is looking for and build new solutions. But even then, sometimes I’m just wrong and I’m sorry about that.


#18

It’s interesting to hear this because one of the issues with a Github-based system is that it doesn’t have a standard for code quality and it doesn’t have any accountability. Developers have asked for things like “curated packages” because of this lack of accountability in Github repos, and currently only “registered packages” have a (minimum) level of accountability that is enforced during the tagging process (tests need to exist and pass, and a few common errors are checked for).

It doesn’t require using Git at all either: you can just paste code into the web browser, or upload using the web interface. Gists are even quicker and made to be solely used through the browser.

But even then, the other example which was brought up, the MATLAB File Exchange, has things like version control. That’s actually an interesting case because these days it seems more and more of the code on the MATLAB File Exchange is actually hosted on Github, and then the file exchange is just a search engine for that. But if that’s what’s being asked for, then Julia Observer pretty much fits that bill since @djsegal has recently added unregistered repos to its searches.

https://juliaobserver.com/

That makes that the main difference between the MATLAB File Exchange and Julia Observer is that Julia Observer is auto-generated instead of requiring people to opt into having their repo shown in the search results.


#19

Well I suppose people’s preferences and impressions don’t change that easily! I believe we get that impression from seeing so many well written and maintained packages on Github which may be a little offputting when comparing my totally unoptimized one-time-use code to. So yes I get that Github is a no strings attached platform but perhaps the impression is that it is not. And I believe what further supports this impression is that the GUI is usually treated as the exception rather than the rule, so there is that additional alienation factor. Anyways I was just giving my opinion, I am personally comfortable enough with Github, but I won’t mind searching a slightly less formal platform for more pieces of code and hacks that I might need in my work.


#20

@ChrisRackauckas
To tell you the truth , when I wrote about levenberg marquedt can be found “somewhere” , I didn’t mean I don’t know where it is , I meant: I know where it is , you can find it too using google , and the search is beneficial as it exposes you to more tools that might come handy.

Anyway I will not dive into psychology of communication and cultural differences.
I appreciate the honest effort you make constantly.

Regarding julia exchange:

I imagine a centralized space to post codes/algorithms , in any state, without the hassle of source control.
Each upload can contain several files but it shall be common practice that there is a single file to include to enable the functionality.
And it is preferd that the code will be contained in one file if possible.

There shall be a readme describing functionality with examples.
The code may contain additional data that is not code upto some limit( for example test images)

Although the functionality I describe is available through some hassle with github or bitbucket or whatever.

It differs in feeling , in GitHub I own every repository I create and maintain … it feels an overkill for a single file conatining template matching code.

It will create another deeper form of interaction between users , and it may help many people in their first steps.

It is a valuable tool when using some language for RnD. Especially in the field of algorithmic research were we disect modify compose and simplify other algorithms.

Most of the published papers and algorithms are useless or meaningless not useful in real world applications .I often just want to check something. quick and dirty.I don’t need a package for that .

I think it is a good idea that may help the language spread , I think Julia is lost in the vastness of stack overflow github space