I’ve started a GitHub wiki geared towards MATLAB users wanting to migrate to Julia. It’s organized around mapping functionality from MATLAB (including the core language as well as the various add-on toolboxes) to the equivalent Python functionalities. Right now it’s largely just a skeleton but as I learn about Julia I’ll keep adding new content. Editing is allowed by default (you just need to be logged in to GitHub) and contributions from the community are of course very welcome!
Wiki is great and all, but when it is down to the function mapping it would be nice to have a script like MATLAB to Julia translator | MATLAB to Julia converter instead of going through the process manually.
5 Likes
Very cool, I wasn’t aware of that resource!
As I’ve been populating the wiki it has occurred to me that I could use the ‘main’ part of the repo to host a proper Julia package that would allow the wiki’s information to be accessed from the Julia REPL. While it wouldn’t be very useful unless/until the wiki has a lot more actual content it seems like a fun learning exercise.
If anyone is interested: Actual Julia package to access wiki entries · Issue #1 · pbouffard/matlabtojulia · GitHub
Is wiki really the way to go these days? Why not just have it in markdown, let github display it, and then people can give PRs, etc.?
1 Like
Thanks for the feedback. I gave this a bit of thought before starting the wiki and decided to err towards making it very easy to contribute; this way all that is required is a github account. Quite a lot of users of MATLAB have never used git or know what a PR is.
It would be pretty easy to migrate to a more standard repo in the future, though.
Actually, come to think of it since a github wiki is just a git repo itself (albeit one that, AFAICT, can’t be accessed via the usual github interface) it should be possible to create another “normal” github repo which is a clone of the wiki one and accept PRs into it.
Just playing around with that idea now it’s not hard to set up but there are some drawbacks to it. The main one seems to be that for wiki-internal links you still have to use wiki-style links which the regular github markdown renderer can’t handle, so you just see a bunch of [[PageName]] when viewing the markdown in the cloned repo. It’s actually pretty frustrating since the wiki syntax they use is so close to the markdown syntax that will render properly in .md files but not close enough. Also there are some features of the wiki mode that are nice, e.g. the list of pages (and ability to search by page name). Finally keeping the ‘shadow’ git repo for the wiki in sync with the actual wiki requires manual intervention (pulling from the shadow repo to a local clone and then pushing to the ‘real’ wiki repo).
Overall I think the wiki is a bit more accessible to more people and since the idea of this is to be something widely usable I think I’ll stick with the github wiki for now. As the Home page of the wiki mentions if someone has bulk changes to make they can open an issue to discuss the best way to do so.
All this being said I just defaulted to GitHub since it was close at hand and I’ll admit I didn’t really check if there were better options that might allow both a wiki-type easy editing experience and the option to use PRs. A few minutes of searching for something just now didn’t turn up anything for me but if you know of something like this, I’d be interested to hear about it!
4 Likes
Ideally, I feel that if the community can make a MATLAB to Julia transpiler (like this MATLAB to NumPy transpiler: https://github.com/victorlei/smop), it’s gonna be a killer app. MATLAB is closed source, but Octave, which shares perhaps 95% of the syntax with MATLAB, may be a good starting point for implementing the parser in Julia.
2 Likes
I’d share your enthusiasm about transpilation if I thought anything like that was remotely possible but I’m going to have to sound like a wet blanket here. First of all even if 95% of Octave syntax truly is the same as MATLAB (a claim that I think may be dubious and really one should define exactly what metric is being used here), that last 5% isn’t to be trifled with. Octave is a nice project and I have a lot of respect for its contributors and the work they’ve done, but frankly it just doesn’t move the needle, especially not in most industrial settings. Well, fine–I guess I can only say that for sectors that I am familiar with, but if there’s a big sector of industry that uses much more Octave than they do MATLAB then I’d love to hear about it! I mean, even in academic settings you see MATLAB used way more often that Octave (again, in my experience, YMMV).
Secondly remember that a huge part of why Mathworks has the stranglehold on the market that they do is the toolboxes. These contain much of the functionality that organizations that are willing to plunk down some serious cash for software are making those make vs. buy decisions about. And the reason that they do this is that either 1) the engineers they have can’t replicate the functionality they can buy for less NRE time than the cost of just buying the licenses, or 2) no amount of NRE using the skills that are available to the organization (it’s a tight market for engineering talent) would result in the functionality being replicated. Usually probably a a combination of both. How would a transpiler stack up against the toolbox functionality? It’s not always implemented solely in MATLAB and by the way even when it is it often isn’t visible to the end users (.p files…).
All that aside I’d assume that the kind of information that could be captured in the wiki (“hey, I know how to do this in MATLAB, how would I go about doing it in Julia?”) would be quite useful to many before such a transpiler were built, and to the builders of the transpiler as well. So it seems like a useful first step regardless.
(NRE = non-recurring engineering [costs], if you’re not familiar with that one)
2 Likes
I also think this Wiki is a really good idea and would be useful for lots of people coming from Matlab.
You are perfectly right that Matlab has still a stranglehold in industry and academia partly because of the large functionality of their toolboxes, which is not available in Octave or other alternatives.
Having documentation in a single place which Matlab toolboxes are covered (not at all, partly or fully) by which Julia ones, is already of great value.
We had a related discussion in the thread https://discourse.julialang.org/t/designated-target-audience-of-julia-1-0/11804/94 where I argued along similar lines to yours why Matlab is not dead yet and why it is not so easy to fully replace it with Octave, R, Python or Julia.
1 Like
In my wiki at http://julia.cookbook.tips/doku.php, just before “Background.” I have a placeholder for migration, too. I used dokuwiki with markdown, but I am happier to point to your wiki for this.
we could use similar wikis and skeleton transpilers for python, perl, and R, too. it would be great if a transpiler could just do the easy and tedious manual mappings, even if the code still needed hand-massaging.
I think I agree with you on pretty much all the issues you pointed out. Maybe a transpiler isn’t a feasible idea. As far as I know, there isn’t a big industry depending on Octave. I myself use MATLAB from a subscription paid by my department, and I only occasionally use Octave, like when I have trouble connecting to my university. You’re right that having to deal with differences in Octave and MATLAB syntax is gonna be an enormous challenge, which renders this kind of transpiler unfeasible. I also agree that the opaque MATLAB toolboxes are big stumbling blocks for the said transpiler. It may never happen like that.
But suffice it to say, not all MATLAB code uses all the complicated stuff and toolboxes (well, depending on the project actually). If a partial translator (not transpiler) can do the translation of basic syntax, and warns the user when it fails to parse the code, like toolbox functions and some arcane syntax, then perhaps it would still be of some use in porting the code. Right now, I share your vision that community Wiki pages would be a suitable venue to inform people on how to port MATLAB code to Julia. Then, I think Julia should make a dedicated Wiki site for these things (like Haskell has a Haskell Wiki).
1 Like
Yep, totally agree, in general there won’t be a one size fits all answer so multiple resources are good when they aren’t completely redundant to one another.
If/when there’s a Julia community wiki to where the one I started could be migrated then I’ll be all for doing that. I’d step up to create that larger-scoped wiki if I thought I could devote sufficient attention to it but for now that’s not the case.
1 Like
IMO Julia requires a very different approach (small, modular functions, loops or broadcasting instead of vectorization, attention to type stability, etc), so while it is relatively easy to make a “literal translation” of simple Matlab scripts, the performance is often disappointing because of these gotchas. I have seen some people do this and conclude that Julia is super-slow.
Also, while there are of course exceptions to this, many Matlab scripts that I have seen are just badly written from a software engineering perspective — brittle to implicit or hidden assumptions, composed of large, monolithic functions, makign extensive use of global variables, and consequently lacking unit tests.
I am not suggesting that this is a feature of Matlab, but rather the culture that has grown around it, with books listing code snippets as solutions and suggesting that you solve problems by copy-pasting and modifying previous code. There is a lot of Matlab code out there, but perhaps the part that should be saved for posterity should be rewritten. This is in sharp contrast to, say, Fortran code I see floating around: most of it is carefully documented, organized, and tested.
7 Likes
The julia wikibook exists and is well maintained (largely thanks to @cormullion).
I suggest you could create a chapter there and migrate it.
4 Likes
I want to second this opinion. I recently spent a week or two converting Matlab code for an intensive project into Julia. The code is mainly data processing with some fminunc optimization. I had thought I would impress my collaborators by speeding up the computation by a factor of 2 or more. When it was all over, the Julia code ran at about half speed relative to the Matlab code. Since none of my collaborators knows Julia, I just gave up and went back to Matlab.
I spent awhile trying to weed out the gotchas from my code (for example, the previous iteration of the Matlab code was all done in global scope—I have since “functionized” everything in Matlab, so my exercise wasn’t a complete waste), but in the end I don’t have quite enough skill with Julia to understand why the performance is bad. Is there an inherent performance difference between fminunc and Optim.jl? Is there a performance difference between Julia’s loop fusion syntax and loops? I just didn’t know where to start to understand the performance issues.
My point in sharing this is simply to illustrate the common pitfalls of someone coming from Matlab to Julia. Providing specific examples of Julia gotchas that are ingrained in Matlab users’ coding habits would be a huge help. And maybe like @HCrepaz said, there may not be a perfect Julia replacement for every Matlab project (though, for me, there ought to be, since I’m only processing data and calling fminunc!).
As a side note: does anyone know of a resource to understand the output of the code profiler? Whenever I have a question about performance, I come on here, and it seems that everyone already knows how to do it. I haven’t seen anything on the main Julia documentation, other than to say, “Use the code profiler.” A tutorial on this theme, perhaps geared towards Matlab users, would also be useful (if it doesn’t already exist).
EDIT: One other feature that might be useful to distinguish is that A = B in Matlab makes a deep copy of B, whereas in Julia you would need to say A = deepcopy(B) to get the same behavior.
1 Like
This comes probably because Matlab and Fortran are used in different stages of the software development cycle. Being an interpreted and dynamically typed language like Python, Matlab is much better suited for quick and sometimes messy prototyping and early algorithm development than the compiled and strongly typed Fortran. So programs which are encoded in Fortran tend to be naturally more mature.
You could try your luck with
I’ve found it useful optimizing code in the past. It’s really dead simple to use
5 Likes
Reading the output files from the profiler directory is impossible.
They are in a standard format, uses by many other languages.
But those programmers also act like everyone already knows how to read them.
ProfileView.jl makes them graphical and trivially understandable
3 Likes
For those using Juno, @profiler is excellent: http://docs.junolab.org/latest/man/juno_frontend.html#Profiler-1
It gives a nice pannable/zoomable visual output and clicking on the relevant regions takes you to the corresponding line of code.
4 Likes
Wow… that may make me switch back from vscode 