# Designing a Paths Julep

**URL:** <https://discourse.julialang.org/t/designing-a-paths-julep/124335>\
**Category:** Internals & Design\
**Tags:** proposal, filesystem, path, rfc\
**Created:** [January 1, 2025, 12:37pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335 "2025-01-01T12:37:25Z")\
**Posts on this page:** 20\
**Page:** 5

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [January 11, 2025, 11:21pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/82 "2025-01-11T23:21:05Z")

</div>

This is a fascinating discussion! Thanks everyone for their input. I have some opinions, but won’t weigh in on most of the controversies because all of the suggestions seem better than the status quo, and other people have more informed opinions than me.

Two things I do want to mention. First, a couple of times file extensions have been mentioned, and I just wanted to flag that I remember having issues in python a decade or so ago when dealing with multiple extensions (eg `my_file.csv.gz`) or less frequently but very annoying, when dots were used in file names (eg `my.file.name.txt`). I don’t know if there are elegant solutions here, just an observation.

But I do have somewhat strong opinions regarding the relationship with strings. I really love the idea behind FilePaths.jl, and have tried to use it many times, but have run into way too many situations where I have to covert back and forth between strings to make modifications, and I always abandon it.

As an example, a situation comes up routinely in practice is that I need to rename a file from one form of annotation to another. For example

```julia
conversions = Dict(
    "sample1" => "db967",
    "sample2" => "db888",
   #etc
)

files =[
    "some/dir/sample1_1.fastq",
    "some/dir/sample1_2.fastq",
    "other/dir/sample1_processed.csv"
    "some/dir/sample2_1.fastq",
    "some/dir/sample2_2.fastq",
    "other/dir/sample2_processed.csv"
    #etc
]

for oldfile in files
    dir = dirname(file)
    filename = basename(file)
    sample = match(r"^sample\d", filename).match
    newname = replace(filename, sample => conversions[sample]
    newfile = joinpath(dir, newname)
    mv(oldfile, newfile)
end

```

Because I need things like regex matching and string replacements, in order to use FilePaths.jl I was constantly needing to convert to strings and back to paths, so whatever benefit I was getting using explicit path types was swamped by boilerplate.

I think the idea of explicit path types is compelling, but whatever the merits of “a path is not a string”, enforcing that super rigidly is, in practice, annoying.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [January 12, 2025, 8:28am UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/83 "2025-01-12T08:28:26Z")

</div>

> [@kevbonham](#):
>
> As an example, a situation comes up routinely in practice is that I need to rename a file from one form of annotation to another.

To be honest, that seems more like a missing feature of Base / FilePaths.jl. It really shouldn’t be an issue to provide a flexible `rename` function. I suppose in filesystem terms this is just a `move`, but still, there shouldn’t be any issue with doing that.

Perhaps easier filtering of lists of paths would enable this?

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [January 12, 2025, 10:25am UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/84 "2025-01-12T10:25:15Z")

</div>

> [@Sukera](#):
>
> a flexible `rename` function

Regarding naming, on Julia nightlies, `Base.rename` is the low-level primitive used for implementing `mv`. `Base.rename` is now `public`: [Make `rename` public by nhz2 · Pull Request #55652 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/55652)

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [January 12, 2025, 4:07pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/85 "2025-01-12T16:07:46Z")

</div>

> [@nsajko](#):
>
> `Base.rename` is the low-level primitive used for implementing `mv`. `Base.rename` is now `public`

I suppose that’s nice, but AFAICT, it just changes the last line of my loop there from `mv` to `rename`.

> [@Sukera](#):
>
> It really shouldn’t be an issue to provide a flexible `rename` function

If there was a function that takes a function as the first argument to determine the rename, and allows one to treat the file name as a String w/r/t regex matching / replacing, and handles the conversion into and out of whatever file path types, that could definitely work.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [January 12, 2025, 4:27pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/86 "2025-01-12T16:27:38Z")

</div>

I agree that it’s not good if people need to do a bunch of path-string-path roundtripping just to do the transformations they need. Maybe it could be interesting to define `replace(path, ...)` in a way that it works segment by segment if you pass a string or a regex match. If you include path separators in your string logic then you’re back to manually handling forward slash / backward slash.

Maybe another option could be a multipart `replace` where you pass a tuple or vector of parts and they match the path segments sequentially, each one taking a separate substitution string or so. Like `replace(p"a/b/c", (r"a", r"b") => (s"x", s"y"))` or so.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [January 13, 2025, 2:57pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/87 "2025-01-13T14:57:37Z")

</div>

> [@kevbonham](#):
>
> But I do have somewhat strong opinions regarding the relationship with strings. I really love the idea behind [FilePaths.jl](https://juliaregistries.github.io/General/packages/redirect_to_repo/FilePaths), and have tried to use it many times, but have run into way too many situations where I have to covert back and forth between strings to make modifications, and I always abandon it.

I think my list of “potential helper functions” from earlier may be relevant here:

> [@tecosaur](#):
>
> I’m tempted to take a leaf from other ecosystems with more path functions, the rather nice CL library I reference in the design doc [fosskers/filepaths: Modern and consistent filepath manipulation for Common Lisp. - Codeberg.org](https://codeberg.org/fosskers/filepaths) has:
> 
> ```julia
> > (filepaths:with-base #p"/foo/bar/baz.txt" "jack")
> #p"/foo/bar/jack.txt"
> 
> > (filepaths:with-name #p"/foo/bar/baz.txt" "jack.json")
> #p"/foo/bar/jack.json"
> 
> > (filepaths:with-parent #p"/foo/bar/baz.json" #p"/zing")
> #p"/zing/baz.json"
> 
> > (filepaths:with-extension #p"/foo/bar/baz.txt" "json")
> #p"/foo/bar/baz.json"
> 
> > (filepaths:add-extension #p"/foo/bar/baz.txt" "zip")
> #p"/foo/bar/baz.txt.zip"
> 
> > (filepaths:drop-extension #p"/foo/bar/baz.json")
> #p"/foo/bar/baz"
> 
> ```
> 
> Rust has the subset: `with_file_name`, `with_extension`, `with_added_extension`.

With a `withname` function, your example would become:

```julia
for oldfile in files
    filename = basename(file)
    newname = replace(filename, r"^sample\d" => s -> conversions[s])
    mv(oldfile, withname(oldfile, newname))
end

```

Without `withname`, you’d have to do `newfile = p"$(parent(oldfile))/$newname"` (note that this doesn’t handle one of the motivating edgecases for `withname`: an `oldfile` with no parent).

You’d still have to do String ↔ Path conversion if you want to modify the parent path with regex, but I don’t see any major usability hurdles in general from this.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [January 13, 2025, 2:59pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/88 "2025-01-13T14:59:58Z")

</div>

Without jumping too far ahead here, I think there’s definitely a nice `replace(::Path, ...)` method along the lines you’re thinking 🙂

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [January 13, 2025, 3:28pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/89 "2025-01-13T15:28:31Z")

</div>

> [@tecosaur](#):
>
> Rust has the subset: `with_file_name`, `with_extension`

Note that we already have convenient interface for these operations with current `String` paths with Accessors.jl:

```julia
julia> using Accessors

julia> p = "/home/aplavin/file.txt"
"/home/aplavin/file.txt"

julia> @set basename(p) = "some.jpg"
"/home/aplavin/some.jpg"

julia> @set dirname(p) = "/root"
"/root/file.txt"

julia> @set splitext(p)[2] = ".jpg"
"/home/aplavin/file.jpg"

julia> @modify(uppercase, basename(p))
"/home/aplavin/FILE.TXT"

```

Basically, this is the general solution for all “I want a similar object but with `f(x) = newval`” problems. And this interface will (of course) remain possible for any new path implementation.

Seems much cleaner than creating and learning new functions for everything: you don’t really need `basename` + `set_basename` if you have `set` and `basename` separately 🙂

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [January 13, 2025, 3:55pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/90 "2025-01-13T15:55:28Z")

</div>

That’s nice an all, but I don’t think the answer for “how to perform basic path modifications” should be to use a 3rd party package.

FWIW, I am familiar with this approach, namely `setf` and generalised variables in Lisp-land. If there was something like that in base Julia, I’d be more onboard with this approach.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [January 13, 2025, 4:14pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/91 "2025-01-13T16:14:53Z")

</div>

> [@tecosaur](#):
>
> I don’t think the answer for “how to perform basic path modifications” should be to use a 3rd party package

That’s fair, but `Base` itself just has to provide _some_ interface – generic, but not necessarily the one with the least boilerplate or convenient in _all_ situations. That’s a common situation already, and paths aren’t really special.

Like, you can do `joinpath(dirname(p), "some.jpg")` with just `Base`, but `@set basename(p) = "some.jpg"` requires a package.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [January 13, 2025, 4:18pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/92 "2025-01-13T16:18:46Z")

</div>

> [@aplavin](#):
>
> Like, you can do `joinpath(dirname(p), "some.jpg")` with just `Base`, but `@set basename(p) = "some.jpg"` requires a package.

This is exactly the problem, without a `withname` function (and friends) the “obvious” code you write without a 3rd party package is wrong. `parent(p"old.jpg")` is `nothing` when there is no parent, rather than the empty string. Currently this example “works” because of what is essentially a quirk in how `joinpath` interprets empty strings. This is a large part of the motivation for functions like `withname`, `withsuffix`, etc.

I think it’s worth noting that these approaches aren’t mutually exclusive, there can be a `withname` (or similar) function in Base, and `@set basename(p) = "new"` can use `withname` itself.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [January 13, 2025, 4:20pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/93 "2025-01-13T16:20:16Z")

</div>

> [@tecosaur](#):
>
> `parent(p"old.jpg")` is `nothing` when there is no parent, rather than the empty string

Not sure if that’s a good thing, having a neutral element for functions like `joinpath` is generally useful.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [January 13, 2025, 4:26pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/94 "2025-01-13T16:26:23Z")

</div>

More generally, I skimmed through this thread but still not sure I understand the scope. Is this julep expected to accomodate stuff like remote paths?

Current Julia filesystem API makes it kinda-possible to define the Base FS interface for remotes, so that users can write functions that work with either local or remote files. But it definitely requires some care.  
One example is my package to work with [OSF.io](http://OSF.io): [GitHub - JuliaAPlavin/OpenScienceFramework.jl](https://github.com/JuliaAPlavin/OpenScienceFramework.jl).

Would be great not to lose this ability at least, and make it more principled and convenient at best.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [January 13, 2025, 4:27pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/95 "2025-01-13T16:27:19Z")

</div>

> [@aplavin](#):
>
> More generally, I skimmed through this thread but still not sure I understand the scope. Is this julep expected to accomodate stuff like remote paths?

I’d recommend looking at the HackMD doc rather than this thread for the proposal itself.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [January 13, 2025, 4:27pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/96 "2025-01-13T16:27:57Z")

</div>

I didn’t find any mention of “remote” or its synonyms there though.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [January 13, 2025, 4:29pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/97 "2025-01-13T16:29:04Z")

</div>

Right, because they’re not an explicit part of the current proposal.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [January 13, 2025, 4:29pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/98 "2025-01-13T16:29:22Z")

</div>

Another piece of prior art, a popular Python package [fsspec: Filesystem interfaces for Python — fsspec 2024.10.0.post13+gdbed2ec.d20241115 documentation](https://filesystem-spec.readthedocs.io/en/latest/).

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [January 13, 2025, 4:32pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/99 "2025-01-13T16:32:59Z")

</div>

> [@tecosaur](#):
>
> Right, because they’re not an explicit part of the current proposal.

Is it because you expect this interface to automatically be feasible for remotes without extra effort, or explicitly don’t want to cover those usecases?

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [January 13, 2025, 4:36pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/100 "2025-01-13T16:36:54Z")

</div>

> [@aplavin](#):
>
> Is it because you expect using this interface to automatically be feasible for remotes without extra effort, or explicitly don’t want to cover those usecases?

I think it’s worth keeping in mind that this is a _Path_ interface not a _Filesystem_ interface. That would be a rather different (and more involved) proposal altogether. Not to say it’s a bad idea (I think in recent years Go added a nice generic FS interface IIRC), it’s just not this idea.

This proposal introduces a structured way of working with local filesystem paths, and attempts to find a sensible and minimal abstraction for other paths (URIs, S3, etc.) in `AbstractPath`.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [January 13, 2025, 4:43pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/101 "2025-01-13T16:43:03Z")

</div>

> [@tecosaur](#):
>
> I think it’s worth keeping in mind that this is a _Path_ interface not a _Filesystem_ interface.

To reduce confusion, you may want to clarify that in the proposal and clearly define the scope – especially given that in Julia the [Filesystem · The Julia Language](https://docs.julialang.org/en/v1/base/file/) docs page describes both.

> [@tecosaur](#):
>
> attempts to find a sensible and minimal abstraction for other paths (URIs, S3, etc.)

That’s what I tried to find in the proposal, but didn’t see any mention whatsoever.

Now, it’s kinda-possible to write code that works with paths, and later apply it to remote files or files within archives. Is this intended to be possible within this proposal as well?

[Previous page](https://discourse.julialang.org/t/designing-a-paths-julep/124335.md?page=4)

[Next page](https://discourse.julialang.org/t/designing-a-paths-julep/124335.md?page=6)